So I wanted readline to 'just work' when entering paths in conjunction with raw_input(). Turns out that this is rather strange, undocumented and full of tripwires. This is how i did it:

import os
import sys
import readline


def list_folder(path):
    """
    Lists folder contents
    """
    if path.startswith(os.path.sep):
        # absolute path
        basedir = os.path.dirname(path)
        contents = os.listdir(basedir)
        # add back the parent
        contents = [os.path.join(basedir, d) for d in contents]
    else:
        # relative path
        contents = os.listdir(os.curdir)
    return contents


def completer(text, state):
    """
    Our custom completer function
    """
    options = [x for x in list_folder(text) if x.startswith(text)]
    return options[state]

readline.set_completer(completer)

if sys.platform == 'darwin':
    # Apple uses libedit.
    readline.parse_and_bind("bind -e")
    readline.parse_and_bind("bind '\t' rl_complete")
else:
    # Some tweaks for linux
    readline.parse_and_bind('tab: complete')
    readline.set_completer_delims(' \t\n`~!@#$%^&*()-=+[{]}\\|;:\'",<>?')


### Try out the stuff:
while True:
    last_string = raw_input('? ')
    print 'Last input:', last_string

Basically, readline would not work at all under MacOS until you use the little dance above.
Assuming it would work out of the box under linux was wrong as well, since os.path.sep (/ for that matter) is in the list of completer_delims, resulting in the fact that it is not being passed to the completer function. For example, sending it /home will just make it receive home. Fortunately, you can override this with readline.set_completer_delims().
For even more user-friendlyness, you could add a trailing slash in the list_folder function, like: contents = [d + os.path.sep for d in contents if os.path.isdir(d)].