Can you unlink a directory if it's non-empty? I think this will fail if there are *.pyc files laying around