Remove dirent.h when WITHOUT_FILE is defined.