Don't call closedir() on directories that didn't exist