Filter out ignored files, not ignored dirs