Optimizes using fnmatch only if needed

Skips fnmatch'ing if there is only a '*' in filter.