okcompute/p4clean — pull request #18 by johntconklin (2014-03-11 04:02:30+00:00)
Title: Compile exclusion_list into a regular expresion object

Since the regular expression match occurs against every file, I was looking to see if it could be made any faster. I thought that using fnmatch.fnmatch() against each element in the exclusion_list might be faster, since fnmatch() is usually much faster than regular expression in C code. I wrote benchmark that compared the two with the list of files from the Python-2.7 distribution. I was surprised to find that fnmatch() was much slower, until I looked at the implementation and found that it converted the glob pattern into a regular expression under the hood. For 200 iterations, fnmatch() took 4.44 seconds, while re.match() took 1.47 seconds.

But l then realized that re.match() was recompiling the regex for each comparison. Since we're using the same pattern for each file, we can compile it once and match against that regular expression object. The same benchmark completed in 0.57 seconds, nearly 3 times as fast.

I also used os.path.join() to create the path for excluding the .p4clean config file, figuring that one pattern in the list would be faster than two.

This change took about 5 seconds off the runtime of "p4clean -n" off my benchmark dirty workspace.