mozilla/mozregression — pull request #78 by cpeterso (2014-01-06 03:17:05+00:00)
Title: Optimize inbound finder parsing

`inboundfinder.py`'s `getInboundRevisions` function had a number of optimization opportunities:

1. The *.txt regex was also matching *.txt.gz log files, so the script would download ~20 ~1MB *.txt.gz log file *per revision* then attempt to search the .gz file for "/rev/". [This directory](http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-macosx64/1388768786/) is a good example of many *.txt.gz log files.

2. Once the *.txt regex found the correct revision txt file, the function did not break from the loop early, so it would continue download all the mismatched *.txt.gz files from #1.

3. Replacing urllib2 with the [requests library](http://docs.python-requests.org/en/latest/), which will automatically opt-into and handle "Content-Encoding: gzip". This is a small help for downloading the ftp directory pages, but not help for download the already-compressed Firefox installers.

4. I didn't implement it, but a more difficult `inboundfinder.py` optimization would be to lazily parse the revision directories because most of them will not be needed. The build timestamp directories are in chronological order and, assuming the given --good-rev will precede the --bad-rev, `inboundfinder.py` could binary search the timestamp directories for the good and bad revisions' directories. Then the `Bisector` could binary search the timestamp directories, parsing just the directories' revision txt files needed.