twitter/commons — pull request #211 by msabramo (2014-02-10 23:13:50+00:00)
Title: Prevent pants.bootstrap failure when crawl a page and get BadStatusLine

Fixes GH-210.

Without:

```
marca@marca-mac2:~/dev/git-repos/twitter.commons_2$ ./pants.bootstrap
Build operating on targets: OrderedSet([PythonBinary(src/python/twitter/pants/BUILD:pants)])
Building PythonBinary PythonBinary(src/python/twitter/pants/BUILD:pants):
Exception in thread Thread-1:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/marca/dev/git-repos/twitter.commons_2/src/python/twitter/common/python/http/crawler.py", line 44, in execute
hrefs, rel_hrefs = self.execute(url)
File "/Users/marca/dev/git-repos/twitter.commons_2/src/python/twitter/common/python/http/crawler.py", line 150, in execute
return self._remote_execute(url)
File "/Users/marca/dev/git-repos/twitter.commons_2/src/python/twitter/common/python/http/crawler.py", line 135, in _remote_execute
with contextlib.closing(self._open(url, conn_timeout=self._conn_timeout)) as index_fp:
File "/Users/marca/dev/git-repos/twitter.commons_2/src/python/twitter/common/python/http/http.py", line 206, in open
self.cache(url, conn_timeout=conn_timeout)
File "/Users/marca/dev/git-repos/twitter.commons_2/src/python/twitter/common/python/http/http.py", line 195, in cache
self.encode_url(url, conn_timeout=conn_timeout)
File "/Users/marca/dev/git-repos/twitter.commons_2/src/python/twitter/common/python/http/http.py", line 168, in encode_url
with contextlib.closing(self.really_open(url, conn_timeout=conn_timeout)) as http_fp:
File "/Users/marca/dev/git-repos/twitter.commons_2/src/python/twitter/common/python/http/http.py", line 161, in really_open
return self._opener.open(url, conn_timeout=conn_timeout)
File "/Users/marca/dev/git-repos/twitter.commons_2/src/python/twitter/common/python/http/http.py", line 114, in open
return urllib_request.urlopen(url, **kw)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 404, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 422, in _open
'_open', req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1214, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1187, in do_open
r = h.getresponse(buffering=True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1045, in getresponse
response.begin()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 409, in begin
version, status, reason = self._read_status()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 373, in _read_status
raise BadStatusLine(line)
BadStatusLine: ''
```

With:

```
marca@marca-mac2:~/dev/git-repos/twitter.commons_2$ ./pants.bootstrap
Build operating on targets: OrderedSet([PythonBinary(src/python/twitter/pants/BUILD:pants)])
Building PythonBinary PythonBinary(src/python/twitter/pants/BUILD:pants):
Wrote /Users/marca/dev/git-repos/twitter.commons_2/dist/pants.pex
Pants 0.0.10 @ PANTS_BUILD_ROOT: /Users/marca/dev/git-repos/twitter.commons_2

Available subcommands:
py	Python chroot manipulation.
help	Provides help for available commands or a single specified command.
build	Builds a specified target.
setup_py	Generate setup.py-based Python projects from python_library targets.
goal	Lists installed goals or else executes a named goal.

Default subcommand flags can be stored in ~/.pantsrc using the 'options' key of a
section named for the subcommand in ini style format, ie:
[build]
options: --log-exit
pants.pex version 16d0fe7c66ba15033533d6b927b516e22b215d00
```