BadStatusLine on Server restart

Hi,

from time to time we this error:

```
File "/home/foo_esm_di226/lib/python2.7/site-packages/requests/api.py", line 49, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/home/foo_esm_di226/lib/python2.7/site-packages/requests/sessions.py", line 461, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/foo_esm_di226/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/home/foo_esm_di226/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine("''",))
```

This happens if the request library tries to connect during an Apache server restart.

I think it would be nice if the request library would retry if this happens.

But I guess you see it different. 

Would you accept a patch which retries on an empty HTTP Status line?


1. We don't retry most errors by default
2. If you use a session, you can [configure retries on your own](http://www.coglib.com/~icordasc/blog/2014/12/retries-in-requests.html) without having to wait for a release of requests
3. Not every `BadStatusLine` error is retry-able. In fact, for the most, a `BadStatusLine` can mean a very large number of things and if we retry those automatically for users it could end up in their request taking much longer than they intend. Timeouts would apply to each request in the retried chain as opposed to the whole chain so they wouldn't even be able to set a timeout to limit the execution of the retries without relying on a third-party library.
4. I don't remember if urllib3's retries (which power requests' retries) will even work with a `BadStatusLine` error or if they need to be based around HTTP status codes


BadStatusLine usually indicates the request made it to the server, so it's
a tricky retry situation; you can generally retry idempotent requests (GET,
PUT, DELETE) but not anything side effecting.

The "Connection aborted" message is a little misleading, the connection
worked just fine I believe, the server just sent back the empty string
instead of an HTTP response..

## 

Kevin Burke
phone: 925.271.7005 | twentymilliseconds.com

On Mon, Jun 15, 2015 at 7:17 AM, Ian Cordasco notifications@github.com
wrote:

> 1. We don't retry most errors by default
> 2. If you use a session, you can configure retries on your own
>    http://www.coglib.com/%7Eicordasc/blog/2014/12/retries-in-requests.html
>    without having to wait for a release of requests
> 3. Not every BadStatusLine error is retry-able. In fact, for the most,
>    a BadStatusLine can mean a very large number of things and if we retry
>    those automatically for users it could end up in their request taking much
>    longer than they intend. Timeouts would apply to each request in the
>    retried chain as opposed to the whole chain so they wouldn't even be able
>    to set a timeout to limit the execution of the retries without relying on a
>    third-party library.
> 4. I don't remember if urllib3's retries (which power requests'
>    retries) will even work with a BadStatusLine error or if they need to
>    be based around HTTP status codes
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2642#issuecomment-112085194
> .


> The "Connection aborted" message is a little misleading, the connection worked just fine I believe

I disagree. We rely on `httplib`/`http.client` still and in [some cases](https://hg.python.org/cpython/file/b2a37f0ef915/Lib/http/client.py#l264) it closes the connection before raising a `BadStatusLine`. We cannot always introspect whether it has so `Connection aborted` is fairly accurate.


`BadStatusLine` is extremely unclear. It can occur in a number of situations, and makes no guarantees about whether the request made it through. It just fundamentally means httplib failed to parse the header line.

In _this_ instance it's unlikely that the server "sent back the empty string" because that's not a thing you can meaningfully do in TCP (I guess you could try emitting a zero length packet? not sure if that's valid TCP). What this means is that the connection got closed, likely by the shutdown of the server, without any data left in the send buffer. That will have caused a zero-length read.

A good example of why we _shouldn't_ blindly retry in this case is that plenty of servers will reject requests they don't like by just closing the connection, which will cause exactly this error. This kind of thing tells you nothing about whether the request succeeded. This means @kevinburke's advice is right (only retry idempotent requests), but not quite for the right reason.

</pedantry>


What is your advice for me - a requests lib user?


My advice is that you should either follow [this blog post](http://www.coglib.com/~icordasc/blog/2014/12/retries-in-requests.html), or you should be catching exceptions around your requests and choosing to retry yourself.

Requests cannot protect you from the network exploding underneath you. The best we can do is give you tools to help manage it, which that blog post does. =) Sadly, networks are unreliable beasts, and it's dangerous to pretend otherwise.


Do you have an advice how to create a BadStatusLine in a test?

I know who to use mocking, but I am not sure where to inject a matching "raise ..." line.

The exception in real world looks like this:

```

  File "/home/foo_esm_d/src/foo/footests/test_foo.py", line 35, in testAAA_need_authentication
    response=requests.get(url, verify=False)
  File "/home/foo_esm_d/lib/python2.7/site-packages/requests/api.py", line 69, in get
    return request('get', url, params=params, **kwargs)
  File "/home/foo_esm_d/lib/python2.7/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/home/foo_esm_d/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/foo_esm_d/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/home/foo_esm_d/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine("''",))
```


Yup, it's easy. Have a socket server that does this:

``` python
import socket

s = socket.socket()
s.bind(('0.0.0.0', 8080))
s.listen(5)

while True:
    b = s.accept()

    while True:
        data = b.recv(65535)
        if b'\r\n\r\n' in data:
            break

    b.close()
```

Then, hit the server with a call from requests. Should cause exactly the bug you're looking at.


On our continuous integration server several runs happen in parallel. It is likely that port 8080 is already in use. :-(

Is there a way to use the requests lib (client part) via unix-domain sockets?


Port 8080 is not special, you can change it to whatever you want in your system. Alternatively, you can set it to zero which means 'any port', and then use `s.getsockname()[1]` to find out what port you were assigned.


@Lukasa  thank you very much! I was not aware of the "zero means any port" method.

For the unix-sockets way I found this:

https://pypi.python.org/pypi/requests-unixsocket/