Unexpected keyword argument in packages/urllib3/connectionpool.py

This is what I get during a requests session under python 3.3:

Traceback (most recent call last):
  File "/.../python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 319, in _make_request
    httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'


@jmandreoli I'm certain that is not the entire of your traceback and it is likely related to #1915 and #1289. Your issue is not that, because we were handling that exception when a different one occurred (with certainty). You need to give us the version of requests and what the _entire_ traceback is.


On 04/07/14 19:20, Ian Cordasco wrote:

> @jmandreoli https://github.com/jmandreoli I'm certain that is not the
> entire of your traceback and it is likely related to #1915
> https://github.com/kennethreitz/requests/issues/1915 and #1289
> https://github.com/kennethreitz/requests/issues/1289. Your issue is
> not that, because we were handling that exception when a different one
> occurred (with certainty). You need to give us the version of requests
> and what the /entire/ traceback is.

I send you enclosed the full traceback.
- For me, the "unexpected keyword argument 'buffering'" error may not be 
  the main cause of the problem, but it must still be a bug (invoking a 
  method with a keyword argument which does not exist).
- The version of python/requests is

> Python 3.3.5 |Continuum Analytics, Inc.| (default, Mar 10 2014, 11:19:31)
> [GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> 
> > > > import requests
> > > > requests.**version**
> > > > '2.3.0'
- I must also point out that the error does not always happen.
- If it can help, I also send you enclosed the relevant section of the 
  code. Note that the generator "egxmonitor" is enumerated only at low 
  frequency (once every 2 secs), so it is not like the server is bombarded 
  with requests in the "while True" loop.

Thanks for looking into this issue anyway, and if you find something, 
please let me know.

```
Jean-Marc
```

Traceback (most recent call last):
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 319, in _make_request
    httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 493, in urlopen
    body=body, headers=headers)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 321, in _make_request
    httplib_response = conn.getresponse()
  File "/.../anaconda/envs/py3k/lib/python3.3/http/client.py", line 1147, in getresponse
    response.begin()
  File "/.../anaconda/envs/py3k/lib/python3.3/http/client.py", line 358, in begin
    version, status, reason = self._read_status()
  File "/.../anaconda/envs/py3k/lib/python3.3/http/client.py", line 328, in _read_status
    raise BadStatusLine(line)
http.client.BadStatusLine: ''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/adapters.py", line 327, in send
    timeout=timeout
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 543, in urlopen
    raise MaxRetryError(self, url, e)
requests.packages.urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='13.202.222.220', port=80): Max retries exceeded with url: /UE/Post__PL__Data (Caused by <class 'http.client.BadStatusLine'>: '')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/.../chronorec.py", line 161, in newsession
    for itern,x in enumerate(source,1):
  File "/.../chronorec.py", line 526, in atinterval
    for x in it:
  File "/.../src/pwr/egx.py", line 30, in egxmonitor
    r = session.post(url,auth=login,data=data)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/sessions.py", line 498, in post
    return self.request('POST', url, data=data, *_kwargs)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, *_send_kwargs)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/adapters.py", line 375, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='13.202.222.220', port=80): Max retries exceeded with url: /UE/Post__PL__Data (Caused by <class 'http.client.BadStatusLine'>: '')


@jmandreoli It is not a bug. The relevant code is reproduced here in full (it lives [here](https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/connectionpool.py#L318-L321) for context):

``` python
try: # Python 2.7+, use buffering of HTTP responses
    httplib_response = conn.getresponse(buffering=True)
except TypeError: # Python 2.6 and older
    httplib_response = conn.getresponse()
```

This code block exists because we want to use the `buffering` kwarg if we can on platforms that have it, but it isn't present on Python 2.6 or some versions of Python 3 (where it's the default). The style (attempt to use it, fail if we hit the exception and try without it) is a programming style usually summarised as "it's easier to ask forgiveness than permission". It's a very common Python programming style.

_Unfortunately_, the chained tracebacks feature in Python 3.X trips people up when using this style because you frequently see exceptions that are of that form. In this case, the TypeError is totally expected.

The _relevant_ exception is the second one: `BadStatusLine`. The subsequent two exceptions are just wrapper exceptions and can be ignored. Looks like you're receiving a malformed response: you should use Wireshark or tcpdump to examine what is actually being received when you're hitting this error.

`<digression>`

I am not a fan of Python 3's chained exceptions being on by default. They're a very useful debugging feature, but they're also a classic example of an expert-level interface. If you don't expect to see them they're hugely difficult to read.

This is the perfect example: you posted four tracebacks. It's ridiculous to ask a non-expert requests user to work out which the hell one of those is the exception that matters. All you can know is that the last one is the one you'd have had to catch.

_I_ know which one matters, but I've worked on this library for three years. This means that when I debug I can tell the difference between things that should happen and things that shouldn't, and so the extra context is helpful. For people who can't tell the difference (i.e. non-experts), the extra context is noise that makes debugging harder. On Python 2, you'd have seen this:

```
Traceback (most recent call last):
  File "/.../chronorec.py", line 161, in newsession
    for itern,x in enumerate(source,1):
  File "/.../chronorec.py", line 526, in atinterval
    for x in it:
  File "/.../src/pwr/egx.py", line 30, in egxmonitor
    r = session.post(url,auth=login,data=data)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/sessions.py", line 498, in post
    return self.request('POST', url, data=data, **kwargs)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/.../anaconda/envs/py3k/lib/python3.3/site-packages/requests/adapters.py", line 375, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='13.202.222.220', port=80): Max retries exceeded with url: /UE/Post__PL__Data (Caused by <class 'http.client.BadStatusLine'>: '')
```

That exception is much more immediate: we had an error during connection (`ConnectionError`), which is the result of hitting our retry limit with a specific URL (`Max retries exceeded`), and the things that caused us to retry were `BadStatusLine` errors.

For all of Python 3's moves to make things easier for non-experts, this one made it harder for them. I find that inconsistency frustrating.

`</digression>`


I suspected the server, but I was indeed under the impression that the 
"unexpected keyword" error was what prevented the package from 
recovering from the server error. Your explanation made it clear what 
was happening (I was not aware of this Python3 change in chained traceback).

I have just added a try-catch around the session.post, and forced a 
repeat (up to some limit) if there is a RequestException. This works. 
Thanks very much for your help.

```
Jean-Marc
```


No problem, I'm glad you were able to resolve it. =)


@jmandreoli I am facing a similar issue, can you tell how you hadled it