ConnectionResetError on some urls

I'm building Django app for translating english words to polish. I'm trying to make a request to dictionary website. When requesting for url: 'http://pl.bab.la/slownik/angielski-polski/' after few minutes it throws ConnectionResetError, but Wireshark sees 200 OK response less than second after request. 
The strange thing is that when url is 'http://pl.bab.la/slownik/' it works.

```
url = 'http://pl.bab.la/slownik/angielski-polski/'
response  = requests.get(url)
```

Using urllib everything works fine.

```
import urllib.request
req = urllib.request.Request(url)
response = urllib.request.urlopen(req)
```

[See full traceback](https://gist.github.com/Alexander3/6b2fe33fd1b98c31a2008fa98ad6d896)
I'm using Python 3.5.1 on Windows 10 64 bit


Interestingly, this seems to be a Python 3 bug: I can reproduce this repeatably on Python 3.5.1, but am unable to reproduce it on Python 2.7. Not a Windows bug though: I can reproduce on macOS.


Got it. Revealed, as always, by using `stream=True`. The problem is the headers. The server is sending a _gigantic_ Link header that is UTF-8 encoded:

```
Link: <http://www.babla.cn/英语-波兰语/>; rel="alternate"; hreflang="zh-Hans", <http://cs.bab.la/slovnik/anglicky-polsky/>; rel="alternate"; hreflang="cs", <http://da.bab.la/ordbog/engelsk-polsk/>; rel="alternate"; hreflang="da", <http://de.bab.la/woerterbuch/englisch-polnisch/>; rel="alternate"; hreflang="de", <http://www.babla.gr/αγγλικα-πολωνικα/>; rel="alternate"; hreflang="el", <http://en.bab.la/dictionary/english-polish/>; rel="alternate"; hreflang="en", <http://eo.bab.la/vortaro/angla-pola/>; rel="alternate"; hreflang="eo", <http://es.bab.la/diccionario/ingles-polaco/>; rel="alternate"; hreflang="es", <http://fi.bab.la/sanakirja/englanti-puola/>; rel="alternate"; hreflang="fi", <http://fr.bab.la/dictionnaire/anglais-polonais/>; rel="alternate"; hreflang="fr", <http://www.babla.in/अंग्रेज़ी-पोलिश/>; rel="alternate"; hreflang="hi", <http://hu.bab.la/szótár/angol-lengyel/>; rel="alternate"; hreflang="hu", <http://www.babla.co.id/bahasa-inggris-bahasa-polandia/>; rel="alternate"; hreflang="id", <http://it.bab.la/dizionario/inglese-polacco/>; rel="alternate"; hreflang="it", <http://ja.bab.la/辞書/英語-ポーランド語/>; rel="alternate"; hreflang="ja", <http://www.babla.kr/영어-폴란드어/>; rel="alternate"; hreflang="ko", <http://nl.bab.la/woordenboek/engels-pools/>; rel="alternate"; hreflang="nl", <http://www.babla.no/engelsk-polsk/>; rel="alternate"; hreflang="no", <http://pl.bab.la/slownik/angielski-polski/>; rel="alternate"; hreflang="pl", <http://pt.bab.la/dicionario/ingles-polones/>; rel="alternate"; hreflang="pt", <http://ro.bab.la/dictionar/engleza-poloneza/>; rel="alternate"; hreflang="ro", <http://www.babla.ru/английский-польский/>; rel="alternate"; hreflang="ru", <http://sv.bab.la/lexikon/engelsk-polsk/>; rel="alternate"; hreflang="sv", <http://sw.bab.la/kamusi/kiingereza-kipolishi/>; rel="alternate"; hreflang="sw", <http://www.babla.co.th/english-polish/>; rel="alternate"; hreflang="th", <http://tr.bab.la/sozluk/ingilizce-lehce/>; rel="alternate"; hreflang="tr", <http://www.babla.vn/tieng-anh-tieng-ba-lan/>; rel="alternate"; hreflang="vi"
```

This works fine on Python 2, because it handles all the headers as byte strings. This means it searches for the literal newline byte and parses appropriately. You can see this in the headers dict on Python 2:

``` python
>>> r = requests.get('http://pl.bab.la/slownik/angielski-polski/', stream=True)
>>> r.headers
{'Content-Length': '9774', 'Content-Language': 'pl', 'Content-Encoding': 'gzip', 'Set-Cookie': 'PHPSESSID=4jh22qivih8dl3l3arllsed1j1; path=/; domain=.bab.la, settings=pl.enpl.en.es.0..; expires=Thu, 17-Nov-2016 11:13:03 GMT; path=/; domain=.bab.la', 'Expires': 'Mon, 26 Jul 1997 05:00:00 GMT', 'Vary': 'Accept-Encoding', 'Keep-Alive': 'timeout=5, max=70', 'Server': 'Apache', 'Last-Modified': 'Tue, 09 Aug 2016 11:13:03 GMT', 'Connection': 'Keep-Alive', 'Link': '<http://www.babla.cn/\xe8\x8b\xb1\xe8\xaf\xad-\xe6\xb3\xa2\xe5\x85\xb0\xe8\xaf\xad/>; rel="alternate"; hreflang="zh-Hans", <http://cs.bab.la/slovnik/anglicky-polsky/>; rel="alternate"; hreflang="cs", <http://da.bab.la/ordbog/engelsk-polsk/>; rel="alternate"; hreflang="da", <http://de.bab.la/woerterbuch/englisch-polnisch/>; rel="alternate"; hreflang="de", <http://www.babla.gr/\xce\xb1\xce\xb3\xce\xb3\xce\xbb\xce\xb9\xce\xba\xce\xb1-\xcf\x80\xce\xbf\xce\xbb\xcf\x89\xce\xbd\xce\xb9\xce\xba\xce\xb1/>; rel="alternate"; hreflang="el", <http://en.bab.la/dictionary/english-polish/>; rel="alternate"; hreflang="en", <http://eo.bab.la/vortaro/angla-pola/>; rel="alternate"; hreflang="eo", <http://es.bab.la/diccionario/ingles-polaco/>; rel="alternate"; hreflang="es", <http://fi.bab.la/sanakirja/englanti-puola/>; rel="alternate"; hreflang="fi", <http://fr.bab.la/dictionnaire/anglais-polonais/>; rel="alternate"; hreflang="fr", <http://www.babla.in/\xe0\xa4\x85\xe0\xa4\x82\xe0\xa4\x97\xe0\xa5\x8d\xe0\xa4\xb0\xe0\xa5\x87\xe0\xa4\x9c\xe0\xa4\xbc\xe0\xa5\x80-\xe0\xa4\xaa\xe0\xa5\x8b\xe0\xa4\xb2\xe0\xa4\xbf\xe0\xa4\xb6/>; rel="alternate"; hreflang="hi", <http://hu.bab.la/sz\xc3\xb3t\xc3\xa1r/angol-lengyel/>; rel="alternate"; hreflang="hu", <http://www.babla.co.id/bahasa-inggris-bahasa-polandia/>; rel="alternate"; hreflang="id", <http://it.bab.la/dizionario/inglese-polacco/>; rel="alternate"; hreflang="it", <http://ja.bab.la/\xe8\xbe\x9e\xe6\x9b\xb8/\xe8\x8b\xb1\xe8\xaa\x9e-\xe3\x83\x9d\xe3\x83\xbc\xe3\x83\xa9\xe3\x83\xb3\xe3\x83\x89\xe8\xaa\x9e/>; rel="alternate"; hreflang="ja", <http://www.babla.kr/\xec\x98\x81\xec\x96\xb4-\xed\x8f\xb4\xeb\x9e\x80\xeb\x93\x9c\xec\x96\xb4/>; rel="alternate"; hreflang="ko", <http://nl.bab.la/woordenboek/engels-pools/>; rel="alternate"; hreflang="nl", <http://www.babla.no/engelsk-polsk/>; rel="alternate"; hreflang="no", <http://pl.bab.la/slownik/angielski-polski/>; rel="alternate"; hreflang="pl", <http://pt.bab.la/dicionario/ingles-polones/>; rel="alternate"; hreflang="pt", <http://ro.bab.la/dictionar/engleza-poloneza/>; rel="alternate"; hreflang="ro", <http://www.babla.ru/\xd0\xb0\xd0\xbd\xd0\xb3\xd0\xbb\xd0\xb8\xd0\xb9\xd1\x81\xd0\xba\xd0\xb8\xd0\xb9-\xd0\xbf\xd0\xbe\xd0\xbb\xd1\x8c\xd1\x81\xd0\xba\xd0\xb8\xd0\xb9/>; rel="alternate"; hreflang="ru", <http://sv.bab.la/lexikon/engelsk-polsk/>; rel="alternate"; hreflang="sv", <http://sw.bab.la/kamusi/kiingereza-kipolishi/>; rel="alternate"; hreflang="sw", <http://www.babla.co.th/english-polish/>; rel="alternate"; hreflang="th", <http://tr.bab.la/sozluk/ingilizce-lehce/>; rel="alternate"; hreflang="tr", <http://www.babla.vn/tieng-anh-tieng-ba-lan/>; rel="alternate"; hreflang="vi"', 'Pragma': 'no-cache', 'Cache-Control': 'no-store,no-cache,must-revalidate, post-check=0,pre-check=0', 'Date': 'Tue, 09 Aug 2016 11:13:03 GMT', 'P3P': 'CP="CAO DSP CURa ADMa DEVa TAIa OUR BUS IND UNI COM NAV INT"', 'Content-Type': 'text/html;charset=UTF-8', 'X-Frame-Options': 'SAMEORIGIN, SAMEORIGIN'}
```

On Python 3, `http.client` gets all high-and-mighty about headers and tries to decode the header block _using Latin-1_. That goes totally wrong:

``` python
>>> r = requests.get('http://pl.bab.la/slownik/angielski-polski/', stream=True)
>>> r.headers
{'Pragma': 'no-cache', 'P3P': 'CP="CAO DSP CURa ADMa DEVa TAIa OUR BUS IND UNI COM NAV INT"', 'Expires': 'Mon, 26 Jul 1997 05:00:00 GMT', 'Content-Language': 'pl', 'Link': '<http://www.babla.cn/è\x8b±è¯\xad-æ³¢å\x85', 'Date': 'Tue, 09 Aug 2016 11:12:39 GMT', 'Cache-Control': 'no-store,no-cache,must-revalidate, post-check=0,pre-check=0', 'Server': 'Apache'}
```

As you can see, the Link header is truncated. This causes a problem, because it means that all headers _after_ the Link header are simply dropped (or rather, treated as part of the body), and that includes the Content-Length header. Thus, `http.client` falls back on connection-termination framing: that is, it assumes that TCP FIN will mark the end of the body. The server is expecting to try to keep the connection alive, so Requests blocks until the connection is timed out by the server, several minutes later.

Now, the server is definitely misbehaving here: it really _shouldn't_ be sending plain UTF-8 in its header block like this. However, a better HTTP client that wasn't so obsessed with translating things into unicode strings wouldn't have this problem either.

I think this counts as a CPython bug for the moment, I'm afraid: Requests is simply not in a position to replace httplib at the moment. We're working on it, but it's going to take a _long_ time before that happens.


This is now tracked as [Python issue 27716](https://bugs.python.org/issue27716).


Thanks, I've found out that `urllib.request` uses `http.client` too (https://docs.python.org/3/library/http.client.html). Is documentation outdated or `urllib` is doing some magic to overcome this issue?


@Alexander3 urllib.request doesn't connection pool. This means that the remote server closes the connection at the end of the request, which is what httplib was expecting. You can try doing this by setting the `Connection` header value to `close`. This is generally a bad idea, as it's very wasteful, but it may help in this case.