ConnectionResetError on Windows 7/Python 3.3

I get this error whenever I use any URL with .get() on Windows 7 x64 and Python3.3

http://bpaste.net/show/cavZW7pfXISmFdNMga4n/

Edit: I forgot to mention that I get the error after waiting a while.. around 5 mins.. just waiting for the command to respond

Stacktrace:

``` python
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    r = requests.get('http://www.wwenews.us/1334995', allow_redirects=True)
  File "C:\Python33\lib\site-packages\requests\api.py", line 55, in get
    return request('get', url, **kwargs)
  File "C:\Python33\lib\site-packages\requests\api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python33\lib\site-packages\requests\sessions.py", line 279, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
  File "C:\Python33\lib\site-packages\requests\sessions.py", line 374, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python33\lib\site-packages\requests\adapters.py", line 222, in send
    r.content
  File "C:\Python33\lib\site-packages\requests\models.py", line 550, in content
    self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
  File "C:\Python33\lib\site-packages\requests\models.py", line 496, in generate
    chunk = self.raw.read(chunk_size)
  File "C:\Python33\lib\site-packages\requests\packages\urllib3\response.py", line 148, in read
    return self._fp.read(amt)
  File "C:\Python33\lib\http\client.py", line 499, in read
    return super(HTTPResponse, self).read(amt)
  File "C:\Python33\lib\http\client.py", line 534, in readinto
    n = self.fp.readinto(b)
  File "C:\Python33\lib\socket.py", line 297, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
```


@shazow any clues?


On linux, requests fails to parse the response (It does parse the status code). The first header sent by the server is this one:
(If I read the RFC correctly spaces in header names aren't allowed)

```
Secured by Excellent Serv: 
```

It seems this one breaks the response parsing. Raw httplib (which is used by urllib3) can't handle it either.

This is the whole response:

``` http
HTTP/1.1 301 Moved Permanently
Secured by Excellent Serv : 
Date: Thu, 07 Feb 2013 20:48:01 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://wn.arabseed.com/1334995
Cache-Control: private
```


This isn't giving me any issues oddly enough. Perhaps this is another issue based on your physical location as well?


Uh, hang on. Two different users have posted in this issue. The traceback in the original post does not indicate what the URL was, so I can't tell whether this is one issue or two. @norey @t-8ch, are you two talking about the same problem, and if so, how on earth do you know? :confused: 


In fact, rereading this, there's simply no way these are the same issue. @norey claims that **any** URL fails, which is clearly not a header related problem. @t-8ch, can you please open a new issue instead of hijacking this one? =)


Following up on @norey's problem:

I can't reproduce this. I'm running Windows 7 x64, Python 3.3 (64-bit version), and can quite happily send get requests to Google.


I just tried requesting Google and some other URLs, and it worked fine. But that URL still gives me the same error... I don't get the reason especially when @sigmavirus24 didn't have any issues.


@norey: Just to be totally clear, what exact URL is causing you problems?


@Lukasa the one shown in the paste I posted.

I just tried it on Python2.7.3 and it works !! but not on Python3.3 !! 
Looks like something FUBARed at my end but can't seem to locate it.


Aha, no, I don't think it is. I ran the same code, once in Python3.3 and once in 2.7.

In Python 2.7:

``` python
>>> import requests
>>> r = requests.get('http://www.wwenews.us/1334995')
>>> r.status_code
200
>>> r.headers
{'content-encoding': 'gzip', 'secured by excellent serv ': '', 'x-powered-by': 'PHP/5.2.17', 'vary': 'Accept-Encoding', 'transfer-encoding': 'chunked', 'connection': 'close', 'cache-control': 'private', 'date': 'Fri, 08 Feb 2013 09:55:43 GMT', 'content-type': 'text/html'}
>>> r.url
u'http://wn.arabseed.com/1334995'
```

Whereas, in Python 3.3:

``` python
>>> import requests
>>> r = requests.get('http://www.wwenews.us/1334995')
>>> r.status_code
301
>>> r.headers
{}
```

So I actually think this is the same, and that header is choking on Python 3.3. Not sure why yet.


I get the same output ( @Lukasa's output of 3.3)  on my Linux box running Python 3.2.3

So seems that it's an issue in Python 3, right ?


Wireshark chokes on that input too, and fails to parse the rest of the body into headers. Frankly, I think the server is doing something stupid here.


no doubt, but why can't I use requests on Py3.3/Windows with that URL.. whereas Py2.7 handles it normally.. very strange... I think I should format the PC :P


I don't think its your PCs problem, as I can repro it.


You can reproduce the error I get in my paste?


Yeah just checked. I get the 301 as well. I was just checking for an exception last night.


Right, so @norey, your exception seems to be specific to you. @sigmavirus24 and I are getting the "can't parse headers" issue.


With python 3 responses are parsed by `email.parser.Parser`.
Python 2 used `mimetools.Message`, a subclass of `rfc822.Message`, both now
deprecated.

The differences begin in `HTTPResponse.begin()`.

Put this in `headers.txt`:

```
Secured by Excellent Serv : 
Date: Fri, 08 Feb 2013 16:12:22 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: close
Location: http://wn.arabseed.com/1334995
Cache-Control: private
```

And run this (python3):

``` python
from email.parser import Parser
f = open('headers.txt')
print(Parser().parse(f).items())
```

Now remove the broken header and repeat.

Edit: new testcode


The root of the problem lies in `email.feedparser.FeedParser._feedgen()` where it aborts on the first non-header line.
If a line is a header is determined by matching a regex (`email.feedparser.headerRE`), which by default doesn't allow spaces.

If you change this regex from

```
headerRE = re.compile(r'^(From |[\041-\071\073-\176]{1,}:|[\t ])')
```

to something like the following it should work.

``` python
#                                      space here  v
headerRE = re.compile(r'^(From |[\041-\071\073-\176 ]{1,}:|[\t ])')
```

This snippet would allow spaces in headers:

``` python
import re
import email
import requests

email.feedparser.headerRE = re.compile(r'^(From |[\041-\071\073-\176 ]{1,}:|[\t])')
r = requests.get('http://www.wwenews.us/1334995')
print(r.status_code)
print(r.content)
```

@norey
This is a workaround, not a bugfix.
The whole problem is caused by the server not sending conformant HTTP responses.
Requests and python do the right thing.
Anyway, I hope this helps you.


@t-8ch if this isn't standards compliant report it upstream at bugs.python.org please. You seem to have a good understanding of what is happening.

On a separate topic, @norey, you said this happens with any URL you use. Were you exaggerating or were you being serious? The reason I ask is because using requests on python 3.3 works perfectly fine for me.


Ah I wasn't clear.
This is a workaround for the horrible brokenness of this server side 'security' feature (ha, maybe this is scrape protection).

I am fairly sure, that python is compliant.

[RFC 2616 Section 4.2](https://tools.ietf.org/html/rfc2616#page-31) defines the header names as a single `token`.
[Section 2.2](https://tools.ietf.org/html/rfc2616#section-2.2) about tokens:

```
[..]
SP = <US-ASCII SP, space (32)>
[..]
token = 1*<any CHAR except CTLs or separators>
separators = "(" | ")" | "<" | ">" | "@"
           | "," | ";" | ":" | "\" | <">
           | "/" | "[" | "]" | "?" | "="
           | "{" | "}" | SP | HT
```

[RFC822 Section 3.2](https://tools.ietf.org/html/rfc822#section-3.2), also
citet in RFC 2625 Section 4.2, defines headers in this way:

```
field = field-name ":" [ field-body ] CRLF
field-name = 1*<any CHAR, excluding CTLs, SPACE, and ":">
field-body = [....]
```

So no space in header names (looks horrible anyways, imho).


@sigmavirus24 I actually tried with a couple of URLs not everything, so I just assumed. But I tried again on Google and it works fine the other URL that gives the exception is ( it is the same server I guess, giving that crap)

``` python
requests.get('http://forum.arabseed.com/showthread.php?p=1202597#post1202597')
```

Ok now, let's disregard its behavior and concentrate on the exception I am getting, what is its reason? (it is in my first post). 


Could you run the snippet I posted in my previous comment (https://github.com/kennethreitz/requests/issues/1171#issuecomment-13300401)
Does this fix the exception?


I ran it, and no more exception! great!

The question is why didn't you guy get that exception while I did...


Maybe it is some broken proxy which tries to parse the headers before it forwards the status code.
If you really want to know it, take _working_ website and examine the headers (or post them here).

``` python
import requests
r = requests.get("http://test.t-8ch.de/")
print(r.headers)
```

Could you also paste the stack trace directly into your first post for future reference?


sure, done!

headers:

``` python
{'connection': 'Keep-Alive', 'content-type': 'text/html; charset=utf-8', 'server': 'nginx', 'expires': 'Fri, 08 Feb 2013 21:16:04 GMT', 'content-encoding': 'gzip', 'transfer-encoding': 'chunked', 'cache-control': 'no-cache', 'last-modified': 'Sun, 15 Apr 2012 16:53:58 GMT', 'date': 'Fri, 08 Feb 2013 21:16:05 GMT'}
```


Looks fine. I'd still blame a misbehaving proxy. You can try it over a different connection (tethered from you mobile phone) or from a different machine. As nobody can reproduce this there aren't many possibilies left.
I don't know if the bugtracker is the right place for this back and forth. If you want you can contact me via mail or xmpp (same address as mail) and we can post the final result here again.


@norey

``` pycon
~/sandbox/requests (master) python3.3
Python 3.3.0 (default, Dec 15 2012, 17:19:46) 
[GCC 4.5.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> r = requests.get('http://forum.arabseed.com/showthread.php?p=1202597#post1202597')
>>> r
<Response [200]>
>>> r.headers
{}
```

What I did notice is that for that website, the headers dictionary was completely empty. That is horribly uncharacteristic.

It seems like every website you're looking at does something wacky with their headers. I'm +1 on closing this issue right now, but if you can find a different URL that also causes this error for you, I don't mind keeping it open.


Thank you all for helping me out with this.. but what can I do if my luck went with bad websites =)

I will close it now, and thanks again


@norey I didn't mean you should close this, just that the evidence so far is that some people don't know how to run servers. If you can find other sites that do this, feel free to contact @t-8ch or myself via email, or just post it back here.