Connection Reset by Peer

## Summary

While making an HTTPS GET request to fema.gov I get a Connection Reset Error, this is confirmed on multiple machines not to happen in Python 2 but does happen on Python 3, both are using the same version of requests. Very bizarrely (in this case) the failure occurs when trying to read the 3704 byte. (discovered by using lower level functions). Below is the URL:

`https://hazards.fema.gov/gis/nfhl/rest/services/public/NFHL/MapServer/16/query?where=&text=&objectIds=&time=&geometry=-9240000%2C4400000%2C-9230000%2C4410000&geometryType=esriGeometryEnvelope&inSR=102100&spatialRel=esriSpatialRelIntersects&relationParam=&outFields=&returnGeometry=true&maxAllowableOffset=&geometryPrecision=&outSR=&returnIdsOnly=false&returnCountOnly=false&orderByFields=&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&returnDistinctValues=false&returnTrueCurves=false&resultOffset=&resultRecordCount=&f=pjson`

Things I have tried:

* User-Agent strings
* Disabling verify
* Using machines with different IP addresses
* Matching all libraries except openssl_version to the working Python 2.7 environment

## Reproduction Steps

```
python
import requests
url ="https://hazards.fema.gov/gis/nfhl/rest/services/public/NFHL/MapServer/16/query?where=&text=&objectIds=&time=&geometry=-9240000%2C4400000%2C-9230000%2C4410000&geometryType=esriGeometryEnvelope&inSR=102100&spatialRel=esriSpatialRelIntersects&relationParam=&outFields=&returnGeometry=true&maxAllowableOffset=&geometryPrecision=&outSR=&returnIdsOnly=false&returnCountOnly=false&orderByFields=&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&returnDistinctValues=false&returnTrueCurves=false&resultOffset=&resultRecordCount=&f=pjson"
response=requests.get(url)
```

## System Information

```   
$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.2.2"
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.4"
  },
  "platform": {
    "release": "4.13.0-21-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010008f",
    "version": "18.0.0"
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "1010006f"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": true
}
```

As noted above the request works fine in Python 2.7

```
$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.9"
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.14"
  },
  "platform": {
    "release": "4.13.0-21-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1000207f",
    "version": "16.2.0"
  },
  "requests": {
    "version": "2.18.1"
  },
  "system_ssl": {
    "version": "1000207f"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": true
}
```

I have tried to get the versions of my Python 3 to match but even doing this the request still fails in Python 3.

```
$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.9"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.4"
  },
  "platform": {
    "release": "4.13.0-21-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010006f",
    "version": "16.2.0"
  },
  "requests": {
    "version": "2.18.1"
  },
  "system_ssl": {
    "version": "1010006f"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": true
}
```

Have you ever tried that using a browser to get the response of the API?
It seems that the problem was caused by the remote server error.

Works in a browser, works in CURL, works in python 2.7. Works in Python 3.6 using this awful, awful code

```        
cmd = "curl -o {cache_file} {url}".format(
    url=url, cache_file=cache_file)
p = subprocess.Popen(cmd.split())
try:
    p.communicate(timeout=30)
    if p.returncode != 0:
        raise Exception()
except subprocess.TimeoutExpired:
    p.kill()
    p.communicate()
    raise Exception()
```

@etothepii Before we get started, your GitHub handle is *excellent*.

Next, I've noticed a few discrepancies:

1. Your version of Requests is different across your test environments
1. Your urllib3 versions are different
1. Your pyOpenSSL and Cryptography versions are different
1. As a result of the discrepancy in pyOpenSSL and Cryptography, the *actual* OpenSSL versions you're using are different.

Can you retest with the same versions of requests, urllib3, pyOpenSSL, and cryptography?

I attempted on a Mac with Python 3.6 to retrieve that URL and it works fine.

@etothepii We haven't heard back from you in a few days and it doesn't seem like this is entirely reproducible with what you've given us. As a result, I'm proposing we close this after a few more days. Let us know if you have further information that points to this being a bug in requests.

Cheers,
Ian

Hi Ian, 

Thank you for you help, it is very much appreciated.

As discussed above I have satisfied the immediate problem by running curl in a sub process but that seems like a horrible solution.

I really appreciate your help and I agree with you that it works on some other setups. As evidenced by it working on Python 3. To me this suggests that it is most likely not the server that is at fault (as would be the normal response to a Connection Reset by Peer). Doesn't it seem so very odd to you that I am able to make the request and get a 200 response using lower level functions but get the connection reset if trying to read past char 3704, despite the message clearly being 15000 chars long?

Please accept my apologies for the confusion above - with diagnostics. I was trying to show that I had already tried to align my non-working Python 3 system with a working Python 2 system, by showing what I had changed it to. As far as I can see I have managed to sync everything up between the working between them except for the system SSL version which is 1000207f (working) rather than 1010006f (non-working). Everything else is the same according to my version of Beyond Compare (image attached).

Any help or suggestions for further avenues of inquiry would be very welcome.

Kind regards,

James

![image](https://user-images.githubusercontent.com/2609402/40588261-192492d8-61b1-11e8-9fa7-e63344926193.png)

So is it correct to say that the "Broken" side of your screenshot is what the current state of your system is after trying to make things match?

Yes, I could not figure out a good way to change the system_ssl version. The words are too common to search well on Google. I would not object to closing the issue if no-one else has reported similar behavior, using curl in a sub-process does solve the problem - even if not in a very pythonic way. If you can think of anything I can do that might help to diagnose the problem I will do so happily but if nothing is immediately coming to mind then I would rather not waste your time. 

Thank you for your help.

@etothepii Can you also share how you've installed the following:

- python2
- python3
- requests (python 2)
- requests (python 3)
- pyOpenSSL (python 2)
- pyOpenSSL (python 3)
- openssl

Are both python 2 and python 3 installed on the same system?

I really appreciate your help.

I had hoped it would be something obvious and that by talking it out the answer would present itself. It has not, and with curl working as a viable solution I will have to stick with that and move on.

Everything is running in Docker with installation via pip.

Thanks for checking back in @etothepii. Since it doesn't look like there is more information to work with, and this seems to be a local configuration issue, I'm going to close this out.

Please feel free to reopen if you have more information later. Thanks!