Raibek/requests-github-issues huggingface dataset:
Response 412 from this website

I keep getting response 412 and a blank webpage with only 4 blank lines from this webpage:

http://wjw.hubei.gov.cn/

## Expected Result

Both Chrome and Firefox can open the page fine. But I notice when the browser loads the page, it is a blank page at first and the browser reloads / loads more and the complete page is loaded eventually. 

## Actual Result

On both Windows 10 (Python 3.7.4) and Raspberry Pi (Python 3.5.3), requests cannot return the page like Chrome / Firefox did. 

## Reproduction Steps

```python
import requests
url = 'http://wjw.hubei.gov.cn/'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36'}
res = requests.get(url,headers=headers)
print(res)
```

<Response [412]>

If you use print(res.content) you will also notice that the server return a bunch of HTML but just that not of the webpage returned by Chrome / Firefox.

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.7.1"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.13"
  },
  "platform": {
    "release": "4.9.59-v7+",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "101000af",
    "version": "16.2.0"
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "101000cf"
  },
  "urllib3": {
    "version": "1.25.3"
  },
  "using_pyopenssl": true
}

```
I also ask this over in stackoverflow but so far no luck:

https://stackoverflow.com/questions/59657613/response-412-when-using-the-requests-python-package-to-access-this-webpage-ho#59661448

A helpful fellow suggested that a valid cookie has to be sent along with the request, but so far this cannot solve the issue on my platforms.

Is there a way to read this particular webpage?

Supplement:

On the Raspberry Pi, even lynx cannot open the webpage.

This does not sound like an issue with Requests, but the server's configuration interacting with the request you send. If you could provide (e.g.) a cURL request like the below that *does* work or give unexpectedly different results than Requests could provide, that may be an issue.

```
curl --verbose http://wjw.hubei.gov.cn/ \
    -H"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36"
...
< HTTP/1.1 412 Precondition Failed 
...
```

I might recommend doing a request with Chrome with the Network dev tools open, the [copy the request as cURL](https://www.google.com/search?q=chrome+copy+request+as+curl). Verify that also works in the command line. You can convert that to requests then with [this tool](https://curl.trillworks.com/). Note that you'll generally end up with many many more headers.
Getting ConnectionResetError(10054) while using Requests

Summary.
I am getting ConnectionResetError(10054) while running a POST against a webservice. I am getting successful output every time when I am using POSTMAN to do it.
## Expected Result
Should get HTTP Code:200 and successful output

## Actual Result

What happened instead.
urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/Admin/Services/Webservices/Testing/new_test.py", line 77, in <module>
    response = requests.request("POST", url, headers=headers, data = payload,verify=False)
  File "C:\Python37\lib\site-packages\requests\api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python37\lib\site-packages\requests\sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python37\lib\site-packages\requests\sessions.py", line 686, in send
    r.content
  File "C:\Python37\lib\site-packages\requests\models.py", line 828, in content
    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
  File "C:\Python37\lib\site-packages\requests\models.py", line 753, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

## Reproduction Steps

```python
import requests
response = requests.request("POST", url, headers=headers, data = payload,verify=False)
print(response.text.encode('utf8'))
```

## System Information

    $ python -m requests.help

```
λ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.4"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "1010103f"
  },
  "urllib3": {
    "version": "1.25.6"
  },
  "using_pyopenssl": false
}
```
[error.txt](https://github.com/psf/requests/files/4041827/error.txt)



Did you get out of this problem?  i get this issue too and don't know what happened, can you share how to resolve this problem?
ModuleNotFoundError: No module named 'requests'

Mac os,Python 3.7.3
```
$  pip3 install requests
Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (2.22.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests) (2019.6.16)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from requests) (1.25.3)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests) (2.8)
```

```
$ ls /usr/local/lib/python3.7/site-packages|grep requests
requests
requests-2.22.0.dist-info
requests_file-1.4.3.dist-info
requests_file.py
requests_toolbelt
requests_toolbelt-0.9.1.dist-info
```
```
$ python3 upJsp.py -h
Traceback (most recent call last):
  File "upJsp.py", line 4, in <module>
    import requests as requestsSs
ModuleNotFoundError: No module named 'requests'
```

fix:
sys.path add /usr/local/lib/python3.7/site-packages
Enable copr builds and add packit config

Let us introduce [packit service](https://packit.dev) to you - the automation to integrate upstream open source projects into Fedora operating system.

After merging this PR, you are just a few steps away from RPM builds being automatically triggered on your PR's.
It means, that you'll be able to try and play with your change, packaged as an RPM.

But there is more. By using packit, you can for example enable adding new releases into Fedora Rawhide.

What are the next steps?
* Install [Packit-as-a-Service github application](https://github.com/marketplace/packit-as-a-service) in your repo
* In case you are first user, wait for account approval
* Enjoy the built RPMs!

For more info, please:
 * check out the documentation: https://packit.dev/docs/
 * contact [@packit-service/the-packit-team](https://github.com/orgs/packit-service/teams/the-packit-team)

Thanks for the PR, @dhodovsk! I don't think we're going to do any distro specific build integration at this time. Looks like a pretty cool idea though!
Update supported Python versions

Supported versions are 2.7 and 3.5+ (currently to 3.8):

https://github.com/psf/requests/blob/eedd67462819f8dbf8c1c32e77f9070606605231/setup.py#L82-L100

3.4 was dropped in https://github.com/psf/requests/pull/5092.

Writing "3.5+" means the docs don't need updating when 3.9 is released, only when 3.5 is dropped.
Fix test_conflicting_post_params to work on pytest 5

The non-contextmanager form of pytest.raises was removed in pytest 5.
http://doc.pytest.org/en/latest/deprecations.html#raises-warns-with-a-string-as-the-second-argument

It was used here to support Python < 2.7, but that is no longer needed.
https://github.com/psf/requests/pull/1503#issuecomment-22333666

Fixes https://github.com/psf/requests/issues/5304

What can I do to help getting this merged? I consider the change fairly trivial, self contained and easy to grasp. But the more this remains open, the chances of diverging are getting higher.

If there is anything else required from my side, please do let me know.


Thanks @hroncok!

Thanks for merging @nateprewitt.
pytest.raises with a string as the second argument removed in pytest 5

See http://doc.pytest.org/en/latest/deprecations.html#raises-warns-with-a-string-as-the-second-argument

The non-comntextmanager form of `pytest.raises` was removed in pytest 5. It is used in:

https://github.com/psf/requests/blob/eedd67462819f8dbf8c1c32e77f9070606605231/tests/test_requests.py#L774-L778

The error is:

```
self = <tests.test_requests.TestRequests object at 0x7f68030882b0>
httpbin = <function prepare_url.<locals>.inner at 0x7f68034c8f70>

    def test_conflicting_post_params(self, httpbin):
        url = httpbin('post')
        with open('Pipfile') as f:
>           pytest.raises(ValueError, "requests.post(url, data='[{\"some\": \"data\"}]', files={'some': f})")
E           TypeError: 'requests.post(url, data=\'[{"some": "data"}]\', files={\'some\': f})' object (type: <class 'str'>) must be callable

tests/test_requests.py:777: TypeError
```

The code was added in https://github.com/psf/requests/pull/1503 [to support Python < 2.7](https://github.com/psf/requests/pull/1503#issuecomment-22333666):

> You can't use the context manager form of assertRaises() on Pythons earlier than 2.7. You'll need to use one of these two forms...

Since 2.6 is not supported anymore, this could be converted to:

```python
        with open('Pipfile') as f:
            with pytest.raises(ValueError):
                requests.post(url, data='[{"some": "data"}]', files={'some': f})
            with pytest.raises(ValueError):
                requests.post(url, data=u('[{"some": "data"}]'), files={'some': f})
```

## Expected Result

Test pass with pytest 5.

## Actual Result

Test `TestRequests.test_conflicting_post_params` fails with pytest 5.

## Reproduction Steps

```console
$ rm .tox -rf
$ tox -e py38
```

(I've actually needed to install pygments into the environment, otherwise test collection has failed.)

## System Information

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.1"
  },
  "platform": {
    "release": "5.3.16-300.fc31.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "1010104f"
  },
  "urllib3": {
    "version": "1.25.7"
  },
  "using_pyopenssl": false
}

```

pytest 5.3.2

## The Travis CI

As a side note, the Travis CI job runs on ancient pytest version (3.x), because it is preinstalled, and not updated.


https://github.com/psf/requests/pull/5305
Add Python 2 support entry in FAQ

Fixes: #5211

Signed-off-by: Sumana Harihareswara <sh@changeset.nyc>

@nateprewitt and @hugovk could you take a look? I'm happy to have my PR edited or superseded in the interests of phrasing the FAQ item accurately, and you're the experts, not me!

(And Happy New Year of course!)

I'm not a member of the Requests project, but looks about right to me. Happy new year!
Change <Response [XXX]> to self.text

From talking to people and personal experiences, I always feel this is more intuitive. Only downside is inside a REPL, but I think there is a way to detect that?

This would spam up a console if it were a long response body and also be a breaking change. It's unlikely that this will be merged.

Hi @JonPizza,

Seth is right here. This would add a minor short cut to `text` but risks printing very long bodies into the console unexpectedly.

I think we’ll pass on this for now, thanks for the suggestion!
reading multipart/x-mixed-replace binary stream 

I need to read a http stream (with head multipart/x-mixed-replace), the content-type is image/jpeg,how could i do?
r = requests.get('http://192.168.1.100:4747/mjpegfeed?1920x1080', stream=True)
c0 = r.content

code [c0 = r.content] will wait until response finish, but this response is not going to finish.
so what can I do?



Hi @fangyong125,

As per the message in the issue you opened, we don't support these kinds of requests for help on this issue tracker. Please open a question on Stackoverflow. Thanks!
Document `requests.utils`

Are the methods imported by `requests.utils` supposed to be public?

https://github.com/psf/requests/blob/eedd67462819f8dbf8c1c32e77f9070606605231/requests/utils.py

https://github.com/psf/requests/blob/eedd67462819f8dbf8c1c32e77f9070606605231/requests/compat.py

Is it okay to do 

```python
import requests.utils
requests.utils.urlparse(whatever)
```

in my code? If yes, shouldn't they be documented as if they were part of requests? Or at least the docs for them should be copy/pasted from the Python docs. I [don't see them mentioned anywhere](https://www.google.com/search?q=site%3Ahttps%3A%2F%2Frequests.readthedocs.io%2F+urlparse), just the three encoding-related methods https://requests.readthedocs.io/en/master/api/#requests.utils.get_encodings_from_content

Or should I always import these methods directly `from urllib.parse import urlparse`, etc.?

I've been asked to comment on this. The answer in short is "No." those were never intended to be public parts of the API but people still do use them at their own risk. We've broken them unintentionally before and I'm sure whoever ends up maintaining this will do so again as they well should.
requests raise broken pipe, but postman return 500 status

when upload big file(bigger than server limit). I expect return response with 500 status

1、I tried postman to post api, it return 500 error,that what I need

2、I tried requests to post api

here is requests code:
```
with open('invalid_big_license', 'rb') as f:
    test = s.post(
        url,
        headers=headers,
        data=None,
        files={'file': ('invalid_big_license', f)},
        timeout=1200
    )
``` 
requests raise 

`requests.exceptions.ConnectionError: ('Connection aborted.', OSError(41, 'Protocol wrong type for socket'))`
what I need is response return with 500 status

if upload file is under server limit(like 1m),requests is success

Hi @EmptyRabbit, this is bad data being returned by the server. The first thing I'd check is if this is specific to the Requests User-Agent. You may need to use something like Wireshark to see what exactly is coming across the wire. From the information provided so far, it doesn't look like this is a defect in Requests though.

I'm going to close this for now since we don't have a reproducible example. Feel free to reopen if you have more information or have found this is reproducible outside of this specific server.
`Session().get` not usuable with `with`

Summary.

## Expected Result

Able to use `with session.get() as response`, same as `with requests.get() as response`.

## Actual Result

Throws exception:
```
AttributeError: __exit__
```

## Reproduction Steps

```python
import requests
session = requests.Session()
with session.get('http://some.domain') as response:
  print(response.headers)
```

## System Information

```
Linux Mint
Python 3.5.2
python-requests_2.9.1-3ubuntu0.1_all.deb
```

upgrade requests version to >=2.18

@songguoxiong 2.9.1 is the latest available via `apt-get`

try to install via `pip`?
```bash
python3 -m pip install "requests>=2.18"
```

Okay, that works.

Any chance of submitting something recent to get the `apt` repositories updated? I see now that 2.9.1 is 4 years old.

Requests contributors aren't in control of `apt` repositories. Please open an issue on the Linux Mint tracker.
requests.get of this URL causes timeout and network failure (router reboots)

Summary.
when I use requests.get for a particular url it causes a network crash (lose connection and my router reboots). The only return message is a timeout error.   Here are two example URLs that cause the issue:
https://images.harrods.com/product/gucci/large-gg-tweed-rajah-tote-bag_000000006389623001.jpg
https://images.harrods.com/product/gucci/small-leather-soho-disco-bag_000000000005256593.jpg

I now understand how to avoid the issue, but without any error message it took a long time to debug, and crashed the network a number of times.  I ended up running the get in urllib3 and I got the SSL error message.  Doing the request.get with a custom header for 'User-Agent', the network does not crash.

## Expected Result
Timeout error with network error result 
In order to debug I did the same get with urllib3 - I got this error when I used urllib3 for the same URL:
MaxRetryError: HTTPSConnectionPool(host='images.harrods.com', port=443): Max retries exceeded with url: /product/gucci/large-gg-tweed-rajah-tote-bag_000000006389623001.jpg (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))


## Actual Result
Time out error with network router rebooting and wifi card disconnecting.  Exception has no mention of any network error except timeout.

## Reproduction Steps

```python
import requests
url = 'https://images.harrods.com/product/gucci/large-gg-tweed-rajah-tote-bag_000000006389623001.jpg'
r  =  requests.get(url, timeout=4)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.8"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.5"
  },
  "platform": {
    "release": "4.15.0-72-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010104f",
    "version": "19.1.0"
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "1010104f"
  },
  "urllib3": {
    "version": "1.25.7"
  },
  "using_pyopenssl": true
}

```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Hi @filipmu,

The issue with your network card and router aren't related to Requests. Attempting a repro from what you provided correctly returns a read timeout error. It appears Harrods may be modifying server behaviour in some cases to stop web scraping. All non-trivial requests from Requests should be provided a timeout to catch cases like this.

The urllib3 exception you've provided doesn't seem to be related to this case.
Ignore normalization URL

In cases I would like to test Path Traversal vulnerabilities in web applications, the target URL is usually `https://example.com/../../path`. Using the GET method in requests, that URL will be normalized as below. Consequently, this will lead to unintended results in the test.

```
r = requests.get('https://example.com/../../path')
print(r.url)
# https://example.com/path
```
With `curl`, we have an option to ignore the normalization : `--path-as-is`. So, do we have an equivalent option for `requests`?


@EDjur Thank you for your information. Do you know a workaround to resolve this?

Try using prepared request

```
url = "http://example.com/../something.txt"
s = requests.Session()
req = requests.Request(method='POST' ,url=url, headers=headers, data=data)
prep = req.prepare()
prep.url = url
r = s.send(prep, verify=False)
```

@akmalhisyam Thank you very much. It works well.

@akmalhisyam is correct here. Using the PreparedRequests workflow will typically solve issues like this if you MUST explicitly set something.
2.python-requests.org can't be reached

https://2.python-requests.org/ isn't loading on my browser, via curl, or Down Detector (https://www.isitdownrightnow.com/2.python-requests.org.html).

## Expected Result
Site to load

## Actual Result
```
This site can’t be reached2.python-requests.org’s server IP address could not be found.
```
What happened instead.

## Reproduction Steps
Try to load homepage in browser.

## System Information
Ubuntu 18.04, Firefox

It's back now for me.
respect no_proxy settings

fix issue #4871 

test fails
```
    def test_bypass(self, url):
>       assert get_environ_proxies(url, no_proxy=None) == {}
E       AssertionError: assert {'__bypass_proxies': True} == {}
E         Left contains more items:
E         {'__bypass_proxies': True}
E         Use -v to get the full diff
tests/test_utils.py:170: AssertionError
_____ TestGetEnvironProxies.test_bypass[no_proxy-http://192.168.0.1:5000/] _____
```
If they do't have same length, there is no need to create two dict to compare.


Hi @sunsongpy, checks like [this](https://github.com/python/cpython/blob/109fc2792a490ee5cd8a423e17d415fbdedec5c8/Objects/dictobject.c#L2751) are baked into the base objects, so there’s no need for us to reimplement it here. Thanks for the suggestion! 

Apologies, I glanced over the PR too quickly. If you can provide benchmarks for this change showing a significant improvement, I think we can look at it. Otherwise, I think we’ll pass for now.

> Yes this can improve performance .

thanks

> 
> 
> > Apologies, I glanced over the PR too quickly. If you can provide benchmarks for this change showing a significant improvement, I think we can look at it. Otherwise, I think we’ll pass for now.
> 
> Sorry, I know this change won't bring much of a significant improvement, but it certainly makes the code more rigorous. If the length is different, it's definitely False (two dict comparison of Cpython source code is the first step is to compare the length, why we don't do like that first).

After seeing the @nateprewit link . I agree with him. Adding extra lines of code just to do what is already being done, is not justified.
mypy is failing on type checking Response

We're using mypy internally to check for type errors, on an specific interaction with `requests.models.Response` object it gives you an error about a missing attribute, which is declared on the class.

Minimal example



## Expected Result
Type check successful

## Actual Result
```
$ python3.6 -m mypy test.py
test.py:3: error: "Response" has no attribute "next"
```

## Reproduction Steps
```txt
# requirements.txt
certifi==2019.11.28
chardet==3.0.4
idna==2.8
mypy==0.720
mypy-extensions==0.4.1
psutil==5.4.3
requests==2.22.0
typed-ast==1.4.0
typing==3.7.4.1
typing-extensions==3.7.4
urllib3==1.25.7
```

```python
# test.py
import requests
response = requests.get('https://google.com')
print(response.next)
```

Running:
```
python3.6 -m mypy test.py
```

## System Information

    $ python -m requests.help

```
$ python3.6 -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.9"
  },
  "platform": {
    "release": "3.10.0-862.14.4.el7.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.25.7"
  },
  "using_pyopenssl": false
}
```


This was fixed in typeshed on October 18th in this commit: https://github.com/python/typeshed/commit/bf862d907957e46b97419565a5bb90314d074ee4

The requests repo doesn't maintain it's own type hints. Maybe you need to upgrade your MyPy version to get a newer typeshed version? Not sure.

Yes, upgrade resolved thanks for info!
incorrect coding of location under 302 response

When I use headers.get('location') I got http://xxx.xxx.xxx/2019/é????°????1????????-|?????3?o????????-|?????|?±????è??.[è?±] ???????·é????·???..2018-1.epub

Obviously there is problem of encoding. It should be Chinese after '/2019/'

What shall I do to sovle this problem?

HTTP/1.1 302 Found
Server: nginx/1.10.3 (Ubuntu)
Date: Wed, 04 Dec 2019 08:36:48 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Set-Cookie: advanced-frontend=4lo57sauj74js561pjth714bq2; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
location: http://xxx.xxx.xxx/2019/é????°????1????????-|?????3?o????????-|?????|?±????è??.[è?±] ???????·é????·???..2018-1.epub
Connection: keep-alive


Hi @billwyy, can you clarify if the server is returning an indiscernible location or if Requests is performing incorrect decoding on it?

We want to do the best we can with non-english encodings, but we'd only expect iso-8859-1 and utf-8 with the current HTTP RFCs for headers.

Resolving since we haven't heard back.
Unified check version compatibility

I have noticed that there is a `# noqa: F811` comment at `__init__.py` when we check the version compatibility of urllib3.

And there exist duplicate code to check the compatibility of different packages.

So I write a method to handle it.

Hi @Mellcap,

Thanks for contributing to Requests! This is an internal implementation detail that we're using to validate a limited number of dependencies. While I think this is a reasonable abstraction, we don't intend to increase the number of dependencies we use. I'm a bit worried this may adds too many constraints on us modifying things going forward. I think we'll pass on this for now.
Test case, test_total_timeout_connect failed

Summary.
When getting the latest release, v2.20, and run the test case it failed in the case test_total_timeout_connect. The test case is expecting, ConnectTimeout exception but instead it gets ConnectionError. The error message is as follows:

```
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
```
## Expected Result
I would expect all the test cases should be passed without any modification.

## Actual Result
4 test case fail.
Error message is consistent with the following:

```
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
```
## Reproduction Steps
It seems a symptom only happens when using WiFi. You need to make sure to use WiFi not landline.

Run the following test case:
pytest tests/test_requests.py::TestTimeout::test_total_timeout_connect

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": ""
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.5"
  },
  "platform": {
    "release": "4.15.0-70-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "1010100f"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": false
}

```


Upon investigation, it is a mismatch of exception is thrown and the expected exception of test case.

Exception thrown:  requests.exceptions.ConnectionError
Expected exception in the test case: ConnectTimeout

```
    def test_total_timeout_connect(self, timeout):
        try:
            requests.get(TARPIT, timeout=timeout)
            pytest.fail('The connect() request should time out.')
        except ConnectTimeout:
            pass
```

Furthermore, ConnectionTimeout is child class of ConnectionError.

```
class ConnectTimeout(ConnectionError, Timeout): 
```

This is the root cause. A fix can be throwing ConnectTimeout specifically as follow:

```
        except (ProtocolError, socket.error) as err:
            raise ConnectTimeout(err, request=request)
```

Hi @templetontsai, this appears to be something on your wireless connection is either blocking the connection or routing it to another endpoint. It may be a firewall, proxy or the symptom of a MITM. The exception we're expecting for these test cases is correct (a timeout). The exception you're getting is the connection being forcibly closed by whatever server receiving the request.

I'd double check your host/network configuration, and maybe see which host you're connecting to on your wireless network.
Cannot keep 302 using session

Requests can not keep 302 using session，I need to get some cookies in 302
`s = requests.Session()
response = s.post(__login_url,data=__login_data,headers=__login_headers,allow_redirects=False)`

@navysummer `s.cookies` will give you the cookie for that request made to `login_url`.

@navysummer you can access the `history` variable on your response object to see all of the redirects. They will each have the cookies for that stage attached to them.
requests can't handle http response header correctly

when there were two status lines in http response,for example
```
HTTP/1.0 200 OK
Date: Thu, 28 Nov 2019 01:27:10 GMT
Server: Boa/0.94.13
Connection: close
HTTP/1.0 200 OK
Connection: close
Content-Length: 75746
Content-Type: image/jpeg

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```
The requests lib can only parsed the headers before the second status line as follows,
```
import requests
req = requests.get("http://121.156.161.92/cgi-bin/get_jpeg?ch=1")
print(req.headers)
{'Date': 'Thu, 28 Nov 2019 01:22:44 GMT', 'Server': 'Boa/0.94.13', 'Connection': 'close'}
```
is there a way to parse all http headers and skip the bad http status line? 

This is a malformed http response if you’re expecting the headers to occur in a single request. Interpreting that response any differently would be incorrect in this case.

You can reach out to the owner of the server to see what’s going on with the responses they’re emitting.
Fix the syntax highlighting in the example

Previously, dots were not included, which was breaking multiline syntax highlighting in for example in interactive mode. Example:
![Screenshot from 2019-11-27 16-06-00](https://user-images.githubusercontent.com/2760196/69730070-17958a00-1130-11ea-8be2-86728d993478.png)

Here is how it looks after the fix:
![Screenshot from 2019-11-27 16-02-36](https://user-images.githubusercontent.com/2760196/69730115-2aa85a00-1130-11ea-959d-5b54896bcb92.png)
How to use Session

```
import requests
import time
from requests.packages import urllib3

#Disable https insecure warnings for verify=False
urllib3.disable_warnings()

with open('1.ts','rb') as fh:
    file_data = fh.read()

def chunk_trans():
    for i in range(0, 3):
        time.sleep(0.1)
        yield file_data

req = requests.Request('POST', 'http://192.168.123.151/1.ts', data=chunk_trans(), headers={'Host':'a.b.c.d'})
prep = req.prepare()
s = requests.Session()
s.send(prep, timeout=2, verify=False, allow_redirects=False)
```

On Server, I am seeing two "Host" headers are being sent. is that expected or I am not doing it correctly?


can anyone look into this please and help?

Closing as duplicate of #5274.
Two Host header are being sent when doing chunk transfer with Session

Two Host header are being sent when doing chunk transfer with Session
---------------
>>> import requests
>>> requests.__version__
'2.22.0'
```
import requests
import time
from requests.packages import urllib3

#Disable https insecure warnings for verify=False
urllib3.disable_warnings()

with open('1.ts','rb') as fh:
    file_data = fh.read()

def chunk_trans():
    for i in range(0, 3):
        time.sleep(0.1)
        yield file_data

req = requests.Request('POST', 'http://192.168.123.151/1.ts', data=chunk_trans(), headers={'Host':'a.b.c.d'})
prep = req.prepare()
s = requests.Session()
s.send(prep, timeout=2, verify=False, allow_redirects=False)

```

Actual Output: Two host headers are seen on server side.
```
Host: 192.168.123.151
Host: a.b.c.d
Transfer-Encoding: chunked
```
-------------

only "a.b.c.d" host header should come.

can anyone look into this and help?

I see that there is a PR for this (and has been for a while) but it is not yet merged.

I am also having this issue. I can apply the patch on my own fork, but is there any reason this hasn't been merged? Just forgotten about maybe?

I guess it is kind of an edge-case..
Requests supports urllib3 v1.25.7 ?

Hi Guys,
based on [https://2.python-requests.org/en/master/community/updates/#id1](https://2.python-requests.org/en/master/community/updates/#id1) requests should support urllib3 v1.25.2 and greater(?)
The `__init__.py` file doesn't reflect that.

## Expected Result

Support for urllib3 v1.25.7 

## Actual Result

/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25.7) or chardet (2.2.1) doesn't match a supported version!

## Reproduction Steps

```python
import requests

```

## System Information

```
python -m requests.help
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.25.7) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
{
  "chardet": {
    "version": "2.2.1"
  },
  "cryptography": {
    "version": "2.8"
  },
  "idna": {
    "version": "2.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.5"
  },
  "platform": {
    "release": "3.10.0-1062.4.3.el7.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "100020bf",
    "version": "0.13.1"
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.25.7"
  },
  "using_pyopenssl": true
}
```


It's your chardet version that isn't valid, should be >=3.0.2.

Fix with `$ python -m pip install chardet>=3.0.2`

Thanks, pip list was showing chardet as 3.0.4 so that's why I thought it's fine. I did force reinstall it and now there is no error.
Using the request library to match the requests_toolbelt library request to upload a file, the second request cannot be passed

# Using the request library to match the requests_toolbelt library request to upload a file, the second request cannot be passed.

```python
from requests_toolbelt import MultipartEncoder
import requests

m = MultipartEncoder(fields={'upload': open('test.txt', 'rb')},
                     boundary='----WebKitFormBoundarytZTJQrWcjjcJIMVQ')
params = {'path': 'test.txt',
          'token': '123456',
          'num': 0, 'offset': 0,
          'limit': 8}
response = requests.post('http://httpbin.org/post',
                         params=params,
                         data=m,
                         headers={'Content-Type': m.content_type})
# print("1: ", response.text)
# print("2: ", response.request.body)
# print("3: ", response.request.headers)

print(2)
response1 = requests.post('http://httpbin.org/post',
                         params=params,
                         data=m,
                         headers={'Content-Type': m.content_type})
```

- Note：The request for the response1 variable cannot be completed, where test.txt is an empty file

Using the fiddler grab package, you find that after the second request, the body is missing after you enter the requests library

First request:
![image](https://user-images.githubusercontent.com/22344864/69218326-721c6c80-0bab-11ea-9087-e1ed73eb5165.png)

Second request:
![image](https://user-images.githubusercontent.com/22344864/69218370-9415ef00-0bab-11ea-9f42-34af5afc2451.png)



Hi @longweiqiang, I believe the issue you’re seeing here is that you’re trying to resend a file after you’ve already read to the end. If the toolbelt isn’t automatically doing this, you’ll either need to move the pointer to the beginning of the file or reopen it.

> Hi @longweiqiang, I believe the issue you’re seeing here is that you’re trying to resend a file after you’ve already read to the end. If the toolbelt isn’t automatically doing this, you’ll either need to move the pointer to the beginning of the file or reopen it.

thanks，It's working fine
Session module taking much longer than expected

Session module is running slower than normal requests. 
No errors logged on either side and everything functions properly, but its consistantly 17.5 times slower.  

## Expected Result

s.head(addr) returns in under .01 seconds

## Actual Result

s.head(addr) takes around .1 seconds

## Reproduction Steps

```python
import requests
s=requests.Session()
s.head('http://192.168.0.1:8080/')
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.6.1"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.9"
  },
  "platform": {
    "release": "12.0-RELEASE-p10",
    "system": "FreeBSD"
  },
  "pyOpenSSL": {
    "openssl_version": "1010101f",
    "version": "19.0.0"
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "1010101f"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": true
}


send: b'HEAD / HTTP/1.1\r\nHost: 192.168.1.76:8080\r\nUser-Agent: python-requests/2.21.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep
-alive\r\n
session_id=303c361737b70c6a6aacb86896a4900091da7555\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: text/html;charset=utf-8
header: Server: CherryPy/18.3.0
header: Date: Mon, 18 Nov 2019 20:51:08 GMT
header: Allow: GET, HEAD, POST, PUT
header: Content-Length: 2
DEBUG:urllib3.connectionpool:http://192.168.1.76:8080 "HEAD / HTTP/1.1" 200 0
```




Hi @unthinkman,

We'd need a lot more information here to see what is going on. If you're looking at [`Requests.head()`](https://github.com/psf/requests/blob/master/requests/api.py#L60-L61) for comparison, it's effectively identical to the code you're using.

If you're trying to compare Requests to another library, we're not actively working on improving Requests speed. Its primary focus is ease of use and has extra overhead to provide that.

I'm going to resolve this for now since there doesn't appear to be anything actionable.
3.python-requests.org documentation says it's for version 2.21.0

Documentation at https://3.python-requests.org/ says it's for Release 2.21.0 which is confusing for users. (#5265).

![image](https://user-images.githubusercontent.com/5297556/68976963-b9c28300-07bc-11ea-8f41-0cc50bfc4c55.png)

It's not clear to new users that "Requests III" means "the pending 3.x release of Requests" (e.g. other libraries like urllib3 are on a 1.x release).

The version stated at the top should be 3.x.y, not an actual version to which the data doesn't correspond. I don't know what branch the 3.x docs are built from, so I don't know how to fix this.

Alternatively, a banner on all the pages mentioning it's for 3.x, a prerelease (how do you get it?)

Also, the download link on this page leads to this Github repo, so it's not clear how you can actually download version 3 or if it's vaporware.

Hi @nicktimo,

We don't have access to that domain or control of the contents on it. You might be able to reach out to @kennethreitz on how it's being updated.

@nateprewitt what is the canonical domain/URL to be used? I'm slightly confused about it, and I often find links on StackOverflow that are broken.

Both http://python-requests.org and https://requests.readthedocs.io/en/master/ should point to current documentation.
Documentation for trust_env not accessible.

[The link for trust_env documentation](https://2.python-requests.org//en/latest/api/#requests.Session.trust_env) fails to load.


Duplicate of #5214

[Here's a working link](https://requests.kennethreitz.org/en/latest/api/#requests.Session.trust_env)

The proper / canonical link should be https://requests.readthedocs.io/en/master/api/#requests.Session.trust_env however both indexation and intra-doc linking is currently a mess.
docs - clarify json() error exception for Python 3

python 3 exception is `JSONDecodeError:  Expecting value: line 1 column 1 (char 0)`

https://github.com/psf/requests/issues/4908

This should absolutely not be merged.

> This should absolutely not be merged.

why ? 

Hi @wesinator, I think we’ll pass on this one. If you take a look at Python 3, you’ll find that JSONDecodeError to be a ValueError. Including the simplejson stack trace doesn’t provide information specific to Requests, just general stacktrace handling.

> 
> 
> Hi @wesinator, I think we’ll pass on this one. If you take a look at Python 3, you’ll find that JSONDecodeError to be a ValueError. Including the simplejson stack trace doesn’t provide information specific to Requests, just general stacktrace handling.

ok, fair. I know it's general stacktrace

thanks,
Safely handle, when uri contains IPv6 link local with %zone index 

## Issue:
When URI contains IPv6 link local with zone index (Ex: fe80::726f:8a26:222a:2bf3%eth0), requote_uri encodes all % to %25. Because of it, requests.get was requesting different resource.

Screenshot: 
![image](https://user-images.githubusercontent.com/1691611/68458332-8b76fd80-0228-11ea-9c6f-8e439f257180.png)

I'm proposing to ignore domain part in utils.unquote_unreserved.

## Actual Result
<pre>
>>> requests.utils.requote_uri('http://[fe80::726f:8a26:222a:2bf3%eth0]:8080/a%20b/index.html')
'http://[fe80::726f:8a26:222a:2bf3%25eth0]:8080/a%2520b/index.html'
</pre>

## Expected Result
<pre>
>>> requests.utils.requote_uri('http://[fe80::726f:8a26:222a:2bf3%eth0]:8080/a%20b/index.html')
'http://[fe80::726f:8a26:222a:2bf3%eth0]:8080/a%20b/index.html'
</pre>

## System Information
<pre>
# python3 -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.2.2"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.5.2"
  },
  "platform": {
    "release": "4.4.71-UNRELEASED-v4-00050-g76f27ecf9a52",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1000207f",
    "version": "18.0.0"
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "1000207f"
  },
  "urllib3": {
    "version": "1.24.1"
  },
  "using_pyopenssl": true
}
</pre>
[FEATURE REQUEST] Fire and forget

In my project i need to perform fire and forget for REST-API post and patch calls.

My current solution is:

```
try:
    requests.post(my_uri, params=data
                  timeout=1e-4)
except requests.exceptions.Timeout:
    pass
```
I think a nicer solution could be:
`requests.fire_and_forget(my_uri, method='post', params=data)`
Where method default argument is `'post'`

This feature is requested by other users see this reddit thread 

> https://www.reddit.com/r/learnpython/comments/8ljftm/fire_and_forget_http_request/

My current solution is not perfect since the size of the timeout is relevant. If I set it to 1e-8 and include data it will break before I have sent the data.

You can't actually "forget" about the connection because you'll need to hold it open, and possibly even wait for TLS negotiation (which requires a few RTT's). Run the connection in a thread, or use async libraries like [aiohttp](https://aiohttp.readthedocs.io/en/stable/client_quickstart.html#make-a-request) or Requests 3 and wrap it in a task (`asyncio.create_task`)

Hi @PMLP-novo, the Requests API is frozen right now, so we won’t be adding any new functions at the moment. Thanks for the suggestion!
Docsearch on the left menu bar is not working

Enter any text into the search bar on the left where it says "Search the doc" (see screenshot)
Press enter

![Screen Shot 2019-11-04 at 8 03 17 PM](https://user-images.githubusercontent.com/5844587/68234016-439c7000-ffb5-11e9-9fa5-fd1dffdf186d.png)


## Expected Result

I expected to see some search results.

## Actual Result

It sends me to another url like https://2.python-requests.org/en/master/...

I think that search is hosted by community.algolia.com, and the url probably should be updated from that website.


Using the sphinx quick search works, but since that search box appears first on the top, that was the one I tried first.

It would be great to have it fixed. Thanks.

Forgot to attach screenshot. It's been added.

This was fixed last year but this wasn't resolved.
RFC8288 support for link relations

Comply to RFC8288 for relation type (section 3.3)
https://tools.ietf.org/html/rfc8288#section-3.3

Changes:
- Only parse first "rel" parameter
- Allow multiple link relation types, i.e.
  rel="start http://example.net/relation/other"

In an effort to clean up the issue tracker to only have issues that are still relevant to the project we've done a quick pass and decided this issue may no longer be relevant for a variety of potential reasons, including:

* Applies to a much older version, unclear whether the issue still applies.
* Change requires a backwards incompatible release and it's unclear if the benefits are worth the migration effort from the community.
* There isn't a clear demand from the community on the change landing in Requests.

If you think the issue should remain open, please comment so below or open a new issue and link back to the original issue. Again, thank you for opening the issue and for the discussion, it's much appreciated.
http://docs.python-requests.org/ and associated is down

I cannot retrieve the docs for python-requests without using non-TLS http.

## Expected Result

The following should return the page content at the end:

`curl -L http://docs.python-requests.org/ -v`

## Actual Result

Instead, I get a TLS error:

```
curl -L http://docs.python-requests.org/ -v
*   Trying 3.13.31.214...
* TCP_NODELAY set
* Connected to docs.python-requests.org (3.13.31.214) port 80 (#0)
> GET / HTTP/1.1
> Host: docs.python-requests.org
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Content-Type: text/html; charset=utf-8
< Location: https://2.python-requests.org/
< X-Redirector-Version: 2.6.0-4f37d77
< Date: Thu, 24 Oct 2019 13:48:03 GMT
< Content-Length: 65
< Connection: close
< 
* Closing connection 0
* Issue another request to this URL: 'https://2.python-requests.org/'
*   Trying 3.13.31.214...
* TCP_NODELAY set
* Connected to 2.python-requests.org (3.13.31.214) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, Server hello (2):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* stopped the pause stream!
* Closing connection 1
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
```

## Reproduction Steps

Run the above curl.


Seems like this is a duplicate of https://github.com/psf/requests/issues/5212. The fix just hasn't gone live.
Android and ios project

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.
Hi there! i am new in app development but i have to submit an android and ios app within 2 weeks i do not know where to start. please help me . thank you


Hi @faout, this is a support forum for Requests. I would suggest posting on StackOverflow with questions about getting started with these topics. Thanks!