Raibek/requests-github-issues huggingface dataset:
Ignoring ssl certificate does not suppress urllib3 log warning entries

Specifying ```verify=False``` to ignore ssl certificate does not suppress urllib3 warning log entries, causing the log to fill up with unwanted warnings.

## Expected Result
When certificate verification is purposely suppressed, no log entries warning about this condition should be emitted.


## Actual Result

Even with ```verify=False ```, log entries such as the following are written to the log:
```
Feb 17 11:57:27 rpi4 python[8811]:   warnings.warn(
Feb 17 11:57:27 rpi4 python[8811]: /root/pyvenv/<...>/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.178.216'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
```
This can easily fill up the log when there are many such requests.

## Reproduction Steps

```python
import requests
requests.get('https://kennethreitz.org', verify=False)
<Response [200]>
```
Then check syslog, e.g ```journalctl -u <applicable unit>``` and note log entries such as the above


## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.11"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.1"
  },
  "platform": {
    "release": "5.10.63-v7+",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "1010104f"
  },
  "urllib3": {
    "version": "1.26.8"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

```

In the future, please search **closed and** open issues before creating new ones that are duplicates.

Oh apologies, I couldn't find a duplicate, otherwise I wouldn't have posted. Would you kindly share the link to the original ? Many thanks.
How to set multi same key in request header

To Bypass some WAF, I Wanna Send a Malformed HTTP request like this:
```
POST /123123 HTTP/1.1
Host: a.b.com
Hello: hello1
Hello: hello2

a=b
```
Request have two same key in headers.

Try:
```
headers = [('interests', 'football'), ('interests', 'basketball')]
headers = {'interests': ['football', 'basketball']}
```
are all execept.

Try:
```
from urllib3._collections import HTTPHeaderDict
headers = HTTPHeaderDict()
headers .add('interests', 'football')
headers .add('interests', 'basketball')
```
The request will be :
```
POST /123123 HTTP/1.1
Host: a.b.com
interests: football, basketball
```

Please help me to know , how to set same key with diff value in request header, thanks.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
request using a proxy never returns a response 

a request using a wrong content-length header and a proxy never returns a response 

## Expected Result

```python
header = {
        "content-length": "9999999",
        }
r = requests.get("https://www.github.com", headers=header, verify=False, proxies={"https": "https://localhost:8080"})
```
a normal response should be returned

## Actual Result

the certificate verification alert is printed and a response is never returned, the request is not sent since it never shows up in mitm proxy 

```python

header = {
        "content-length": "9999999",
        }
r = requests.get("https://www.github.com", headers=header)

```
if i remove the proxy a request is sent and a response is returned normally

## System Information

```json
{
  "chardet": {
    "version": "3.0.4"
  },
  "charset_normalizer": {
    "version": "2.0.6"
  },
  "cryptography": {
    "version": "3.2.1"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.9"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "1010108f",
    "version": "19.1.0"
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "1010107f"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}
```
also im not sure if this a bug or expected behaviour since it took me 2 days to identify the issue


This is likely your proxy's fault because you're sending the wrong content-type header. It's likely trying to continue reading from the socket while we're also trying to read a response. This isn't a bug in requests. Don't set headers that Requests usually manages for you so you don't run into issues like this
Is there anyway that I can set system proxy using requests on Windows?

When I get a url by setting proxy in requests, it doesn't change the proxy on my computer.
 How can I change the global proxy on my computer so that I can always use this proxy next time ?
  I know how to set proxy to my computer manually. I want to do that by coding.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Catch `urllib3.exceptions.SSLError` in `Response.iter_content`

I believe requests should catch that and raise `requests.exceptions.SSLError`:

```
Traceback (most recent call last):
  File "urllib3\response.py", line 438, in _error_catcher
  File "urllib3\response.py", line 519, in read
  File "http\client.py", line 465, in read
  File "socket.py", line 705, in readinto
  File "ssl.py", line 1273, in recv_into
  File "ssl.py", line 1129, in read
ssl.SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2536)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tkinter\__init__.py", line 1921, in __call__
  File "tkinter\__init__.py", line 839, in callit
  File "updater_gui.py", line 421, in metadata_cached
  File "concurrent\futures\_base.py", line 438, in result
  File "concurrent\futures\_base.py", line 390, in __get_result
  File "concurrent\futures\thread.py", line 58, in run
  File "updater\updater.py", line 314, in get_metadata
  File "pzd.py", line 256, in get_info
  File "pzd.py", line 216, in _get_central_dir
  File "pzd.py", line 176, in _get_chunk
  File "pzd.py", line 170, in _get_range
  File "requests\models.py", line 836, in content
  File "requests\models.py", line 758, in generate
  File "urllib3\response.py", line 576, in stream
  File "urllib3\response.py", line 512, in read
  File "contextlib.py", line 153, in __exit__
  File "urllib3\response.py", line 449, in _error_catcher
urllib3.exceptions.SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2536)
```

This is requests 2.26.0.

That's some edge case and I've seen that happen only once (this is automatic error report from my program so I don't know any details, it's anonymous). I'm not trying to solve the problem, I just want to be able to catch `requests.exceptions.SSLError` instead of `urllib3.exceptions.SSLError`.

Thanks for reporting the issue @anadius. We've definitely had a few cases where either new or rarely raised exceptions from urllib3 leak through. I've added #6057 as a proposal to handle this.

We've merged #6057, so I'd expect this change to be available in the next release.
Help! RemoteDisconnected('Remote end closed connection without response') 

I'm using residential proxies from https://www.dreamproxies.io/ 
The proxies seem to be all live, checked it on multiple proxy checkers but i keep getting Proxy error
`HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))`

**Here's my code:**
```
import requests
try:
    proxies = {
        "http": 'http://<username>:<password>@<ip>:<port>',
        "https": 'http://<username>:<password>@<ip>:<port>'
    }
    r = requests.get('https://www.google.com/', proxies=proxies)
    print(r.status_code, "Proxy works")
except Exception as e:
    print(e)
```

**Output for: `python -m requests.help`**
```
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.10"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.2"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "101010df"
  },
  "urllib3": {
    "version": "1.26.8"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

Further, you have patted credentials into a public issue and should revoke those as quickly as you can to prevent abuse
BlackDuck complaining about the use of urllib3 version older than 1.26.5

We are using Python requests in our project and we run BlackDuck to scan for vulnerabilities from the usage of third-party components. In case of usage of requests library (the latest version 2.27.1), it complains about the usage of old version urllib3, with the following message. 

Description
An issue was discovered in urllib3 before 1.26.5. When provided with a URL containing many @ characters in the authority component, the authority regular expression exhibits catastrophic backtracking, causing a denial of service if a URL were passed as a parameter or redirected to via an HTTP redirect.

CVE-2021-33503 ((BDSA-2021-1688))

## Expected Result

Requests lib should not continue using the urllib3 old version with this security risk.

## Actual Result

Requests lib is still using an old version of urllib3 with this security risk.

## Reproduction Steps

We need to run the BlackDuck (Synopsis) tool.

## System Information

 Python 3

Requests doesn't constrain your maximum urllib3 version to patch releases, you can upgrade urllib3 safely without issue.

python -m pip install -U urllib3

Thanks @sethmlarson .  Shouldn't request by default use a version of urllib3 equal to or greater than 1.26.5? That would avoid all the problems from the start imho and is probably the correct way to solve this issue.

No. That would require more work for operating system packagers who already apply security patches without changing version strings and thus secure software for their users. 

Perfect. Makes sense. Thanks @sigmavirus24
why requests speed is very slow in same environment!

when i get response by curl !
![image](https://user-images.githubusercontent.com/34080816/151486453-01cb38d7-f8e0-4077-aaca-d1bbd4b24962.png)

when i get response by requests !
![image](https://user-images.githubusercontent.com/34080816/151486473-a8229d51-bc98-4153-9df7-ffc6e55ff7e1.png)


Please fill out the template when creating an issue.

You can't compare two completely different tools in two different ways and expect a result that makes sense. If you need the performance that curl provides then use curl.

Also noting you're running an ancient version of Python, try upgrading for some free speedups.
Timeline for v3.0

Is there a timeline on 3.x?  or is it dead?  

Hi @achapkowski, we don't have any timelines to share on a new major version.

In the event one is released, I'd like to temper expectations that it will not resemble what was originally promised by previous maintainers. We don't have the resources currently to deliver on those changes.

@nateprewitt thank you for the update.
Request JSON body in custom auth contains extra spaces

<!-- Summary. -->
The request.body argument of custom auth is different than the bytes that the server receives when using json. Specifically, when inside the custom auth handler, the request.body includes spaces, e.g. `{"myKey":<SPACE>"myValue"}` where the final request sent does not include this space.

This is a problem for custom auth which tries to hash the body and add that as a header, as the hash will not match

## Expected Result
See the demo code below. The main issue is that if you pass in json=(someJson) then inside of MyCustomAuth r.body contains a string (well really bytes but...) which include spaces. However, the final request sent does not include spaces. I would expect the payload to match.

For the demo code below, x-hash returns
x-hash is f667340ebfa34b192bc594772679c8d461eb5455022442cdfcdb21ba30bdf0d5 which is the hash of `{"myKey":<SPACE>"myValue"}`


<!-- What you expected. -->

## Actual Result

the actual HTTP body has a hash of 090150119e53c870ca0da54fe88605c6ed57139e068d0ccbd2483ae212ea383b which is the hash of `{"myKey":"myValue"}` (no space)
<!-- What happened instead. -->

## Reproduction Steps

```python
import requests
import requests
from hashlib import sha256

class MyCustomAuth:
  def __call__(self, r):
    r.headers['x-hash']=sha256(r.body).hexdigest()
    return r

requests.post('https://example.com', json={"myKey": "myValue"}, auth=MyCustomAuth())
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.10"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.1"
  },
  "platform": {
    "release": "5.15.12-1-MANJARO",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "101010df"
  },
  "urllib3": {
    "version": "1.26.8"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Ah. Actually the client is doing the correct thing, but it's the server middleware that's changing the request body. Sorry for the noise
Why isn't requests_ntlm included in requests library?

I'm wondering if there's any reason that requests won't include https://pypi.org/project/requests_ntlm/ in the official repo?


Hi @CaledoniaProject, Requests specifically has opt-in functionality like oauth and ntlm created in separate repos because they're not part of the library's core http functionality. You'll also find other tools in the requests toolbelt.

The primary reason is to avoid unnecessary bloat and overhead for non-critical functionality.

Understood, thanks.
I got the following error  ImportError: cannot import name 'FastAPI'

[cymetricx@ip-172-31-30-0 fast]$ sudo python3 -m pip install fastapi
Collecting fastapi
  Using cached fastapi-0.72.0-py3-none-any.whl (52 kB)
Requirement already satisfied: starlette==0.17.1 in /usr/local/lib/python3.6/site-packages (from fastapi) (0.17.1)
Collecting pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2
  Using cached pydantic-1.9.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.0 MB)
Requirement already satisfied: anyio<4,>=3.0.0 in /usr/local/lib/python3.6/site-packages (from starlette==0.17.1->fastapi) (3.5.0)
Requirement already satisfied: contextlib2>=21.6.0 in /usr/local/lib/python3.6/site-packages (from starlette==0.17.1->fastapi) (21.6.0)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.6/site-packages (from starlette==0.17.1->fastapi) (4.0.1)
Requirement already satisfied: dataclasses>=0.6 in /usr/local/lib/python3.6/site-packages (from pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2->fastapi) (0.8)
Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.6/site-packages (from anyio<4,>=3.0.0->starlette==0.17.1->fastapi) (3.3)
Requirement already satisfied: contextvars in /usr/local/lib/python3.6/site-packages (from anyio<4,>=3.0.0->starlette==0.17.1->fastapi) (2.4)
Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.6/site-packages (from anyio<4,>=3.0.0->starlette==0.17.1->fastapi) (1.2.0)
Requirement already satisfied: immutables>=0.9 in /usr/local/lib64/python3.6/site-packages (from contextvars->anyio<4,>=3.0.0->starlette==0.17.1->fastapi) (0.16)
Installing collected packages: pydantic, fastapi
Successfully installed fastapi-0.72.0 pydantic-1.9.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[cymetricx@ip-172-31-30-0 fast]$ python3 fastapi2.py 
Traceback (most recent call last):
  File "fastapi2.py", line 2, in <module>
    from fastapi import FastAPI
  File "/home/cymetricx/fast/fastapi.py", line 11, in <module>
    from fastapi import FastAPI
ImportError: cannot import name 'FastAPI'

This has nothing ot do with the library requests.

I have the same problem! Version 0.74.0.
How do i request this full page?

When I request this page
`https://dictionary.writtenchinese.com/#sk=yi&svt=pinyin`
I only get half of it.

Is there a way to get the search results as well?



Hi @John-ison, Requests isn't a browser and won't execute JavaScript which is likely the issue you're hitting. We can't provide general web troubleshooting support on this issue tracker though. Your best option is to check https://StackOverflow.com for what you're trying to do.
Allow HTTPResponse subclassing

- defining a new static variable in  HTTPAdapter : _clsHTTPResponse
- using this variable instead of direct HTTPAdapter object creation


We're under a feature freeze and this is adding an undocumented, private attribute that users might be expected to override. We will not be accepting this

Maybe it need to be discussed ... 
If it is only a documentation problem, tell me where to add it.

Reason of this pull-request: 
https://stackoverflow.com/questions/16903528/how-to-get-response-ssl-certificate-from-requests-in-python/70642124#70642124
mohon bantuannya untuk melakukan pemulihan keamanan

<!-- Summary. -->

## Expected Result

<!-- What you expected. -->

## Actual Result

<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```json
{
  "paste": "here"
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


mohon bantuannya untuk melakukan pemulihan keamanan
Can't unset the User-Agent

<!-- Summary. -->

Impossible to unset the User-Agent, if I pass `headers = { 'User-Agent': None }`, urllib3 UA will be set as a fallback

<!-- What you expected. -->

I expect to not have any User-Agent sent to the webserver

<!-- What happened instead. -->

![image](https://user-images.githubusercontent.com/2185992/148385822-0a88da4f-9165-4f66-94e1-6769c6607f9b.png)

## Reproduction Steps

```python
import requests

headers = {
    'User-Agent': None
}

proxies = {
    'https': 'http://127.0.0.1:8888'
}

response = requests.post(
    'http://localhost:8888/',
    data={'name': 'test'},
    headers=headers,
    proxies=proxies,
    verify=False
)

exit()
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.9"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.1"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "101010cf"
  },
  "urllib3": {
    "version": "1.26.7"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


I thought this was a duplicate of the other issue you commented, sorry for the misclick

> I thought this was a duplicate of the other issue you commented, sorry for the misclick

No problem, I have the same issue even without session

Yup. I see that from the other issue too. Thanks for the clear issue

@sigmavirus24 Your PR #5693 solves the issue, why not merging it ?

I merged your PR with the latest commit on my local branch, only one test fails : TestRequests.test_header_remove_is_case_insensitive

Hi @marwanpro, there are still some usability concerns with that patch. For the time being, you are free to do the same thing and pass urllib3's SKIP_HEADER in your request to disable their defaulted values.

```
headers = {'user-agent': SKIP_HEADER, ... }

req = Request('GET', someurl)
prep = req.prepare()
prep.headers = headers
```

This does appear to be a duplicate of #5671 too. Is there a subtlety I'm missing @sigmavirus24? If not, I'll port the current workaround to that ticket and resolve this one so we can keep tracking centralized.
parsing of proxy url broken with requests 2.27.0

It seems like this change https://github.com/psf/requests/pull/5917 unfortunately broke production for us.

We are using a proxy url that follows this scheme: `http://randomstring:randomstring@eu-west-static-04.example.com:9293` (from a service that provides static IPs)

## Expected Result

In 2.26.0 this still works:
```
>>> from requests.utils import prepend_scheme_if_needed
>>> prepend_scheme_if_needed('http://randomstring:randomstring@eu-west-static-04.example.com:9293', 'http')
'http://randomstring:randomstring@eu-west-static-04.example.com:9293'
```

## Actual Result
In 2.27.0 it's broken:
```
>>> from requests.utils import prepend_scheme_if_needed
>>> prepend_scheme_if_needed('http://randomstring:randomstring@eu-west-static-04.example.com:9293', 'http')
'http://eu-west-static-04.example.com:9293'
```

This leads in the end to `ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required')))` because the url was cut.

## Reproduction Steps

```python
>>> from requests.utils import prepend_scheme_if_needed
>>> prepend_scheme_if_needed('http://randomstring:randomstring@eu-west-static-04.example.com:9293', 'http')
```

## System Information

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.9"
  },
  "cryptography": {
    "version": "36.0.1"
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.9"
  },
  "platform": {
    "release": "5.14.21-2-MANJARO",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "101010df",
    "version": "21.0.0"
  },
  "requests": {
    "version": "2.27.0"
  },
  "system_ssl": {
    "version": "101010df"
  },
  "urllib3": {
    "version": "1.26.7"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": true
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Hi, @anneFly. Thanks for the report! This is actually a duplicate of #6027 which has been patched in #6028 and will be released sometime today. You can track status in #6029.
Requests doesn't add `boundary` when passing `Content-Type: multipart/form-data` header manually

Requests with `Content-Type: multipart/form-data` need to also have a `; boundary=someUniqueString` in that header. I think this is actually true for all `multipart/*` MIME types.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types#multipartform-data

This is how curl works, if you pass `Content-Type: multipart/form-data` it'll still add a boundary because users shouldn't need to know how all the HTTP headers work, that's the library's job. We are running into this issue when trying to convert curl commands into requests code https://github.com/curlconverter/curlconverter/issues/248 and it would be convenient if Requests did the same thing as curl.

I don't see why anyone would ever want to pass `Content-Type: multipart/form-data` without a `boundary`, unless they just don't know about it. If you do actually need to not have a boundary you would have to explicitly add an empty one `; boundary=` to your header. But I can also see how it might surprise users that even if they pass a header manually it gets modified. 

## Expected Result

Requests still adds a `boundary` to my header if I don't, i.e. when I pass `headers={'Content-Type': 'multipart/form-data'}` and `files=`

## Actual Result

Requests uses the header as-is and the request fails.

## Reproduction Steps

```pycon
>>> import requests
>>> 
>>> files = {
...     'hello': (None, 'world'),
... }
>>> requests.post('http://example.com', files=files).request.headers['Content-Type']
'multipart/form-data; boundary=498931185311c522b8ca559425b2b965'
>>> 
>>> headers = {
...     'Content-Type': 'multipart/form-data',
... }
>>> 
>>> requests.post('http://example.com', headers=headers, files=files).request.headers['Content-Type']
'multipart/form-data'
```

Compare with curl:

```sh
$ curl -vvv example.com -H "Content-Type: multipart/form-data" -F "hello=world"
[...]
> Content-Type: multipart/form-data; boundary=------------------------ff92aa2ea98d6cc3
```

Hi @verhovsky,

Thanks for reaching out. I believe you may be looking for something like the [MultipartEncoder](https://toolbelt.readthedocs.io/en/latest/user.html) which is part of the [Requests Toolbelt](https://github.com/requests/toolbelt) package. We don't provide this directly through Requests and I'm not sure we have intentions to change that currently. That would be our recommendation for what you're currently trying to accomplish.

In the future, please search **closed and** open issues before creating new ones that are duplicates.
Requests: Python 2.7 Support and Deprecation

# Overview

As we enter our third year of Python 2.7 reaching end-of-life, Requests has decided it's time to start deprecating our support. While we have yet to confirm a date, we want to provide early notice that this is coming at some point in 2022. This follows as the majority of our critical downstream consumers have finished their deprecation campaigns. While we understand this may be viewed as an inconvenience for those still using Python 2.7, I'll try to layout the steps we're taking to ensure a smooth transition.

In January, we will be releasing Requests 2.27.0. This is intended to be the last minor release with guaranteed support, including all 2.27.x hotfix releases. We've taken steps to ensure this release series includes fixes for some of the longer standing pain points and provide as much long term stability as we're able. Users who intend to continue Python 2.7 use should review the `Mitigation Steps` section of this issue.

Going forward, we're still working through what a Python 3-only Requests looks like. We'd like to preface now that it's unlikely to contain many of the more exciting features that may have previously been promised by former maintainers. In future releases, we'll look to add quality-of-life improvements as they make sense while maintaining the primary focus of stability in the library. We'll provide more on that as information is available.

# Mitigation Steps

For Requests users that intend to remain on Python 2.7 through 2022, we recommend taking the following mitigation steps.

1. For users of `pip` 9.0+, we'll also be releasing an update to the `python_requires` metadata to `>=3.7` in Requests. This will prevent unintended installation on unsupported Python versions.
2. For users who want to codify this requirement across all environments, ensure that you've explicitly pinned Requests in your workspace or project to `requests<2.28`. This will prevent unintended interruptions with future releases.
3. For users of LTS operating systems providing Python 2.7 support through 2023, you'll need to work with their support teams for  bug or security issues for any releases after 2.27.

It's good to hear that requests decided to drop Python 2.7. I would propose to release version 3 without much refactoring but mainly dropping Python 2.7 compatibility, so that Python 3 users can benefit from less dependency and less code to run. Then we can do refactors or non-breaking changes in minor releases. Also it's possible to have another major release with more breaking changes. Otherwise it'll take much longer to drop Python 2.7 and do refactoring at the same time.

I wanted to provide a quick update on timelines. We're currently targeting May for the next release barring any unforeseen circumstances. That will be a minor release (2.28.0) with minimal functional changes but will likely be breaking for Python 2.7 users. We'd advise making sure you have appropriate pinning/`pip` versions supporting `python_requires` prior to then.

Resolving now that we have 2.28.0 out the door.
CVE-2021-33503: bump up urllib3 version

| Severity   | High                                                                             |
| Identifier | CVE-2021-33503                                                                   |
| URL        | https://nvd.nist.gov/vuln/detail/CVE-2021-33503                                  |
| Scanner    | gemnasium                                                                        |
| Message    | Uncontrolled Resource Consumption in urllib3                                     |
| Package    | urllib3 1.25.11                                                                  |
| Solution   | Upgrade to version 1.26.5 or above.                                              |
| Path       | requests 2.26.0 > urllib3  1.25.11                                                                          |
| File       | pipdeptree.json 

https://github.com/psf/requests/blob/d09659997cd1e3eca49a07c59ece5557071c0ab9/setup.py#L48 

This allows users to upgrade to 1.26.5 if they wish. We aren't restricting users from installing this
v2.27.0

2.27.0 (2022-01-03)
-------------------

**Improvements**

- Officially added support for Python 3.10. (#5928)

- Added a `requests.exceptions.JSONDecodeError` to unify JSON exceptions between
  Python 2 and 3. This gets raised in the `response.json()` method, and is
  backwards compatible as it inherits from previously thrown exceptions.
  Can be caught from `requests.exceptions.RequestException` as well. (#5856)

- Improved error text for misnamed `InvalidSchema` and `MissingSchema`
  exceptions. This is a temporary fix until exceptions can be renamed
  (Schema->Scheme). (#6017)

- Improved proxy parsing for proxy URLs missing a scheme. This will address
  recent changes to `urlparse` in Python 3.9+. (#5917)

**Bugfixes**

- Fixed defect in `extract_zipped_paths` which could result in an infinite loop
  for some paths. (#5851)

- Fixed handling for `AttributeError` when calculating length of files obtained
  by `Tarfile.extractfile()`. (#5239)

- Fixed urllib3 exception leak, wrapping `urllib3.exceptions.InvalidHeader` with
  `requests.exceptions.InvalidHeader`. (#5914)

- Fixed bug where two Host headers were sent for chunked requests. (#5391)

- Fixed regression in Requests 2.26.0 where `Proxy-Authorization` was
  incorrectly stripped from all requests sent with `Session.send`. (#5924)

- Fixed performance regression in 2.26.0 for hosts with a large number of
  proxies available in the environment. (#5924)

- Fixed idna exception leak, wrapping `UnicodeError` with
  `requests.exceptions.InvalidURL` for URLs with a leading dot (.) in the
  domain. (#5414)

**Deprecations**

- Requests support for Python 2.7 and 3.6 will be ending in 2022. While we
  don't have exact dates, Requests 2.27.x is likely to be the last release
  series providing support.

Currently pending #5894 and #5917.

Alright, things should be finalized for release notes. I'll move it out of draft for approval but let's wait to merge. I'll do that with the release and tagging process.
Auth is rarely working for anything

I find this module pretty much useless as for its auth:

```
# Even the example from the docs is not working:
In [36]: requests.get('https://api.github.com/user', auth=('108806', 'my_password'))
Out[36]: <Response [401]>
In [37]: with requests.Session() as sess:
    ...:     print ( requests.get('https://api.github.com/user', auth=('108806', 'my_password')).status_code )
Out[37]: <Response [401]>
```

Also, a lot of websites are returning 200 on everything, so how can I know if I'm logged or not?
Idk, checking cookies? Some sites are working without any, with ie JWT.
But even with the simplest scenario this module can't log me in when I pass the correct credentials on 9/10 of websites.




Hi @108806, thanks for the feedback. It sounds like there may be some misunderstandings about how authentication works in the HTTP protocol. You'll find Github [hasn't supported Basic Auth](https://developer.github.com/changes/2020-02-14-deprecating-password-auth/) for almost 2 years, which is the behavior you're observing.

We'd recommend doing some reading on how HTTP auth works and directing further questions to StackOverflow. We won't be able to provide support for that on this issue tracker.

I've added #6018 to remove the current ambiguity in the documentation.
requests 2.26.0 is forbidding NaN values in JSON

It is not possible to send NaNs (and I suppose also Infinity) double values through JSON.
The cause is in file https://github.com/psf/requests/blob/main/requests/models.py, where, at line 473, you have `allow_nan=False`.
Please restore the previous behavior, because it is now impossible for me to pass NaNs to computational microservices that need NaNs to mark not recorded values.
Setting `allow_nan=True` fixes the issue.
If this is not possible, at least allow the user to specify this flag from request calls.

Thanks.
Best regards.

## Expected Result

It should be possible to pass NaNs (and +/- Infinity) as numeric floating point values.


## Actual Result

An exception is raised every time I have a NaN into a floating point value.

## Reproduction Steps

```python
import requests
import numpy as np

input_dict = { 'value': np.nan }
endpoint = 'whatever is your endpoint'

requests.post(endpoint,json=input_dict)

```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "3.0.4"
  },
  "charset_normalizer": {
    "version": "2.0.6"
  },
  "cryptography": {
    "version": "2.8"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.10"
  },
  "platform": {
    "release": "5.11.0-43-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010106f",
    "version": "19.0.0"
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.26.7"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}
```


99.9% of people don't need or want `NaN` and those that do can serialize the data themselves, e.g.,

```py
import json

import requests
import numpy as np

input_dict = { 'value': np.nan }
endpoint = 'whatever is your endpoint'

requests.post(endpoint,data=json.dumps(input_dict), headers={"Content-Type": "application/json"})
```


@sigmavirus24 it's often times not in our control as to whether a `NaN` or an `inf` can be transformed to something else.

In addition, I think this is a regression between 2.25.1 and the newer versions. The `requests` package supported `NaN` and `inf` in 2.25.1 and moving to 2.26.0+ is causing brokenness. So, exposing a knob to help users who were dependent on that behavior is reasonable? Thoughts?


I've expressed my thoughts. Further, requests is in feature freeze and the tiniest but if looking into past similar issues shows a consistent attitude - if you don't like how requests encodes your request data in any given format, the are documented ways to do it yourself. Knobs will not be added
AttributeError: module 'collections' has no attribute 'MutableMapping'

Requests cannot be installed under Python 3.10. Python 3.10 removed importing abc from collections directly.

## Expected Result

Successful `pip install requests`

## Actual Result

```
  File "/usr/lib/python3/dist-packages/pipenv/vendor/requests/cookies.py", line 172, in <module>
    class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'
```

## Reproduction Steps

```sh
pip install requests

```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.9"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.0"
  },
  "platform": {
    "release": "5.10.60.1-microsoft-standard-WSL2",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.26.7"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Hi @unjello, this is not an issue with Requests. As you can see from the provided stacktrace, this is an old copy of requests vendored in `pipenv`. You'll need to either upgrade your version of `pipenv` to one that supports Python 3.10 or file a ticket in the [pypa/pipenv](https://github.com/pypa/pipenv) repo to get support.

Hey, sorry for the alarm. You're absolutely right. I have for some reason changed manually installed `pipenv` to `apt` installed one, which happened to be (surprise, surprise) old. I haven't connect the dots so I thought it was requests fault when I tried to install it in a new project :)
Extraordinarily slow performance on Mac

I'm trying to fetch a website using requests. While this is fast (<200ms) on Linux, it takes very long on my Macbook Air (>1min). Importantly, this does not appear to be the general networking capabilities of the two machines, because cURL is roughly equally fast.

## Expected Result

I would expect both systems to be equally fast and in both cases to be similarly fast as cURL.

## Actual Result

As explained above, requests is much slower on Mac than on Linux and also much slower than the equivalent request using cURL.

## Reproduction Steps

```python
import requests
import subprocess
import time

url = 'https://en.wikipedia.org/wiki/Kalman_filter'

t0 = time.time()
r = requests.get(url)
t1 = time.time()
subprocess.check_output(['curl', url])
t2 = time.time()

print('Requests:', t1-t0)
print('cURL:', t2-t1)
```
On Linux, the output is:
```
Requests: 0.1399681568145752
cURL: 0.1920619010925293
```
On Mac, the output is:
```
Requests: 75.28015923500061
cURL: 0.5037477016448975
```

## System Information

    $ python -m requests.help

On Linux (Ubuntu 20.04, running in virtual environment):
```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.9"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.5"
  },
  "platform": {
    "release": "5.11.0-40-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.26.7"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```
On Mac (Big Sur, M1 silicon, running in virtual environment):
```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.9"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.8"
  },
  "platform": {
    "release": "20.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.26.7"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```


I don't have the same problem on any of the macOS machines I have access to. It works seem something else is up with your specific environment on that computer.
No Error when iteration over streamed content randomly stops

<!-- Summary. -->

## Expected Result
An error is thrown when iterating over streamed content and the connection is closed/timed out by server.

<!-- What you expected. -->

## Actual Result
Iterating over streamed content randomly think it has all the content when it doesn't.

<!-- What happened instead. -->

## Reproduction Steps
If you try to use this function to download a large file with and the download speed is slow it will sometimes eventually just think it has completed writing all the content when it has not. I have tried including different headers and it didn't prevent this from happening. Sorry if this doesn't count as a bug, I can not find any similar issues online.

edit: found that setting chunk_size=1 on a large video file will have the same effect as a slow connection.

```python
import requests

def requests_download(url:str, file_name:str):
    responce = requests.get(url=url, stream=True)
    with open(file_name, 'wb') as f:
        for chunk in responce.iter_content(chunk_size=1024):
            f.write(chunk)
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {        
    "version": "4.0.0"
  },
  "cryptography": {   
    "version": ""     
  },
  "idna": {
    "version": "2.10" 
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.5"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.26.4"
  },
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


In no case do we raise an error when a server sends less data than they promised, streaming or otherwise. Further, we don't raise an error when it sends more than it promised. Case in point, if a server compresses its response, the Content-Length header will be the length of the compressed response. If we raised an error after decompressing that the lengths were different, that would be nonsensical.
Python requests library call works on Windows but not Linux... WHY?

Long time user of python requests here. Trying to do a simple call to this endpoint: 
`https://www.overstock.com/api/product.json?prod_id=10897789`

My current code:
```
import requests

headers = { 'User-Agent': 'Mozilla/5.0', 'Accept': 'application/json' }
url = 'https://www.overstock.com/api/product.json?prod_id=10897789'
r = requests.get( url, headers=headers )
result = r.json()
print( result )
```

## Expected Result
(Shortened)
```
{'categoryId': 244, 'subCategoryId': 31446, 'altSubCategoryId': 0, 'taxonomy': {'store': {'id': 1, 'name': 'Rugs', 'apiUrl': 'https://www.overstock.com/api/search.json?taxonomy=sto1', 'htmlUrl': 'https://www.overstock.com/Home-Garden/1/store.html'}, 'department': {'id': 3, 'name': 'Casual Rugs'...
```

## Actual Result

Unfortunately, from that same script on Linux, I am not getting the identical result. So far I am stumped as to why this is happening...

Here is the ugly Linux error:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/.local/share/virtualenvs/online-project-7j1lNF7P/lib/python3.6/site-packages/requests/models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
```


## Reproduction Steps

```
import requests

headers = { 'User-Agent': 'Mozilla/5.0', 'Accept': 'application/json' }
url = 'https://www.overstock.com/api/product.json?prod_id=10897789'
r = requests.get( url, headers=headers )
result = r.json()
print( result )

```

## System Information

1. Windows 10
2. Ubuntu 18.04.6 LTS (GNU/Linux 4.15.0-163-generic x86_64)



Hi @doverradio,

Something is triggering the server to return a non-JSON response when calling the service on Linux. Using the `json()` api requires that the payload be valid json, and you may need to add code to verify that before invoking `json()`. You're likely encountering some form of bot detection which is changing the response for Linux.

Unfortunately this isn't an issue with Requests. We'd recommend opening an issue on StackOverflow if you're interested in finding alternative solutions to your issue. Thanks!
Rejecting URLs with Unverified SSL Certs

Fixes an issue with unverified SSL certificates not being rejected

Non-TLS URLs will not send certificates to verify. Why do you think this library should reject those URLs no matter what?

Spoiler: We will *not* be accepting this PR

Thanks for the proposal @Mr1Vgy, but unfortunately this breaks all non-https URIs, which is not something we intend to change. I'm resolving since we won't be able to accept this.