Raibek/requests-github-issues huggingface dataset:
response.reason is None instead of the expected reason string

<!-- Summary. -->

`response.reason` is `None` instead of the expected reason string.

## Expected Result
Print `'INTERNAL SERVER ERROR'` or something similar

<!-- What you expected. -->

## Actual Result

`'None'` is printed
<!-- What happened instead. -->

## Reproduction Steps

```python
    requests_mock.get(
        'https://google.com',
        json={},
        status_code=500
    )

    response = requests.get('https://google.com')
    print(response.reason)
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.1.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.0"
  },
  "platform": {
    "release": "22.1.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010113f"
  },
  "urllib3": {
    "version": "1.26.13"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


If the mocking library doesn't set it on the Response it creates, why do you think this is a bug in requests? 

Because the [docs](https://requests.readthedocs.io/en/latest/api/?highlight=reaso#requests.Response.reason) claim it will be there and the response reason phrase is [optional](https://stackoverflow.com/questions/17517086/can-an-http-response-omit-the-reason-phrase).

If this is indeed not planned, this should at least be reopened as a docs request to improve the documentation, claiming that the value will be `None` unless the server specifies a value.

You don't understand. Requests is not creating the response, requests-mock a separate unrelated project does https://github.com/jamielennox/requests-mock/blob/c19c3343d3a70a57427bd37fbb5fcffb0b7def1c/requests_mock/response.py#L142
Fix the proxy_bypass_registry function all returning true in some cases.

With the context code,i think we should avoid this situation with empty strings,thx!

There's no clear description of a bug, how to reproduce or even tests to show this fixes something. Further you speak of empty strings and are filtering on None which are not the same thing. 

The value of proxyOverride is controlled by a third-party program on the computer, so it is not controllable.
For example, when I use fiddler on my computer, this value is `<-loopback>;`.(Not all computers)
When proxyOverride splits, it's equal to `['<-loopback>', '']`.
So `re.match('', host, re.I)` will return true,that bypassing the proxy.

And filter(function, iterable) is equivalent to the generator expression (item for item in iterable if item) if function is None,that is, all elements of iterable that are false are removed.
Ref:https://docs.python.org/3/library/functions.html#filter

I think should be avoided for code robustness.

> The value of proxyOverride is controlled by a third-party program on the computer, so it is not controllable. For example, when I use fiddler on my computer, this value is `<-loopback>;`.(Not all computers) When proxyOverride splits, it's equal to `['<-loopback>', '']`. So `re.match('', host, re.I)` will return true,that bypassing the proxy.
> 
> And filter(function, iterable) is equivalent to the generator expression (item for item in iterable if item) if function is None,that is, all elements of iterable that are false are removed. Ref:https://docs.python.org/3/library/functions.html#filter
> 
> I think should be avoided for code robustness.

Yes, I also noticed this situation. This wastes a lot of my time discovering this problem. ProxyOverride sometimes ends with a semicolon and is difficult to reproduce. But it has not yet been merged into the main branch.

> > The value of proxyOverride is controlled by a third-party program on the computer, so it is not controllable. For example, when I use fiddler on my computer, this value is `<-loopback>;`.(Not all computers) When proxyOverride splits, it's equal to `['<-loopback>', '']`. So `re.match('', host, re.I)` will return true,that bypassing the proxy.
> > And filter(function, iterable) is equivalent to the generator expression (item for item in iterable if item) if function is None,that is, all elements of iterable that are false are removed. Ref:https://docs.python.org/3/library/functions.html#filter
> > I think should be avoided for code robustness.
> 
> Yes, I also noticed this situation. This wastes a lot of my time discovering this problem. ProxyOverride sometimes ends with a semicolon and is difficult to reproduce. But it has not yet been merged into the main branch.

I'm glad someone else is having the same problem as me, but I also hope that no one will waste time on this issue in the future, haha.

@flysee can you add tests for this change so it does not regress? 

> @flysee can you add tests for this change so it does not regress?

Of course, test cases have been added.
Specify complete list of all available phony targets

The list of phony targets was incomplete. Add a list of the phony targets explicitly at the end of the file.

I don't believe we're using this any longer. Thanks for trying to fix it up but I would just as soon delete it

If that's true, then there's a case for removing the Makefile from the repository. No point in keeping it around if any energy put towards maintaining it is dismissed.

If the Makefile is in fact used, and one can make a good argument for maintaining such a file in the repository (as analogously NPM based repositories offer a 'scripts' directive in the package.json file), then I maintain that the proposed changes are relevant.
How to change Keep-Alive parameters?

We have tried setting these headers ->

Connection: keep-alive
Keep-Alive: timeout=5, max=2

But it is not working as expected

https://requests.readthedocs.io/en/latest/user/advanced/#keep-alive

Documentation does not specify how to change the keep-alive parameters 

Hi there,

Thanks for reaching out. Unfortunately, there's too little detail in this issue to be able to help you and it's not reproducible. That indicates a likely problem in your environment which we are not responsible for debugging and which the maintainers of this project do not have the time to help you with. As a result, I'm closing this issue.
Conflicting version with charset-normalizer

Hello,

I would like to update my virtual environment.
I am using python 3.9, requests 2.28.1 and charset-normalizer 2.1.1
When I update charset-normalizer:
`    $  python3 -m pip install -U charset-normalizer`
I get this answer:
`ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
requests 2.28.1 requires charset-normalizer<3,>=2, but you have charset-normalizer 3.0.1 which is incompatible.
`

Is a new version of the Requests library expected in a near future?

Thank you.

In the future, please search **closed and** open issues before creating new ones that are duplicates.

Already fixed in #6261, pending a release.
JS Render

Greetings to all! I have a suggestion to create the possibility of rendering js scripts in requests. Yes, I know that there is a requests-html project, but it is abandoned, and somehow you need to cope with js

Requests is just a lib for (as the name suggests) http requests.Not more. It is not a Browser and will never be. You should take a look at Selenium, which allows you to control a Browser like Firefox or Chrome.

In the future, please search **closed and** open issues before creating new ones that are duplicates.
requests.put() does not respect timeout when data is chunked

`requests.put(url, data=some_generator, timeout=5.0)` does not timeout if the server accepts the connection but does not send a response.

## Expected Result

`requests.put(url, data=some_generator, timeout=5.0)` would time out after 5 seconds if the server does not send a response.

## Actual Result

`requests.put(url, data=some_generator, timeout=5.0)` can hang indefinitely if the server does not send a response and does not close the connection.

## Reproduction Steps

```python
import requests

def data_chunked():
    data = [b'hello', b'world']
    for chunk in data:
        yield chunk

# This can block forever if the server accepts the connection but does not respond
requests.put("http://localhost:8080/", data=data_chunked(), timeout=5.0)

# This times out as expected
requests.put("http://localhost:8080/", data=b'hello', timeout=5.0)
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.10"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.8"
  },
  "platform": {
    "release": "3.10.0-1062.18.1.el7.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.26.8"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```


Thanks for the report, @missing64001. We moved to use urllib3's chunking behavior in 2.30.0 which should have resolved this. I'll close this out along with the PR since it no longer applies to the code base. Please let us know if you're still seeing the issue with the latest release.
how to increase the chunk size when sending an http stream-upload request through request.post and receiving it through fastapi

This is fastapi http side:
      
```
       from fastapi import Request
       import aiofiles

       @app.post('/upload')
        async def upload(request: Request):
            try:
                filename = request.headers['filename']
                async with aiofiles.open(filename, 'wb') as f:
                    async for chunk in request.stream():
                        await f.write(chunk)
            except Exception:
                return {"message": "There was an error uploading the file"}
         
            return {"message": f"Successfuly uploaded {filename}"}
```

this is the request side:
```
    def gen_file_stream():
        with open(file_name, "rb") as f:
            while True:
                data = f.read(1024*1024)
                if not data:
                    break
                yield data
          resp=requests.post(url,data=gen_file_stream())
```

when I check the size of the chunk its 64k and not 1MB (as Im trying to send here).
If I use TestClient from fastapi it is working, I mean Im able to send a chunk of 1MB, so it seems to me requests package is dividing the size of the chunk internally to 64KB.
is there any way I can make request.post to send a chunk of 1MB?
   


In the future, please search **closed and** open issues before creating new ones that are duplicates.
charset-normalizer upgraded to 3.x

charset-normalizer has been upgraded to version 3. requests requires version >= 2 and < 3.

## Expected Result

support for charset-normalizer 3

## Actual Result

warning of incompatibility with charset-normalizer 3

## Reproduction Steps

```shell
% pip install --upgrade charset-normalizer
Requirement already satisfied: charset-normalizer in /usr/local/lib/python3.10/site-packages (3.0.0)
Collecting charset-normalizer
  Downloading charset_normalizer-3.0.1-cp310-cp310-macosx_10_9_x86_64.whl (124 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.2/124.2 kB 2.5 MB/s eta 0:00:00
Installing collected packages: charset-normalizer
  Attempting uninstall: charset-normalizer
    Found existing installation: charset-normalizer 3.0.0
    Uninstalling charset-normalizer-3.0.0:
      Successfully uninstalled charset-normalizer-3.0.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
requests 2.28.1 requires charset-normalizer<3,>=2, but you have charset-normalizer 3.0.1 which is incompatible.
Successfully installed charset-normalizer-3.0.1
```

## System Information

    $ python -m requests.help

```shell
% python -m requests.help
/usr/local/lib/python3.10/site-packages/requests/__init__.py:109: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (None)/charset_normalizer (3.0.1) doesn't match a supported version!
  warnings.warn(
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.0.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.8"
  },
  "platform": {
    "release": "21.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010113f"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


This is addressed in https://github.com/psf/requests/pull/6261 but hasn't been released yet.

For what it's worth, this is on the 2.29.0 milestone.
Requests sends garbage when requesting to a web server on the same machine 

## Summery

When sending to a web server located on the same machine as the python code is executing, it will send garbage data on all of the fields, the request itself, exiting with the error  `Invalid SSL version`.  The server logs show what looks like garbage data.

![image](https://user-images.githubusercontent.com/62214409/202252392-8fa13e5a-91a1-4f9c-8d66-c33d21a8b703.png)

## Expected Result

I thought that it would work normally and just send a normal web request rather then garbage data.

## Actual Result

It sent a lot of garbage data and the server returned Status Code 400.
## Reproduction Steps
Shell Commands:
```bash
python -m http.server 8080
```
Python Code:
```python
import requests

requests.get('https://0.0.0.0:8080')

# you could also use https://127.0.0.1:8080, and https://localhost:8080 and you will get the same result. Using HTTP protocol also produces the same result. 
```

## System Information

    $ python -m requests.help

```json
{
  {
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.1.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.12"
  },
  "platform": {
    "release": "5.15.0-1021-gcp",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "101010df"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com/) for general usage and network fundamentals questions.
Move to urllib3 version 2.0.0 and Move Requests to version 2

With urllib3 releasing the alpha of 2.0.0, is it time to consider moving requests to version 2.0.0 and make the long standing changes that are needed to improve the library?

Requests is already on 2.x. Thanks for the feedback, we're aware of the upcoming changes to urllib3 and are finishing up preparations for the release. We may look at major version bumping Requests at a later date but have no immediate plans.
downloading 125 KB file hangs lib

https://raw.githubusercontent.com/MycroftAI/mycroft-skills-data/21.02/skill-metadata.json

# hangs at this line
r = requests.get(market_info_url)
# never gets here.

Checked OS memory and have like 2 GB free.

Any clue how to fix this and me debug the lib?

is ver 2.28.1

thank you.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com/) for general usage questions.
requests 2.28.1 is imcompatible with charset-normalizer 3.0.0 

I am using python 3.10.8 in Arch linux. When I update all pip packages,  `requests 2.28.1`  become broken and throws this error-  `requests 2.28.1 requires charset-normalizer<3,>=2, but you have charset-normalizer 3.0.0 which is incompatible.`

In the future, please search **closed and** open issues before creating new ones that are duplicates.

> 

requests 2.28.1 requires charset-normalizer<3,>=2, but you have charset-normalizer 3.0.0 which is incompatible.
HTTPS Request through HTTP Proxy is sent incorrectly when using chunked body

I am using the chunked request body feature (`data=generator`). When defining a HTTP proxy and sending a request to a HTTPS URL, `requests` is trying to perform a TLS handshake with the (HTTP) proxy server, instead of sending a CONNECT request and performing the handshake with the target server through the tunnel. This behaviour prevents the request from completing successfully.

## Expected Result

`requests` should send a CONNECT request to the proxy and negotiate TLS with the target server through the tunnel.

## Actual Result

`requests` tries to negotiate TLS directly with the HTTP proxy server, and fails to do so.

## Reproduction Steps

Run an HTTP proxy on `127.0.0.1:8080` and execute the following code:
```python
  import requests
  
  def body_generator():
    yield "test"
    return
  
  requests.post("https://www.google.com/",
    proxies={"https": "http://127.0.0.1:8080/"},
    data=body_generator(),
  )
```

## More info

The chunked encoding generator feature is implemented in `requests/adapters.py`:
```python
try:
    if not chunked:
        resp = conn.urlopen(
            ...
        )

    # Send the request.
    else:
        if hasattr(conn, "proxy_pool"):
            conn = conn.proxy_pool

        low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)

        try:
            ... Send request with chunked body ...
```

In the case of chunked encoding, instead of using `conn.urlopen` to send the request, it uses `conn._get_conn` and sends the request manually.
It seems like, for some reason, `conn._get_conn` is not handling the proxy correctly.

I can confirm this is still an issue on latest release.
I filed a duplicate issue (now closed) here: https://github.com/psf/requests/issues/6371

I am using requests 2.28.2 with urllib3 as 1.26.14, and getting below issue when calling the api (proxy is not added).

`requests.exceptions.SSLError: HTTPSConnectionPool(host="my client host", port="my client port"): Max retries exceeded with url: /v1/omni-channel/message (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1129)')))`

Is there anything else that I need to check?

Seems there is an issue with the SSL/TLS connection when making the API call. 

You can check few things - 

1. Verify the URL: Ensure that the URL you are using is correct and corresponds to the intended API endpoint.
2. Updating the libraries 'requests' and its dependencies 'urllib' and 'ssl' to the latest one.
3. Verify the SSL/TLS version compatibility with the client-server versions
"proxy_request_buffering off" in NGINX proxy causes ConnectionAbortedError

Hello guys. I've searched a lot about this error and ended up not finding any answer elsewhere. So I decided to ask here, in case anyone can help.
I'm getting this error whenever I use lib Requests on NGINX proxies with the parameter "proxy_request_buffering off" configured in nginx.conf:

The error happens when uploading any file.
At first, I take an HTTP 200 (OK) when I establish the connection and a 307 (Temporary Redirect) to send the file to the final destination.
But after a few seconds, I get the error: ConnectionAbortedError; [WinError 10053] An established connection was aborted by the software in your host machine

By Curl I can send the file normally without problems.

Any suggestions that don't involve stripping the "proxy_request_buffering off" parameter from the NGINX proxy? (I need him)


Thanks


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
[Feature request or doc oversight?] Apparently, there's still no way to read uncompressed content as file-like object #465 

This was originally discussed here: https://github.com/psf/requests/issues/465 For some reason the ticket was however closed without a file-like object approach being added here: https://requests.readthedocs.io/en/latest/user/quickstart/ (Other than the somewhat odd `.raw` that returns something I suspect most people that just want a file-like stream aren't interested in.) There doesn't seem to be an explanation why that ticket was closed, and the discussion suggesting *something* was fixed but I can't really figure out what.

Since it's not that easy to wrap `iter_content()` to a file like object with arbitrary read amounts and just wrapping `.content` for that generically seems to be unsafe to me due to denial of service, it would be nice if this was provided and then documented here: https://requests.readthedocs.io/en/latest/user/quickstart/ (With a section alike to "Raw Response Content".)

The requests version I tested on is: 2.27.1
RFE: support for `charset_normalizer` >= 3

Any plans to provuide support for `charset_normalizer` >= 3?

```
requires = [
    "charset_normalizer>=2,<4",
```

it's already supported in main branch

There are as well checks in `__init__.py`

OK I see that changes in `__init__.py` are added as well.
BTW: does really all those checs really necessary? 🤔 

Already fixed in #6261, pending a release.
Using a custom CA certificate

hi, i was hoping to use the `SSL_CERT_DIR` env var to use a custom ca, but it seems like this feature isn't actually in requests just yet based on this comment https://github.com/psf/requests/pull/2903#issuecomment-279012869. is there a way to use a custom ca with requests?

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Fix setuptools deprecation warnings

Update keys used in `setup.cfg` in order to fix the following setuptools deprecation warnings:

> The license_file parameter is deprecated, use license_files instead.

> Usage of dash-separated 'provides-extra' will not be supported
> in future versions. Please use the underscore name 'provides_extra'
> instead

> Usage of dash-separated 'requires-dist' will not be supported
> in future versions. Please use the underscore name 'requires_dist'
> instead

Thanks for the efforts, but I have updated as you instructed but still getting this error:

C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\site-packages\setuptools\dist.py:771: UserWarning: Usage of dash-separated 'provides-extra' will not be supported in future versions. Please use the underscore name 'provides_extra' instead
  warnings.warn(
C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\site-packages\setuptools\dist.py:771: UserWarning: Usage of dash-separated 'requires-dist' will not be supported in future versions. Please use the underscore name 'requires_dist' instead
  warnings.warn(
C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\site-packages\setuptools\config\setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
  warnings.warn(msg, warning_class)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: no commands supplied
Requests __init__.py emits warnings about version numbers

## Background

I have the following package versions installed alongside requests:

```
urllib3            1.26.12
charset-normalizer 3.0.0
```
This complies with the version constraints in the requests `setup.py`.

The `__init__.py` code in requests does not match what is in `setup.py` and so warns about the versions of both of these packages. `__init__.py` wants `urllib3 >= 1.21.1, <= 1.26` and `charset_normalizer >= 2.0.0 < 3.0.0`.

## Expected Result

No warning.

## Actual Result

A slightly confusing runtime warning appears!

## Reproduction Steps

```
$ python3 -m requests.help
/home/bje/source/nemo/myenv/lib/python3.10/site-packages/requests/__init__.py:109: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (None)/charset_normalizer (3.0.0) doesn't match a supported version!
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.0.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.6"
  },
  "platform": {
    "release": "5.15.0-47-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

Hi @bje-, please check open and closed issues before opening new ones. The current release of Requests doesn't support charset_normalizer 3.0, hence the warning. It will be updated in the next release.

Thanks, but why isn't this reflected in `setup.py` so that `pip` would have alerted me to the issue?

It is, the supported range in setup.py for 2.28.1 is `<3`. You've upgraded the package independently of Requests which is creating the conflict.
charset-normalizer released version 3.0.0, requests support <3 - unable to keep up to date packages as a result

<!-- Summary. -->

charset-normalizer released version 3.0.0, requests require charset-normalizer <3, as a result packages can't be kept up to date (possible security issues left unpatched going forward)

```
The conflict is caused by:

    The user requested charset-normalizer==3.0.0

    requests 2.28.1 depends on charset-normalizer<3 and >=2
```

this is due to https://github.com/psf/requests/blob/main/setup.py#L62

Seems quite similar to and idna issue I fixed a while back at https://github.com/psf/requests/issues/5710

## Expected Result

Be able to install requests with the latest version of the dependent package

## Actual Result

Fails installing with the latest version of charset-normalizer

## Reproduction Steps

```shell
pip install charset-normalizer==3.0.0
pip install requests
```
Watch it fails


And just now noticed that was fixed at https://github.com/psf/requests/pull/6261 just not released yet with a new version so gonna close this
Allow charset normalizer >=2 and <4

charset-normalizer 3.0.0 has been released.

v3.0.0 up to 4x faster than v2.1
https://github.com/Ousret/charset_normalizer/releases/tag/3.0.0
https://charset-normalizer.readthedocs.io/en/latest/community/speedup.html



Yes, and given this isn't critical path for most users it's not critical to force people through a major version change to get that. If they use character detection of any sort and need performance they can force it to upgrade, but that's not our job

Thanks @deedy5!

Any estimation on a new release with this fix? 

This should now be available on PyPI in Requests 2.28.2.
Why isn't `JSONDecodeError` included in the documentation?

[The documentation](https://requests.readthedocs.io/en/stable/api/#exceptions) not include the exception [`JSONDecodeError`](https://github.com/psf/requests/blob/main/requests/exceptions.py#L31), which means that we can't reference it in our docs (i.e. ``:external:py:class:`requests.exceptions.JSONDecodeError` `` fails). Why is this? And can you include it in the docs?

@nbro10 
Looks like already fixed in https://requests.readthedocs.io/en/latest/api/#:~:text=exception%20requests.JSONDecodeError

@PratikMMohite Thanks for pointing this out. But [the stable version](https://requests.readthedocs.io/en/stable/api/#exceptions) still doesn't contain this change. We're using the stable documentation.

Yes, this was fixed in https://github.com/psf/requests/pull/6262 as a result of this bug report. @nateprewitt I believe this can be safely closed now.
How to disable only hostname verification

Summary.
How to disable only hostname verification

## Expected Result
Communicate via SSL/TSL connection without hostname verification only

What you expected.
Disable hostname verification

## Actual Result
Communicate via SSL/TSL connection without hostname verification only

What happened instead.
Tried with Verify=False. But it disables entire certificate validation 

## Reproduction Steps

```python
import requests
res = requests.get(<url>, headers=<headers>, timeout=30)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.6"
  },
  "platform": {
    "release": "3.10.0-1160.53.1.el7.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.0"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.26.9"
  },
  "using_pyopenssl": false
}

```
This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Add an Example for automatic retries to the Advanced Usage docs

While Requests doesn't automatically retry failures, this is an extremely common advanced use case in real world applications.

Although there's a mention of this capability in the [`HTTPAdapter` API docs](https://requests.readthedocs.io/en/latest/api/#requests.adapters.HTTPAdapter), it's a bit buried and not very specific.

The ubiquity of the use case fits perfectly in the [Transport Adapters section of the Advanced Usage docs](https://requests.readthedocs.io/en/latest/user/advanced/#transport-adapters), so I thought it would be useful to have a simple Example of this capability.

I took a stab at such an addition, and tried to follow the style of the rest of the docs pretty closely.

Linking over to the [`urllib3.util.Retry` docs](https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.Retry) seems like it should reinforce the fact that Requests isn't the owner of that code and that library should be consulted with questions about that class (rather than Requests' support of it).

I'm curious if maintainers have any feedback about the content or code example. Specifically I was curious whether you have any opinions as to the appropriate standard for the `Retry` import. While `urllib3.util.Retry` is where the class lives and what the `urllib3` docs reference, `from urllib3 import Retry` also works, and in some StackOverflow posts I've seen people use `from requests.adapters import Retry`. I felt that last one would telegraph an ownership by Requests that would be undesirable, but I'm wondering if anyone is particularly picky about the best practice for that import.

Also, let me know if this is non-trivial enough to add a HISTORY entry for.

@nateprewitt and @sethmlarson, apologies for the tag-summon (I find that a bit discourteous sometimes) but I think this is a pretty low-effort, low-risk, high-value docs change that folks would find useful, and I'm guessing it probably just got lost in the rest of the activity on this project.

Do either of you (or any other maintainers) have any thoughts or feedback on the change in general, or the questions I posed above in the original post? Is this something you'd consider accepting into the docs?

Hi @matthewarmand, thanks for checking in. This does look reasonable at first glance, it appears to have fallen off the review radar. I'll do a quick review this evening and let you know if anything needs to change. Otherwise, I'm alright with adding this.
Auth not working for post/put but works for get

Auth works fine with get requests, but failing in posts and puts

## Expected Result
Get a http status code

## Actual Result

` File "/Users/erik.mathis/.pyenv/versions/3.10.5/lib/python3.10/site-packages/requests/api.py", line 130, in put
    return request("put", url, data=data, **kwargs)
  File "/Users/erik.mathis/.pyenv/versions/3.10.5/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/erik.mathis/.pyenv/versions/3.10.5/lib/python3.10/site-packages/requests/sessions.py", line 573, in request
    prep = self.prepare_request(req)
  File "/Users/erik.mathis/.pyenv/versions/3.10.5/lib/python3.10/site-packages/requests/sessions.py", line 484, in prepare_request
    p.prepare(
  File "/Users/erik.mathis/.pyenv/versions/3.10.5/lib/python3.10/site-packages/requests/models.py", line 372, in prepare
    self.prepare_auth(auth, url)
  File "/Users/erik.mathis/.pyenv/versions/3.10.5/lib/python3.10/site-packages/requests/models.py", line 603, in prepare_auth
    r = auth(self)
TypeError: 'dict' object is not callable`

## Reproduction Steps

```python
import requests
auth = HTTPBasicAuth('X', 'X')
headers = {
    'Content-Type': 'application/json'
}
url="abc.com"

def get(path: str=None, auth: tuple=auth, headers: dict=headers):
    uri=f'{url}{path}'
    return requests.get(uri, auth: tuple=auth, headers=headers)

def post(path: str=None, auth: tuple=auth, headers: dict=headers, data: dict=None):
    uri=f'{url}{path}'
    return requests.post(uri, auth=auth, headers=headers, data=json.dumps(data))
```

## System Information

    $ python -m requests.help

```json
$ python -m requests.help
{
  "chardet": {
    "version": "5.0.0"
  },
  "charset_normalizer": {
    "version": "2.1.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.5"
  },
  "platform": {
    "release": "21.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010111f"
  },
  "urllib3": {
    "version": "1.26.11"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Your reproduction steps are not working and your code looks woefully broken. You're also passing the wrong type in which means you'll never get a HTTP response code since we can't create a valid request for you with what you've given us. Please read the exception in the future and ask for help on Stack overflow
Mention pickling in the docs for creating custom Transport Adapters

While subclassing `HTTPAdapter` as well as `Session` the class variable `__attrs__` needs to be appended with any custom instance variables. Otherwise, the instance breaks during pickling, which can happen in many different scenarios - for example, during Django's `setUpTestData` deep copying/in-memory data isolation. I think it would be nice if this is mentioned in the documentation. 

There's a classic example of a custom adapter for setting a timeout value that is mentioned in multiple blog posts. Unfortunately, all of these examples are essentially broken because of a lack of information about pickling and `__attrs__` class variable. 

And yet the pattern of overriding `__attrs__` exists in many open source libs, for example:

https://github.com/requests/toolbelt/blob/master/requests_toolbelt/adapters/ssl.py#L46
https://github.com/docker/docker-py/blob/main/docker/transport/unixconn.py#L57

If this pattern is so commonly used and needed, it might be a good idea to add it to the docs. Again, this is because so many people can stumble upon this issue.
python context.verify_mode is ignored by requests module

In the following code the response contains ssl error `SSL: CERTIFICATE_VERIFY_FAILED` although the default verify_mode `ssl.CERT_NONE` is set in the context by calling context=create_ssl_context() as hown below. I expected that I to get no ssl error.

It seems that `verify_mode` in the context is ignored by the requests module. It is not clear to me what attributes from the context are not ignored by the requests module? will it take context.minimum_version? Thanks

Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))

```
def create_ssl_context(verify_mode=ssl.CERT_NONE,
                   key_file=None, cert_file=None,
                   cafile=None,
                   tls_min_version=None):
    context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    context.verify_mode = verify_mode
    if tls_min_version:
        context.minimum_version = tls_min_version
    if key_file or cert_file:
        context.load_cert_chain(cert_file, key_file)
    if cafile:
        context.load_verify_locations(cafile)
    else:
        context.set_default_verify_paths()
    return context

class SSLContextAdapter(requests.adapters.HTTPAdapter):
   def __init__(self, ssl_context=None, **kwargs):
      self.ssl_context = ssl_context
      super().__init__(**kwargs)

   def init_poolmanager(self, *args, **kwargs):
      kwargs['ssl_context'] = self.ssl_context
      return super(SSLContextAdapter, self).init_poolmanager(*args, **kwargs)


def client_side(context, hostname, data, headers):
  s = requests.Session()
  context = create_ssl_context()
  s.mount('https://', SSLContextAdapter(context))
  s.mount('http://', SSLContextAdapter(context))
  print(context.verify_mode)
  response = s.post(hostname, data=data, headers=headers)
  print(response.text)



I identified the issue it is that the kwargs argument sends `verify` as True despite the adapter's verify_mode is is set to None.
The solution was to add a check to disable SSL certificate verification when the adapter's `ssl_context.verify_mode` is set to `ssl.CERT_NONE`, by setting the `verify` parameter to False in the `kwargs` argument.
Check the following [branch](https://github.com/eth7/requests/tree/fix-ssl-certification-validation-when-set-to-none).
Response library in python give me back a wrong response status code when trying to request a JavaScript file

<h3>Describe the bug</h3>
Hello,<br>
When I use the response library in python I get the wrong response status code but I get the right one when I use urllib.request

<br>Code Using Response library:

```
import requests
try:
    r = requests.head("https://www.codecademy.com/cdn-cgi/challenge-platform/h/g/scripts/alpha/invisible.js")
    print(r.status_code)
    # prints the int of the status code. Find more at httpstatusrappers.com :)
except requests.ConnectionError:
    print("failed to connect")
```

This URL `https://www.codecademy.com/cdn-cgi/challenge-platform/h/g/scripts/alpha/invisible.js` gives me back `404` status code even though it gives me 200 when I visit the URL in the browser and I can see the javascript response.

When I used the urllib.request library it gave back the right status code which is 200

Code Using urllib.request library:

```
from urllib.request import urlopen
urls = ['https://www.codecademy.com/cdn-cgi/challenge-platform/h/g/scripts/alpha/invisible.js','https://www.codecademy.com/idontexist.js']
results={}
for url in urls:
    try:
        ret = urlopen(url)
        if ret.code == 200:
            state="Exists"
            print(url+' : {}'.format(state))
    except:
        state="Not exists"
        print(url+' : {}'.format(state))

    results[url]=state
```

<h3>Additional context</h3>
Maybe there are more situations where this happens than just JavaSript files I didn't try more URLs.

<h3>Version of responses</h3>
0.21.0

<h3>Steps to Reproduce</h3>

1- Go to this URL https://www.codecademy.com/cdn-cgi/challenge-platform/h/g/scripts/alpha/invisible.js in your browser and you will get a 200 response.

2- Run this code and you will get 404 not found
```
import requests
try:
    r = requests.head("https://www.codecademy.com/cdn-cgi/challenge-platform/h/g/scripts/alpha/invisible.js")
    print(r.status_code)
    # prints the int of the status code. Find more at httpstatusrappers.com :)
except requests.ConnectionError:
    print("failed to connect")
```

<h3>Expected Result</h3>
The code should have given back a 200 status code.

<h3>Actual Result</h3>
The code gives back 404 a status code.

You are sending a HEAD request, change it to a GET request

`r = requests.get("https://www.codecademy.com/cdn-cgi/challenge-platform/h/g/scripts/alpha/invisible.js")`

Also you haven't even bothered to change your issue https://github.com/getsentry/responses/issues/591

Thanks, it's working now. If may I ask why HEAD doesn't give the right status code?

The server hasn't implemented consistent responses for different http methods. We aren't able to control this as it all happens server side.

I'm going to resolve this as it's unrelated to Requests. If you have further questions, StackOverflow is likely the best place to check.
Lib version for Python 3.4 ?

For CudaText editor with Python 3.4 on Windows XP, we need 'requests' for py 3.4.
 Current version uses f-strings - not easy to fix all places. and it uses JSONDecodeError which is missing in py 3.4.
Pls help to find old release.

@spyrosmas needs it.

Found on pypi.
Accept-Encoding header can't be completely removed

I am essentially reposting #5564 (which is similar to #5671) with more details, since it was closed as a question but it isn't a question. It's reporting a mismatch between what the user wanted/expected and what actually happened.

In one terminal tab, I ran this command

```sh
nc -l 8888
```

in another I ran this Python code:

```pycon
>>> import requests
>>> requests.get('http://localhost:8888')
```

Which sent this HTTP request:

```none
GET / HTTP/1.1
Host: localhost:8888
User-Agent: python-requests/2.28.0
Accept-Encoding: gzip, deflate, br
Accept: */*
Connection: keep-alive
```

Next, I tried modifying my Python to not send any headers, which [the documentation](https://requests.readthedocs.io/en/latest/user/advanced/#session-objects) says you can do like this:

> Sometimes you’ll want to omit session-level keys from a dict parameter. To do this, you simply set that key’s value to `None` in the method-level parameter. It will automatically be omitted.

So I ran this Python code

```pycon
requests.get('http://localhost:8888', headers={
    'Host': None,
    'User-Agent': None,
    'Accept-Encoding': None,
    'Accept': None,
    'Connection': None
})
```

## Expected Result

I expected to see an HTTP request with no headers:

```
GET / HTTP/1.1
```

## Actual Result

The `Accept` and `Connection` headers are removed, the `Host` header is kept, but `Accept-Encoding` and `User-Agent` do weird things.

```
GET / HTTP/1.1
Host: localhost:8888
Accept-Encoding: identity
User-Agent: python-urllib3/1.26.9
```

The solution here is the same as #5671:

```py
import requests
import urllib3

requests.get('http://localhost:8888', headers={
    'Host': urllib3.util.SKIP_HEADER,
    'User-Agent': urllib3.util.SKIP_HEADER,
    'Accept-Encoding': urllib3.util.SKIP_HEADER,
    'Accept': None,
    'Connection': None
})
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.1.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.6"
  },
  "platform": {
    "release": "21.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.0"
  },
  "system_ssl": {
    "version": "1010111f"
  },
  "urllib3": {
    "version": "1.26.9"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

Closing this since this is already known and there's even #5693