Raibek/requests-github-issues huggingface dataset:
Import error with werkzeug

When I run `make ci` on the main branch, it throws this import error with werkzeug

```
ImportError: cannot import name 'parse_authorization_header' from 'werkzeug.http' (/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/werkzeug/http.py)
make: *** [Makefile:8: ci] Error 1
```


## Expected Result

To run make ci successfully on the main branch. 

## Actual Result

Import error with latest version of werkzeug. I tried downgrading to a 2.3.X version of werkzeug and it ran without any issue. 

## Reproduction Steps

``make ci``


This is being tracked in psf/httpbin
CA-certs bundles and requests.certs.where - Improved documentation

I initially thought `requests.certs.where` (as well as `certifi.where`) returned the location of the cacert file _actually_ being used, and I could use it to verify that the correct file was loaded. I am not the only one to make this mistake: https://stackoverflow.com/a/42982144/735070.
![Screenshot from 2023-11-01 05-05-56](https://github.com/psf/requests/assets/4488298/1ce95ea1-958a-445a-9792-e2aa9b53bbd3)

I have realized though, that it simply return the path of the bundled cacert file that comes included with the library.

### Demonstration of this behavior:

    #test.py
    import requests, certifi,os
    print('os.environ.get("REQUESTS_CA_BUNDLE") -> ',repr(os.environ.get("REQUESTS_CA_BUNDLE")))
    print('os.environ.get("CURL_CA_BUNDLE")     -> ',repr(os.environ.get("CURL_CA_BUNDLE")))
    print('requests.cert.where())               -> ',repr(requests.certs.where()))
    print('certifi.where())                     -> ',repr(certifi.where()))

Running test.py in shell with REQUESTS_CA_BUNDLE set:

    $ REQUESTS_CA_BUNDLE=/home/jdoe/catest/my_cacert.pem python test.py
    os.environ.get("REQUESTS_CA_BUNDLE") ->  '/home/jdoe/catest/my_cacert.pem'
    os.environ.get("CURL_CA_BUNDLE")     ->  None
    requests.cert.where())               ->  '/home/jdoe/.local/lib/python3.10/site-packages/certifi/cacert.pem'
    certifi.where())                     ->  '/home/jdoe/.local/lib/python3.10/site-packages/certifi/cacert.pem'

### Improve documentation - requests.certs.where-function

Currently `requests.certs.where.__doc__` returns null. How about

_returns the path of the CA-certs bundle that is included with the requests package, ie not necessarily the bundle actually being used._

### Improve documentation - How is CA-bundle chosen

Maybe the simplest way of making this clear is to add a section with a list or decision tree that shows which file (if any) will be used as CA-certs bundle.


### Add some way of getting the bundle _actually_ being used

In my case a static function, `requests.cacerts_loaded()`. I realize this may return a file different from the one being used, if the user specifies `verify` (and maybe its possible to change in other ways too). 

### Demonstration run on

- certifi version: 2023.05.07
- requests version: 2.25.1
- Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux


I was so confused when I set REQUEST_CA_BUNDLE and the where() functions didn't change.
HTTP/2 support - For real (even HTTP/3 is a reality right now!)

Hi!,

[HTTP/2](https://en.wikipedia.org/wiki/HTTP/2) was introduced 8 years ago!.

Is there any intention to support it, its characteristics, in the **short** term?.

If not: is it for lack of time, interest, people?. Perhaps it would be interesting, in this case, to transfer this library to an organization that would have the necessary resources to update it; or, at least, to provide it with the required resources: has it been tried, is it of interest?. Anchoring in the past, I think, has never been a good idea.

If the answer is "**no**", or **"never**", I think the best option would be to abandon, and give up, this project so that others can make it even **bigger**.

Thank you (really) from someone who appreciates your work.


Going to mark this as a "answered" by https://github.com/urllib3/urllib3/pull/3030, which has an extremely thorough description of what it would take for HTTP/2 support in urllib3, the underlying library of Requests. There is a lack of people and time, so if you have access to those things and want to solve the problem it would be much appreciated.

Thank you for your response. I still think that investing time in maintaining an already obsolete library is a waste of time.

But it is your time. I assure you it won't be mine :).

Thanks for speeding up my decision. And good luck, always.

Regards,

> maintaining an already obsolete library is a waste of time.

Given it's the 4th most downloaded Python package globally, it must be serving _some_ purpose :) We wish you well.

Hi!,

Of course it is, of course this library is important, of course it is (or was?) necessary ... but you are eight years late. At least you are. And that, in our young ecosystem, is too long. And living on past achievements has never ensured either the present or the future.

And I am sorry, my apologies, if I sounded rude, but I insist on my idea: I sincerely believe that you are wasting your time. And, by extension, ours.

Of course I am not able to match what you have achieved, I do not have that knowledge, but I have ideas, some absurd: that this library, today, does not even allow me to carry them out.

Therefore, I repeat: Thank you, ample. For everything. And that perhaps it would be interesting to point out in your page that you have no intention to support neither the present nor the future, at least in the short term.

Since we are sincere, let's be completely sincere. Don't you agree? :).

Regards,
Optimized 2 functions + added Doc Strings + fixed formatting issues(non-technical just prettyness issues)

Optimized the following functions
- `help._implementation()` - Made it faster, doesn't really make an effect but nice to have. `Confirmed to still work on my system`
- `utils.urldefragauth(url)` -  removes the fragment and authentication parts from a URL in a more concise and efficient manner by directly modifying the `netloc` variable and avoiding unnecessary assignments. `Confimed to still work on my system`

Added or modified doc strings:
- `exceptions.JSONDecodeError()` - Capitalized json -> JSON so it matches the rest of the code in that file.
- `status_codes.py` - Modified some of the formatting for the code, so they all match. (No random lines of code that use multiline args )
- Added a few doc strings to functions in `utils.py`
- Added a doc string to `test_lowlevel.test_json_decode_compatibility_for_alt_utf_encodings()

Thanks for the contribution. Unfortunately we don't accept cleaned charges and changing the names of exception classes is a backwards incompatible change we will not accept. Finally the formatting changes you made conflict with black which we use to format the entire codebase
every chardet package maps to requests.packages.chardet.* package respectively

Added a fix for the issue mentioned here - https://github.com/psf/requests/issues/6223

Checked in the code, only one package of chardet was being mapped to requests.packages.chardet despite the for loop. 

Have added the fix so each chardet package maps to it's corresponding requests.packages.chardet.* respectively.

Your PR as is will _still_ have the same issue you reported and does not fix it. I've added a suggestion

Same checks are failing on the main branch - https://github.com/psf/requests/issues/6566

I think the issue is with werkzeug latest version which gets downloaded due to dependency with `httpbin`

Is there anything more required in this PR to be merged? Or is it on standby until the httpbin issue gets resolved @sigmavirus24 

closing & reopening to trigger new build
Include NOTICE file in distribution

Since the Apache license requires inclusion of the copyright notice, it would be really helpful to have the NOTICE file included in the distribution along with the LICENSE file so it can be collected from there by scripts instead of having to retrieve it separately from the source repository.

Hi @stevecj, the NOTICE file is currently available in the source distribution on PyPI and the published release assets on GitHub:
* [PyPI .tar.gz](https://files.pythonhosted.org/packages/9d/be/10918a2eac4ae9f02f6cfe6414b7a155ccd8f7f9d4380d62fd5b955065c3/requests-2.31.0.tar.gz)
* [GitHub release .tar.gz](https://github.com/psf/requests/archive/refs/tags/v2.31.0.tar.gz)


Both of these should be straight forward to automate, could you clarify where you're expecting to see it?

Resolving since we haven't heard back.
URL parameter is added when the value is an empty string but not when it is an empty list

<!-- Summary. -->
Both an empty string `''` and an empty list `[]` are not of NoneType and are considered a falsy value and therefore should behave the same when being added as a URL parameter. But in requests when passing an empty string `''` as a parameter value the final URL contains the parameter but not when an empty list `[]` is passed as the value.

## Expected Result
When passing an empty list as parameter value, the final URL **should contain** the parameter.

<!-- What you expected. -->

## Actual Result
When passing an empty list as parameter value, the final URL **does not contain** the parameter.

<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

base_url = 'https://jsonplaceholder.typicode.com/todos'

params = {'_limit': 5, 'id__in': ''}
response = requests.get(base_url, params)
print(response.request.url)
# Output: https://jsonplaceholder.typicode.com/todos?_limit=5&id__in=

params = {'_limit': 5, 'id__in': []}
response = requests.get(base_url, params)
print(response.request.url)
# Output: https://jsonplaceholder.typicode.com/todos?_limit=5
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.1.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.11"
  },
  "platform": {
    "release": "22.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.2"
  },
  "system_ssl": {
    "version": "1010115f"
  },
  "urllib3": {
    "version": "1.26.15"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Yes as observed this is the case when using any other empty datatype, except a **str()**, as it gives an empty string.

From Documentation:
*\" Note that any dictionary key whose value is None will not be added to the URL’s query string. \"* 

But as known, an empty list is not of None type, so i guess since it does not know what to do with empty datatypes it just treats then as None.

Proposed a fix here - https://github.com/psf/requests/pull/6561
Certificates are printed to stdout when running requests.get

<!-- Summary. -->

## Expected Result

<!-- What you expected. -->

Any warnings not to be printed to stdout, (stderr instead)

## Actual Result

Certificate line is printed to stdout

<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

a = requests.get("https://github.com")
b = requests.get("https://github.com", verify=False)
```

Returns

```
/home/cwl_ica_user/.icav2-cli-plugins/pyenv/lib/pypy3.9/site-packages/certifi/cacert.pem None
/etc/ssl/certs/ca-certificates.crt /etc/ssl/certs
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "PyPy",
    "version": "7.3.13"
  },
  "platform": {
    "release": "6.2.0-35-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30100030"
  },
  "urllib3": {
    "version": "1.26.18"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


I can't reproduce this. Something is clearly missing from your reproduction steps

Hmm, seems to be an issue with Python 3.9.18 when packaged by conda-forge (not an issue when using 3.9.17).  My clipboard from terminal also appears to be broken so I cannot currently share my outputs. 

I don't believe this is a requests library issue, but something underneath.  

Closing for now.  



@sigmavirus24 I have found the offending code - nothing to do with the requests library - but from https://github.com/pypy/pypy/issues/4009. Credit to @elliotwutingfeng for the link
docs(socks): same block as other sections

strange formatting:
https://docs.python-requests.org/en/latest/user/advanced/#socks

The current formatting looks correct to me. What looks wrong to you? 

i have some kind of strange underscore

![image](https://github.com/psf/requests/assets/28842337/af8403bd-2ff9-4919-b0d8-49da2b0644de)

In a private firefox without dark theme
![image](https://github.com/psf/requests/assets/28842337/103064f2-39f8-4baa-b3e5-9a942219bb5b)
![image](https://github.com/psf/requests/assets/28842337/565692bd-9290-4a3e-b2c8-321f6cd038f0)

***

https://github.com/psf/requests/blob/main/docs/user/advanced.rst

In the rst preview you can see that this bash code block is not the same as the other bash code bloc


I cannot replicate that on Firefox on macOS or Linux

I cannot replicate that on Firefox on macOS or Linux.

I would be willing to entertain a PR, however that adds sphinx-prompt to our docs dependencies and switches this to using `.. prompt::` instead because that's more correct 

I am actually able to replicate this on macOS in Firefox 115.3.1esr (64-bit). It appears whitespace in these blocks are being replaced with `span` tags with the `w` CSS class from pygments. This adds a faint white (#f8f8f8) underline to each whitespace.

<img width="327" alt="Screenshot 2023-10-18 at 12 26 22 PM" src="https://github.com/psf/requests/assets/5271761/249056bc-c256-4784-8d51-0128b9eca4e2">

I don't however immediately see why this is being added specifically for the bash codeblock. I'm fine moving to the simplified syntax (`::`) (or `.. prompt::` as Ian noted) in this case since the bash formatting doesn't materially change the output, but it would be interesting to understand why it's happening.

To find out why it's happening I suspect we'll need a pygments bug to investigate.

The span doesn't change what you copy out of the text though
Missing assert in test_requests.py test_header_validation

[requests/tests/test\_requests.py at main · psf/requests](https://github.com/psf/requests/blob/main/tests/test_requests.py#L1705)

```
    def test_header_validation(self, httpbin):
        """Ensure prepare_headers regex isn't flagging valid header contents."""
        valid_headers = {
            "foo": "bar baz qux",
            "bar": b"fbbq",
            "baz": "",
            "qux": "1",
        }
        r = requests.get(httpbin("get"), headers=valid_headers)
        for key in valid_headers.keys():
            valid_headers[key] == r.request.headers[key]
```

`valid_headers[key] == r.request.headers[key]` should probably be `assert valid_headers[key] == r.request.headers[key]`.

I would like to contribute to solve this issue. @sigmavirus24  can I take this up?

Resolved in #6552
@ethanroy No additional consideration beyond my suggestion of using a session-level auth handler, in the comment directly above yours.

> @ethanroy No additional consideration beyond my suggestion of using a session-level auth handler, in the comment directly above yours.

_Originally posted by @Lukasa in https://github.com/psf/requests/issues/2949#issuecomment-234553490_

I'm not sure if this used to work, or if something broke in the interim, but attaching an auth handler to a session does not cause the handler to get invoked for each redirect after the first request. Oddly, it seems like that's also the case for adapters, so it doesn't seem that there's a way to correctly set auth headers after a redirect.

Responding to a conversation by opening an issue is not a valid bug report. If you need assistance with your auth handler, please use StackOverflow
Conflicting requirements for urllib3

In origin/main ee93fac6b2f7 I find conflicting required urllib3 version information:

```
setup.cfg:10:    urllib3>=1.21.1,<1.27
setup.py:64:    "urllib3>=1.21.1,<3",
```

Shouldn't both files have the same restrictions?

Best regards

Heinrich

They should, good catch xypron. We'll work on getting those aligned.
EOF occurred in violation of protocol starting Python3.10 on large requests

# Bug report

### Bug description:

We have identified a regression starting with Python 3.10 when making a large request over TLS.

```python
import requests
requests.post('https://google.com', data=b'A'*1000000)
```

```python
urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:2426)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2426)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:2426)')))
```

The same request works fine on Python 3.9.

### CPython versions tested on:

3.10, 3.11, 3.12

### Operating systems tested on:

Linux

Closing to track on https://github.com/urllib3/urllib3/issues/3148 which you opened simultaneously.
Cookies with `Secure` are not sent for `localhost` via unencrypted `http`

See the title, `Secure` cookies are not sent in requests to `localhost` if those requests are not encrypted. There should probably be an exception for `localhost` (as there is in browsers, as well) to aid in developing servers.

## Expected Result

The cookies should be sent with the request.

## Actual Result

They aren't.

## Reproduction Steps

```python
import requests

session = requests.Session()
session.post("http://localhost:8080/login")
session.get("http://localhost:8080/only-with-login").raise_for_status()
```

and

```python
from flask import Flask, request, make_response

app = Flask(__name__)


@app.route("/login")
def login():
    resp = make_response("Logged in")
    resp.set_cookie("token", "token", secure=True)
    return resp


@app.route("/only-with-login")
def only_with_login():
    if request.cookies.get("token") != "token":
        return "Not logged in", 403
    return "Logged in"
```

Running the flask app with `flask --app server run --port 8080` and then running the requests with `python3 do_requests.py` throws an exception in `do_requests`, while in a browser the page does show "Logged in" after visiting `http://localhost:8080/login`.

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "5.2.0"
  },
  "charset_normalizer": {
    "version": "3.2.0"
  },
  "cryptography": {
    "version": "41.0.3"
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.5"
  },
  "platform": {
    "release": "6.5.4-arch2-1",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "30100020",
    "version": "23.2.0"
  },
  "requests": {
    "version": "2.28.2"
  },
  "system_ssl": {
    "version": "30100030"
  },
  "urllib3": {
    "version": "1.26.15"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


No we'll not be introducing code to handle this. Use something like trustme to generate certs and validate them with requests instead and you'll get the correct behavior 100% of the time. 

This resolution is counterintuitive since it puts Requests at odds with browser behaviour:

* Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1648993
* Chromium: https://issues.chromium.org/issues/40120372

[MDN also documents this special-case](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie#secure):

> The `https:` requirements are ignored when the `Secure` attribute is set by localhost (since Chrome 89 and Firefox 75).
Versions at requests.readthedocs.io flyout menu are outdated

The https://requests.readthedocs.io/ webpage flyout menu versions are currently:

- [latest](https://requests.readthedocs.io/en/latest/) [stable](https://requests.readthedocs.io/en/stable/) [v3.0.0](https://requests.readthedocs.io/en/v3.0.0/) [v2.9.1](https://requests.readthedocs.io/en/v2.9.1/) [v2.8.1](https://requests.readthedocs.io/en/v2.8.1/) [v2.7.0](https://requests.readthedocs.io/en/v2.7.0/) [v2.6.2](https://requests.readthedocs.io/en/v2.6.2/) [v2.5.3](https://requests.readthedocs.io/en/v2.5.3/) [v2.4.3](https://requests.readthedocs.io/en/v2.4.3/) [v2.3.0](https://requests.readthedocs.io/en/v2.3.0/) [v2.2.1](https://requests.readthedocs.io/en/v2.2.1/) [v2.1.0](https://requests.readthedocs.io/en/v2.1.0/) [v2.0.0](https://requests.readthedocs.io/en/v2.0.0/) [v1.2.3](https://requests.readthedocs.io/en/v1.2.3/) [v1.1.0](https://requests.readthedocs.io/en/v1.1.0/) [v1.0.4](https://requests.readthedocs.io/en/v1.0.4/)

Screenshot:

![image](https://github.com/psf/requests/assets/17479683/0ec2240b-6b41-42a0-8448-6391321fa2b2)

There are few problems with the versions:
- First, some of the selected versions in the flyout menu are really old. The 2.9.1 [is from December 2015](https://github.com/psf/requests/releases/tag/v2.9.1). I would expect to see 2.31.0, 2.30.0, 2.29.0 and perhaps 2.28.2 on the list.
- The 3.0.0 links to documentation for v.2.10.0.


I would like to work on this issue. 
I am not much clear about the issue description, so could please elaborate on it.

The flyout menu is the black box in the bottom right corner:
![image](https://github.com/psf/requests/assets/17479683/31296583-000b-4112-bec2-bdbd955148e5)

This has old versions listed. It is missing recent versions like 2.31.0, 2.30.0, 2.29.0. In addition, the link for 3.0.0 in the flyout menu -- this:

![image](https://github.com/psf/requests/assets/17479683/6a9d76af-48ca-404d-b260-b303c65059a2)

..points to documentation of v.2.10.0:

![image](https://github.com/psf/requests/assets/17479683/949a6ba1-884e-4e87-96c2-9643c3369b68)


So issue says that, 
- Add the remaining versions like 2.31.0, 2.30.0, 2.29.0. 
- Correct the link for 3.0.0

Is it correct or i left with anything else ?

I think what summarizes the issue well! :)

Hey while going through the code i am not able to locate the flyout menu.
Could you please locate it.

I'm not familiar with the requests repository, but it could be also that it is configured somewhere in the readthedocs settings. I hope someone else here knows what should be edited.

These settings are managed form the administration panel in RTD. The older versions have been removed and we'll only be providing flyout links for stable (latest release) and latest (main branch). Thanks for the issue @fohrloop.
Requests are blocked by cloudflare inside docker

Hello to all, i need some help here

I have a code where i need to post a URL to get the autorization of the cookies and then i get this cookies and do another request in the website

It's a goverment thing, and i really need some help

When i use this script on my computer i can use it normally but when i upload in the container on docker (in my vps) i get "403" and i cannot continue my code

Someone can help me?

Anyone had an similar issue?

Everything is correct, the certificate path, the user and password, everything... But i keep being blocked by cloudflare

![image](https://github.com/psf/requests/assets/51841352/55cbeb4d-5cf7-4dbb-8a55-b49990acf398)

The HTML message of cloudflare and the 403 on docker:

![image](https://github.com/psf/requests/assets/51841352/955d84ef-ec2b-4b8b-81e0-6fadec01da5a)

Thank you all!



Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Getting incorrect response.

I was downloading an image using following python code:

```python
import requests
from PIL import Image
from io import BytesIO

link = "https://coloredmanga.com/wp-content/uploads/WP-manga/data/manga_6173003824249/85d69f8667f073e69cd03154b09f1412/149.jpg"

res = requests.get(link)
img = Image.open(BytesIO(res.content))    # PIL is unable to parse the image because of wrong response body.
```

## Expected Result

This code should download the image successfully, as the image can easily be downloaded using browser and postman.

## Actual Result

I tried running the same code two times. First time it was successful. But when I again ran it, it failed. 
I printed first 100 bytes of the response in both cases.

**Case 1: Successful download**
```python
>>> res.content[:100]
b'\xff\xd8\xff\xe1)\xdcExif\x00\x00MM\x00*\x00\x00\x00\x08\x00\x0c\x01\x00\x00\x03\x00\x00\x00\x01\x05\xf0\x00\x00\x01\x01\x00\x03\x00\x00\x00\x01\t`\x00\x00\x01\x02\x00\x03\x00\x00\x00\x03\x00\x00\x00\x9e\x01\x06\x00\x03\x00\x00\x00\x01\x00\x02\x00\x00\x01\x12\x00\x03\x00\x00\x00\x01\x00\x01\x00\x00\x01\x15\x00\x03\x00\x00\x00\x01\x00\x03\x00\x00\x01\x1a\x00\x05\x00\x00'
```

**Case 2: Failure**
```python
>>> res.content[:100]
b'\xa5\xff\x7f1\n\xe9iq\xc6\x83\x88jR\x0f\x80:R\x17\xfe\xfc\xf9\xf7G\xc80\xb7\xff\xfc\xba\xff_\xfb\x0c\xef\xf5<@I`\x03\x9c\xe0\n\xb5\x0f\xb0JI\xf2\x0c\xe0\xdc\xa6\xa8\x18\x84T^\x04$"\xae"\xd1\x02\x12-\xd5Ry\x15\x89h\x91\xa8\xb8R)\xd5\xd2\x12\xf5S\x87\xff\x1b\x9e\xe6\xf6\x8f\x0c\x03\x8c\x9a\xb3\x89\x15,\x98\x0c'
```

## Reproduction Steps

Use the snippet above

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "5.2.0"
  },
  "charset_normalizer": {
    "version": "3.2.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.18"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "300000a0"
  },
  "urllib3": {
    "version": "2.0.4"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
How can I translate a page to English when scraping with requests python

I'm trying to scrape the site [roca](https://roca.com.br/imovel/locacao/casas/sao-carlos/jardim-sao-carlos/66857?hl=en). This page is in Portuguese language, but I want to have translated into English so that I can capture English content when scraping with requests.

I have tried doing the below option of using Accepted-Language as I saw on [stackoverflow] (https://stackoverflow.com/questions/43572267/i-get-translated-text-when-i-do-a-get-request-in-python-how-to-get-english-co) 

```
url = r'https://roca.com.br/imovel/locacao/casas/sao-carlos/jardim-sao-carlos/66857'
headers = {"Accept-Language": "en-US,en;q=0.5"}
response = requests.get(url, headers=headers).text # get content of URL (webpage), save to variable response
soup = BeautifulSoup(response,'html.parser')    # create a new object using html parse
name = soup.select('div.Divisor__StyledDiv-sc-gc9tdc-1 h1')
print(f'House Name: {name[0].text}')
```
But when I run the above code, it still prints in Portuguese which I don't want. Please help

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Automatically Close Requests Session After Specified Timeout, Even with Retries

**Desc**
Currently, when using the requests library in Python to create a session with a timeout and there are retries implemented within the request, the session remains open even after the specified timeout has elapsed. This behavior can lead to resource leakage and unexpected behavior when managing sessions with timeouts.

**Proposal:**
Add functionality to automatically close a requests session after a specified timeout, even if there are retries in the request itself. This enhancement would improve the library's usability and ensure that sessions are properly managed, even in cases involving retries.

**Use Case:**
As a developer, I often need to make HTTP requests with retries in case of transient errors. However, I also need to ensure that the session is closed after a certain timeout to prevent resource leaks. This feature would enable me to have both retry functionality and automatic session closure without having to implement custom solutions.

**Suggested Implementation:**
One way to implement this feature is to introduce an optional session_timeout parameter when creating a session or making requests. This parameter would specify the maximum duration the session can remain open, and if the timeout is reached, the session would be closed automatically, regardless of ongoing retries.



In the future, please search **closed and** open issues before creating new ones that are duplicates.
Why encoding hard coding to "ISO-8859-1"

<!-- Summary. -->

## Expected Result
UTF-8 is better

<!-- What you expected. -->

## Actual Result
ISO-8859-1

<!-- What happened instead. -->

## Reproduction Steps

I wanna download subtitle content, here is the code
```python
import requests
requests.get('https://events-delivery.apple.com/1008yunfyueoofvvgxpjmmvcblgnjsel/vod_main_bTVAkbqTcJuRovcmMRYungakLuiQyTWT/cc/zh/zh_0.webvtt',
                    headers={
                        'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8',
                        'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36',}.text
```
I found response.text is garble
through read code, found this snippet in "utils.py" on "def get_encoding_from_headers(headers)"
```python
    if "text" in content_type:
        return "ISO-8859-1"
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.1.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.4"
  },
  "platform": {
    "release": "21.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30100020"
  },
  "urllib3": {
    "version": "2.0.2"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


You need to look at the response headers. The response is not providing an explicit `charset` otherwise we'd exit with this condition: https://github.com/psf/requests/blob/881281250f74549f560408e5546d95a8cd73ce28/src/requests/utils.py#L548

UTF-8 has increasingly become the "default" for newer sites, but Requests has to handle all content old and new. The easiest thing for any newer site to do is to properly specify it's `charset` because that's been supported since HTTP/1.0 if I remember correctly. 

Since your site is not specifying an encoding explicitly, Requests uses the next most common encoding if it can predict that the content is either `text/plain` or `text/html`. After that it'll use an external library to guess. You can always override the encoding yourself if you know what you care to use.

In the future, please search **closed and** open issues before creating new ones that are duplicates. There are **many** past issues discussing exactly tihs.
open core legacy patcher

hello, i am facing a error while downloading the ventura file from open core legacy patcher, it downloads 2GB from 11.9 gb  and a error comes     Download failed: 
HTTPSConnectionPool(host='swcdn.apple.com', port=443): Read timed out

please help me with this 

thankyou

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Getting 500 with Requests.get whereas Postman gives 400

Summary: For a Get API, The same url and token gives 400 status code in postman and when tried with pytest-bdd with requests library, I get 500

## Expected Result

 For a invalid param, should be giving status code 400 in the response

## Actual Result

We get 500 status code though the request is exactly the same used in both places, please see below images

## Reproduction Steps





<img width="1273" alt="image"
<img width="694" alt="Screenshot 2023-09-11 at 6 28 37 PM" src="https://github.com/psf/requests/assets/42774628/5f66eacc-5fe6-4af6-b489-84f32081752f">
 src="https://github.com/psf/requests/assets/42774628/86da23bb-5943-4bbb-9a11-84361155474b">

## System Information
pytest --version:  6.2.5
Python3 --version: 3.9.6
MacOS

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

The server is returning a 500, not requests. Something is wrong in your setup

<img width="1273" alt="image" src="https://github.com/psf/requests/assets/42774628/a016bd9d-51ea-4afa-9bc8-c125f8930faa">


But my request is same in both places, please refer this image.

Your images do not prove what you want them to nor are they searchable or accessible to those who need to use assistive devices like a screen reader. The requests are not identical and there are any number of prior issues here and answers on StackOverflow that can help you diagnose what the difference is.
Requests honor env no_proxy but will not respect no_proxy when passed as proxies dictionary

When working in a environment that uses proxies, most servers and clients will have proxy information set in their environment:

```
HTTP_PROXY=http://some-proxy.com:1234
HTTPS_PROXY=http://some-proxy.com:1234
NO_PROXY=example.com
```
Requests will respect these environment variables by default, meaning the following will work just fine.

```python
import requests

requests.get("https://example.com")
```

The snippet above will not use proxy, as specified with the `NO_PROXY` variable.

In requests we can also pass a proxy dict to requests, like this:

``` python
proxies = {
    "http": "http://some-proxy.com:1234",
    "https": "http://some-proxy.com:1234",
    "no": "example.com"  # Don't use the proxy for requests to example.com
}

requests.get("https://example.com", proxies=proxies)
```

But this wont work, request does not honor the `no` directive when passing proxies directly.


## Expected Result

I expect requests to bypass proxy when issuing a get towards example.com

## Actual Result

Requests tries to contact the proxy

## Reproduction Steps

See example above

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.2.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.3"
  },
  "platform": {
    "release": "6.2.0-31-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30000020"
  },
  "urllib3": {
    "version": "2.0.4"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

```


Duplicate of #5000
Response Object Code cleanup suggestions

Any thoughts here? 

I felt like the exception driven logic of `ok()` wasn't ideal, so I went in an added a new method: `is_raisable_status()`, which basically checks if the response code is 400 or more (could be improved?). 

Also put `reason = self.decode_reason()` reason decoding into its own method.

If theres interest in getting this change pushed thru, I'll put in the effort to get unit tests updated and written, and update some of the method documentation 😂  

Hi @xxdunedainxx,

I don't think this is something we'd accept as it breaks the established behavior of the API which we wouldn't do in the current minor version.

We'll pass on this for now, but thanks for the contribution.
upload a photo and copy link

import requests
url = 'https://scanqr.org/'
responde2 = requests.get(url)
files = {'file': open('ph.jpg', 'rb')}
response = requests.post(url, files=files)
print(response.text)
print(responde2.text)


This is a site for scanning the QR code, that is, you upload the picture of the QR code, then it shows the link related to the QR code.
I uploaded a photo using the requests library and got the output before and after, but there is no difference between the outputs
If I expected, give me the link in the output after uploading


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Failed to validate certificate

I try to access a self signed web server with python requests. While everything works (firefox, openssl), my python program still fail to validate the certificate.

For testing the certificate works I use:
openssl s_client -CAfile pathtomyca.crt -servername fqdn -connect fqdn:443
openssl accept the certificate.
If I omit the CAfile option, openssl normaly said its self-signed

Now the python script is:
#!/usr/bin/env python3
import requests
import OpenSSL
import ssl
url = 'https://fqdn'
verify = 'pathtomyca.crt'
try:
    r = requests.get(url, verify=verify)
except requests.exceptions.RequestException as e:
    cert = ssl.get_server_certificate(('fqdn', 443))
    print(cert)
    cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, cert)
    print(cert.get_issuer())
    print(cert.get_subject().get_components())
    print(e)

The exeption is
HTTPSConnectionPool(host='fqdn', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'fqdn'. (_ssl.c:1010)")))

But all cert.get_xxx show me the right fqdn

So there is only requests which dont like my certificate, but I do not find why.
Any help appreciated.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

It is why I selected "Request for Help" in issue choice. So it dont see why you closed it...

Or just remove "request for help" in issue choice of you dont want them...

https://github.com/psf/requests/blob/main/.github/ISSUE_TEMPLATE/Custom.md tells you to go to stack overflow when you select it.
Proper TypeAnnotation for proxies

## Feature Request: Proper Type Annotations for Proxies in Requests Library

### Proposal

I propose enhancing the type annotations for proxy settings in the `requests` library to specify a dictionary of string keys and string values. This would provide clearer type expectations to users and improve the development experience when configuring proxies for HTTP requests. As it is not much needed, but it would be great if proper Annotations are present.


Currently, the type annotation for proxies in the `requests` class is defined as: 

```python
proxies: Any | None
```

### Desired State

I suggest updating the type annotation for proxies to be more specific:

```python
proxies: Optional[Dict[str, str]]
```


There are no type annotations in this library. The error likely lies in typeshed if there's anything
I'm trying to Install macOS Monterey on Older Unsupported Mac( macbook pro 13" late 2012) using Open-core patcher but keep getting this "Error Download failed:  HTTPSConnectionPool(host='swcdn.apple.com', port=443): Read timed out." ca somebody help me please.

Summary.

## Expected Result

What you expected.

## Actual Result

What happened instead.

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```
<paste here>
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).
Unable to import requests - No module named `http.client`

I am unable to import the requests library. I get a ModuleNotFoundError with `No module named 'http.client'

## Expected Result

Running `import requests` should not throw an error.

## Actual Result

Running `import requests` throws an error:
```
Traceback (most recent call last):
  File "/home/mahmoud/requests_test.py", line 1, in <module>
    import requests
  File "/home/mahmoud/.local/lib/python3.10/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/home/mahmoud/.local/lib/python3.10/site-packages/urllib3/__init__.py", line 11, in <module>
    from . import exceptions
  File "/home/mahmoud/.local/lib/python3.10/site-packages/urllib3/exceptions.py", line 3, in <module>
    from .packages.six.moves.http_client import IncompleteRead as httplib_IncompleteRead
  File "/home/mahmoud/.local/lib/python3.10/site-packages/urllib3/packages/six.py", line 199, in load_module
    mod = mod._resolve()
  File "/home/mahmoud/.local/lib/python3.10/site-packages/urllib3/packages/six.py", line 113, in _resolve
    return _import_module(self.mod)
  File "/home/mahmoud/.local/lib/python3.10/site-packages/urllib3/packages/six.py", line 82, in _import_module
    __import__(name)
ModuleNotFoundError: No module named 'http.client'
```

## Reproduction Steps

```python
import requests
```
Obviously, not everyone is running into this issue. I'm not sure what's special about my system that would cause this.  I haven't done anything special or out of the ordinary with my server setup or python installation. Everything was working fine up until a few days ago when this issue just appeared unexpectedly for no apparent reason.

## System Information

Ubuntu Server 22.04, python3 version 3.10.12

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com/) for general usage questions instead and only report defects here.

```
ModuleNotFoundError: No module named 'http.client'
```
From this specific error, it seems you may have something shadowing the `http` namespace. You may have a file called `http.py` in your project. The [`http`](https://docs.python.org/3/library/http.html) and [`http.client`](https://docs.python.org/3/library/http.client.html#module-http.client) modules are available in all version of Python 3 we support in Requests.

I do not have anything shadowing the http namespace. I have no files named http.py or anything like that.

I can run `import http` then `dir(http)` yields `['HTTPStatus', 'IntEnum', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']`

However, running `import http.client` yields the `ModuleNotFoundError`
TypeError: __init__() got an unexpected keyword argument 'strict'

Hi all, getting this error when trying to run all scripts. Any ideas on how to fix would be greatly appreciated.

many thanks


TypeError: __init__() got an unexpected keyword argument 'strict'



Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com/) for general usage questions instead and only report defects here.

For your Stackoverflow issue, you'll want to include the full stacktrace because there isn't enough information here. You may want to look at your version of urllib3 and any adapters your code may have interacting with it.