Raibek/requests-github-issues huggingface dataset:
I get confused using python documentation HELP

for an example:
https://docs.python-requests.org/en/latest/api/#requests.cookies.RequestsCookieJar.set
`set(name, value, **kwargs)`
`Dict-like set() that also supports optional domain and path args in order to resolve naming collisions from using one cookie jar over multiple domains.`

But where I found `**kwargs` information's like what other arguments its take :-(
I mean about these arguments domain, path, expires, max_age, secure, httponly

All other functions are same as like this, I got total confused what to pass as parameters.

I am sorry for NOOB like question. but I am really straggle with it a lot.

in php.net they are describe all parameters properly. Where I found all parameters are hidden with `**kwargs`.

Thank you

Hi @abhijit420kb,

This is specified at the top of the class:

```
Compatibility class; is a cookielib.CookieJar, but exposes a dict interface.
```

You can find the parameters supported by default in the constructor for Python's [Cookie](https://github.com/python/cpython/blob/3.11/Lib/http/cookiejar.py#L743-L801). This may vary depending on Python version/underlying CookieJar implementation which is why this information is defined to be optional `**kwargs`.
Allow str/bytes subclasses to be used as header parts

Closes https://github.com/psf/requests/issues/6159 Adapted from https://github.com/nateprewitt/requests/commit/66fcc9c88cf5a53e85322305f41006032744cbe3 but also supports mixed type tuples.

If you can rebase this onto the linting change, I think this is set.

@nateprewitt Done, ready for review!
TCP Keep Alive doesn't seem to be enabled by default

In the documentation, it stated the TCP Keep Alive are enabled by default when using `requests`, but looking at the code it doesn't seem to be the case.

## Expected Result

TCP Keep Alive be enable by default.

## Actual Result

I could not find anywhere in the code where the TCP Keep Alive are enabled. Maybe I missed something ? In the urllib3 library it seems to me that TCP Keep Alive are not enabled by default, see https://github.com/urllib3/urllib3/blob/5a24dc55d86e4c5ca7fe0bd496da768caf617591/src/urllib3/connection.py#L109.

It would also be good to have the possibility to pass socket arguments to the `HTTPAdapter`, for my use case where I needed TCP keep alive, I had to overwrite the `HTTPAdapter` like this in order to pass socket options:

```python
class HTTPKeepAliveAdapter(HTTPAdapter):

    def __init__(self, max_retries):
        self._pool_kwargs = {
            'socket_options':
                HTTPConnection.default_socket_options + [
                    (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1),  # enable keep alive
                    (socket.SOL_TCP, socket.TCP_KEEPIDLE, 120),  # idle time 120s
                    (socket.SOL_TCP, socket.TCP_KEEPINTVL, 120)
                ]  # interval time 120s
        }
        super().__init__(pool_connections=40, pool_maxsize=60, max_retries=max_retries)
        self.init_poolmanager(
            self._pool_connections, self._pool_maxsize, block=self._pool_block, **self._pool_kwargs
        )

    def __setstate__(self, state):
        super().__init__(state)

        self.init_poolmanager(
            self._pool_connections, self._pool_maxsize, block=self._pool_block, **self._pool_kwargs
        )
```

But I may have missed something as well ? 





If you'd like to specify custom socket options more conveniently, https://toolbelt.readthedocs.io/en/latest/adapters.html#socketoptionsadapter

I believe TCP Keep-Alive caused more issues than benefits but the references in the docs were never removed. There's also: https://toolbelt.readthedocs.io/en/latest/adapters.html#requests_toolbelt.adapters.socket_options.TCPKeepAliveAdapter to make this easier for folks who know they need it
Request don't return value while curl does

Hi, I have an issue while making a post request.
I'm doing the following curl:
curl -X POST MY_ADDRESS -d SOME_JSON --connect-timeout 2000

After 15 minutes I get the response from the server but when I try to do so in python, nothing returns (request was timed out).


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
@albayraktaroglu Same story. Upgrading `pyopenssl` does not works for me. 👎

              @albayraktaroglu Same story. Upgrading `pyopenssl` does not works for me. 👎

_Originally posted by @maldil in https://github.com/psf/requests/issues/4246#issuecomment-695162681_
            

I fixed it by inserting "verify=False"
response = session.get(url, params=parameters, verify=False)
Use context manager to open files

In the documentation for `POST a Multipart-Encoded File` the example code opens the file inline but never closes it. This leaves the file open until the script ends. It is better to use a context manager even in example code.

Original:
```
>>> url = 'https://httpbin.org/post'
>>> files = {'file': open('report.xls', 'rb')}

>>> r = requests.post(url, files=files)
>>> r.text
```

Replace with:
```
>>> url = 'https://httpbin.org/post'
>>> with open('report.xls', 'rb') as fd:
...     files = {'file': fd}

...     r = requests.post(url, files=files)
...     r.text
```

I've just got bitten by this example, and was just about to create a PR for this too 😃 

(Maybe
```python
    ...     r = requests.post(url, files=multiple_files)
    ...     r.text
```
could be
```python
    ...     r = requests.post(url, files=multiple_files)
    ... 
    >>> r.text
```
Help defeat requests. Only urllib3 works.

[pcapng.zip](https://github.com/psf/requests/files/10534054/pcapng.zip)
[Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.](https://stackoverflow.com/questions/75147860/post-request-to-send-file-file-fw-not-working)
Good afternoon! I have been unable to make a working POST request to send the firmware for many months now.
Works only through urllib3:
```
import urllib3
http = urllib3.PoolManager()
ip="10.254.255.105"
name_file="boot_v1.5_app_v2.27.8.fw"
uri = "/firmware/update_from_page.htm"

with open(name_file, 'rb') as fp:
    file_data = fp.read()
    send = http.request(
        'POST',
        'http://' + ip + uri,
        fields={'new_firmware': (name_file, file_data),}
    )`
```
Problem line for the requests library:
```
send = requests.post(
                             url='http://'+ip+uri,
                             data={'new_firmware': (name_file, file_data),},
                                       )
```
Tried using it in different ways:
1) using sessions
2) Adding headers, including from the working request urllib3
3) Adding various parameters verify = False, allow_redirects=True\False
4) Changing data= to files= resets the connection
I also did Wireshark dumps, maybe this will help determine the differences in requests.
At first glance, they differ in size (requests are twice as large)

When using the requests library, the process does not stop, the connection is not interrupted. Only after the script is forced to stop, the device restores availability.

Did you try to open the file before sending the POST request ? 
like this:
```

import requests

url = '<your url>'
yourfiles = {'file': open('<name off file>' ,'rb')}
#You can also add a timeout parameter 
send= requests.post(url, files = myfiles,timeout=5)


```
no_proxy env doesn't handle '*' in domain name

It is common to use `*` in no_proxy env var to avoid using proxy for whole range of subdomain.
Current implementation doesn't handle `*`.

## Expected Result

Avoid proxy when url match no_proxy with a `*` in it.

```
>>> import requests
>>> requests.utils.should_bypass_proxies("https://requests.readthedocs.io", "*.readthedocs.io")
True
```
## Actual Result

Use proxy even if no_proxy contains a value that would match domain with a `*`.

```
>>> import requests
>>> requests.utils.should_bypass_proxies("https://requests.readthedocs.io", "*.readthedocs.io")
False
```

## Reproduction Steps

```python
import requests
requests.utils.should_bypass_proxies("https://requests.readthedocs.io", "*.readthedocs.io")
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "4.0.0"
  },
  "charset_normalizer": {
    "version": "2.0.4"
  },
  "cryptography": {
    "version": "3.3.2"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.2"
  },
  "platform": {
    "release": "5.10.0-19-amd64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010109f",
    "version": "20.0.1"
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "101010ef"
  },
  "urllib3": {
    "version": "1.26.5"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}
```


It must be `.readthedocs.io`
session.verify not working when variable 'REQUESTS_CA_BUNDLE', is set

I defined the verify parameter to False in the session object, I have the env 'REQEUSTS_CA_BUNDLE' defined as './ca_bondle.cer', in the request, it should respect the value that was assigned in the session object

## Use the verify parameter that was defined in the session object, which should be mandatory

when I assign a value to the verify parameter of the session object, the api should use this parameter to make the request, and not the value that was defined in the environment variable 'REQUESTS_CA_BOUNDLE'

## It is currently using the 'REQUESTS_CA_BOUNDLE' environment variable to define the value of the verify parameter during the request

Usou a variável de ambiente 'REQUESTS_CA_BOUNDLE', para definir o valor do parâmetro verify durante a requisição

## Define the 'REQUESTS_CA_BUNDLE' environment variable and make two requests, one that must use the certificate, and the second that must not be used, before making the second request, define the verify parameter of the session object as False, this value should be respected , however, the api is using the parameter that was defined in the environment variable, damaging the request!

```python
import requests

conf={'url': required(KAFKA_REGISTRY), 'ssl.ca.location': False}
session = Session()
session.verify = False
response = session.request(method, url="/".join([self.base_url, url]), headers=headers, data=body, params=query)
```

in the sessions.py line 532
```python
verify = merge_setting(verify, self.verify)
```

in the sessions.py line 532
```python
settings = self.merge_environment_settings(
    prep.url, proxies, stream, verify, cert
)
```

in the sessions.py line 701
```python
verify = merge_setting(verify, self.verify)
```

in the sessions.py line 717
```python
if verify is True or verify is None:
    verify = (os.environ.get('REQUESTS_CA_BUNDLE') or
              os.environ.get('CURL_CA_BUNDLE'))
```

in the sessions.py line 724
```python
verify = merge_setting(verify, self.verify)
```

in the sessions.py line 50 the session_setting is not null, need return this value
```python
if session_setting is None:
    return request_setting

if request_setting is None:
    return session_setting

# Bypass if not a dictionary (e.g. verify)
if not (
        isinstance(session_setting, Mapping) and
        isinstance(request_setting, Mapping)
):
    return request_setting
```

## System Information

    $ python -m requests.help

```json
{
  "paste": "here"
}
```

- Requests v2.26.0
- Python 3.7


In the future, please search **closed and** open issues before creating new ones that are duplicates.
Requoting does not unquote `%27`

When requoting a redirection URL, the single quotation mark does not get unquoted: `%27` => `%27`:
```python
In [1]: from requests.utils import requote_uri

In [2]: url = "https://netloc/path?param=param%27%27value"

In [3]: print(url)
Out[3]: 'https://netloc/path?param=param%27%27value'

In [4]: print(requote_uri(url))
Out[4]: 'https://netloc/path?param=param%27%27value'
```

However, the `aiohttp` library uses `yarl.ULR` and this does unquote `%27`: `%27` => `'`
```python
In [5]: from yarl import URL

In [6]: print(url)
Out[6]: 'https://netloc/path?param=param%27%27value'

In [7]: print(str(URL(url)))
Out[7]: "https://netloc/path?param=param''value"
```

## Expected Result

I think both libraries should have the same behavior.

## Actual Result

The `requests` library does not unquote `%27`, whereas the `aiohttp` library (that uses `yarl`) does unquote `%27`.

## Reproduction Steps

```python
from requests.utils import requote_uri
from yarl import URL

url = "https://netloc/path?param=param%27%27value"

print(url)

print(requote_uri(url))

print(URL(url))
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.8"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.7"
  },
  "platform": {
    "release": "5.14.0-1055-oem",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "101010cf"
  },
  "urllib3": {
    "version": "1.26.7"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Arguably the RFCs are in Requests' favor. It's _safer_ to leave those quotes because they should be quoted by RFC 3986 and even the What WG URL spec suggests they should be. Not all servers are going to reject aiohttp's behavior but some will. It's safer to be conservative in what is sent (quote reserved characters, etc.) then it is not to. 

Thank you for your explanation @sigmavirus24.

Indeed, we found this discrepancy between `requests` and `aiohttp` when we got a 403 error from our server at Hugging Face in streaming mode (that uses `aiohttp`) and no error in non-streaming mode (that uses `requests`)

See:
- huggingface/datasets#5459
Using requests to get a file with the contents 1.1.1 on windows fails

On windows using urllib, I get the correct result. On linux using requests, I get the correct result. It seems to be a specific requests + windows issue. 

## Expected Result

I expect to get the string "1.1.1"

## Actual Result

I get some garbage, non-ascii data that I cannot paste here accurately.

## Reproduction Steps

- Create a file on a http server somewhere
- Put in it exactly the characters "1.1.1"
- Try to get that file with requests

```python
import requests
r = requests.get("http://your.url/here")
print(r.text)
```

- Change the text of the file to read anything else, such as "1.1.2"
- The previous code succeeds now, and prints `1.1.2`.

In case it makes any difference, I tested it with the url `http://<myserver>.com/asdfasdf`

The following things cause this bug not to occur:
- Running the code on Linux
- Using `urllib.requests` instead of `requests`
- Opening the URL in a browser (this shows the file contents is in fact the correct value)

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.0.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.1"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.2"
  },
  "system_ssl": {
    "version": "1010111f"
  },
  "urllib3": {
    "version": "1.26.14"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

@sigmavirus24 I'm struggling to see how this *isn't* a defect in this project, since it's a specific file contents causing weird and unpredictable behaviour in the library. I have exhausted every possibility other than a bug in requests, or some windows API that I am unaware of. Sorry if i missed anything, but it definitely seems like a bug. I'll edit the issue for some clarification, too.

There are probably close to 100 issues of "works in Linux, doesn't work in Windows" both here and on stack overflow. They are all environmental problems in your setup, not bugs in the library. That's why it's not a defect.
Can't properly DL this particular image although it works with urllib

This code would normally work for other images but for this [specific image URL](https://i.discogs.com/UzkDSQACNmbykQvWO0B9JGVajps7_MULzdwP9RK-o94/rs:fit/g:sm/q:90/h:591/w:589/czM6Ly9kaXNjb2dz/LWRhdGFiYXNlLWlt/YWdlcy9MLTk3NzE1/LTE2MjA2NjkwNzQt/MjMxMS5qcGVn.jpeg) it doesn't. The image is downloaded but the file is corrupt.

```python
url = "https://i.discogs.com/UzkDSQACNmbykQvWO0B9JGVajps7_MULzdwP9RK-o94/rs:fit/g:sm/q:90/h:591/w:589/czM6Ly9kaXNjb2dz/LWRhdGFiYXNlLWlt/YWdlcy9MLTk3NzE1/LTE2MjA2NjkwNzQt/MjMxMS5qcGVn.jpeg"

# doesn't work with requests
import requests  # 2.28.1
img_data = requests.get(url).content
with open('image_requests.jpeg', 'wb') as handler:
    handler.write(img_data)

# it works with urllib
import urllib
urllib.request.urlretrieve(url, "image_urllib.jpeg")
```

Result:

<img width="562" alt="Screenshot 2023-01-22 at 12 52 00" src="https://user-images.githubusercontent.com/1932338/213916740-0611c0c3-3660-47b9-a15f-a8c4d54b5aac.png">

We can see the `requests` one doesn't have a preview and doesn't have the same size.

<img width="372" alt="Screenshot 2023-01-22 at 12 52 46" src="https://user-images.githubusercontent.com/1932338/213916766-fde189c4-5390-40ca-a3de-09dadaef67f8.png">

Here are other examples:

- https://i.discogs.com/O0MU8c-zTjW0H_YWugum_KKdJzk1UNZok1xbeY0C1wo/rs:fit/g:sm/q:90/h:580/w:580/czM6Ly9kaXNjb2dz/LWRhdGFiYXNlLWlt/YWdlcy9MLTg4OTQ5/LTE0MzQ2NDQ2MTct/Nzc1Mi5qcGVn.jpeg
- https://i.discogs.com/b9vVXUMd_KhK8fZsoZ9nZR47HQ15m_t0fmG1GqHjl2Q/rs:fit/g:sm/q:90/h:356/w:472/czM6Ly9kaXNjb2dz/LWRhdGFiYXNlLWlt/YWdlcy9MLTQ4LTE2/MTAwNTEzNjgtOTQ4/OS5qcGVn.jpeg

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
urllib3 breaking changes

<!-- Summary. -->

The Requests library does not pin urllib3 to a version, but it exposes its classes. this means that if you use the exposed classes you might get different behavior, depending on which urllib3 version you have installed. Specifically, google Colab has an old version of urllib3 installed, so when using imports from `requests.adapters` it broke our library

## Expected Result

No error when using the `allowed_methods` param with Retry

## Actual Result

`TypeError: unexpected keyword argument 'allowed_methods'

## Reproduction Steps

```
pip install urllib3==1.24.3 # the version that Google Colab uses
pip install requests
```

```python
from requests.adapters import Retry

Retry(allowed_methods=['GET']) # a param that replaced `method_whitelist` in one of the versions of urllib3
```


> The Requests library does not pin urllib3 to a version, _but it exposes its classes_.

Hi @mkozakov, I'm not sure this statement makes sense. Requests does [pin urllib3](https://github.com/psf/requests/blob/main/setup.py#L64) to a supported range. Requests imports a class into the adapters module for internal use within the module. That's how any import works. We explicitly document if you want to change retries you should import `Retry` from urllib3 and pass it to the adapter. That's a dependency that you need to manage within your code at that point.

> If you need granular control over the conditions under which we retry a request, import urllib3's ``Retry`` class and pass that instead.

Using transitive imports of undocumented classes is a high risk decision and will likely lead to unexpected breakages like this. If this is code you've written, I'd advise following our documentation and ensure you're pinning for your API requirements. If this is managed by Google, you'll need to reach out to them for adjustments.

@nateprewitt got it. Should I not be importing `HTTPAdapter` either?
This is how we added retries, but maybe that's wrong? 

```python
from requests.adapters import HTTPAdapter, Retry

session = requests.Session()
retries = Retry(
    total=self.max_retries,
    backoff_factor=0.5,
    allowed_methods=['POST', 'GET'],
    status_forcelist=[429, 500, 502, 503, 504]
)
session.mount('https://', HTTPAdapter(max_retries=retries))
session.mount('http://', HTTPAdapter(max_retries=retries))

response = session.request('POST', url, headers=headers, json=json, **self.request_dict)
```

Our documentation clearly states what is our code and safe to import. Anything not documented you shouldn't be importing. We will not answer questions one by one about what you're importing and whether or not it's safe
Support missing SNIMissingWarning in tests

urllib3 2.0 only supports OpenSSL 1.1.1+ which means SNI is mandatory, and removed the warning altogether.

Looks like there's a small linting issue. Could you update the PR using our pre-commit hooks?

Although where it's moving the test import doesn't look right to me 👀. I'll investigate that separately.

Oops, sorry about that. Fixed the lint.

> Although where it's moving the test import doesn't look right to me 👀. I'll investigate that separately.

This is because I used `from tests import X` instead of `from . import X` and isort does not know that `tests` is a [known first party](https://pycqa.github.io/isort/docs/configuration/options.html#known-first-party). I switched back to `from . import X`.
Add `mime_type` property to response object

Added a `mime_type` property to the response object to allow for quick figuring out of the type of content of a response. This is a tiny QOL improvement, as it makes it easier to extract the mime type from the header without having to clutter up code just to extract it.

Hi @MarkusHammer, this is pretty specific behavior that I don't believe has a broadly applicable use case.

The end goal can be accomplished with a few lines of user written code when needed, so I think we'll stick with that as our advisement for now. Thanks!
fix: Remove '<4' from python_requires

Resolves #6332

The 

https://github.com/psf/requests/blob/61c324da43dd8b775d3930d76265538b3ca27bc1/setup.py#L97

cap was added in PR #6091.

As discussed in https://discuss.python.org/t/use-of-less-than-next-major-version-e-g-4-in-python-requires-setup-py/1066 and [other places](https://iscinumpy.dev/post/bound-version-constraints/) by the PyPA, use of upper bounds with `python_requires` for _future_ versions of Python is unintended use of `python_requires` and actively discouraged.

c.f. https://github.com/pypa/packaging.python.org/pull/850 for further detail.

This is ready for review.

It looks like this is a pretty consistent standard you've helped drive across projects. Given the comments in the discussion from the other PyPA maintainers, I think this is reasonable. Thanks @matthewfeickert!
Remove '<4' from python_requires

## Summary

As discussed in the discuss.python topic ["Use of “less-than next-major-version” (e.g., `<4`) in `python_requires` (setup.py)"](https://discuss.python.org/t/use-of-less-than-next-major-version-e-g-4-in-python-requires-setup-py/1066) and [other places](https://iscinumpy.dev/post/bound-version-constraints/) there can be some side effects of adding `<4` to a library's `python_requires`. Following these discussions the use of `<4` for `python_requires` was removed from the PyPA's packaging guide in https://github.com/pypa/packaging.python.org/pull/850. While 

https://github.com/psf/requests/blob/61c324da43dd8b775d3930d76265538b3ca27bc1/setup.py#L97

doesn't pose an immediate breaking threat in 2023, it would be good to remove this upper cap and is using `python_requires` in a way that was not intended.
Add example and improve docs for RequestException

I added a brief example of exception handling for `RequestException` and documented the `request` and `response` attributes. PR does not overlap with #6237

Hi @edward70,

Thanks for the contribution! While this is all correct, it's also information that can be understood reading the code. We try to reserve usage comments for areas that are particularly complex or unintuitive, otherwise it can make code harder to follow due to comment clutter. For that reason, I think we'll pass on this PR for the moment.
Please do not put upper bounds on your requirements constraints

See https://iscinumpy.dev/post/bound-version-constraints/ as for why this is a problem.
This is a source of byzantine issues such as https://github.com/nexB/scancode-toolkit/issues/3195

For reference these are looking like this for now https://github.com/psf/requests/blob/2745db131a667a3d5457957cb3ed6adeb9b9949a/setup.py#L61:L65
```
requires = [
    "charset_normalizer>=2,<4",
    "idna>=2.5,<4",
    "urllib3>=1.21.1,<1.27",
    "certifi>=2017.4.17",
```

In the future, please search **closed and** open issues before creating new ones that are duplicates.

@sigmavirus24 sorry for the churn then! I guess you meant this https://github.com/psf/requests/issues/6328#event-8200957254 , repasted here for reference:

> To be clear, jessemyers our position is the following:
> 
> - Requests is critical infrastructure for many projects and the larger Python community (pip, pipenv, twine, and so many other core workflows rely on it).
> 
> - urllib3 has the right to make minorly breaking changes to itself in between releases. Requests digs into urllib3 in certain places for certain workflows. Given that, we have to ensure things don't break when a new version of urllib3 came out, and we no longer vendor urllib3 into Requests, we need to be certain that the rest of the world won't burst into flames.
> 
> No software can guarantee future proof compatibility. We're very explicitly stating what we support. We could leave the upper bound off but we'd be implicitly supporting everything urllib3 releases. And when that would break, it would inevitably end up with us being painted as the worst people in the world.
> 
> Furthermore, there are two of us working on this project actively and both of us have very little time to keep up with other projects' release schedules. The most honest way we can communicate what we are capable of supporting at a given time is with these version constraints. We're far from the only open source project to take this stance. It would seem, then, that the more projects your project(s) use, the more they will encounter this and the more you will need to take a more measured approach to upgrading your dependencies.
[...]


There are _many_ past discussions of this. That's but one
From Expecting value: line 1 column 1 (char 0) to Response content not in json format

Hi Please run:

        res = self.session.post(url=self.url, data=json.dumps(data), headers=self.h, verify=False)
        try:
            js = res.json()
            token = js['token']
            self.cookie['reese84'] = token
            print(fg + ' (success)')
            return token
        except ValueError:
            print("Response content is not valid JSON")

I am getting this :

[*] Fetching TokenResponse content is not valid JSON

We can't help you debug general Python code, this issue tracker is only for defects in the Requests library. I recommend trying stackoverflow.com/ or the [Python Discord](https://www.pythondiscord.com/).
Allow charset-normalizer 3.x

requests does not work with charset-normalizer 3.0.0 or 3.0.1, blocking users of requests from updating to the latest charset-normalizer.

## Expected Result

<!-- What you expected. -->

## Actual Result

<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```json
{
  "paste": "here"
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Duplicate #6261. Please check existing issues before opening new ones, thanks.
Validation of header type makes it imposible to use a str subclass

Hi,
the validation of header type makes it impossible to use a str subclass.

code of validation:
https://github.com/psf/requests/blob/ec553c275fe915ab8c179a530907b564892d2922/requests/utils.py#L1036

I am rendering a the header by using the Django Template engine which returns a django.utils.safestring.SafeString. When using this as a header I get the

requests.exceptions.InvalidHeader: Header part ('application/xml') from {'accept': 'application/xml'} must be of type str or bytes, not <class 'django.utils.safestring.SafeString'>
exception.

May be using a construct with isinstance(ret, tuple(HEADER_VALIDATORS)):
could solve the problem.

thx & br
Matthias


In the future, please search **closed and** open issues before creating new ones that are duplicates.
Bump certifi dependencie to v2022.12.7 or higher due to Security Vulnerability

Affected versions of certifi<=2022.12.7 package are vulnerable to Insufficient Verification of Data Authenticity resulting in Certifi root certificate removal from TrustCor. The root certificates are being removed pursuant to an investigation prompted by media reporting that TrustCor's ownership also operated a business that produced spyware. 

## Expected Result

You must update the `certifi` version in `requests` dependencies.

## Actual Result

"certifi>=2017.4.17" in https://github.com/psf/requests/blob/ec553c275fe915ab8c179a530907b564892d2922/setup.py#L65

## More info

See: https://security.snyk.io/vuln/SNYK-PYTHON-CERTIFI-3164749
See: https://groups.google.com/a/mozilla.org/g/dev-security-policy/c/oxX69KFvsm4/m/yLohoVqtCgAJ


Hi @MefhigosetH, Requests already supports the patched version of Certifi. Raising the floor will unnecessarily break users as they may be unaffected by the TrustCor issue.

Hey there, I contacted privately not having seen this and didn't want raise an issue as it was linked to a vuln (_albeit, moderate_) so apologies for the unnecessary message Nate. So I've been trying to upgrade to the patched version of Certifi on one of our projects but get `Because requests (2.28.1) depends on certifi (2022.12.07) which doesn't match any versions, requests is forbidden`. Where could I go from here?

Hi @Masquerainsley, looking at the screenshot you provided, it appears you need to update your poetry.lock file. certifi `2022.12.7` (note 7 not 07) is available on [PyPI](https://pypi.org/project/certifi/2022.12.7/) with the fix. Requests will use that or any other version you install. You'll want to make sure you're using Python 3.6+ and running `poetry upgrade` to get the latest version.

I changed it to 7 instead of 07 after checking the PyPi site and still got the same error, I was in the process of upgrading the lock file when this happened. Turns out I had some weird caching thing - deleted and regenerated the file and it's fine now! thanks for your support :).
Result from browser JS blocked site does not replicate iin Requests

I'm having trouble understanding a java issue.
My browser has javascript blocker extension, and the site comes up normal with JS fully blocked.
When using requests, however I get 403.

I know Selenium should be used for Java driven sites, but my understanding is that Requests should work if the site comes up with al JS blocked in browser?

Perhaps it's a cookies issue? But the initial / first request can't have any cookies, so...
Also, does `requests.Session()` automatically retain cookies through iterations or do cookies need to be extracted and passed-on in the next iteration?

EDIT: Perhaps this issue is caused by an https problem? I notice that only system_ssl is populated in System Information below.

## PY Code
```
session = requests.Session()
session.headers.update(headers)
cookies = ''
response = session.get(url, cookies =cookies, headers=headers)
```

## Actual Result
<403 forbidden>
Enable JavaScript and cookies to continue

## System Information

```
{
  "chardet": {
    "version": "4.0.0"
  },
  "charset_normalizer": {
    "version": "2.1.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.7"
  },
  "platform": {
    "release": "5.19.0-26-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "30000050"
  },
  "urllib3": {
    "version": "1.26.9"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": false
}
```

Thank you.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Duplicate #6261

        Duplicate #6261

Nate: Sorry i posted duplicate.  Probably posted in the wrong place. Newbie and my first post on the forum.
You closed #6314 as duplicate of #6261. I was not making a feature request but seeking help.  
I was hoping to get help in solving the issue I am experiencing with RequestsDependencyWarning error messages.
      

Hi @thetechnodino, your last issue was closed because the warning you're encountering is discussed in the issue linked. For the time being you can downgrade to `charset_normalizer<3.0` if you're trying to get rid of the warning.

For future reference, please follow the instructions provided when you opened this issue. Thanks!

> Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.
RequestsDependencyWarning

Windows 11 (Professional); Pycharm 2022.3
Getting dependency warning in both Python 3.10 (latest) and Python 3.11 (latest). The code produces the desired results as well as a  dependency warning is generated.
[python code]
<import requests
quote = requests.get(url="https://api.kanye.rest")
quote.raise_for_status()
kanye = quote.json()
print(kanye)>


Error message (copy & paste from console)
C:\Users\XXX\AppData\Local\Programs\Python\Python310\python.exe E:\Python_Projects\Kanye\scratch.py 

C:\Users\XXX\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\__init__.py:109: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (None)/charset_normalizer (3.0.1) doesn't match a supported version!
  warnings.warn(

{'quote': "I don't wanna see no woke tweets or hear no woke raps ... it's show time ... it's a whole different energy right now"}

Process finished with exit code 0

Installed Packages

![Packages_2022-12-19 201204](https://user-images.githubusercontent.com/47829077/209002429-52793a32-b48f-4127-81e2-ac88c2195c6e.png)




Duplicate #6261
requests ignores no_proxy which contains a IPv6 CIDR 

python requests supports a no_proxy environment variable with a IPv4 CIDR e.g.

e.g
```
+ export http_proxy no_proxy
+ http_proxy=http://localhost
+ python -c 'import requests.utils ; print(requests.utils.should_bypass_proxies("http://1.2.3.4", None))'
False
+ no_proxy=1.2.3.0/24
+ python -c 'import requests.utils ; print(requests.utils.should_bypass_proxies("http://1.2.3.4", None))'
True
```

but doesn't support a IPv6 CIDR e.g.

```
+ http_proxy=http://localhost
+ python -c 'import requests.utils ; print(requests.utils.should_bypass_proxies("http://[1:2:3::4]", None))'
False
+ no_proxy=1:2:3::/64
+ python -c 'import requests.utils ; print(requests.utils.should_bypass_proxies("http://[1:2:3::4]", None))'
False
```


## Expected Result

For proxy to be bypassed if a the url matches the IPv6 CIDR in no_proxy

## Actual Result

IPv6 CIDR ignored

## Reproduction Steps

as above

## System Information

[derekh@laptop ~]$  python -m requests.help
/usr/lib/python3.11/site-packages/requests/help.py:24: DeprecationWarning: 'urllib3.contrib.pyopenssl' module is deprecated and will be removed in a future release of urllib3 2.x. Read more in this issue: https://github.com/urllib3/urllib3/issues/2680
  from urllib3.contrib import pyopenssl
{
  "chardet": {
    "version": "5.1.0"
  },
  "charset_normalizer": {
    "version": "2.1.0"
  },
  "cryptography": {
    "version": "37.0.2"
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.1"
  },
  "platform": {
    "release": "6.0.13-300.fc37.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "30000050",
    "version": "21.0.0"
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "30000050"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}


Proposed a fix here https://github.com/psf/requests/pull/5953
TypeError: 'ABCMeta' object is not subscriptable

The error is thrown  when linting with mypy and if `CaseInsensitiveDict` being used as a function return type hint with python 3.7 and 3.8.
The error is not thrown with python 3.9, 3.10 and 3.11.

## Expected Result

The error is not thrown 

## Actual Result

See example:
https://github.com/python-gitlab/python-gitlab/actions/runs/3726391254/jobs/6319811114


Hi @lmilbaum, you'll want to report this to GitLab. This is not code maintained by this project.

> Hi @lmilbaum, you'll want to report this to GitLab. This is not code maintained by this project.

@nateprewitt nateprewitt  Apologies. i'm not following. Which code is not maintained by this project?
The issue I reported refers to a class which is declared by the `requests` package and being used by another project. 

The type definition in the GitLab code for requests_backend.py is incorrect. Mypy [does not support abc.collections](https://mypy.readthedocs.io/en/latest/builtin_types.html#generic-types) before 3.9, so your definition is only compatible in 3.9+.

We don't intend to change the superclass for CaseInsensitiveDict, so the typing will need to be updated in the GitLab code. There are multiple options to work with this but we'll leave that decision to the code owners.

Thank you for your feedback. I will work out with the project maintainers.
requests.patch does not work, REST app does

Hello. I am attempting to update records through the Directus API with requests.patch.

Given the following code:

```
#!/usr/local/bin/python3

import requests

id = 1000
api_url_root = "[redacted]" 
api_access_token = "[redacted]"

api_url = api_url_root + "/items/filmstrips/" + str(id) + "?access_token=" + api_access_token

response = requests.patch(api_url, data={'problem':True}) 
```

The Directus server responds 200, and the Directus logs state:

```
[18:52:03] TRACE (4106 on uncommonephemera): [1.144ms] update `filmstrips` set `date_updated` = ?, `user_updated` = ? where `id` in (?) [Wed Dec 14 2022 23:52:03 GMT-0500 (Eastern Standard Time), [userid UUID redacted], 1000]
```

...which does not indicate the field "problem" is altered. Going in to Directus shows the field is not set to True.

I downloaded RapidAPI and sent the same request as shown in the screenshot:

[https://imgur.com/a/1IesDhD](https://imgur.com/a/1IesDhD)

The Directus logs show:

```
[19:03:17] TRACE (4106 on uncommonephemera): [0.876ms] update `filmstrips` set `problem` = ?, `date_updated` = ?, `user_updated` = ? where `id` in (?) [true, Thu Dec 15 2022 00:03:17 GMT-0500 (Eastern Standard Time), [userid UUID redacted], 1000]
```

....which shows that the field "problem" is updated. Upon going into Directus, the field is set to True.

Everything I've been able to read or search for about requests says this is how I'm supposed to do it. I've been at this all afternoon, trying different things, nothing works. Using version 2.28.1. How can I make this work?


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here. 

Please also closely read the documentation about inspecting the data you've sent in a request, how to send JSON data, and how to send urlencoded data as I expect these will help you find your solution.

Also note that screenshots are useless for offering help as many people rely on screen readers and further they aren't searchable to help others encountering similar issues

I'd say if all the documentation available says to do it this way, and it doesn't work, that's a defect.

I'm not entirely sure how to describe what I've done in a GUI app that wouldn't generate a thousand questions about what I inadvertently left out of the description or imply I didn't do something I should have.

I tried json.dumps(payload), json=payload, and adding the Content-Type header, none of that made any difference.

Either way, it sounds like everyone here is certain the library is problem-free (talk about needing a screen reader), so I'll be on my way. HTTP for humans, my ass
Please make a new release

The last release of requests was from June 29.

Since then charset-normalizer 3.x has been released, which the release is incompatible with. This was already fixed in #6261, but is unreleased.

Clearly this affects many people, as this issue has been re-reported **so** many times:
* #6268
* #6272
* #6298
* #6288
* #6305
* #6281
* #6273
* #6272

If even just to stop people from wasting their time re-reporting this, please make a release.


Thanks for the feedback, @intgr. As already stated in the other issues, this has been merged and will be in the next release 2.29.0.

Why are requests project maintainers so dismissive... I am fully aware that the PR was merged and will be in the next release, I already stated as much in my message.

But there is no planning/tracking issue or discussion about a release that I could find.

What are the plans for the next release? Is it going to happen within days or months? Or is it still waiting for some blockers?


Why do you feel entitled to demanding we do something in our free time?

Why do you think we don't know we need to release it? Issues like this just take what little time we might otherwise have had to work on the release and wastes it. 

You argue that we're wasting other people's time when the issues have been indexed by Google and are easily searchable on GitHub. So they're eating their time as well as ours. We're not responsible for people choosing to create duplicate issues when the original is easily found. We're not responsible for the 2 or 3 duplicates we received as soon as charset-normalizer released before it could be tested or fixed. 

> What are the plans for the next release? Is it going to happen within days or months? Or is it still waiting for some blockers?

Requests has released twice yearly usually December and June since 2018, except for exceptional circumstances. You can find the full list of open issues attached to the milestone for 2.29.0.

As Ian has stated this is all easily discoverable through the tools GitHub provides to manage projects. It's linked from the original PR you've provided as well.

I think there's some confusion about being dismissive v. needing to reiterate the same thing repeatedly. We've answered all of the above questions previously but there's an entitlement that we provide this information to you directly instead of taking the time to do minimal research. This isn't scalable to the millions of people using Requests, so responses are often curt.

We'll plan to do the next release in the coming week or after the new year, as is the normal release cadence. We're currently waiting to resolve final blockers for urllib3 2.0's release.
Broken dependency with latest charset-normalizer

pip's dependency resolver generates the following error:
`requests 2.28.1 requires charset-normalizer<3,>=2, but you have charset-normalizer 3.0.1 which is incompatible.`

## Expected Result

<!-- What you expected. -->

## Actual Result

<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.0.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.7"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010111f"
  },
  "urllib3": {
    "version": "1.26.13"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


In the future, please search **closed and** open issues before creating new ones that are duplicates.
Update `certifi` dependency to patch known vulnerability.

`certifi>=2017.11.05, < 2022.12.07` has a known security vulnerability, described [here](https://github.com/advisories/GHSA-43fp-rhv2-5gv8). The currently supported `certifi` range `certifi>=2017.4.17` includes these vulnerable versions.

This pull request updates `certifi` to `2022.12.07` which patches the vulnerability.

Since we're not constraining upgrading to the latest version I think we're okay to keep the version range as-is, users should manage their dependencies and vulnerable versions. Going to close this.