Raibek/requests-github-issues huggingface dataset:
Download almost empty files

**Hey, I have some urls which will be instently download just by paste it in browser. But when i use os and requests to download it,
it doesn't download properly**

## I Expected:
Download a 72 M.b with .hdf format.

## But the result:
download a 11 k.b .hdf file! 

## Reproduction Steps

```python
import os
import requests


def download_file(url, output_path):
    try:
        response = requests.get(url, stream=True)
        print(response)
        if response.status_code == 200:
            filename = url.split("/")[-1]
            file_path = os.path.join(output_path, filename)
            
            with open(file_path, "wb") as file:
                for chunk in response.iter_content():
                    file.write(chunk)
            
            print(f"Downloaded {filename} to {file_path}")
        else:
            print(f"Failed to download {url}. Status code: {response.status_code}")
    except Exception as e:
        print(f"An error occurred: {e}")

download_file("https://ladsweb.modaps.eosdis.nasa.gov/archive/allData/61/MOD021KM/2022/152/MOD021KM.A2022152.1930.061.2022153073644.hdf", "H:/__________Basir/2- Download_Modis_Images/Some_Sample_Downloaded/_Test 2.0.0")

```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.1.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.12"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "30100020"
  },
  "urllib3": {
    "version": "1.26.11"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Fix monthly download badge in README

This PR fixes the monthly download badge in the README.

**At the moment:**
<img width="791" alt="image" src="https://github.com/psf/requests/assets/11357413/189384fd-d24b-4e36-8e6d-a763581d207e">


**With this PR:**
<img width="765" alt="image" src="https://github.com/psf/requests/assets/11357413/0174ed1a-82f1-475f-8245-dad2c57c11fc">
Move to src directory

This is more of a pulse check/RFC from the other maintainers. Using a `src` directory has been the standard advice for package structure [1][2] for quite a while for proper test encapsulation. This PR proposes we move Requests to follow suit.

I don't think we have anything requiring this move beyond providing a better example since Requests is still cited as an example for projects (despite being relatively out of date from community standards).

1. https://packaging.python.org/en/latest/tutorials/packaging-projects/
2. https://docs.pytest.org/en/latest/goodpractices.html
Upgrade to httpbin 0.10.0

This PR will upgrade the Requests test suite to use the recently released httpbin 0.10.0 that now supports the latest major version of Flask. This will let us remove our pins on pallets projects and hopefully allow distro maintainers to no longer need to support these older versions.

Thanks @mgorny and @kevin1024 for helping get this unblocked and closing out #6070!

The PyPy 3.8 Windows build seems to be consistently timing out at 10 min because it's trying to build wheels for a handful of dependencies and installation is taking ~8 min of the run. We may need to either bump the time limit or see if there's a way we can get the installation step sped up.

Edit: It succeeded on the third retry. zope.interfaces, flasgger, and rpds_py seem to be the time intensive builds for wheels. I'm not sure if we can optimize this at all or if this is just a requirement to operate with PyPy. If this becomes a reoccurring issue, I'd be fine dropping back to fewer PyPy builds if there's not a clear solution.

> Edit: It succeeded on the third retry. zope.interfaces, flasgger, and rpds_py seem to be the time intensive builds for wheels. I'm not sure if we can optimize this at all or if this is just a requirement to operate with PyPy. If this becomes a reoccurring issue, I'd be fine dropping back to fewer PyPy builds if there's not a clear solution.

Why can't we cache the wheels that are built so subsequent runs are faster/less likely to time out?

TIL [this](https://github.blog/changelog/2021-11-23-github-actions-setup-python-now-supports-dependency-caching/) was an option. I've pushed up a change to enable it and we'll see if speed improves.

Looking at the builds, the expensive wheels don't look like they're going to take advantage of the cache. They all have unique ABI for each platform and version of PyPy.

`pp39-pypy39_pp73-win`
`pp38-pypy38_pp73-win`
`pp39-pypy39_pp73-macosx_10_9`
etc.

The issues seem isolated to Windows and showed up on the 3.12-dev builds too. It also was happening with other builds so it's not unique to the httpbin changes. I'll merge the cache change because it appears to have some minor improvements for the other builds.

I also see @mgorny is working on a more involved fix in psf/httpbin to deal with windows. I'd rather forego those for now and bump the timeout to 12 min which seems to be the upper bound for the builds that are currently failing. We can reevaluate if it becomes a nuisance.
Wrong HTTP verb is used when sending a request

<!-- Summary. -->

I'm not sure what is causing this, but when I try to send a POST request, the library sends a GET instead.

<!-- What you expected. -->

I expected requests lib to send a POST.

<!-- What happened instead. -->

```python
Python 3.10.3 (v3.10.3:a342a49189, Mar 16 2022, 09:34:18) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> response = requests.post("https://docs.python.org/")
>>> response.status_code
200
>>> response.request.__dict__
{'method': 'GET', 'url': 'https://docs.python.org/3/', 'headers': {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}, '_cookies': <RequestsCookieJar[]>, 'body': None, 'hooks': {'response': []}, '_body_position': None}
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.2.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.3"
  },
  "platform": {
    "release": "20.3.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "101010ef"
  },
  "urllib3": {
    "version": "2.0.4"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

In the future, please search **closed and** open issues before creating new ones that are duplicates.


It's likely you're encountering a redirect and the semantically correct behavior is to change the method to a GET. Check the history of the response 

When I print `response.is_redirect` the value is `False`

Is this the expected behavior as well ?

> When I print `response.is_redirect` the value is `False`
> 
> Is this the expected behavior as well ?

Check `response.history` as I said. `response.is_redirect` doesn't indicate a response is the result of following a redirect.

Just some complementary informations : 
- it's a private API, I don't have any control on how the backend works
- When I use curl or urllib3 on the same endpoint, I don't get redirected. Using requests I get redirected.
There is something wrong with your request url when sending a link with Chinese in it.

When I using this module, I sent a URL with Chinese character in it.This link is "https://bestgirlsexy.com/china/pans%e5%86%99%e7%9c%9f/" , 

!!BUT!! when I sent "https://bestgirlsexy.com/china/pans%e5%86%99%e7%9c%9f/" to the requests module,
 it gave me a wrong result. 

So the pocket's URL sent on the network is not "https://bestgirlsexy.com/china/pans%e5%86%99%e7%9c%9f/", it should be "https://bestgirlsexy.com/china/pans%E5%86%99%E7%9C%9F/" or somethinf else.

 So how can I solve this problem? Should I build a module with cpp and use python to use it?

Visually I see nothing different in the URLs and you've not shown what you expected versus what you're actually seeing. There's not enough information here to help you.

I suggest you ask for more help on Stack overflow since there is no bug here based on what you've shown. 

> 

You should notice that the character after the "%".The upper and lower character after that URL will affect the response status code! But requests haven't give me a function to handle it.
Add Cloudflare-specific HTTP status codes

### Add Cloudflare-specific HTTP status codes

This pull request adds the Cloudflare-specific HTTP status codes to the project. Cloudflare uses these status codes to represent various error scenarios that are specific to their services. The new codes range from 520 to 533.

### Changes Made
- Added constants for Cloudflare-specific HTTP status codes to the existing `_codes` dictionary.

### New Status Codes
- 520: UNKNOWN ERROR
- 521: WEB SERVER IS DOWN
- 522: CONNECTION TIMED OUT
- 523: ORIGIN IS UNREACHABLE
- 524: A TIMEOUT OCCURRED
- 525: SSL HANDSHAKE FAILED
- 526: INVALID SSL CERTIFICATE
- 527: RAILGUN ERROR
- 530: ORIGIN DNS ERROR
- 531: ORIGIN CONNECTION TIMEOUT
- 532: ORIGIN READ TIMEOUT
- 533: ORIGIN UNCERTAIN ERROR

These additions expand our support for a wider range of recognized HTTP status codes, especially relevant for integrating with Cloudflare services.

Please review and merge this pull request if the changes are deemed appropriate for the project.


Hi @mahdirahimi1999, the status_codes module is largely considered deprecated and we don't intend to add any more codes to it that aren't explicitly defined in an HTTP RFC.

This would be a good opportunity if someone wants to start and maintain a separate package with these as reference if needed. That will allow the user set that may need this functionality to opt into it. Thanks!
NewConnectionError 

When I pass in a request url for a url that does not respond I get a 

```python3
request = requests.get(
            "url"       
            proxies=proxies,
        )
```

```
Error: SOCKSHTTPSConnectionPool(host='atxdyqbqutne3q0hoidwnjkxzxzyogtw0boudm7ztiwrdxrucedrsolw.onion', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0xffff8fbf9690>: Failed to establish a new connection: 0x01: General SOCKS server failure'))
```

But when I also kill my proxies I get the same error. This should not be, how can I catch the different errors and understand what I'm dealing with? Is this behaviour expected? If so, why?

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

Hi, I'm not very well versed with the requests module and the error seems like a urllib3 error of some sort. Is this expected behaviour or a bug?
proxies set in the session will not affect  `session.get`

<!-- Summary. -->

## Expected Result

proxies and verify and so on will affect the session.get / session.post

## Actual Result

it does not affect.

## Reproduction Steps
### 1
```python
import requests

def get_session(proxy):
    session = requests.Session()
    return session

get_session().get(xxxxxx)
```
this will get a 407 error.
### 2
```python
import requests

proxy={"http_proxy":"xxxxx","https_proxy":"xxxxx"}
def get_session(proxy):
    session = requests.Session()
    session.proxies = proxy
    session.verify = False
    return session

get_session().get(xxxxxx)
```
this will get a 407 error.
### 3
```python
import requests

proxy={"http_proxy":"xxxxx","https_proxy":"xxxxx"}
def get_session(proxy):
    session = requests.Session()
    session.proxies = proxy
    session.verify = False
    return session

get_session().get(xxxxxx,proxies=proxy,verify=False)
```
It works well.


## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "4.0.0"
  },
  "charset_normalizer": {
    "version": "2.0.4"
  },
  "cryptography": {
    "version": "39.0.1"
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.9"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "1010114f",
    "version": "23.0.0"
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "1010114f"
  },
  "urllib3": {
    "version": "1.26.14"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


In the future, please search **closed and** open issues before creating new ones that are duplicates.
PayLoad trucated when using french special characters like é, è, à

Hi,

I m using requests-2.31.0

Here is 3 examples of my Python variable used to send a payload : 

Below with no special character :
Python variable = **{"updateGroups":true,"groups":[],"user":{"displayName":"TONI eric","email":"xxxxxx@xxxxxxx.com","name":"anto1","password":"anto1","accountRoleIds":[1677,1680,1683]}}**

Below with 2 special characters :
Python variable = **{"updateGroups":true,"groups":[],"user":{"displayName":"TONé éric","email":"xxxxxx@xxxxxxx.com","name":"anto3","password":"anto3","accountRoleIds":[1677,1680,1683]}}**

Below with 3 special characters :
Python variable = **{"updateGroups":true,"groups":[],"user":{"displayName":"èTONé éric","email":"xxxxxx@xxxxxxx.com","name":"anto3","password":"anto3","accountRoleIds":[1677,1680,1683]}}**

The 3 variables are well formatted than i expect with good informations.

But here is the content sent by the module requests-2.31.0 WHEN THE PAYLOAD CONTAINS special french characters (Fiddler RAW)

## Actual Result

{"updateGroups":true,"groups":[],"user":{"displayName":"ANTONéN Ayméric","email":"xxxxxx@xxxxxxx.com","name":"anto3","password":"anto3","accountRoleIds":[1677,1680,1683]

{"updateGroups":true,"groups":[],"user":{"displayName":"ANèTONéN Ayméric","email":"xxxxxx@xxxxxxx.com","name":"anto3","password":"anto3","accountRoleIds":[1677,1680,1683


## Expected Result

{"updateGroups":true,"groups":[],"user":{"displayName":"ANTONéN Ayméric","email":"xxxxxx@xxxxxxx.com","name":"anto3","password":"anto3","accountRoleIds":[1677,1680,1683]}}

{"updateGroups":true,"groups":[],"user":{"displayName":"ANèTONéN Ayméric","email":"xxxxxx@xxxxxxx.com","name":"anto3","password":"anto3","accountRoleIds":[1677,1680,1683]}}


Regarding the count of special characters, the payload is truncated of this count of characters.

Regards


You haven't described how you're sending the payload with requests, the version of python, environment, or anything else that would help diagnose this.

Hi,
sorry for send you few informations.

I precise that when there is NO special characters, the request and the payloas are OK (http200)

I m running Python 3.9.0 on a windows machine

Here is an extract of my code.

    MesHeaders3 = {'X-CSRF-TOKEN': Tolkien,'Accept': 'application/json, text/plain, */*','Content-Type': 'application/json;charset=UTF-8','Accept-Language': 'fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7'}
    PayloadJson = '{"updateGroups":true,"groups":[],"user":{"displayName":"' + UserFullName + '","email":"' + UserEmail + '","name":"'+ UserName + '","password":"' + UserName + '","accountRoleIds":[' + IDRoles2use + ']}}'

    print (PayloadJson)
    
    MyRequest3 = requests.post(Url_UsersCreate, headers=MesHeaders3,cookies=Biscuit2Send, data=PayloadJson)

Regards

Formatting JSON by hand is a perfect way to end up with invalid JSON and unexpected behavior. I strongly suggest you create a dictionary and then pass that via the `json` parameter to `requests.post` and allow that to convert your data to JSON. I expect that will fix your problem.

As the saying goes, junk in, junk out.
Incomplete URL

## Reproduction Steps

```python
import requests

httpbin_host='https://httpbin.org'
# httpbin_host='https://httpbin.org/'
s = requests.Session()
r=s.get(url=f'{httpbin_host}/cookies/set/sessioncookie/123456789')
print(r.url)
print(r.request.url)
```
Executing the above code will get the following incorrect result.
```txt
https://httpbin.org/cookies
https://httpbin.org/cookies
```
<!-- 
If  set `httpbin_host`  to `https://httpbin.org/`, normal results can be obtained.
```txt
https://httpbin.org//cookies/set/sessioncookie/123456789
https://httpbin.org//cookies/set/sessioncookie/123456789
```
 -->
## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "3.1.1"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.5"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "1010108f",
    "version": "19.1.0"
  },
  "requests": {
    "version": "2.24.0"
  },
  "system_ssl": {
    "version": "1010108f"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": true
}
```


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
SSL ERROR on jiotv login in KODI app.

Below error show during login jio tv I  KODI application 

[HTTPSConnectionPool(host='api.jio.com', port=443): Max retries exceeded with url: /v3/dip/user/otp/send (Caused by SSLError(SSLError(1, '[SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:992)')))  ]

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
SSLV3_ALERT_HANDSHAKE_FAILURE Python 3.11 and OpenSSL 3

This is an intermittent exception that is raised under certain conditions and not any other time. There is another closed issue similar to this one:
https://github.com/psf/requests/issues/4753
But I think this is a new issue.

What would be the recommended steps for isolating the source of the problem? The site this request connects to only supports TLS. SSL is not an option.

## Expected Result

Connection success and TLS like all other requests.

## Actual Result

error:
```
SSL Error: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: /query/sha256 (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1002)')))
```

## Reproduction Steps

```python
import requests
response = requests.get(url, auth=(username, password))
```
Just a basic standard request. Nothing special.

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.2.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.4"
  },
  "platform": {
    "release": "6.2.0-24-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30000080"
  },
  "urllib3": {
    "version": "1.26.16"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

This comes directly from the standard library raising the error on the specified line returned from trying to perform the handshake [here](https://github.com/python/cpython/blob/7dead6a33ac786465c3d927d5c2025caf0c1a71b/Modules/_ssl.c#L964). Literally, OpenSSL cannot complete the handshake.

Without knowing anything about the server, it's possible there are multiple backends or multiple intermediaries, one of which is leading to this particular error for you. The client configuration and the server configuration cannot agree on how to connect thus the handshake fails. I recall a time in the past when OpenSSL used `SSLv3` to refer to `TLS1.0` but I don't believe Python 3.11, OpenSSL 3, nor the versions of urllib3 and requests you're using allow for 1.0 or really anything below 1.2. This genuinely seems to be an intermediary or server issue that is unlikely possible for you to resolve.

In the future, though, please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

@utkonos  how to fixed problem, i same error, Help me please
session post set allow_redirects=False failed

I want to get redirected location，But although I set ：
s = requests.Session() 
response = s.post(url,data=datas,headers=headers,allow_redirects=False)
response.status_code is 200， not 302，why???

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
URLQUICK needs adapters.py HTTPResponse

<!-- Summary. -->

## Expected Result

URLQUICK does not fail importing
 from requests.adapters import HTTPResponse

## Actual Result
fails badly:
File "/home/khalistoo/.var/app/tv.kodi.Kodi/data/addons/plugin.video.catchuptvandmore/resources/lib/main.py", line 13, in
import urlquick
File "/home/khalistoo/.var/app/tv.kodi.Kodi/data/addons/script.module.codequick/lib/urlquick.py", line 63, in
from requests.adapters import HTTPResponse
ImportError: cannot import name 'HTTPResponse' from 'requests.adapters' (/home/khalistoo/.var/app/tv.kodi.Kodi/data/addons/script.module.requests/lib/requests/adapters.py)

## Reproduction Steps
Install catchuptvandmore in github. Upadte to the latest request package

## System Information

Android, Kodi Nexus 20.1
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


I've just submited PRs to:
- urlquick module: https://github.com/willforde/urlquick/pull/118
- codequick kodi addon (embeds `codequick.py`): https://github.com/willforde/script.module.codequick/pull/53

Thanks for these PR. Unfortunately, I'm pretty scared these addons are not maintained no longer. 

Well I see activity from willforde on June 2023. He may not develop them anymore, but still be able to merge PRs.

Next step will be Kodi team to package the new version of `script.module.codequick`

That's not Kodi Team who package the modules. Maintainers of a module pull request them into xbmc repo script github.

HTTPResponse was never a documented portion of the adapters module and should never have been imported from there. This is not a bug in requests
support adding headers when using proxies

In some cases, when using a proxy, it is necessary to specify specific header content. Therefore, I have implemented this functionality in a simple way.

usage：
https://github.com/anysoft/requests/blob/e6b05c5edba85b6b383b39fdde4b3ab9b514f6db/tests/test_proxy.py
``` python
import requests


def uc_unicom(ip):
    try:
        headers = {
            'user-agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Mobile Safari/537.36 Edg/111.0.1661.41'
        }
        proxies = {
            'http': f'http://{ip}',
            'https': f'http://{ip}',
            'headers': {
                'Proxy-Authorization': 'Basic dXNlcjpwd2Q=',
                'Q-GUID': 'XXXX',
            }
        }
        response = requests.get('https://api.ip.sb/ip', headers=headers, proxies=proxies)
        print(response.text)
    except Exception as e:
        print(e)



if __name__ == '__main__':
    uc_unicom('127.0.0.1:8080')

```

Thanks for putting together a pull request and sending it. Unfortunately, Requests is under feature-freeze and has been for quite a while.

Regardless, there is a clear way to add headers for your proxy. That said, your example test is specifying `Proxy-Authorization` but near the code you altered, that's already supported. If you can't use basic authentication with your proxy, you can change the adapters to do the right thing based off of what your needs are and replace the adapter.
The header is not change 

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.

header = {
    "Authorizationapikey": "apikey ***************************:******************************",
    'content-type': 'application/json',
}

param={
    "limit": 0
}


response = requests.get(url ,headers=header ,params=param)
print(response.url)
print(response.headers)
print(response)
response_data = response.json()


#The header in this code is not changing and I don't know why so if there is any help to explain that the problem from me or from the module it self .
#And the response keeps giving me Response<401> which is not authorized 

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Support passing request headers as multi-line strings

Title: Support passing request headers as multi-line strings

Description:
Currently, when using the requests library to send HTTP requests, we need to pass the request headers as a dictionary to the headers parameter. However, in practical development, it is common to directly copy request headers from a browser and parse them into a dictionary, rather than manually editing a dictionary, which is more convenient and elegant.

I suggest an improvement to requests that allows the headers parameter to accept multi-line strings as input. This would enable developers to directly copy request headers from a browser, provide them as a string input, and have requests internally parse them into a dictionary.

Here's an example code snippet that demonstrates parsing multi-line strings into a dictionary:

```
headers = """
Host: example.com
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9,en;q=0.8
"""
headers = dict(x.split(": ", 1) for x in headers.splitlines() if x)

```
This improvement would enhance the usability and flexibility of requests, aligning it better with real-world development needs. I hope to see this feature implemented in future versions.

Thank you for considering and accepting this suggestion!



Requests has been under a feature freeze for years now, so the answer is definitely No.

Regardless, if we were implementing new features I would be strongly against this feature because it allows for too many potential issues that could create security problems for users. It may seem like an improvement in user experience but it would be bad for security
Requests changes the case of hex letters in percent encodings

## Summary

Requests changes the case of hex letters in percent encodings in URLs.

Requests should always send the request target exactly as it was passed.

### Reproduction

```py3
import requests

print(requests.Request('GET', 'https://example.com/?example=%2f').prepare().url)
```

## Rationale

I wouldn't even absolutely ever not even in my worst nightmares ever consider this an issue.
That was until I met this beautiful http server, which won't work if you don't send **exactly** the same URL on http redirections.

See screenshots below:
![Screenshot showing successful response to request with lowercase hex](https://i.ibb.co/T2JnYhY/Screenshot-20230626-135150.png "Screenshot lowercase hex")
![Screenshot showing 404 response to request with uppercase hex](https://i.ibb.co/bmpjSG1/Screenshot-20230626-135128.png "Screenshot uppercase hex")
The only difference between the two requests is casing of hex characters, but the response from the server is completely different.

I would want to say that this issue is caused by a very weird design decision of this website (which is a wi-fi login page by the way), but who am I to argue with the server.

I don't have control nor access to the source of the server, but I suspect that the parameter `md` in the request is a hash of some part of the URI. So the request target must be sent exactly as it was passed in the `Location:` header (of a different request, not part of the screenshots).

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.1.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.2"
  },
  "platform": {
    "release": "6.3.8-zen1-1-zen",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30100010"
  },
  "urllib3": {
    "version": "2.0.3"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

Seems like the bug lies in urllib3.

```py3
import urllib3.util

print(urllib3.util.parse_url('https://example.com/?example=%2f').query)
```

I'm not sure there's much we can do here. There was a decision from the urllib3 team a handful of years ago where URLs started getting unilaterally standardized which broke a handful of workflows with PreparedRequests. I saw Quentin's reply on the urllib3 ticket but I don't think we can utilize that escape hatch in a backwards compatible manner.

Given this has only appeared a few times in the last ~5 years, I think we can safely say it's an unfortunate but rare edge case. I'm going to resolve this for now since it's unactionable. If the situation changes with what urllib3 exposes in the future we may be able to revisit it.

In case someone encounters this issue: I switched to httpx, which doesn't use urllib3 and doesn't have this bug.
XML cut off

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.

I'm trying to send a XML with the last version, i read the file and put it in a var, when i send it, request cut off the final 3 caracters of the file, this didn't do the version 25.

The code:

archivo = open(namexml, "r")
xml = archivo.read()
resp=requests.post(url, data=xml, headers=headers,cert=cert,verify=False)

I don't know if it is a bug or i'm do it wrong.


Thankxxxx


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Redirect from doi.org to onlinelibrary.wiley.com fails

If I visit `https://doi.org/10.1002/jccs.200600142` with my browser, everything is fine. But both requests fails:

```sh
python -c "import requests; print(requests.head('https://doi.org/10.1002/jccs.200600142', allow_redirects=True))"

<Response [403]>
```

I also tried accepting cookies and changing the user-agent, which also did not help:
```py
import requests
with requests.Session() as s:
    print(s.get('https://doi.org/10.1002/jccs.200600142', allow_redirects=True, headers={'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/114.0'}))

<Response [403]>
```

Does someone know what `requests` does differently than Firefox? Or should I include more headers?

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

Sorry I did not know that requests for helps are not welcome here @sigmavirus24 . I recommend locking the button here to avoid future help requests:

![image](https://github.com/psf/requests/assets/18174744/5f66344a-28e1-45e1-a65a-3889b1600262)

---

BTW I asked on SO: <https://stackoverflow.com/questions/76466113/redirect-from-doi-org-to-onlinelibrary-wiley-com-fails>
Getting 401 unauthorized status in an API

I was making a POST request in [nutritionix API](https://www.nutritionix.com/business/api) using requests module. In one of the API route which is `https://trackapi.nutritionix.com/v2/natural/nutrients`, which returns all the necessary details for any food item if we pass json as `{"query": <any_name>}` and headers as `x-app-id` and `x-app-key`.

It works fine in Postman, but in Python it shows 401 unauthorized status if I fetch the `x-app-id` and `x-app-key` from .cfg file.

The format of that cfg file is 
```
[nutritionix]
API_ID=<api-id>
API_KEY=<api-key>
```

I am fetching all those keys using `configparser` module, I am able to print those keys in the terminal, but it shows 401 error. But if I directly paste these keys into the code directly (which is not recommended), it is showing 200 status.

I am pasting the code below for reference:

```python
import requests
from configparser import ConfigParser

config = ConfigParser()
config.read('./secrets.cfg')

x_app_id = config['nutritionix']['API_ID']
x_app_key = config['nutritionix']['API_KEY']

headers = {
    'Accept':'application/json',
    'Content-Type': 'application/json',
    "x-app-id": x_app_id,
    "x-app-key": x_app_key,
}


url = 'https://trackapi.nutritionix.com/v2/natural/nutrients/'

def get_details(prompt: str):
    food_details = requests.post(url, headers=headers, json={"query": prompt})
    print(food_details)
```

This issue doesn't look like a problem with the requests library. You probably should ask for further help in stackoverflow, but one hint:

> x_app_id = config['nutritionix']['API_ID']

Here you're using `'API_ID'` string instead of the `API_ID` variable.

Yep, I believe @Mariatta is right. Thank you!

I'm going to close this out because it doesn't appear to be an issue with Requests.

> This issue doesn't look like a problem with the requests library. You probably should ask for further help in stackoverflow, but one hint:
> 
> > x_app_id = config['nutritionix']['API_ID']
> 
> Here you're using `'API_ID'` string instead of the `API_ID` variable.

This solution didn't work out for me and I didn't realise that this issue isn't related to requests library, my bad. I will post this in stack overflow.

Well, thanks @Mariatta for helping me out!
raise ClientError for 4xx and ServerError for 5xx

`raise_for_status()` will now raises 
- ClientError for status code 4xx
- ServerError for status code 5xx

Both ClientError and ServerError are subclass of HTTPError, which should make them backward compatible for users that are catching `HTTPError`

Also kept the error message the same for backward compatibility too in case users are matching the message value.

Ideally the "Client Error" and "Server Error" in the message could be removed given the class name provide the same context.
i.e
`f"{self.status_code}: {reason} for url: {self.url}"`

Thanks for putting this together! We're under feature freeze now, however, and are not accepting new API additions. As such, we will be closing this
Runtime.ImportModuleError after upgraded to 2.31.0

Found below error from logs after auto-upgraded (by Renovate) from 2.28.1 to 2.31.0 :
```
[ERROR] Runtime.ImportModuleError: Unable to import module 'handler': libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory
Traceback (most recent call last):
```


## System Information

Running python version 3.8 in docker image python:3.8-alpine


Hi @jack1220, the error doesn't appear to be related to Requests. You're importing code that's trying to access `libc` files. Requests does not provide any C code, so this is stemming from some other dependency.
No possibility to provide raw urls 

Since requests is the major python library for web requests and python is used for web testing
it is unacceptable, that requests does not provide a way to define raw urls for web requests.

And malformed url behavior is one essential aspect of web testing, so it must be provided.

And to just close a previous issue on this topic by saying it is not our responsibility is brazen and tone deaf.

Closed, unresolved issues regarding this problem:
https://github.com/psf/requests/issues/5739
https://github.com/psf/requests/issues/5907

Please keep this issue open until this basic feature is working again.


Do you all have a character disorder to close it again?

The library is provided as is. Please read the license

Gaslighting won't work, try something more clever,
this bug has nothing to do with the license.

You even documented some workaround here once:
https://github.com/urllib3/urllib3/issues/1790

> this bug has nothing to do with the license.
> Please keep this issue open until this basic feature is working again.

We're closing this because the library is provided as is. It's not gaslighting.
I knew we missed a step...

Got a bit too far ahead of myself on the release. Running Black over the patch to fix quotes.

@nateprewitt , what was the problem of having single quotes ?

There is no problem with single quotes, but Requests uses [black](https://github.com/psf/black/) for formatting which [prefers double quotes](https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#strings).
v2.31.0

2.31.0 (2023-05-22)
-------------------

**Security**
- Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential
  forwarding of `Proxy-Authorization` headers to destination servers when
  following HTTPS redirects.

  When proxies are defined with user info (https://user:pass@proxy:8080), Requests
  will construct a `Proxy-Authorization` header that is attached to the request to
  authenticate with the proxy.

  In cases where Requests receives a redirect response, it previously reattached
  the `Proxy-Authorization` header incorrectly, resulting in the value being
  sent through the tunneled connection to the destination server. Users who rely on
  defining their proxy credentials in the URL are *strongly* encouraged to upgrade
  to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy
  credentials once the change has been fully deployed.

  Users who do not use a proxy or do not supply their proxy credentials through
  the user information portion of their proxy URL are not subject to this
  vulnerability.

  Full details can be read in our [Github Security Advisory](https://github.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q)
  and [CVE-2023-32681](https://nvd.nist.gov/vuln/detail/CVE-2023-32681).
Add os.PathLike support for cert files

## Summary

Adds support for any path-like object, including `pathlib.Path`. See https://github.com/psf/requests/issues/5936#issuecomment-931758148 for more details.
Some things have changed since the linked issue was created, so it may be a good time to add support now.

Fixes #5936 

My main concern is that if we add support here, folks will want us to support it everywhere else we accept file-like objects and that's not exactly tenable for us to support

Maybe it doesn't have to be advertised, and we can add in path-like support slowly where it is feasible.

Should I close this?
Small typo fix in HISTORY.md

I was going through the release history docs to ensure compatibility safety in case of upgrading the package to the latest version. While doing so found this minuscule typo in the docs, where the word `entries` was written as `entires`.

Forgive me for the petty nitpicking but just thought of proposing a fix 😅 

mass kaatiyadhaga unargirar @syed-saif
Adding new tests

Instead of defining prepare_url as a fixture, it could be defined as a function that is used within the fixture definitions. This would make the code more readable and avoid confusion about what prepare_url is doing.

It might be useful to add more tests that use the httpbin and httpbin_secure fixtures to test different HTTP methods, response codes, and request parameters. This would help ensure that the fixtures are working correctly and that the requests library is handling HTTP requests and responses as expected.

Hi @abhinavyedla02, thanks for the contribution. I'm not sure if this provides a meaningful change to the test suite. It's arguable if readability is improved and moves away from the general intention of [conftest.py](https://docs.pytest.org/en/6.2.x/fixture.html#conftest-py-sharing-fixtures-across-multiple-files). I think we'll pass on this PR.
test on py3.12


@graingert Is pytest-httpbin==2.0.0rc1 going to be promoted to stable? 

@sethmlarson hopefully, just blowing the dust off of CI https://github.com/kevin1024/pytest-httpbin/pull/76

can you get someone to look at https://github.com/postmanlabs/httpbin/pull/684/files ?

@graingert, httpbin appears to be in a pretty rough state. It was moved under @postmanlabs when they started hosting the live endpoint but has since been relatively abandoned. They kicked all the previous maintainers off the project and don't appear to have any resourcing allocated to it. I've tried reaching out a few times without any meaningful response.

Going forward, we may either need to fork it or move Requests onto a different testing dependency.

I cut https://github.com/postmanlabs/httpbin/issues/702 to see if we can get a more public response there on maintenance status.

> @graingert Is pytest-httpbin==2.0.0rc1 going to be promoted to stable?

@sethmlarson I've just pushed v2.0.0 to pypi!

Thanks, @graingert!

> They kicked all the previous maintainers off the project and don't appear to have any resourcing allocated to it. I've tried reaching out a few times without any meaningful response.

Well, I seem to have been removed as a maintainer on the github repo, but I am still a maintainer on pypi. Perhaps I can help with a new release!  

I also (occasionally) maintain https://github.com/kevin1024/pytest-httpbin which relies on httpbin, and https://github.com/kevin1024/vcrpy/

Shall we fork under [psf](https://github.com/psf/) or would you like me to fork under my own name?


Hey @kevin1024, thanks for checking in! I believe you, Ian, and myself are the three with release perms on PyPI so we can definitely go that direction. I was going to wait out a response on the issue above for a bit longer before we take on the task of forking.

I think there are also a number of changes Kenneth merge between the last release and now that may need vetting before we cut something new. If we don't hear anything back going into July, I think that's probably time to look at taking action.

Edit: Don't let me stop you from doing this sooner if you're motivated. I just wanted to minimize churn/maintenance overhead if possible.

Hello fellow maintainer!  Works for me.  Let's wait a bit longer for a response before forking.  My main motivation for forking would be to fix:

https://github.com/postmanlabs/httpbin/issues/673

So that I can fix:

https://github.com/kevin1024/pytest-httpbin/issues/72

So that I can fix:

https://github.com/kevin1024/vcrpy/issues/645

Basically: https://www.youtube.com/watch?v=AbSehcT19u0
WinError 10053 recved when send data to client

<!-- Summary. -->
I used stream=True,timeout=60 in client, and when I use iter_content, it's failed for timeout, so I try get it again, but the connection closed by it (the client). I think of that the connection should not be closed so I can use iter_content for many times in while()
 
## Expected Result

<!-- What you expected. -->

## Actual Result

<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```json
{
  "paste": "here"
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'

Recent `urllib3 2.0.2` seems not compatible with `requests`?
## Expected Result

<!-- What you expected. -->

## Actual Result

<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```json
{
  "paste": "here"
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Duplicate of #6432

You're using an outdated and long deprecated attribute from urllib3 retries in your code somewhere. This isn't a bug in requests
dependency conflict: botocore 1.29.127 requires urllib3<1.27,>=1.25.4, but you have urllib3 2.0.2 which is incompatible.

I am a new developer and am having an issue with my API gateway API. 

It returns the following error:

{"errorMessage": "Unable to import module 'app': cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/var/task/urllib3/util/ssl_.py)", "errorType": "Runtime.ImportModuleError"

and I'm pretty sure it has something to do with the dependency conflict: botocore 1.29.127 requires urllib3<1.27,>=1.25.4, but you have urllib3 2.0.2 which is incompatible.

Does this sound familiar to anyone? 

botocore needs to update the setup requirements to have urllib3<2.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.