Raibek/requests-github-issues huggingface dataset:
Wrong JSON output!

If I make a request using Python3 requests lib - the JSON output is completely wrong. But if I make the same response using other language or cURL or Postman or others.. All is good.
I think it's the body decoding trouble in Python3, because all is good using JavaScript request or others... 
 
## Expected Result

```python
print(r.json().get('offer').get('state').get('mileage')) # Should be 170000
```

## Actual Result

```python
print(r.json().get('offer').get('state').get('mileage')) # But its 199000! Wtf?
```

## Reproduction Steps

```python
import requests

url = {
    'url': 'https://apiauto.ru/1.0/offer/cars/1105235269-6558d30b?rid=225',
    'headers': {
        "X-Authorization": "Vertis ios-62ca2575df9c74b3958d118afcbb7602",
    },
}

r = requests.get(**url)
print(r.json().get('offer').get('state').get('mileage')) # Try make this the same request in e.g. Postman, the mileage should be 170000 
```

## System Information

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.6"
  },
  "platform": {
    "release": "19.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.24.0"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": false
}
```


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

You can confirm what is being returned to you by looking at `r.text`. I suspect you'll see `199000` in the response making this an issue with the server.

@sigmavirus24 No. Its not the easy-fix problem. I tried a lot of stuff - from requests headers to response raw bytes. Moreover if python version is 3.4 - everything is ok. I think there is an error in decoding, but I dont know where 

@kransk you've given no details to actually prove this is a requests bug. We rely heavily on the standard library. If there's actually an issue decoding somewhere it's likely there.
requests with Retry, unexpected kwarg allowed_methods 

I want to retry on failed POST requests (questionable pattern, I know). I tried using the `allowed_methods` kwarg in Retry and got an error.

## Expected Result

I expect the `allowed_methods` to work, and to allow for retries on POST requests. 

## Actual Result

 I tried the kwarg with both `requests.packages.urllib3.util.retry.Retry` and `urllib3.util.retry.Retry`. both resulted in the same error:
```
TypeError: __init__() got an unexpected keyword argument 'allowed_methods'
```

## Reproduction Steps

```python
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
# from urllib3.util.retry import Retry

with requests.Session() as r:
    max_retries: Retry = Retry(
        total=3,
        backoff_factor=240,  # factor * (2 ** (num_of_retries - 1))
        status_forcelist=[500, 502, 503, 504, 520, 524],
        allowed_methods=frozenset(['GET', 'POST'])
    )
    r.mount('http://', HTTPAdapter(max_retries=max_retries)) 
    r.mount('https://', HTTPAdapter(max_retries=max_retries))

    # make request
    # raise for status
    # etc

```

## System Information

Mac, Big Sur, Version 11.5.2

### relevant Pipfile details

requests = "==2.22.0"
python_version = "3.6"

### python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.12"
  },
  "platform": {
    "release": "20.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": false
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


You need to be using urllib3 1.26+ to use `allowed_methods`. I recommend upgrading your Requests and urllib3 versions.

Thank you for the prompt response! I'm using a cloud environment with locked versions, so I can't upgrade. I found the old parameter, so all is well.
how to get the dict in server ?

server:
```
app = flask.Flask(__name__)
@app.route("/http/",methods=['GET','POST'])
def tryOnce():
    params = flask.request.data.decode("utf-8")
    print(params)
```
request:
```
params={'appid': '123', 'checkin_type': '6', 'role': '1', 'stunum': '158122','keys':{"k1":'1',"k2":'2',"k4":4}}
requests.post(url3,params,headers={"Content-Type":"application/json"}).json()
```


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Don't mention pip as the reason for supporting py2

since pip no longer supports Python 2

I don't know if that is actually the official plan, but I get the impression that that's what's going to happen from this comment https://github.com/psf/requests/pull/5660#issuecomment-729236559

Hi @verhovsky, thanks for the PR. I'm not sure that comment was related to Python 2.7's deprecation. We had initially kept it as a requirement for `pip` but now that that's gone, we don't have a strong reason to keep it. It's just a matter of figuring out when we decide to formally stop Python 2.7 support. That likely won't happen before a major version bump of Requests.

If you'd like to replace the reference to `pip` with something more general like "and intend to maintain Python 2.7 support within Requests until otherwise announced.", I'm fine merging that.

@nateprewitt how about just removing that part? "We do not have immediate plans" and "[we] intend to maintain Python 2.7 support within Requests until otherwise announced" are basically saying the same thing twice.
request and response to bytearrray and back

Hi,
i want to make a encrypted tunnel and therefore I need to convert request/response to a bytearray, encrypt the bytearray, transport it, decrypt it and then recreate the request/response from bytearray.

So i need a to_bytearray() and a from_bytearray() for both, request and response


Start with #969 this library is under a feature-freeze. New features and API surface area are not being added
I hope the send method of the Session Object can support the Request Object

I will define multiple Request objects in advance, define the request data and parameters in the Request object, and then want to send the Request through the send method of session, but the send method requires preparedrequests object. 

Why not convert the Requests object into a PreparedRequests object when sending

Because there's a different API that's been explicitly defined and well-documented for this and we're not changing it. This project is under a feature-freeze and the API is quite stable. Changing this would destabilize it.

Thank you for your quick reply. Can you clarify what "a different API" is？
Stuck on starting a HTTPS connection

Hi, seems a stupid problem but i cannot figure it out.
This issue seems to happen only with this URL, but i cannot be sure.
Using a different HTTPX on my machine or Requests client on another machine works.
Although i wasn't able to replicate it on another machine, i think it's a good idea to try and resolve the problem.

URL:  https://nominatim.openstreetmap.org/search?q=Milan&format=json&polygon=0

## Expected Result
Get the body of the URL using requests.get.

## Actual Result
It gets stuck and seems to be in an infinite loop.
With logging on DEBUG, the only info it prints is:

`[14-Sep-21 23:17:30] [DEBUG] - Starting new HTTPS connection (1): nominatim.openstreetmap.org:443`

and nothing more.

## Reproduction Steps
```python
import requests
url = "https://nominatim.openstreetmap.org/search?q=Milan&format=json&polygon=0"
response = requests.get(url)
print(response.text)
```

## System Information
```
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.5"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.2"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.5"
  },
  "platform": {
    "release": "5.11.0-7633-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "101010af"
  },
  "urllib3": {
    "version": "1.26.6"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false

```

Note:
I know that issues which are hard to replicate are usually not that helpful, but i also know that reporting a bug (that is happening on my machine) is better than doing nothing.

Have you tried setting a connect timeout?

I can not reproduce this either. The DNS seems to only have 1 entry, so if that server is unreachable via your network or overloaded for some period of time, retrying might work.

I also noted that the response I got back was that I was blocked for abusing the service. Maybe it's seeing your headers (which include the `python-requests` name and version) and throttling you to disincline you from abusing the service? A different client with different `User-Agent` headers and a different machine (with a different IP) would not trigger the abuse detection.

Either way, this isn't a bug in Requests but a problem between you and OpenStreetMap
How pass urllib3 assert_hostname in Requests Adapter?

Hello,

What do I need to customize to prevent host name checking?  I am assuming it's the HTTPSConnectionPool's assert_hostname=False bit of code.

Since I cannot find any documentation on this, I was wondering if the developers can help me decide where to implement this customization?  

My guess was by inheriting from `HTTPAdapter` then modifying the `__init__()` to allow a user to pass in the assert_hostname=True/False.

Once that is done, would I overload the `verify_cert` method?  




Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here. Please be sure to include what you've tried, what you saw as a result, and what you were expecting.

@sigmavirus24 I did try stack exchange, posted code and got down voted.  I came here for help, because I can't find the documentation on where/how this should be done. 

https://stackoverflow.com/questions/69102923/python-requests-setting-assert-hostname-to-false
latest version of requests is not backwards compatible

Summary.

## Expected Result
requests(URL, headers, payload) returns payload

What you expected.

## Actual Result
Error, because new version of requests does not accept 3 parameters, only 2. It causes all python programs that use requests with 3 parameters to crash and need to be reprogrammed. 

What happened instead.

## Reproduction Steps

```python
import requests
r = requests.get(URL, headers=headers, params=payload)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.10"
  },
  "platform": {
    "release": "5.11.0-27-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.25.8"
  },
  "using_pyopenssl": false
}

```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Hi @jbrepogmailcom, could you elaborate more on what the actual issue you're hitting is. Nothing changed in v2.x for Requests around this behavior and the example works without issue. I'd recommend checking to see if you have a requests.py file that's overwriting the namespace.

Hello, I am very sorry but I corrected the script and I do not have original error message.

What I know for sure is that `r=requests.get(URL, headers=headers, payload=payload)` worked for years until few days (or weeks?) ago. It may have been some automatic linux update that I got recently, but the script stopped working. When I debugged it manually, I was getting error that `r` does not exist on next line. So I ran `r=requests.get(URL, headers, payload)` and I got error message like requests.get accepts only 2 parameters, but not 3.

When I deleted `headers` and changed command to `r=requests.get(URL, payload=payload)` then it worked normally.

(headers and payload were variables)

Jan

I tried it for second time manually with all 3 parameters and it worked, so I do not know what was the cause. I am sorry and thanks for your time. You can mark issue as invalid.
Add back logo

Fixes #5926

I’m not a web designer, so the positioning might be adjusted.

However, the logo’s restoration should take priority over bikeshedding.

Should we add it to README.md also?

I think so, and this PR does it (I updated the title to reflect that). Some people will never see the sphinx docs page, so it makes sense to have it in both locations.

Hi @flying-sheep, we already have a PR open for this in #5923. Thanks for the contribution though.

Oh, sorry, didn’t see that one!
Restore logo

@kennethreitz mentioned he had a single condition for transferring stewardship for this repository:

> When I transferred the project to the PSF, per textual agreement with @ewdurbin, my only requirement was that the logo stayed in place.

I therefore assume that the removal in #5562 (along other images) was accidental and this will swiftly rectified. I’ll do a PR.

Hi @flying-sheep, we're aware and this has already been addressed in #5923. Thanks.
Python 3.10 Readiness

Python 3.10 is just a little over a month away and we have some issues that need to be addressed in Requests. Currently our test suite relies heavily on `httpbin` which has been largely unmaintained since 2017. As a dependency, it relies on `brotlipy` which has since migrated to a new package [`brotlicffi`](https://github.com/python-hyper/brotlicffi). Given these are both C-based, we can't compile them in CI environments and rely on wheels being available.

It looks unlikely that we'll get a new wheel for `brotlipy`, so we'll need to either find a way to migrate `httpbin` to `brotlicffi`, or work out another alternative for the test suite. I still have PyPI credentials for `httpbin` to cut a new release, but there's a matter of ownership to be worked out now that it's under [Postmanlabs](https://github.com/postmanlabs/httpbin). It also appears to have several versions in the commit history that have never been published that we'll need to consider.

I was removed from `httpbin` when it was sold/transferred to Postman. Maybe it's time to fork/use something different? #5915 has the start of something we might be able to use instead

@nateprewitt I can get you a new wheel of brotlipy for Python 3.10. I've never built a wheel from a build candidate however. Is the way to go there to build a wheel from a build candidate and then once Python 3.10 is officially released upload a new wheel for 3.10 with an incremented build number? If that works for you I can take care of that step.

If we can push a new wheel from the brotlipy tag, that may be _much_ easier than the alternatives in the short term. I would assume the wheels would be virtually identical between the rc and official release. If we have issues though, we can definitely revisit updating the build number with a wheel from the official 3.10 release. Thank you!!

Uploaded `abi3` wheels for brotlipy which are compatible with Python 3.10 and all future Python 3 versions: https://twitter.com/sethmlarson/status/1433488043529359369

Now that @sethmlarson has been kind enough to get us unblocked, I think we should be set with #5928.

@sigmavirus24 If it's decided that #5915 is the future of requests' testing, I will refactor all of the tests to use it and then remove the existing test server.
Refactor rebuild_proxies to separate resolution and auth handling

This is a proposal for handling #5888 that's come out of discussion in an alternative PR (#5893). Given that we've stalled out on progress there for a few weeks, this PR is to help drive a conclusion. The approach taken here is to mitigate the incorrect behavior of stripping `Proxy-Authorization` headers off all requests sent with `Session.send`. This will not address the performance concerns from #5891 as that's a more significant problem without a clear answer.

The goal of driving this separately from #5891 is that one is an unintended breakage in basic behavior of the library and the second is an unfortunate performance regression. I'd like to address the former more quickly if possible instead of letting the problem fester.

When can we merge this @nateprewitt ?

I believe we're waiting on either some consensus with this proposal or feedback for another direction. @sigmavirus24 or @sethmlarson, do you have any thoughts on how we proceed with #5888?
Adapter uses 0 for max_retries instead of the explicit value in DEFAULT_RETRIES

Summary.

## Expected Result

I'm using a package that uses requests under the hood but doesn't support max_retry config.
So to get around that I came across the constant `DEFAULT_RETRIES`  which is used in the initialization of HttpAdapter here:
https://github.com/psf/requests/blob/main/requests/adapters.py#L117

I expected the Adapters to be initialized with the explicit value in the retry constant.

## Actual Result

I explicitly set this constant to some value in my code, which didn't make any difference to the adapters being initialized in the Session. This is due to the next line in above link which uses `0` instead of the constant to create a Retry object.

## Reproduction Steps

```python
import requests

requests.adapters.DEFAULT_RETRIES = 3

for _, adapter in requests.session().adapters:
    print(adapter.max_retries)

```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.6"
  },
  "platform": {
    "release": "20.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": false
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


This is how Python works, unfortunately. That variable isn't intended for you to change as a way of changing other package's behaviour. The signature for `__init__` has `max_retries=DEFAULT_RETRIES` which may as well be `max_retries=0`. This isn't a Requests bug. We don't advertise this as functionality.
Only one header is shown if there are multiple headers of the same name in response.headers

Summary.

## Expected Result

That the library would automatically change the name of one of the headers in order to store it in response.headers, or have some alternative way to get response headers that allows for duplicate header names.

## Actual Result

Only one of the headers is shown, due to Python not allowing duplicat keys.

## Reproduction Steps

```python
import requests

# fetch some website that sends a response with multiple headers of the same name
response = requests.get("some url")
# {"set-token": "foo", "set-token": "baz"}

# only shows one of the headers
response.headers
# {"set-token": "foo"}
```

## System Information
```
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.2"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.7"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.26.0"
  },
  "system_ssl": {
    "version": "1010107f"
  },
  "urllib3": {
    "version": "1.26.6"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```
Move from urlparse to parse_url for prepending schemes

This addresses the base problem raised in #5855 with the `urlparse` changes in Python 3.9 and potential issues in future 3.7/3.8 releases. We've avoided changing other uses of `urlparse` for now due to issues with `parse_url`'s strict validation requirements when performing parsing.

Thanks for this! I've pulled in this patch into our internal monorepo and some preliminary testing shows that it's working (I can `export https_proxy=some-proxy.example.com:3128`, like a lot of our scripts do, and things work again under Python 3.9). I'll hopefully get it deployed today and can report on whether our users can successfully switch to 3.9.

We've rolled this out and it works in prod.

Thanks for checking, @geofft! I think we just need another maintainer to give this a glance and we can look at moving the review forward.
Appropriately handle urllib3's InvalidHeader exception

#4306 raised an issue in our initial integration with urllib3's InvalidHeader implementation. We've been leaking an unwrapped exception in _some_ occasions through our HTTPAdapter. This patch should consolidate to the single Requests exception in all cases.

This may technically be considered breaking as we're changing the exception type, but Requests `InvalidHeader` was already raised through this API in the common case, so I would consider this as more of a bug.
Test verifying that ChunkedEncodingError is raised on an invalid Transfer-Encoding: chunked response.

This logic is not currently exercised anywhere in requests' tests.

For context, most errors that occur during the process of handling a request can be retried through [urllib3's Retry](https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html).   Currently, chunked encoding bypasses this, leading to a situation where users of requests are left to roll their own custom error handling for chunked encoding.  #5664 seeks to remedy that, but there are two issues:

First, the current logic around chunked encoding has limited tests, and there is concern that accepting #5664 would introduce a regression.
Second, switching from requests' internal exceptions to urllib3's exceptions is a breaking change that will require appropriate notification to existing users.

This is a first step towards addressing both issues by exercising the chunked encoding logic internally, introducing a test that will break when requests switches to using urllib3's exceptions for chunked encoding, and providing a simple example to start with for any who seek to write further tests for the chunked encoding logic.

I appreciate the feedback on what tests are needed, @nateprewitt.  I will look into request serialization tests next.
If the length of the response header is less than the actual length of the body, a read timeout occurs

If the length of the response header is less than the actual length of the body, a read timeout occurs.

Hi @sihan-bit, this is expected behavior. If the server tells us there is more data coming, Requests will wait until it arrives or until we reach our read timeout. There's no way for Requests to disambiguate between the two situations, and it likely shouldn't.
request gives different result 

When running this code on mac:

```
import requests
page = requests.get("https://legacy.haraj.com.sa/city/%D8%A7%D9%84%D8%A8%D8%A7%D8%AD%D8%A9")
print(page.text)
```
the result is different from running the same script on Linux. Any idea?

Systems:
```
Mac system info:
  Model Name:	MacBook Pro
  Model Identifier:	MacBookPro12,1
  Processor Name:	Intel Core i5
  Processor Speed:	2.7 GHz
  Number of Processors:	1
  Total Number of Cores:	2
  L2 Cache (per Core):	256 KB
  L3 Cache:	3 MB
  Memory:	8 GB
  Boot ROM Version:	182.0.0.0.0
  SMC Version (system):	2.28f7


{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "3.3.1"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.0"
  },
  "platform": {
    "release": "18.2.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "1010109f",
    "version": "20.0.1"
  },
  "requests": {
    "version": "2.23.0"
  },
  "system_ssl": {
    "version": "1010109f"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": true
}
--------------------------------------------------------------------------------------------

Linux:
    OS: Debian 
     Kernel: x86_64 Linux 4.14.238-182.422.amzn2.x86_64
   Uptime: 11d 9h 59m
   Packages: 439
  Shell: bash 5.0.3
  CPU: Intel Xeon E5-2676 v3 @ 2.4GHz
   GPU: 
   RAM: 342MiB / 983MiB


{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.4"
  },
  "platform": {
    "release": "4.14.238-182.422.amzn2.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.23.0"
  },
  "system_ssl": {
    "version": "1010104f"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": false
}

```

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here. You haven't provided enough information to help either
DOC,BUG: 404: Spanish translation links require .html

At least the Spanish translation links in the RTD widget are 404'ing.

1. https://docs.python-requests.org/en/master/user/quickstart/
2. Click 'es' under Languages in the RTD widget
3. 404

From @astrojuanlu 
https://twitter.com/juanluisback/status/1425742688595464200 :

> It is indeed available, but notice the .html at the end of the URL https://docs.python-requests.org/es/latest/user/quickstart.html
> 
> Looks like the configuration of the projects is not uniform. Would you open an issue on the requests tracker? Feel free to ping me, happy to help!

Thanks for the ping @westurner!

I see that https://github.com/requests/requests-docs-es has not received any attention in 5 years, not sure about the other translations. In any case, even if those docs are very outdated, I wonder if some config changes on the RTD or Sphinx side can at least amend the most obvious issues, like this one.

> I wonder if some config changes on the RTD or Sphinx side can at least amend the most obvious issues, like this one.

I believe the configuration is literally in the sphinx conf file in that translation repo which I do not have access to merge things to. I don't believe RTD can just fix this

What's interesting here is the pages are actually live. You can see that the prefixed domains in the sidebar work fine (i.e. https://de.python-requests.org redirects to https://docs.python-requests.org/de/latest/).

It looks like the primary docs for Requests are using `/en/master` instead of `/en/latest`. This is breaking the language navigator because the translations only exist at `/<language code>/latest`. If you navigate [here](https://docs.python-requests.org/en/latest/), the widget works correctly. This is almost certainly a bad configuration in [RTD](https://readthedocs.org/projects/requests/).

I don't think any maintainers have had access to that account since at least 2018, so we'll need @ewdurbin or @kennethreitz to help with this.

Alright, it looks like we’ve got the pointers sorted. https://docs.python-requests.org should now point to “latest” renabling the correct behavior in the nav widget.

Thanks again for the report!

Thanks!

On Sun, Sep 12, 2021, 18:43 Nate Prewitt ***@***.***> wrote:

> Alright, it looks like we’ve got the pointers sorted.
> https://docs.python-requests.org should now point to “latest” renabling
> the correct behavior in the nav widget.
>
> Thanks again for the report!
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/psf/requests/issues/5902#issuecomment-917723958>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAAMNS7H6QJREW35Q35RHYLUBUUHZANCNFSM5CAU4WOA>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
>
Cannot import requests on python 3.10 rc1

Cannot import requests on python 3.10. The problem is that python 3.10 requires urllib3 >= 1.26.

## Expected Result

I expected the module to import without errors.

## Actual Result

Import failed due to urllib3 version. You get the following callstack:
```    from requests import Session
.../site-packages/requests/__init__.py:43: in <module>
    import urllib3
.../site-packages/urllib3/__init__.py:8: in <module>
    from .connectionpool import (
.../site-packages/urllib3/connectionpool.py:29: in <module>
    from .connection import (
.../site-packages/urllib3/connection.py:39: in <module>
    from .util.ssl_ import (
.../site-packages/urllib3/util/__init__.py:3: in <module>
    from .connection import is_connection_dropped
.../site-packages/urllib3/util/connection.py:3: in <module>
    from .wait import wait_for_read
.../site-packages/urllib3/util/wait.py:1: in <module>
    from .selectors import (
.../site-packages/urllib3/util/selectors.py:14: in <module>
    from collections import namedtuple, Mapping
ImportError: cannot import name 'Mapping' from 'collections' (.../Python/3.10.0-rc.1/x64/lib/python3.10/collections/__init__.py)
```

## Reproduction Steps

```python
import requests
```

## System Information

GitHub CI ubuntu-latest with python version 3.10.0-rc.1


Hi @rkhwaja, the failure you’re receiving is for a code path that hasn’t existed in urllib3 for over 4 years, so the solution would be to update to a newer version (>1.23 at the minimum).

This isn’t an issue with Requests and we can’t raise our pin version range without breaking a large portion of the Python ecosystem. There shouldn’t be anything inhibiting you from installing the correct version of urllib3 though.
get invalid url in InvalidSchema

Requests is not accepting feature requests at this time.
Hi,
I do a get request on a url that redirects to a intent:// or market:// url which are the meaningful url in android.
After raising requests.exceptions.InvalidSchema i want to get this invalid url and do something else with it but there isn't a perfect way to do it so i do something like 
``` e.args[0].split("'")[1] ```
that is a very bad and weird way to do it.
python=3.9 and requests = "2.23.0"

> Requests is not accepting feature requests at this time.

Also this isn't cogent so it's not as if we could assist you with suggesting what to do instead. For questions, please use StackOverflow.
ProxySchemeUnknown on Python 3.9

During an upgrade from python 3.7 to 3.9 I face an error similar to the one reported on #5297.

The proxies are defined on the system level (export http_proxy and export https_proxy).

It works fine on 3.7.

## Expected Result

Requests respect the proxy

## Actual Result

Proxy causes the request fo fail

## Reproduction Steps

```python
import requests
headers={}
payload={}
url='https://www.someurl.com'
response = requests.request("POST", url, headers=headers, data = payload)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.6"
  },
  "platform": {
    "release": "20.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": false
}
```

Traceback

```
   response = requests.request("POST", url, headers=headers, data = payload)
 File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 60, in request
   return session.request(method=method, url=url, **kwargs)
 File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 533, in request
   resp = self.send(prep, **send_kwargs)
 File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 646, in send
   r = adapter.send(request, **kwargs)
 File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 412, in send
   conn = self.get_connection(request.url, proxies)
 File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 309, in get_connection
   proxy_manager = self.proxy_manager_for(proxy)
 File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 193, in proxy_manager_for
   manager = self.proxy_manager[proxy] = proxy_from_url(
 File "/usr/local/lib/python3.9/site-packages/urllib3/poolmanager.py", line 492, in proxy_from_url
   return ProxyManager(proxy_url=url, **kw)
 File "/usr/local/lib/python3.9/site-packages/urllib3/poolmanager.py", line 429, in __init__
   raise ProxySchemeUnknown(proxy.scheme)
```

Hi @iwalucas, can you please provide the proxies you’re using, or at least an identical format with sensitive info substituted? There isn’t enough information to determine if this is the same issue as #5297 or #5855.

In either case, this is almost certainly a duplicate.

@nateprewitt I think its a duplicate of #5855... 
here are the proxies:
```
HTTP_PROXY=proxy.mydomain.com:3128
HTTPS_PROXY=proxy.mydomain.com:3128
```

I've added an update in #5855, we'll track further conversations around this there. I'll resolve this as a duplicate for now. Thanks!
Clean up HTTP status code definitions

Follow-up to the conversation in issue https://github.com/psf/requests/issues/5892.

As is, this change is not a breaking change and produces the exact same `_codes` dictionary. However, I’ve separated out the official HTTP status codes from third-party status codes and added references to relevant documentation.

It would probably make sense to at least complete the list of [nginx codes](https://www.nginx.com/resources/wiki/extending/api/http/); [IIS codes](https://docs.microsoft.com/en-us/troubleshoot/iis/http-status-code) have non-integer sub-codes which may be problematic to represent here.

I think the feedback here is the same as in the issue. We don't have an interest in expanding on what we already feel was a poor decision. It encourages users to continue (or start) building on a non-core feature, which will create more user pain if it is removed/consolidated in the future.
fix: improve proxy handling

* do not rebuild proxies if they are configured
* do not remove Proxy-Authorization on initial request
* internal refactoring


This is my attempt to resolve https://github.com/psf/requests/issues/5891 as well as https://github.com/psf/requests/issues/5888.

Note: there is even more worthwhile performance tuning to be done here, most notably - `getproxies_environment` is called multiple times in this flow. Caching it might also be a good idea, but I don't know this repo too well, so please let me know if that's something we want or not.
In any case, I didn't want to expand the scope of this PR too much so I did minimal changes :)

I don't believe this refactoring is what nate or I had in mind and it breaks a public API without adding additional testing

> Note: there is even more worthwhile performance tuning to be done here, most notably - `getproxies_environment` is called multiple times in this flow. Caching it might also be a good idea, but I don't know this repo too well, so please let me know if that's something we want or not.
> In any case, I didn't want to expand the scope of this PR too much so I did minimal changes :)

As a user of the library, I found similar & also noticed that setting `trust_env` to false can also help improve the situation (caching other `trust_env` related things would also likely be of use). (https://github.com/psf/requests/pull/5496)

> I don't believe this refactoring is what nate or I had in mind and it breaks a public API without adding additional testing

Like I said, I'll try to avoid breaking the public API (I obviously don't want to introduce breaking changes). 
If we decide to proceed with this, I'll add tests (although I didn't add any functionality, so tests not breaking is the best indication that it's working as intended). 
Is there any other approach you'd recommend before I start changing the PR? Or do you want me to abandon it so you guys can take it up internally?
I thought this could be released quickly because it's a major regression for a lot of use-cases, but I completely understand if you think otherwise.

I would like to point out a solution for the proxy problem. I have been applying this solution every time I need to build an venv behind proxy.

I propose the following change at line:
https://github.com/psf/requests/blob/913880c45a3a8c3bf6b298e9c38709cd95a9c97c/requests/sessions.py#L530

Replace it by:

```python
proxies = proxies or self.proxies
```

This change will make this hierarchy `kwargs` -> `Session args` -> `environment` to be respected with no need to change anywhere else. Solving problems also with pip as discussed [here](https://github.com/pypa/pip/issues/9691#issuecomment-791608247) 

Also discussed here: https://github.com/psf/requests/pull/5735#issuecomment-792559731

Resolving this in favor of the merged #5924. Thanks again for working through this with us @omermizr!

@nateprewitt Sure thing! Thanks for addressing this 😃
Fix creating non-listening sockets in tests on some platforms

Fix the listen() invocation for the test server not to pass a backlog
value of zero.  The value of zero means no backlog which effectively
means that the socket can not accept any connections.  This does not
matter for the majority of platforms since the value is only advisory
and the platform tends to go with a bigger backlog anyway.  However,
a few platforms (e.g. alpha or riscv Linux) do take the value literally,
and therefore the tests fail since they are unable to connect to
the server.  Let Python use a 'default reasonable value' instead.

I'm sorry about the failure, it didn't occur to me to test on Python 2.
Prepare URL coerces to lower-case

https://github.com/psf/requests/blob/916d171f2bf01155f2c115159ec475e8d76a690b/requests/models.py#L369

When passing in a URL with an upper-case letter after a `-` character the host is coerced to lower case.

Python Version: 3.9.1
Requests Version: 2.26.0
OS: MacOS

Steps to reproduce
--------------------
```
from requests.models import PreparedRequest
p = PreparedRequest()
url = "http://service-A/abc"

p.prepare_url(url, None)
print(p.url)
"http://service-a/abc"
```

Notice how the `A` has been changed to `a`. This behavior is not expected. I have also submitted [this bug report](https://github.com/urllib3/urllib3/issues/2337) as the behavior arises from the use of `urllib3`.

This is 100% correct. RFC 3986 describes the part of any URI as such:

- scheme (part of the URL preceding the first `:`)
- authority (part of the URL starting with `//` before the path; this includes username/password, hostname, and port)
- path (part of the URL starting with `/`)
- query (part of the URL starting with `?`)
- fragment (part of the URL starting with `#`)

`"http://service-A/abc"` thusly gets parsed as `scheme=http, authority=service-A, path=/abc`. The host portion of the authority is to be treated case-insensitively. In other words, `service-A` is actually functionally equivalent to `service-a`.

Finally, spamming multiple projects is not appropriate, especially when you've done 0 research into how the standards define the behaviour here.

Imagine not realizing that we're getting to a point with Web where there are vastly more companies handling stuff like authorizations, header orders, and even case sensitivity per request, even in a production environment and dismissing a very real issue that is now becoming more common.

@TaquitoSlayer Your sarcastic comment isn't appreciated. I suggest you read up on existing standards for the web before claiming you know how the web works:

- [WHATWG URL](https://url.spec.whatwg.org) which is the most modern standard for URLs [calls for lowercasing the host section of URLs](https://url.spec.whatwg.org/#idna).

- [RFC 3986](https://datatracker.ietf.org/doc/html/rfc3986) which is the standard that Requests and urllib3 primarily use as a reference for their URL parsers also calls for normalization of the host component via lowercasing (quote from [Section 3.2.2](https://datatracker.ietf.org/doc/html/rfc3986#section-3.2.2)):

  > Although host is case-insensitive, producers and normalizers should use lowercase for registered names and hexadecimal addresses for the sake of uniformity, while only using uppercase letters for percent-encodings.

Anyone depending on any different behavior should expect their services to break completely as they're throwing 20+ years of standards out the window.
Set encoding for +json media type suffix responses like application/json

https://tools.ietf.org/html/rfc6839#section-3.1

While this may be correct to spec, I'm pretty hesitant with making this change. We're using this function in the hot-path for all responses and how they interact with `.text`. Currently, these all rely on chardet/charset_normalizer for encoding, anything that isn't encodable with utf-8 will break with this change. That's likely a small subset, but blast radius is hard to gauge.

I'm fine with declining this one too, detecting UTF-8 is pretty straightforward and existing path works fine.

Thanks for the PR, @scop. While you're correct this is how we _should_ function, it deviates far enough from our previous behavior that I don't think we can accept the change. Please feel free to open an issue with the request and we can add it as part of a tracking story for if we ever decide to make breaking changes.
proxy with custom hostname on proxy (possible bug ?)

Hi, I'm running zaproxy behind an nginx to perform some audit on our customer website,
infortunally I have few issues with requests proxying.

```
curl -x https://xxx.xxx.xxx -L http://zap
```

```
<head>
<title>ZAP API UI</title>
</head>
<body> ... </body>

```

but with requests (not exactly the same request):

```
req = requests.post(
    "http://zap/JSON/spider/view/scans",
    proxies={"https": settings.owasp_proxy_url},
    data={"apikey": settings.owasp_api_key.get_secret_value()},
)
```

```
requests.exceptions.ConnectionError: HTTPConnectionPool(host='zap', port=80): Max retries exceeded with url: /JSON/spider/view/scans (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb1f84c2730>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
```

Python: 3.9
requests: 2.25.1

Best regards

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

`nodename nor servname provided, or not known` means we could not resolve the name via DNS for `zap`. Requests delegates to the standard library (which delegates to the operating system) to resolve DNS. We do not use the proxy to find DNS servers so you need to ensure that your system is configured appropriately.
Unify interface definition

With this pull request I am trying to get rid of some differences in the interface definition and doc strings of `requests.api` and `requests.sessions.Session`.

In not all cases there was an obvious right choice and some discussion might be necessary.

Changing function argument definitions to attempt to match things is not something we're going to accept anytime soon, especially not before the next major version. Further, attempting to try to synchronize these docstrings is going to be a futile effort unfortunately as they're going to drift by nature of them being separate strings.
Remove legacy version check for cryptography

It's unlikely that someone would be running [a version of cryptography from before June 2016](https://pypi.org/project/cryptography/1.3.3/) these days.

Given that CentOS 7 was released in 2014 and many companies use that instead of Ubuntu or RHEL, I'm not entirely certain we can make that assumption

I think I'm also not inclined to remove this yet. People can still manually reenable PyOpenSSL in requests via urllib3, and this check would still be needed/useful in that case.

I agree with @sigmavirus24 and @nateprewitt and that makes a complete set. Thanks for opening this though, this may be something we can drop in the future when urllib3 drops pyOpenSSL/cryptography but that's a ways off.
Support for Certificate Transparency

[OpenSSL 1.1 supports checking of Signed Certificate Timestamps](https://www.openssl.org/docs/man1.1.0/man7/ct.html) (SCTs) when establishing a TLS connection, but `requests` does not provide a way to enable or configure this. Doing so would allow for a higher degree of security for users that wish to opt into this.

Requests can't take advantage until this API it's exposed via the standard library `ssl` module. I recommend opening an issue in [Python's issue tracker instead](https://bugs.python.org).