Raibek/requests-github-issues huggingface dataset:
How to get IP address of connected host after getting the response?

Hello,

I'm trying to understand how I could access information about host that the request connected to. I have situation when behind a hostname, there is a multiple IP addresses that you can connect to, so for example `x.com` can route me to `1.2.3.4` or `2.3.4.5`. I need that information after receiving response from requests. I'm using python 3.10 if that matters.

My code is as follows:

```
retry_strategy = Retry(total=1,
backoff_factor=0.5,
status_forcelist=[400, 408, 429, 500, 501, 502, 503, 504],
raise_on_status=False)
        
self.adapter = HTTPAdapter(max_retries=retry_strategy)

# other, not important code

s = Session()
s.mount('https://', self.adapter)
response = s.get(url, headers=self.headers, stream=True, timeout=60)
```

Where (if anywhere?) should I look for that kind of information? I found something about `response.raw` but the connection there is declared as None.

As described in the template, we won't be able to answer questions on this issue tracker. Please use [Stack Overflow](https://stackoverflow.com/)
Set verify=True by default as documented

The [documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification) clearly states that the default for `verify` is `True`. Via [merge_environment_settings()](https://github.com/psf/requests/blob/96b22fa18c00831656ee4b286bf1c9062459b00a/src/requests/sessions.py#L766-L771) and [merge_settings()](https://github.com/psf/requests/blob/96b22fa18c00831656ee4b286bf1c9062459b00a/src/requests/sessions.py#L61-L71) the behaviour of `None` is made the same as `True` because `self.verify` is [initialized](https://github.com/psf/requests/blob/96b22fa18c00831656ee4b286bf1c9062459b00a/src/requests/sessions.py#L424) as `True` but this is rather non-intuitive. Let's make the actual default of request() function `True` as per the documentation. 

This is (as far as the library is concerned) semantically incorrect. None here is meant as a (poor) substitute for "not specified explicitly by user". Changing this means that the user ostensibly always wants verification on. The tests that are failing show how broken this change is.
handle failure in apparent_encoding gracefully

Under certain circumstances, we have seen the content of our response come back as a `str` instead of `bytes`.  When that happens, and we use `response.text`, the library is throwing a `TypeError`.  This PR catches that exception, and lets the `text` property gracefully handle coercing the content to a `str`.

In our case, this happened when we received a `ConnectionError`, however, I believe there may be some other situations where this could occur.

Please we need way more information than you have provided before we could merge this including test cases to ensure that _if_ this is a bug that has gone unreported for nearly a decade, that it doesn't regress.

Please look at the issue template and provide as much of that detail as possible 

Nearly a month with no more explanation of what's going on here that somehow a string is being returned as content. As a result, I'm closing this.
Refresh Test PyPi Package

The Python Package on [TestPyPI has not been updated since 14 March 2015 i.e. 2.5.4.1.](https://test.pypi.org/project/requests/#history) and therefore references the deprecated `python-requests.org`

https://twitter.com/cmlh/status/1747071886029475971 is my recent Twitter thread with @kennethreitz 



No. TestPyPI generally should be evicting those after a period of time thus someone is uploading that. Further TestPyPI isn't intended to have any use besides verifying your upload works. Don't install things from there

@sigmavirus24 wrote:
> No. TestPyPI generally should be evicting those after a period of time thus someone is uploading that. Further TestPyPI isn't intended to have any use besides verifying your upload works. Don't install things from there

@dj stated during 2020 that  "[_currently there is no limit to how long releases might remain on TestPyPI_](https://stackoverflow.com/questions/62714907/how-long-does-a-python-package-stay-on-testpypi)"

Since the above is technically outside of the control of `requests` then I'll escalate this to https://pypi.org/help/#feedback
Closes #6565; Add documentation to `certs.where`.

Closes #6565 by adding documentation to `certs.where`. A wrapper was used so that the `__doc__` attribute of `certifi.where` would be preserved, and to allow for greater intellisense coverage. The second issue brought up in #6565 could not be addressed, as the CA-certs default path is only taken from `certs.where`, and can be overridden on a per-instance level of `Session` instances (for example).

This addresses only one sliver of that issue, not the core of what they're asking for. 

> This addresses only one sliver of that issue, not the core of what they're asking for. 

Yeah, this is my first PR so I'm kinda new to this. From what I was able to see, the `cacerts_loaded` function would not really be possible unless it was a method of Session, as (to my knowledge) the only way to change the bundle path is through `verify` (a Session argument) and through `merge_environment_settings` (a method of Session), meaning there would be no way to get this information globally (again, to my knowledge).
How to use multiple redundant hosts

I would like to understand whether (and how) it is possible to configure requests to target multiple redundant hosts and to use one of them that is reachable.

Following the guideline to post such help requests in SO, I asked that in https://stackoverflow.com/q/76218020/1424462, and got a response there which so far I did not get to work. So I wanted to ask the maintainers directly.

@andy-maier looking at your StackOverflow question you're saying you don't want to use things that are the standard for achieving this. You don't want to use a load balancer because you don't want a single address (which doesn't make a great deal of sense) but fine. Is there a reason you're not leveraging DNS on your internal network? That would allow you to specify `hmc-service` as a DNS name which resolves to `N` addresses. `urllib3` would then retry those until they find one that works. There's countless StackOverflow questions around using dnsmasq and other free software to do exactly this.

In short, none of this is what one would expect of a typical HTTP client library providing the level of interface we provide. And urllib3 isn't exactly the right place for this either. If you want something like this, your issue in the client library is probably the best place because whatever logic you think is "right" here probably isn't right for a large enough percentage of requests or urllib3 users.

---

Put another way, many things operate in a fashion you're describing - databases with redundant servers as one example. In those cases the client libraries for those things do not rely on the underlying TCP libraries to allow users to specify how to connect/what redundant addresses are they either:

* Specify it at the client level (e.g., tell the library you want it to behave a [certain way](https://redis-py.readthedocs.io/en/stable/clustering.html) and either tell it the redundant addresses or have the remote service tell you how to find them)
* Specify it another way outside of the library's control (e.g., DNS; if you're on an IaaS that supports IP swapping, have a healthcheck that dynamically swaps the IP address of the thing you're trying to connect to over to one of the healthy redundant hosts; etc.).

---

Finally, the answer you received will not work likely because it looks like an AI hallucination to me and the person answering likely doesn't understand what is happening. SOCKS is a proxy protocol that is almost certainly completely unrelated to what you were asking so you don't need that at all. In fact, I'm 99% certain that's an LLM hallucination because the [SOCKS Proxy Manager](https://urllib3.readthedocs.io/en/stable/reference/contrib/socks.html#urllib3.contrib.socks.SOCKSProxyManager) does not have any [`resolver` attribute whatsoever](https://github.com/urllib3/urllib3/blob/main/src/urllib3/contrib/socks.py#L175) and neither does the class it inherits from.
python requests.get() stuck

       url=links[x]
        
        try:
            response=requests.get(url,headers=HEADERLISTS,proxies={"http":proxy_lists[prox_index]},timeout=(10,10))
        except Exception as e:
            print("Error : ",e)
            sys.stderr = open('error.log', 'a+', encoding='utf-8')
            traceback.print_exc()
            sys.stderr.close()
            prox_index+=1
            time.sleep(1)
            continue
        if response.status_code==404:
            print("status code 404")
            prox_index+=1
            CURRENT_NUMBER+=1
            x+=1
            time.sleep(1)
            continue
        if response.status_code!=200:
            print("status code: ",response.status_code)
            prox_index+=1
            time.sleep(1)
            continue
        time.sleep(1)
        prox_index+=1
        CURRENT_NUMBER+=1
        x+=1
        


**This is my code. Script running After sometime response is not comming and get stuck. Next iteration is not going. Please give me solution.**

This doesn't look like a bug with requests library but with your code. Please ask in StackOverflow and using the `python-requests` tag.
docs: specify sphinx dirhtml builder, use references instead of absolutes

With the requirement of a configuration file, the default builder of `dirhtml` that RTD used to use is no longer specified. This leads to URLs ending in `.html` now, which
breaks other exisitng references.

Refs: #6603
Refs: https://docs.readthedocs.io/en/stable/config-file/v2.html#sphinx-builder
Refs: https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.dirhtml.DirectoryHTMLBuilder

One example:

https://requests.readthedocs.io/en/latest/user/advanced.html#socks
vs
https://requests--6612.org.readthedocs.build/en/6612/user/advanced/#socks

Probably should revert https://github.com/psf/requests/issues/6604 as well

Please do include a revert of that PR in here. I didn't get a chance to object to it

Revert added

Added references instead of absolutes, so these shouldn't ever need to be changed despite the builder chosen.
Unexpected `UnicodeError` on some URL

Seems that `requests` haven't handle well some url which `curl` handle quite well.

## Expected Result

Should work.

## Actual Result

Crashed as `UnicodeError: label empty or too long`

## Reproduction Steps

```python
import requests

requests.get("https://www.elyricsworld.com/go.php?url=http://894763313no67.%D1%86%D0%B5%D0%BD%D1%82%D1%80-%D1%81%D1%82%D1%80%D0%B0%D1%85%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D1%8F-%D0%BE%D0%BD%D0%BB%D0%B0%D0%B9%D0%BD.%D1%80%D1%84")
```

## System Information

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.2"
  },
  "cryptography": {
    "version": "41.0.5"
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.13"
  },
  "platform": {
    "release": "21.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "30100040",
    "version": "23.2.0"
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30200000"
  },
  "urllib3": {
    "version": "1.26.18"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": true
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


I want to work on this, it would be great if anyone assign this to me!

Duplicate of #6026 #4187 and many others
I don't think the name of this function is appropriate, because the return value of this function is a ConnectionPool object, and the first sentence of the comment is also wrong, which makes it a huge difficulty for me to read!!

    def get_connection(self, url, proxies=None):
        """Returns a urllib3 connection for the given URL. This should not be
        called from user code, and is only exposed for use when subclassing the
        :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.

        :param url: The URL to connect to.
        :param proxies: (optional) A Requests-style dictionary of proxies used on this request.
        :rtype: urllib3.ConnectionPool



adapters.HTTPAdapter.get_connection

You're right. As documented we are returning a connection pool which behaves almost identically to a connection. Alas, that method and that contract has existed for over a decade at this point, so we will not be changing it
refc: remove the if else statement and add redable format.

`
data = [] if data is None else data
`
This statement is little bit complex to read. It's not readable.

`
data = [] or data
`
In the above statement if data is `None` then data store list but if there anything found in the data so, data  some store value. It is readable format.

These are not semantically equivalent statements. This would be a breaking change so we won't be able to accept it.
Links at docs site return 404

At https://requests.readthedocs.io/en/latest/ all links under "Usefull links" lead to 404 page

## Expected Result

Open corresponding docs page

## Actual Result

404 page

## Reproduction Steps

- Open https://requests.readthedocs.io/en/latest/
- Click on "[Quickstart](https://requests.readthedocs.io/en/latest/user/quickstart/)"

## System Information
Not applicable


Looks like `.html` at the end of link is what's missing

Working on it.

Resolved in #6605.
the first response history url is incorrect

<!-- Summary. -->
The first url returned from response.url is different than the url passed to requests.get
## Expected Result

<!-- What you expected. -->
I expected the first history url to be that same as passed into requests.get

```
Redirect #1
URL: http://google.com
```

## Actual Result

The first redirected  url is always the same as the the first response.headers.get('Location')
```python resp.py         
requests.__version__ 2.31.0
initial url: http://google.com

Redirect #1
URL: http://google.com/
Status Code: 301
Redirect: http://www.google.com/

Redirect #2
URL: http://www.google.com/
Status Code: 302
Redirect: https://www.google.com/?gws_rd=ssl

URL: https://www.google.com/?gws_rd=ssl
Status Code: 200
Redirect: None
```
<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

url = "http://google.com"
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"
}
response = requests.get(url, headers=headers, allow_redirects=True)
print(f"requests.__version__ {requests.__version__}")
print(f"initial url: {url}\n")

for index, resp in enumerate(response.history, start=1):
    print(f"Redirect #{index}")
    print(f"URL: {resp.url}")
    print(f"Status Code: {resp.status_code}")
    print(f"Redirect: {resp.headers.get('Location')}\n")

print(f"URL: {response.url}")
print(f"Status Code: {response.status_code}")
print(f"Redirect: {response.headers.get('Location')}")

```

## System Information

 {
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.2"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.12"
  },
  "platform": {
    "release": "22.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "1010115f"
  },
  "urllib3": {
    "version": "2.0.7"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}



Hi @xtremebeing,

I'm assuming you're referring to the trailing `/` in `https://google.com/`? That's a standard semantic of url normalization, and is identical to what you provided. This should be treated the same by a properly functioning server.

>  In general, a URI that uses the generic syntax for authority with an
   empty path should be normalized to a path of "/".

\- [RFC 3986 6.2.3](https://datatracker.ietf.org/doc/html/rfc3986#section-6.2.3)

You may try setting the URL again with the [PreparedRequest workflow](https://requests.readthedocs.io/en/latest/user/advanced.html#prepared-requests), but my hunch without having looked is this is happening in urllib3. If that's the case, this won't be adjustable.
Add note on connection timeout being larger than specified. Fix #5773

[As requested](https://github.com/psf/requests/issues/5773#issuecomment-799787382), here's my take on a PR to improve documentation. I've tried to incorporate the [suggestions made by @sigmavirus24 at a previous PR](https://github.com/psf/requests/pull/5776#discussion_r596043195), by shifting the focus from "IPv4+IPv6" particular case to a more general (and correct) "multiple IPs" scenario.

If approved, this PR will also fix #5760, and supersede PRs #5776 and #6597

Is there anything I can do on my part to fix the build errors, or are the docs test suite broken?

Thank you @MestreLion !
affected urllib3 and certifi packages (older version keeping in setup.py file)

<!-- Summary. --> Please upgrade the urllib3 & certifi(2023.7.22) libraries.
use urllib3  >=1.26.18
certifi >= 2023.7.22

## Expected Result
 Detecting
<!-- What you expected. -->  detecting as vulnerabilites
https://www.cve.org/CVERecord?id=CVE-2023-45803

## Actual Result

<!-- What happened instead. -->

Upgrade to a patched version of urllib3 (v1.26.18 or v2.0.7)
Disable redirects for services that you aren't expecting to respond with redirects with redirects=False.
Disable automatic redirects with redirects=False and handle 303 redirects manually by stripping the HTTP request body.
## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```json
{
  "paste": "here"
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Both of those versions are compatible with the current version of requests. There's nothing to do here
Add Explanation of why Timeout is Doubled.

## Added Explanation for why Timeout is Doubled.
- In this Pull Request I have Tried to Solve #5773 .

./docs/user/advanced.rst
```diff
+ **You need to be careful when using the timeout argument.**
+ If the specified domain has multiple IP addresses, `urllib3`_ will continue to try another IP address when the specified connection timeout is reached. Also, the connection timeout is applied for each attempt.
+ Requests are affected by this because you are using `urllib3`_.
+ This phenomenon can wait longer than you expect. Maybe it's the time that doubled the connection timeout.
+ Of course there may be more.
+ You can also consider extreme solutions, such as having the specified domain name return only one IP address.
+ For example, if DNS returns both IPv4 DNS records (A) and IPv6 DNS records (AAAA), patch it so that it does not return IPv6.
+ However, new issues may arise.
```

If it Suits Your Needs Consider Merging this Pull Request.

Fixed by #6600
Update close-issues.yml

I noticed the auto-labeling was working but not the auto-closing. Looking at recent actions runs I see that we need to specify the token even if we're not giving our own special token. See https://github.com/psf/requests/actions/runs/7057701782/job/19211845073#step:2:13 for additional context, namely

```
 gh: To use GitHub CLI in a GitHub Actions workflow, set the GH_TOKEN environment variable. Example:
  env:
    GH_TOKEN: ${{ github.token }}
```
Is it possible to send async generator as `data` argument of request method?

I'm using `requests` module in Python and I want to send data in request streaming way. so I applied chunked-encoded request according to [this document](https://requests.readthedocs.io/en/latest/user/advanced/#chunk-encoded-requests). 

However, the data to send is provided with another async method. Honestly speaking, I also use Sanic framework, and I read the stream data from `await sanic.request.stream.read()` and I want to relay the streaming data to `request` method of `requests` module.

```
async def generate_from_sanic_request(request):
  while True:
    chunk = await request.stream.read()
    if chunk is None:
      break
    yield chunk

...
requests.request(
  method="POST",
  url="my-own-url",
  data=generate_from_sanic_request, # here is the point of this question.
)
...
```
The document says that I can use the non-async generator function as the `data` argument. However, when I use an async generator there, `requests.request` cannot handle the generator.

I want to do this in a memory-efficient way, so I want to bypass the received streaming request body to requests.request method in one-by-one way.

Can I get any help to solve this problem?

Thanks in advance.

It is not possible to use an async generator with requests. For help with your code, please use [StackOverflow](https://stackoverflow.com).
Limit werkzeug<3 for CI only, fixes #6541

I believe this change will fix CI run failures, such as [these](https://github.com/psf/requests/actions/runs/7038290331).

Hi @bruceadams, thanks for the PR. This is already fixed in psf/httpbin#29. Once that's released the test suite will function normally again. I don't think we'd merge this when the long term fix is already done.

Cool! This certainly felt like an upstream problem and I'd figured out that werkzeug was being pulled in via httpbin. I hadn't gone to look at either werkzeug or httpbin to see if they might be dealing with the problem. I'm glad that it's getting fixed!
Perform correct length measurement for Python strings in HTTP request `data`

This PR addresses the problem detailed in #6586. The fix involves measuring the encoded byte-length of the string as per [RFC 9910](https://www.rfc-editor.org/rfc/rfc9110#field.content-length).

This PR includes two tests that verify the fix addresses the problem. Additionally, all tests pass locally, so no regressions have occurred.

Closing as a duplicate of #6589
Cannot run tests locally

<!-- Summary. -->
I am trying to install the repository and run all the tests as recommended by the [Contribution Instructions](https://requests.readthedocs.io/en/latest/dev/contributing/). I am getting the following error when I perform the following. Importantly, what are the official instructions for installing the dev dependencies for `requests`? I cannot find an official instruction so I'm using the `Makefile`.

## Expected Result

```
Pytest to run successfully.
```

## Actual Result
```
ImportError: cannot import name 'parse_authorization_header' from 'werkzeug.http' (/requests/venv/lib/python3.10/site-packages/werkzeug/http.py)
```
## Reproduction Steps

```
cd requests
python -m venv ./myvenv
make init
pytest tests/
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.2"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.11"
  },
  "platform": {
    "release": "22.4.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "1010114f"
  },
  "urllib3": {
    "version": "2.1.0"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Based on this commit in the `httpbin` repo, it looks like the werkzeug version is constrained, but the dev dependencies do not constrain it to <2.3. Werkzeug removed the `parse_authorization_headers` func in v3.x.

https://github.com/psf/httpbin/commit/496d0b96c3a8c843803e18ac6d8a379b94b4f215

This is a duplicate of other closed issues. Httpbin needs to be released
Status Code 425 named incorrectly

HTTP status code 425 is defined as "too early". While the defining RFC defines the code in the context of TLS, I regularly find APIs that use this status code to indicate to a client that an asynchronous job that has been started is still being processed, so the result can not yet be provided.

requests.status_codes however doesn't know the reason "too early", and instead calls 425 "unordered_collection" and "unordered". I believe, this is a bug.

The defendent: https://github.com/psf/requests/blob/main/src/requests/status_codes.py#L82

I call to the witness stand:
* https://datatracker.ietf.org/doc/html/rfc8470
* https://http.cat/425
* https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#4xx_client_errors

## Expected Result
>>> requests.codes.get("TOO_EARLY")
425
## Actual Result
>>> requests.codes.get("TOO_EARLY")

## Reproduction Steps

```python
import requests
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.6"
  },
  "platform": {
    "release": "6.6.1-arch1-1",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30100040"
  },
  "urllib3": {
    "version": "2.0.7"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

```


I would merge a change to add that to the type definition. Keep in mind unordered was in wide use before that rfc though which is why it has that definition. I doubt anyone is using it from us but we can't remove it. 

Implementing this in an upcoming PR.

Here is a PR with relevant changes that I believe are backwards compatible and generalizable. Could you take a look and give your initial thoughts? @nateprewitt

Resolving now that #6592 is merged.
Fix broken links in docs

Hi,

I noticed a broken link to rfc2988 in the [timeout section of the docs](https://requests.readthedocs.io/en/latest/user/advanced/#timeouts) and thought I would fix it.

While doing that, I thought I would run the sphinx `linkcheck` builder to check for other broken links and fix them.

There were a few, some where I've 'fixed'/updated the link are hopefully uncontroversial. 

Some are more controversial - removing sections written by the original author of requests that now have dead links, removing talks with dead links to slides, removing people's now dead GitHub account links. I wasn't sure what the right thing to do here was, but I went with my intuition, and kept commits pretty discrete so I can drop/amend them if the maintainers want me to do something different. If some are controversial enough to slow down the PR, I can always pull out the uncontroversial ones into their own PR for faster merging.

@sigmavirus24 Thanks for your comments - I think this is ready to look at again whenever you have time. No rush from my side, I'm sure you're busy, just didn't want it to get stuck in process if you thought I was still working on it!

Thanks @Jamim for the review!

It looks like GitHub still wants an approval from a reviewer with write access - I'm guessing you don't have write access (just checking in case something has gone wrong with the process)?

I've just rebased against main so this is fresh and to give the GitHub checks another change to pass - looks like the `build` checks for any python >3.7 are failing with this error:

```
ImportError: cannot import name 'parse_authorization_header' from 'werkzeug.http' (/opt/hostedtoolcache/Python/3.9.18/x64/lib/python3.9/site-packages/werkzeug/http.py)
```

But the 3.7 ones pass, and this error also occurred in the checks for the latest commit to main, so this seems to be an unrelated `wekzeug` dependency-related issue to me.



> I'm guessing you don't have write access

That's true. I just wanted to support your contribution 😃

@EFord36 would you be willing to rebase/resolve the conflicts here?

> @EFord36 would you be willing to rebase/resolve the conflicts here?

Hi,

I've done that now, thanks for the reminder!

Thank you!
"Recommended Packages and Extensions" page asyncio recommendations are outdated

While looking for an asyncio compatible recommendation for requests, I stumbled on the [documentation](https://requests.readthedocs.io/en/latest/community/recommended/) provided by the project. It recommends Request-Threads, which does not seem to be actively developed (no new changes/releases in the past 5 years). txrequests is an alternative Twisted-based package that seems to be more recommended these days: https://pypi.org/project/txrequests/ .

Moreover, requests.Async is not mentioned; this support was moved to https://pypi.org/project/gevent-requests/ (mentioned in History.md). [1]

1. Note to other readers: gevent-requests is the library that requires gevent/greenlet and achieves asynchronous processing via green threads: it doesn't achieve this via native python asyncio coroutines.
fix docstring typo: a -> as

Hi,

I noticed a typo in this docstring, thought I would write a quick PR to fix it.

As far as I can see, this docstring doesn't get added to the rendered html docs on https://requests.readthedocs.io/en/latest/ - they are only visible when viewing the source code.

I've not bothered running tests etc. given its such a small change, but please let me know if there's more you want me to do for this.

Thanks for requests!
Call url.lower() once inside get_adapter

This is a re-proposal of https://github.com/psf/requests/pull/5497 three years later. The typical use-case for adaptaters is to create one and then mount it for both `http://` and `https://`. When using https:// URLs, this result in two calls to `url.lower()`, whereas with the optimization by @dbaxa this is computed only once.
IMHO the `prefix.lower()` thing should also be computed only once when we add the adapter, but I don’t know if this could break some code.

@bfontaine the real issue here is that in the past reasonable, low risk changes like this lead to nonsensical "optimization" PRs that do break things and otherwise look low risk. As with this PR there are no benchmarks showing the actual improvement. And we have a dependency preventing our CI from being reliable. Finally, in general small improvements like this if we start accepting them add up and obfuscate the history of the project which given it's complexity is helpful for Nate and I in our very limited time

Understood. From my quick benchmark the performance gain would be of 0.05s for 1M requests, which is nothing. Sorry for the noise.
Weird behaviour with https and proxy authentication

<!-- Summary. -->

requests.get is getting weird with some combinations of dictionary keys and proxy authentication.

## Expected Result
```python
print(requests.get('https://google.com', proxies={"https" : 'https://user:pass@ip:port'}))
```
Output:
`<Response [200]>`
<!-- What you expected. -->

## Actual Result
Doesn't run? (Maybe timeout)
But it somehow fixes itself with these changes
```python
print(requests.get('https://google.com', proxies={"http" : 'http://user:pass@ip:port'}))
```

Output:
`<Response [200]>`
<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

print(requests.get('https://google.com', proxies={"https" : 'https://user:pass@ip:port'}))

print(requests.get('https://google.com', proxies={"http" : 'http://user:pass@ip:port'}))
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.11"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "1010114f"
  },
  "urllib3": {
    "version": "1.26.9"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
````
    $ python --version
    Python 3.10.11



<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


proxies im using are https capable

misclick

its most likely that your proxies only support http protocol, it should work the way you have it, so its probably proxy issues.

> proxies im using are https capable



> its most likely that your proxies only support http protocol, it should work the way you have it, so its probably proxy issues.
Improper URL serialization of boolean GET parameters

<!-- Summary. -->

Constructing a GET request with a boolean parameter `foo` set to `True` results in `&foo=True` being appended to the URL path, instead of just `&foo`. This makes it inoperable with some APIs (such as the [NIST CVE API](https://nvd.nist.gov/developers/vulnerabilities)) that return an error when a string value is set for a boolean GET parameter.

Example:
```py
>>> import requests
>>> API_URL = "https://services.nvd.nist.gov/rest/json/cves/2.0"
>>> res = requests.get(API_URL, params={"noRejected": True})
>>> res
<Response [404]>
>>> res.headers["message"]
'noRejected parameter cannot have a value.'
```

It's a mystery to me how such a basic feature of the de-facto standard library for making HTTP requests in Python hasn't yet been addressed and I would greatly appreciate if it could be fixed, or at least if a special value/object was added that could be passed as a value in `params` to produce the expected result. Working around this limitation by constructing the URL with all its parameters myself as a programmer does not inspire confidence in the security and correctness of the code.

## Expected Result

```py
>>> res.request.path_url
'/rest/json/cves/2.0?noRejected'
```

## Actual Result

```py
>>> res.request.path_url
'/rest/json/cves/2.0?noRejected=True'
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "5.1.0"
  },
  "charset_normalizer": {
    "version": "3.0.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.11.2"
  },
  "platform": {
    "release": "6.1.0-13-amd64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "300000b0"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": false
}
```

In the future, please search **closed and** open issues before creating new ones that are duplicates.

> In the future, please search **closed and** open issues before creating new ones that are duplicates.

What makes you think I haven't done that? Instead of lecturing me, the minimum you can do is provide a link or an ID. @sigmavirus24
Possible regression with how MaxRetryError is handled

Since requests does not retry failed connections, a somewhat confusion exception is encountered by doing something like this.
```
>>> requests.get('http://localhost:12121')
...
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=12121): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000029B47D92CF8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',))
```

I found in my searches about this exception that years ago this issue was reported and resolved in urllib3 by adding the ability to specify `retries=False` in the request: https://github.com/psf/requests/issues/1198#issuecomment-34582901

The result of the change back then gave a nicer exception output: https://github.com/psf/requests/issues/1198#issuecomment-57952852

Is this a regression, and if possible, could there once again be a way to get a cleaner exception message for this case? Thank you.

There is no regression here. That thread allows that for retries=False but requests always configures retries by default. You'll want to customize an HTTPAdapter to specify different behavior if you don't want to see a MaxRetryError in the exception message 

So how many retries does requests configure? The docs say requests does not do any unless you use Session() https://requests.readthedocs.io/en/latest/user/advanced/#example-automatic-retries

Also I tried configuring HTTPAdapter with max_retries=0 or max_retries=False and the exception did not change. Am I doing something incorrectly?

```
from requests import Session
from requests.adapters import HTTPAdapter

s = Session()
a = HTTPAdapter(max_retries=0)
s.mount('http://', a)
s.get('http://localhost:12121')
```
The connection does not close automatically; it is not possible to close the session manually in recent versions of requests.

## When it worked correctly.
Configuration:
OS debian 10; 
python3.8; 
requests=2.28.0; 
urllib=1.26.14

1. 
`$python`
```python
import requests
r = requests.get("https://fantlab.ru")
# After processing the request, the connection was automatically closed with the status TIME_WAIT.
```
2.
`$python`
```python
import requests
s = requests.Session()
r = s.get("https://fantlab.ru")
# The connection is held until it is closed manually, the status is ESTABLISHED

s.close() #The connection is closed successfully, status is TIME_WAIT
```

## Doesn't work correctly. Update libraries
requests=2.31.0;
urllib3=2.1.0.

1.
`$python`
```python
import requests
r = requests.get("https://fantlab.ru")
#The behavior is now different and the connection is not closed after it has been processed, the status is ESTABLISHED
r.close() #Trying to close the connection manually also doesn't work
#Console exit (ctrl+d) successfully closes connection, status is TIME_WAIT
```

2.
`$python`
```python
import requests
s = requests.Session()
r = s.get("https://fantlab.ru")
# The connection is held until it is closed manually, the status is ESTABLISHED

s.close()
r.close() 
#forced attempt to close connection manually does not work, status is ESTABLISHED

#The connection is closed only when the console exits.
```

**Thus, after library updates, the connection cannot be managed manually (closed), it stopped working.**

Hi @snooppr, can you help limit the scope of changes? Does this combination of dependencies produce the issue? If not, this is likely related to urllib3 2.x.

```
requests=2.31.0
urllib=1.26.14
```

@nateprewitt HI!

Yes, I checked, this problem occurs due to urllib3.
In configuration:
requests=2.31.0
urllib3=1.26.14
The connection is manually closed as usual.

Should we open issues in urllib3 or will the requests developers do it? (requests is completely dependent on urllib3)

upd.
>related to urllib3 2.x.

This problem also occurs due to urllib3.
In configuration:
requests=2.31.0
urllib=**1.26.18**
The connection is not closed either manually or after a request.

Rewrite the documentation, urllib3 behavior has changed.  And it will confuse users of requests.

> close()
Closes all adapters and as such the session

[doc](https://docs.python-requests.org/en/latest/api/#request-sessions)
Document that Response.iter_lines is broken and should be avoided

The `Response.iter_lines` method is seriously broken (it inserts fake blank lines in unpredictable places.)

The earliest report of this I've found is pull #2431 (January 2015).

The behavior was apparently fixed in the 3.0.0 branch, by pulls #3923 and #3984.  (I think #3923 fixes the `delimiter!=None` case and #3984 fixes the `delimiter=None` case.  But I haven't tested it.)

The problem was raised again in issues #3980, #4121, and #5540.

Pull #4629 attempted to partially fix the issue in the master branch, and was rejected.

Personally, I am skeptical that there is any benefit in preserving this broken behavior.  But folks who know better than I do have said that it needs to be maintained.

As long as this *isn't* fixed, people using the library should be informed that method is broken and they shouldn't use it.
HTTPSConnectionPool(port=443): Max retries exceeded with url:(Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1129)')))

I having a long lasting problem with my request connection is getting shutdown using Cloud Run. I couldnt solve it yet so here I am to get support.

Here is the full error:
```
Error occurred: HTTPSConnectionPool(host='google.serper.dev', port=443): Max retries exceeded with url: /search (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1129)')))
```

Here is how I use request:

```
def serper_search(self, arguments):
        query = arguments["q"]
        url = "https://google.serper.dev/search"
        payload = json.dumps({"q": query})
        headers = {
            'X-API-KEY': self.api_key,
            'Content-Type': 'application/json'
        }

        try:
            response = requests.request(
                "POST", url, headers=headers, data=payload)
            # Will raise an HTTPError if the HTTP request returned an unsuccessful status code
            response.raise_for_status()


            # Parse the response and make it ready 
            parsed_response = self.parse_response(response, number_of_results=3)
            return parsed_response  # Assuming the response content is in JSON format
        except requests.exceptions.HTTPError as http_err:
            logger.error(f"HTTP error occurred: {http_err}")
            return None
        except Exception as e:
            logger.error(f"Error occurred: {e}")
            return None

```

This only happens in Cloud Run although the error should be on those side, there must a setting to override this issue on the request side as well. 


**I tried:**
- verify = False

didnt help.

**More info:**
Also I use this with fastapi and adding as a backgroundTask.
fastapi==0.103.1
requests==2.31.0




Saw your comment in:
https://github.com/psf/requests/issues/6429

Just wanted to remind anyone reading that Cloud Run does not allocate CPU outside request unless the revision is explicitly configured to do so; here are the docs:

https://cloud.google.com/run/docs/configuring/cpu-allocation

and here's a sample:

![image](https://github.com/psf/requests/assets/107634081/21afe839-5943-47d0-b2fe-7f15fa9efaaa)
Bug/6294 zero bytes files are chunked

See bug #6294 for original bug report.

This PR provides a possible fix for this, reverting the change in #2896.

The change in #2896 definitely causes some bad side effects, since not all web servers handle "chunked" transfer encoding (i.e. some don't handle it, some don't handle it well).

### Problem demonstration

Given a simple bottle application:
```python
from bottle import run, put, request, default_app

app = default_app()

@app.put("/")
def receive_file():
    upload = request.body.read()
    for head, val in request.headers.items():
        print("%-30s:\t%s" % (head, val))
    clength = request.headers.get("Content-Length", "not-set")
    cencoding = request.headers.get("Transfer-Encoding", "not-set")
    return { 'size': len(upload), 'content-length': clength, 'transfer-encoding': cencoding }

if __name__ == '__main__':
    app.run(host='localhost', port=8880)
```

And a simple requests script:
```python
import requests
import io
empty_obj = io.BytesIO(b'')
resp = requests.put('http://localhost:8880/', data=empty_obj)
print(resp.status_code)
if resp.status_code == 200:
    print(resp.json())
```

## Working Scenario

```
$ python server.py
Bottle v0.12.25 server starting up (using WSGIRefServer())...
Listening on http://localhost:8880/
Hit Ctrl-C to quit.

Content-Length                :	
Content-Type                  :	text/plain
Host                          :	localhost:8880
User-Agent                    :	python-requests/2.31.0
Accept-Encoding               :	gzip, deflate, br
Accept                        :	*/*
Connection                    :	keep-alive
Transfer-Encoding             :	chunked
127.0.0.1 - - [03/Nov/2023 15:51:51] "PUT / HTTP/1.1" 200 65
```
Test script output:
```
200
{'size': 0, 'content-length': '', 'transfer-encoding': 'chunked'}
```

## Failing Scenario
```
$ gunicorn -b localhost:8880 server:app
[2023-11-03 15:52:05 +0100] [1326049] [INFO] Starting gunicorn 21.2.0
[2023-11-03 15:52:05 +0100] [1326049] [INFO] Listening at: http://127.0.0.1:8880 (1326049)
[2023-11-03 15:52:05 +0100] [1326049] [INFO] Using worker: sync
[2023-11-03 15:52:05 +0100] [1326050] [INFO] Booting worker with pid: 1326050
```
Test script output:
```
400
```

### The Original Issue

The change causing this problem in the first place was due to failing to upload data read from a subprocess pipe.

This patch adds a check to `super_len` that avoids returning length 0 for file handles that are not regular files.

closing & reopening to trigger a new build

@eivindt would you be willing to fix the lint errors?

Give me a couple of days and I'll fix them.

Sorry for the delay, there were some windows specific issues I couldn't easily fix without a windows pc.
Requests doesn't work on emscripten / webassembly

This is maybe verging on a feature request, I'm not sure...

Webassembly with emscripten is now an official python platform. However because it runs inside a browser sandbox environment, networking doesn't work normally, and hence nothing in requests works at all. However there is easy access to http(s) requests via the environment it is running in.

Right now, we have a monkeypatch module which patches a new BaseAdapter subclass into `Session.__init__` (see link below).

https://github.com/koenvo/pyodide-http/blob/main/pyodide_http/_requests.py

It isn't masses of code, because most of the stuff that is handled by lower level things in requests is just passed directly off to the browser fetch. 

It would be really nice if a version of this adapter could be added to requests (and the emscripten platform added to requests CI also, so we can see how much functionality works and how much doesn't.)

I did a bunch of work on the monkeypatch code above, so I'd be happy to help on this, but obviously I don't want to spend the time doing a PR for it if it isn't going to have a chance of being merged, hence opening this issue to discuss.

Hey @joemarshall, thanks for opening this issue. We actually have a very similar issue on urllib3 (the underlying library of Requests) that has a bounty available. If you wanted to contribute there then many libraries would gain the ability to use WASM Python.

https://github.com/urllib3/urllib3/issues/2951

I'm going to close this one now that https://github.com/urllib3/urllib3/issues/2951 is closed, Emscripten support isn't stable yet but should be at least possible once urllib3 is released.