Raibek/requests-github-issues huggingface dataset:
apparent_encoding should be cached since chardet can be slow

We have some scraper code that sometimes gets back PDFs and other times gets back HTML. Today we learned that if you access r.text in a large-ish PDF (40MB), chardet is called, which uses a lot of CPU (and a ton of memory):

```
r = requests.get(some_url)
r.text
```

That's more or less fine (best not to try to get the text of a PDF this way), but if you access `r.text` more than once, chardet gets run over and over. 

We have code like this that performs horribly:

```
r = requests.get(some_url)
if bad_text in r.text:
    continue
if other_bad_text in r.text:
    continue
# ...many more tests...
```

When you access `r.text`, it checks if the encoding can come from the HTTP headers. If not, it runs the `apparent_encoding` property, which looks like:

```
    @property
    def apparent_encoding(self):
        """The apparent encoding, provided by the charset_normalizer or chardet libraries."""
        return chardet.detect(self.content)["encoding"]
```

I think that property should probably be cached since it's slow, so that repeated calls to r.text don't hurt so badly.

## Expected Result

I expected the calls to the text property to only calculate the encoding once per request.

## Actual Result

Each call to the text property re-calculates the encoding, which is slow and uses a lot of memory (this is probably a bug in chardet, but it uses hundreds of MB on a 40MB PDF right now).


## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "5.0.0"
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": "36.0.2"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.7"
  },
  "platform": {
    "release": "5.4.209-116.363.amzn2.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "101010ef",
    "version": "20.0.1"
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "101010ef"
  },
  "urllib3": {
    "version": "1.26.5"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}
```


You can do this today by setting `r.encoding = r.apparent_encoding` 

Yeah, or by overriding the `text` or `apparent_encoding` methods. All of those approaches feel like workarounds though, right — or are you suggesting that the `apparent_encoding` method should set the `encoding` property when it finishes, and that could serve as the cache?

One other thought, I noticed that the `content` method actually has caching. Maybe instead of caching the `apparent_encoding` property, the `text` property should be cached to parallel how the the `content` one is.

No, I'm saying that while character detection can be slow, it usually isn't. If it's speed is a problem, the documented way if avoiding it is to set the encoding attribute yourself. This is you want it cached, set the encoding attribute yourself. It's easy and available today
requests.delete does not support body parameter

Requests is not accepting feature requests at this time.
I just figured out that delete method does not support data payload
I have a requirement in my project that accepts json data in delete 
It actually supports in other http clients like axios for frontend but not this one when I am using with Flask
Can you please add this feature ASAP.

Per HTTP RFC's, DELETE with a body has no defined semantics and should be avoided which is the same reasoning we don't support it natively. You can always pass `"DELETE"` and a `body` to the `request()` method yourself.
authentication via netrc has a different outcome than auth argument

NASA Earthdata uses basic authentication and cookies to respond to GET requests. When I use a `Session()` in combination with a `~/.netrc` file, the session object holds the cookies and everything works. When I use a `Session()` but supply my credentials with the `auth` argument, the session cannot hold its cookies. Authentication failure results. It would be great if the `auth` argument could do whatever the session does for netrc authentication.

To reproduce, you will have to have to follow these instructions for a free account: https://disc.gsfc.nasa.gov/earthdata-login

## Expected Result

```
<Response [200]>
```
## Actual Result

```
<Response [401]>
```
## Reproduction Steps

Have a NASA Earthdata account with `<USERNAME>` and `<PASSWORD>` stored in a text file at `~/.netrc` with permissions `-rw-------` and containing the line:
```
machine urs.earthdata.nasa.gov login <USERNAME> password <PASSWORD>
```

The following returns the expected result

```python
from requests import Session

resource = 'https://reason.gesdisc.eosdis.nasa.gov/data/nobm/NOBM_DAY.R2017/2015/day20151201.R2017.nc4'
with Session() as session:
    response = session.get(resource)
response
```

The following returns the actual result
```python
from requests import Session

resource = 'https://reason.gesdisc.eosdis.nasa.gov/data/nobm/NOBM_DAY.R2017/2015/day20151201.R2017.nc4'
with Session() as session:
    response = session.get(resource, auth=('<USERNAME>', '<PASSWORD>'))
response
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.1.1"
  },
  "cryptography": {
    "version": "38.0.1"
  },
  "idna": {
    "version": "3.4"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.6"
  },
  "platform": {
    "release": "21.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "30000050",
    "version": "22.0.0"
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010111f"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": true
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Maybe HTTPBasicAuth helps.. Try: 

from requests.auth import HTTPBasicAuth
auth = HTTPBasicAuth(username, password


Ah, my problem was actually #2949.

For future reference, here is a workaround. I still think it's better to use a netrc file, but if you really want to avoid it...

```python
class EarthdataSession(Session):

    def should_strip_auth(self, old_url, new_url):
        parsed_url = urlparse(new_url)
        if parsed_url.hostname == "urs.earthdata.nasa.gov":
            return False
        else:
            return super().should_strip_auth(old_url, new_url)
```

Use in place of `Session`.
setup.py

<!-- Summary. -->

## This bug does not affect usage, but it does exist
the bug code is in the "setup.py" file, line 78
here is code:
```python
with open(os.path.join(here, "requests", "__version__.py"), "r", "utf-8") as f:
    exec(f.read(), about)
```
## It's an issue about function open()
I got this from Python Documentation
```python
open(file, mode='r', buffering=- 1, encoding=None, errors=None, newline=None, closefd=True, opener=None)
```
The argument ```"utf-8"``` should be passing by ```keyword```, but actually it's passed by ```position```, 
and the third argument is ```buffering```(a int) . 
So the system throws an ```TypeError```: 'str' object cannot be interpreted as an integer

## Fix
```python
with open(os.path.join(here, "requests", "__version__.py"), "r", encoding="utf-8") as f:
    exec(f.read(), about)
```

## Reproduction Steps
open any file with the code, you will get the TypeError.

## System Information

    TypeError: 'str' object cannot be interpreted as an integer

```python
here = os.path.abspath(os.path.dirname(__file__))
with open(os.path.join(here, "requests", "__version__.py"), "r", "utf-8") as f:
    exec(f.read(), about)
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


The function `open()` used in this context was imported from `codecs` and is not the built-in `open()` function in Python itself.
`codecs.open(filename, mode='r', encoding=None, errors='strict', buffering=- 1)` , from [codecs official documentations](https://docs.python.org/3/library/codecs.html#codecs.open)
backport chardet fix for Py3.6 & release 2.27.2

Please backport https://github.com/psf/requests/pull/6179 and make a new release supporting Python 3.6 users.

I know that it is EOL, but https://github.com/common-workflow-language/schema_salad/ and others still have to support Python 3.6.

Thanks,

We don't backport fixes for things we've dropped support for. We simply don't have the time
Mount

<!-- Summary. -->

## Expected Result

<!-- What you expected. -->

## Actual Result

<!-- What happened instead. -->

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```json
{
  "paste": "here"
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->
LookupDict does not implement full contract of the dict class

`LookupDict` absolutely includes all of the dict methods. In particular, it allows the use of `__setitem__` and other things from the dict class. It specifically does only one thing, which is override `__getitem__` to allow returning `None`.

So it's not ok to replace the dict with `object` in this case, even though the requests codebase doesn't use that functionality.

Thanks for reporting this issue though, and I hope you keep reading through the codebase!

_Originally posted by @Lukasa in https://github.com/psf/requests/issues/3848#issuecomment-277197592_

This is unfortunately a false statement. Take a look on the following checks:
```
In [1]: from requests.status_codes import codes

In [2]: codes
Out[2]: <lookup 'status_codes'>

In [3]: codes.keys()
Out[3]: dict_keys([])

In [4]: list(codes.keys())
Out[4]: []
```
If `LookupDict` pretends to quack like a duck it should do so. Or it should not derive from the `dict` if it works even worse:
```
In [8]: from requests.status_codes import codes

In [9]: codes["my_code"] = 700

In [10]: list(codes.keys())
Out[10]: ['my_code']

In [11]: codes["my_code"]

In [12]: 
```

@Lukasa , @sigmavirus24 
`LookupDict` object inherits `dict` object , so it's basically a dictionary. However, when setting a key to a value like `codes["my_code"] = 700` from @michal-klimek example, that key is inaccessible because `__getitem__` and `get` methods have been overridden only to access the object's attributes.
To resolve this issue, we can override the `__setitem__` to write key-value pairs to object's attributes or `__dict__`.
Do you agree with this solution?

1. It doesn't really need to be a dictionary
1. I don't believe we intended the dictionary to be extendable
1. If we did want it to be extendable, we'd have made a good API to allow for that
Improved documentation of exceptions in api.rst

Added use cases and inheritance information for the common types of exceptions that are listed in docs/api.rst. Documentation already exists for the main interface, request sessions, and lower-level classes on the website, so those may want to be moved into api.rst as well for consistent formatting. Alternatively, this information could also be chosen to move to the website. I am open to any changes we might want to make to improve this documentation and help users better understand the exceptions their code may be throwing.
ci: add minimum GitHub token permissions for workflows

### Description
This PR adds minimum token permissions for the GITHUB_TOKEN in GitHub Actions workflows using https://github.com/step-security/secure-workflows.

The GitHub Actions workflow has a GITHUB_TOKEN with write access to multiple scopes.
Here is an example of the permissions in one of the workflow runs:
https://github.com/psf/requests/runs/8179910695?check_suite_focus=true#step:1:19

After this change, the scopes will be reduced to the minimum needed for the following workflows:

- codeql-analysis.yml
- lint.yml
- run-tests.yml

The following workflow already has the least privileged token permission set:

- lock-issues.yml

### Motivation and Context

- This is a security best practice, so if the GITHUB_TOKEN is compromised due to a vulnerability or compromised Action, the damage will be reduced.
- GitHub recommends defining minimum GITHUB_TOKEN permissions.
  https://docs.github.com/en/actions/security-guides/automatic-token-authentication#modifying-the-permissions-for-the-github_token
- The Open Source Security Foundation (OpenSSF) [Scorecards](https://github.com/ossf/scorecard) also treats not setting token permissions as a high-risk issue. This change will help increase the Scorecard score for this repository.

Signed-off-by: Ashish Kurmi <akurmi@stepsecurity.io>

@nateprewitt / @theGOTOguy / @sigmavirus24 : could you please review this PR?
Can not send json and files same as data and files

Some scenarios with requests can not be done straightforward.
For example when I want to send json and files in multipart form, I use `data` and `files` parameters in requests method.
But if data has some `None` values, it will be dropped from request body. So for this problem I use `json` parameter instead of `data`.
Now with `json` parameter, `files` parameter will overwrite request body and json body will be removed from request.

## Expected Result

I want to send json and files with multipart form request, while json body has some None values.

## Actual Result

This expectation can not be done straightforward. Some solutions are messy and will change request body to another schema. like [this](https://stackoverflow.com/a/35946962/11926259)

## Reproduction Steps

```python
import requests
data = {"key": None}
requests.patch(url, json=data, files=files_data)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "4.0.0"
  },
  "cryptography": {
    "version": "3.4.8"
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.4"
  },
  "platform": {
    "release": "5.15.0-47-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "30000020",
    "version": "21.0.0"
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "30000020"
  },
  "urllib3": {
    "version": "1.26.5"
  },
  "using_pyopenssl": true
}

```


That's not how this works and it's documented as multipart only working with data and files parameters. If you need to send JSON inside a multipart field you need to encode it yourself and pass it to data with a field name.
Headers that are subclass of str/byte are not accepted

Requests does not accept headers that are subclasses of string or byte.
They are required i.e. to obfuscate secrets in certain environments, so this is also a security related issue.

The issue was introduced by e36f34597ca96c40dbb81d63a8fe9ac0860b8dc3 at June this year by @nateprewitt 

A possible fix is given in this PR https://github.com/psf/requests/pull/6230

## Expected Result

Headers are used as they are string/byte.

## Actual Result
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/venv/lib64/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/tmp/venv/lib64/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/tmp/venv/lib64/python3.10/site-packages/requests/sessions.py", line 573, in request
    prep = self.prepare_request(req)
  File "/tmp/venv/lib64/python3.10/site-packages/requests/sessions.py", line 484, in prepare_request
    p.prepare(
  File "/tmp/venv/lib64/python3.10/site-packages/requests/models.py", line 369, in prepare
    self.prepare_headers(headers)
  File "/tmp/venv/lib64/python3.10/site-packages/requests/models.py", line 491, in prepare_headers
    check_header_validity(header)
  File "/tmp/venv/lib64/python3.10/site-packages/requests/utils.py", line 1037, in check_header_validity
    raise InvalidHeader(
requests.exceptions.InvalidHeader: Header part (Hidden) from {'token': Hidden} must be of type str or bytes, not <class '__main__.Secret'>

```

## Reproduction Steps

```python
import requests

class Secret(str):
    def __repr__(self):
        return "Hidden"

requests.get("https://google.com", headers={"token": Secret("Confidential")})
```


## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.1.1"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.6"
  },
  "platform": {
    "release": "5.19.6-100.fc35.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010111f"
  },
  "urllib3": {
    "version": "1.26.12"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


Hi @CarliJoy, please check other open issues before opening new ones. I'll resolve this as a duplicate of #6159.

Hi @nateprewitt and Sorry. Indeed I forgot to that. Sorry for the extra work.
`❯` character becomes `â¯` in data after post

Code: https://github.com/pcroland/deew/blob/main/dev_scripts/post/doom9.py
On other sites it worked fine but on this site it changed the input character to gibberish. I guess it's related to urlencoding.

If I urlencode
```
Help:
❯ deew -h
```
urlencoder.org creates
```
Help%3A%0A%E2%9D%AF%20deew%20-h
```
while chrome creates
```
Help%3A%0A%u276F%20deew%20-h
```
Is there a way to mimic how Chrome encodes it?

Hi @pcroland, it looks like this is a request for help. The question here doesn't appear to be related to a defect in Requests. We'd suggest making a post on [StackOverflow](https://stackoverflow.com/) to get further assistance.

requests encodes this character differently than Chrome and Firefox, isn't it possible that it's a bug?

If you have a minimal reproducible example showing a defect in Requests, please provide more details.

What's currently been provided doesn't have any clear indication there's a defect.

requests uses normal urlencoding while Firefox and Chrome uses unicode escape for this character and apparantly some sites don't like the regular urlencoding for this character (and possibly others).

According to https://stackoverflow.com/q/912811/492203 it's a non-standard escape method that has been rejected by the W3C, but as we know Chrome has the monopoly... Maybe it would be worth adding it as an option. Here's some code I threw together quickly for it:

```py
import urllib.parse


def urlquote_chrome(text: str, *_: Any) -> str:
    escaped = (
        urllib.parse.quote(text, encoding="unicode-escape")
        .replace("%5Ca", "%07")
        .replace("%5Cb", "%08")
        .replace("%5Cf", "%0C")
        .replace("%5Cn", "%0A")
        .replace("%5Cr", "%0D")
        .replace("%5Cu", "%u")
        .replace("%5Cv", "%0B")
        .replace("%5Cx", "%")
    )

    # These are probably not needed but just in case
    escaped = re.sub(r"%[a-f0-9]{2}", lambda m: m[0].upper(), escaped)
    escaped = re.sub(r"%u([a-f0-9]{4})", lambda m: f"%u{m[1].upper()}", escaped)

    return escaped


r = session.post(
    url=url,
    data=urllib.parse.urlencode(data, quote_via=urlquote_chrome),
)
```
allow string/byte subtypes for header

We wrap our secret a in special subclass of string, that prevents leaking confidential information in some circumstances (i.e. stack traces) etc.

The current implementation of the header checks does a `type(val) == str` which is not recommended by PEP8.
Due to this implementation we can't pass this variables as headers. Converting them back to a normal string would defeat the purpose of wrapping them in the first place.

This PR fixes both issues.


Related to https://github.com/psf/requests/issues/6159 -> there is also another fix defined there.

Resolving as I believe this was fixed in Requests 2.29.0.
When making a POST request, why doesn't `auth` or `session.auth` work when logging in, but `data=data` does?

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.
I have a website that requires an Email and Password to log in. When I set these values in `session.auth` or `auth=(email, pw)` they don't get the right response data. But, when I pass in a `data` object, it works. 

What is auth or session.auth doing in the background? Is it just assuming to use "username" or something?

login_url = https://prenotami.esteri.it/Home/Login

**data=data** (returns correct response html)
```python3
    with requests.Session() as session:
        data = {"Email": "username@gmail.com", "Password": pw}
        login_response = session.post(
            login_url,
            data=data,
        )
        print(login_response.text) 
```
**session.auth** (doesn't return correct response html)
```python3
    with requests.Session() as session:
        session.auth = (email, pw)
        login_response = session.post(
            login_url,
        )
        print(login_response.text)
```

**auth=()** (doesn't return correct html data)
```python3
    with requests.Session() as session:
        login_response = session.post(
            login_url,
            auth=("username@gmail.com", pw),
        )
        print(login_response.text)
```


> Please refer to our Stack Overflow tag for guidance

You ignored the template. Further, it doesn't seem you understand how HTTP works out the service you're trying to communicate with. The maintainers can't educate each person individually via issues.
Explain the status and the future of the library (in PR template, README, docs site)

Attempting to file a new feature request shows the text:

> Requests is not accepting feature requests at this time.

Fair enough, there must be reason for that (e.g. lack of maintainers).

However, maybe explain a bit more - if new features are not accepted, what is the future of the library in general? You could:

- Pin a ticket on the issue tracker
- Add a note to the README
- And/or maybe to the docs site

Current state leaves people (at least me) searching/googling for the status and explanation, yet nothing is found?

Additionally to this, it would be useful to know what issues are coming into the next release, and a sort of ETA / timeline of releases. I'm currently waiting for the next release, and if it wasn't for someone else also asking that question in an issue "when is the next release coming?" I wouldn't know... It doesn't seem to be documented anywhere (site or repo or otherwise).

@bourgeoisor You should be able to find the currently flagged changes for upcoming releases in our [Milestones](https://github.com/psf/requests/milestone/31), GitHub's standard mechanism for tracking changes. There is no formal release schedule but I'll note the current cadence in a future PR.

That's an oversight on my part-- The changes _are_ indeed in the GitHub Milestones. Thank you for documenting the expected release cadence, that would help a lot!
fix order of verify overrides in Sessions

Currently, `Session` overrides the request-level `verify` parameter by
reading the environment, and then checks to see if it should choose that
or the session-level settings.

Let's consider the scenario where we've provided `REQUESTS_CA_BUNDLE` in
the environment, and then create a session with `s.verify = 'asdf'`. Now,
the final choice is going to be the environment-provided bundle, rather
than the session-provided setting, which is unexpected behavior.

The correct behavior would be to first do the merge of the request and 
session-level parameters, and then use the environment-provided params
if needed. 

See #3829 for why we can't just fix this in a bug patch release
fix order of verify overrides in Sessions

Currently, `Session` overrides the request-level `verify` parameter by
reading the environment, and then checks to see if it should choose that
or the session-level settings.

Let's consider the scenario where we've provided `REQUESTS_CA_BUNDLE` in
the environment, and then create a session with `s.verify = 'asdf'`. Now,
the final choice is going to be the environment-provided bundle, rather
than the session-provided setting, which is unexpected behavior.

The correct behavior would be to first do the merge of the request and 
session-level parameters, and then use the environment-provided params
if needed.
requests.package.chardet* references not assigned correctly?

I found that packages.py is assigning sys.modules['requests.package.chardet'] over and over with different modules, is this intentional or a bug?

I looked at the code and this `target` variable confuses me, it is assigned again in loop and placed with itself(so completely?), looks like a name confliction to me. Code is referenced from installed version 2.28.1.
```
target = chardet.__name__
for mod in list(sys.modules):
    if mod == target or mod.startswith(f"{target}."):
        target = target.replace(target, "chardet")
        sys.modules[f"requests.packages.{target}"] = sys.modules[mod]
```

## Expected Result

every chardet.* package maps to requests.packages.chardet.* respectively

## Actual Result

only requests.package.chardet is assigned at last.

## Reproduction Steps

import requests
import sys
print([m for m in sys.modules if name.startswith('requests.packages.chardet')])


Have proposed a fix here - https://github.com/psf/requests/pull/6562

Resolving as it appears #6562 ended up merged without closing this. Thanks @amkarn258!
request google search news fails

when i run this code
monitored_tickers = ['GME', 'TSLA', 'BTC']
![image](https://user-images.githubusercontent.com/112481361/187419609-584b04eb-1b7e-4d6c-aefb-da8112282d11.png)

i get these results
![image](https://user-images.githubusercontent.com/112481361/187419717-fc2f955a-7346-443f-9e9d-f19bc1072d20.png)


why doesn't return the urls ?
is the problem in the request statement or in google account ?

i am using the greek version of chrome, and i am in greece .





Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
allow_redirect will ignore HSTS(STRICT-TRANSPORT-SECURITY)

while a website set header STRICT-TRANSPORT-SECURITY, it means tell browser use https, but requests is not judged

## Expected Result

sometimes, developer set 302 and redirect website to http . at the same time, someone config header with STRICT-TRANSPORT-SECURITY. To browser, this will be ok, but to requests, not able to access http://example.com because of use http protocol

## Actual Result

![image](https://user-images.githubusercontent.com/100894276/187396909-9a671bd7-c374-4ec1-b461-8f17379e56dc.png)

## Reproduction Steps

```python
import requests
r = requests.get('exmaple.com', allow_redirect=True)
r.status_code ==> 502

In Chrome:
r.status_code == 200
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "3.0.4"
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": "37.0.2"
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.2"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "30000030",
    "version": "22.0.0"
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "101010df"
  },
  "urllib3": {
    "version": "1.26.9"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}
```
updated sha1 hasing to sha256 to avoid collisions

Based on a [security review](https://pypi.openrefactory.com/requests/62ff1f9edd30551cbfff8eeb) of the requests repository, it looks like the `sha1` hashing algorithm should be updated here to avoid collisions. 

I have not added any unit tests since this should be a `no-op` change with the expected behaviour after this change identical to the previous behaviour before the change. I'm also happy to help implement/clean up some of the code here as well. 

Hi @joshmgrant, for future reference, it's considered best practice to always report issues relating to suspected security problems through private channels. Most projects will have either a [SECURITY.md](https://github.com/psf/requests/blob/main/.github/SECURITY.md) or a [specific issue type](https://github.com/psf/requests/security/policy) with instructions on disclosure. This provides appropriate time for projects to mitigate vulnerabilities.

That said, I don't think this is an issue. The hash being used for the cnonce is already being truncated to 16 bytes, so changing this from SHA1 to SHA256 shouldn't have a meaningful effect on collisions. We also use the cnonce to compute the actual digest hash value which can be MD5, SHA1, SHA256, or SHA512.

Can you clarify the practical attack vector you're observing here? This looks like a generic grep scanner report from what's been provided.

@nateprewitt Hello there! Thank you for the feedback on this. I this is my first pull request for a security-related issue (as such) so I was unaware of the instructions for security-related changes and/or vulnerability reporting. 

I was unable to create any sort of (practical) attack related to this change. I completely agree that, as you mention, since this is a cnonce value is used to compute the digest hash, it's not really a meaningful issue to use the SHA1 hash for the conce value. 

My main motivation was that this change was reported by the scanner that I linked to earlier as a _possible_ security vulnerability but without any additional context. After some research and trying to create a meaningful attack, I decided to open the PR first and see what would happen. Now I know :) 

I will close this PR as unmerged but thank you for your patience. Let me know if there's any areas I could look at for contributions (not necessarily security related, I just like Python and Requests). Thank you so much!
return response from `raise_for_status`

I'd guess this has been requested before, but couldn't find any pointers so far.
(I also understand that requests does not accept feature requests, but this is code-wise a trivial one-line change, so I'm going to try anyway :smile: ).

Feature request: return `self` from `Response.raise_for_status()` (instead of returning `None` implicitly). 

That allows using it in a compact chained/fluent manner, for example

```python
data = [
    requests.get(url).raise_for_status().json()
    for url in urls
]
```

With the current `raise_for_status`  you can not use list comprehension like in this examle because you have to use separate statements for the `.get()`, `.raise_for_status()` and `.json()` .

(An alternative solution could be to add some kind of `raise_for_status` opt-in option to `.get()` and alike, but that would require a lot more code changes than a simple `return self` as proposed higher)


This project is in indefinite feature-freeze and not changing the API in ways that may be surprising for existing users. Also I'm certain this has been requested before but am on mobile and can't easily search 

I am junior SDE, and I want to contribute to open source. This is my first time.
Can I work on this ?

> This project is in indefinite feature-freeze

@sigmavirus24 Is this still true?? Why is it in a feature-freeze?

> Why is it in a feature-freeze?

It's documented here: https://requests.readthedocs.io/en/latest/dev/contributing/#feature-requests
Other Language isn't workin as a parameter

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.
when i sending a other langueges parameters requests into a url it retuens
```à¦¬à¦¾à¦�à¦²à¦¾à¦¦à§�à¦¶ à¦ªà§�à¦°à¦¤à¦¿à¦¦à¦¿à¦¨ à¦¸à¦�à¦¬à¦¾à¦¦ à¦ªà¦¤à§�à¦°```
this stuffs
in python requests

@abirabedinkhan Please don't open the same issue multiple times.

If you need assistance with this please consult the documentation and ask for assistance on StackOverflow using the python-requests tag.
à¦¬à¦¾à¦à¦²à¦¾à¦¦à§à¦¶ à¦ªà§à¦°à¦¤à¦¿à¦¦à¦¿à¦¨ à¦¸à¦à¦¬à¦¾à¦¦ à¦ªà¦¤à§à¦°

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.
when i sending a other langueges parameters requests into a url it retuens
``` à¦¬à¦¾à¦à¦²à¦¾à¦¦à§à¦¶ à¦ªà§à¦°à¦¤à¦¿à¦¦à¦¿à¦¨ à¦¸à¦à¦¬à¦¾à¦¦ à¦ªà¦¤à§à¦° ```
 this stuffs
in python requests

Please refer to the top of the issue template. We won't be able to assist with this on this issue tracker.

Sorry I didn't understand what u said. Can u explain a bit? What to do
Merge types-requests into psf/requests with py.typed marker

This issue is to merge the existing type hints available in typeshed ([here](https://github.com/python/typeshed/tree/master/stubs/requests)) into this repository.

Currently, psf/requests supports only Python 3.7+ per https://github.com/psf/requests/blob/main/setup.py#L10

As a result, type-related code would not impact usability by end users. Performance concerns related to additional imports may also be addressed by protecting the type-only imports under the `typing.TYPE_CHECKING` boolean and using the correct syntax downstream.

By making this change, it will keep type information paired with the code, ensuring they get updated together, it will reduce the maintenance burden on the typeshed team (not requested, just a side effect), and will enable more users to get type-hint related information without having to install separate packages.

Depending on your point of view on type hints, they also can reduce the risk of bugs caused by refactoring.

Hi @kkirsche,

We've looked at doing this before. There are a number of issues with the existing typeshed hints we found during our last review. We have a feature branch looking at getting some level of typing integrated in the future but there is currently no timeline for this.

Understandable. Some aspects will certainly depend on your philosophies about type hints (e.g. functionally won't crash vs. expected usage) and there are some weird oddities about semantic meanings in a type checker with types like `Any` vs. `object` that weren't clear to me before starting to contribute more type hints.

Let me know if there's anything you need support on for that goal :)
Python Requests Libarary Removes some Characters in URL when Sending Request

Python **requests** library changes URL in some situations.
it removes some parameters
for example
`
url = "https://www.example.com/test?"
`
it removes `?` in the request when I see it in Burp proxy
it changes to 
`https://www.example.com/test`
because I'm pentesting web applications I need to send for example quotation mark too and other things which will be removed by library automatically.
I need following path:
![image](https://user-images.githubusercontent.com/61124903/182108804-e9811743-2ba4-41d3-8266-9e6d97998e4a.png)
after ? must be empty

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

In the future, please search **closed and** open issues before creating new ones that are duplicates.
Cant download big file

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.
I use requests.get(url) to download a big file from our own cloud box. 
But the response['Content-length'] isnt right, and use 'strem=True' args is wrong, the downloaded file which is a .zip file cant be opened .



Hi @liuweidong88, please refer to the guidance at the beginning of the ticket template. We unfortunately won't be able to help with this.
Remove `__nonzero__` method from `models.Response`

`__nonzero__` was used in python2 as we now use `__bool__`. It is not used at all in python3.
And since [`setup.py` says](https://github.com/psf/requests/blob/177dd90f18a8f4dc79a7d2049f0a3f4fcc5932a0/setup.py#L10) that `reqeusts` supports only `python >= 3.7`, I don't think that it is really required.

I found this while working on `typeshed` types for `requests`: https://github.com/python/typeshed/blob/3fe1f5d6c46b39b8b4632aadc70477a729241d25/stubs/requests/requests/models.pyi#L107

Hi @sobolevn,

Thanks for the PR, but I think we’re unlikely to remove this. The use of `super().__nonzero__()` to implement `__bool__` in subclasses was quite popular when the Python 3 migration started. There’s a high chance this will be breaking if we remove the function entirely.
added from_raw class method to PreparedRequest

Adds a class method to create PreparedRequests from raw HTTP request using Python's builtin BaseHTTPRequestHandler class.

Example usage:

```
>>> from requests import PreparedRequest, Session
>>> prq = PreparedRequest.from_raw('GET / HTTP/1.1\r\nHost: example.com\r\n\r\n', use_https=False)
>>> res = Session().send(prq)
<Response [200]>
```

This would be very useful for someone that wants to use the python requests library but with raw HTTP requests.

Requests is not accepting feature requests or new feature additions at this time. Thank you for putting this together but we won't be accepting it
Remove Python 2 mention on `chardet` behavior

`requests` no longer supports Python 2. A recent commit, 8bce583b9547c7b82d44c8e97f37cf9a16cbe758
removed the `chardet` dependency for Python 2:

```diff
-'chardet>=3.0.2,<5; python_version < "3"',
```

We should edit the docs to remove mention of behavior on Py2.

Thank you both for the extremely speedy review! It does look like I'll need a maintainer to kick off CI though.
Add response object to JSONDecodeError

Add response object to JSONDecodeError.

We used to use the following code, but since 2.27, errors on json parse failure are now caught as RequestException.
Since there is basically `response` in the `.json()` phase, we would like to add it in JSONDecodeError for convenience.

```python
def my_post(url, payload):
    try:
        res = requests.post(url, json=payload)
        res.raise_for_status()
        return res.json()
    except requests.exceptions.RequestException as e:
        logger.warning("Failed to send data %s", e.response.status_code)
    except Exception as e:
        ...
```
```
AttributeError
'NoneType' object has no attribute 'status_code'
```

No, thank you
Suppress context when requests.exceptions.JSONDecodeError is raised.

When a JSON response cannot be parsed, requests catch json.JSONDecodeError (or simplejson.JSONDecodeError) and raise requests.exceptions.JSONDecodeError instead. Because this happens in an except block, unhandled exceptions were printed in a chain:

```
Traceback (most recent call last):
...
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
...
requests.exceptions.JSONDecodeError: [Errno Expecting value] ...
```

This seems unnecessary and it increases cognitive overhead whean examining errors.

As requests.exceptions.JSONDecodeError inherits json.JSONDecodeError and we copy all atrributes from the later to the former, `raise ... from None` seems more approriate than `raise ... from e`.

We don't suppress context anywhere else as far as I remember. I don't understand why it's more valuable here and less helpful elsewhere. (And, no, don't update every exception to suppress context)

Thank you @sigmavirus24, I see now that the error handing for JSON decoding is consistent with the rest of the library.

> (And, no, don't update every exception to suppress context)

What about updating exceptions to use `raise ... from e`? We would get tracebacks with: "The above exception was the direct cause of the following exception:" instead of "During handling of the above exception, another exception occurred:".

I believe that it would follow intentions in [PEP 3134](https://peps.python.org/pep-3134/). If `__cause__` (instead of `__context__`) was set, it would be more apparent that the exception chain is an expected behavior, not some bug.
Fix links in docs to the kenreitz.org domain, which expired.

When I was studying Contributor’s Guide, I found that links to kenreitz.org are broken.

- "The Future of Python HTTP" was archived by Internet Archive, so I linked there.
- "Be cordial or be on your way" was not archived, so I removed the link.
- "A Kenneth Reitz Project." in a footer is broken too, but I did not touch that link.

It appears the links may have been temporarily interrupted but are working presently.
Using `format` as a query parameter for certain URLs causes `requests` to miss the `?` query string separator

<!-- Summary. -->
Using `format` as a query parameter for certain URLs causes `requests` to miss the `?` query string separator

## Expected Result

I expect the `?` separator to be present whenever any query parameters are provided.

<!-- What you expected. -->

## Actual Result

The `?` separator is not present.

<!-- What happened instead. -->

## Reproduction Steps''

```python
>>> import requests
>>> requests.__version__
'2.28.1'
>>> requests.get("https://www.uniprot.org/uniprotkb/search", params={"format": "json"}).url
'https://rest.uniprot.org/uniprotkb/searchformat=json'
>>> requests.get("https://www.uniprot.org", params={"format": "json"}).url
'https://rest.uniprot.org/format=json'
>>> requests.get("https://www.uniprot.org/uniprotkb/search", params={"size": "500"}).url
'https://www.uniprot.org/uniprotkb/search?size=500'
>>> requests.get("https://www.google.com", params={"format": "json"}).url
'https://www.google.com/?format=json'
```

Note how the same issue happens in different subdomains of `uniprot.org` but that it doesn't happen with `google.com`

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.1.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.7"
  },
  "platform": {
    "release": "5.13.0-52-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "101010cf"
  },
  "urllib3": {
    "version": "1.26.10"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


The service is returning the incorrect URL to be redirected to in the `Location` header:

```python
import requests

resp = requests.get(
  "https://www.uniprot.org/uniprotkb/search",
  params={"format": "json"},
  allow_redirects=False
)

# The response was sent to the right URL
print(resp.url) # 'https://www.uniprot.org/uniprotkb/search?format=json'

# The response is indeed a redirect
print(resp.status_code) # 301

# Service redirects to the incorrect URL you're seeing
print(resp.headers["Location"]) # 'https://rest.uniprot.org/uniprotkb/searchformat=json'
```

Going to close this as not a bug in Requests.
Error Partially initialized module 'requests' has no attribute 'post'"

If you have a file named token.py in your path you get the error:

**Error Partially initialized module 'requests' has no attribute 'post'"**

 That is the case with Python 3.8.9 and Requests Version: 2.28.1 on Mac OS 12.4.

I don't believe we import anything directly that would cause that so it's likely an issue with a dependency it the standard libraries we rely on. Therefore, it's not a bug we can fix. Rename your file to not conflict with what (based on my memory, since I'm on my phone) is a standard library package.

Also you ignored the issue template
Adding native support for Bearer token authentication objects

added to auth.py:
class HTTPBearerTokenAuth(AuthBase):
def _bearer_token_auth_str(token):

added 1 test case to test_requests.py

All tests passed locally

I'm very excited for this new feature. Tests also passed locally for me. I'm happy to assist development on this one, please let me know if I can help.

Duplicate of #4437

@sigmavirus24 Thank you for your feedback. We apologize for not noticing https://github.com/psf/requests/issues/4437 , which we inadvertently duplicated, and we apologize to inadvertently reopening a contentious and previously resolved topic. Although we certainly wish this feature could be added - we would find it very useful and we're eager to get involved - we defer to your experience and judgement.

If you don't mind us asking you a couple of questions:

1. This is our first attempt at contributing to open source. How did we do?

2. Can you recommend other ways we could contribute to this repo?

Thank you.