Raibek/requests-github-issues huggingface dataset:
Make use of the 'yield from' keyword whenever possible for a better pythonic look!


Signed-off-by: Harmouch101 <mahmoudddharmouchhh@gmail.com>

We're not accepting cosmetic changes or changes which break compatibility with Python 2.7

Oh! I didn't notice it supports python 2.7. However, I think the project needs to be updated to support only versions > 3.5  of python since other versions have reached the [end of life](https://endoflife.date/python) in terms of security support.
2.25.1: requests requires idna < 3

Looks like requests  modulehas hardcoded dependency to require `idna >= 2.5 and idna< 3` and latest idna is 3.1.
Is it any reason why it cannot be idna is 3.1?
amd/or if it is could requests module can be adjusted to be able use it with latest idna?

Hi @kloczek, there are a number of issues around this including very recent ones like #5804. Please make sure to check open and closed issues before opening new ones in the future. Thanks!

> Hi @kloczek, there are a number of issues around this including very recent ones like #5804. Please make sure to check open and closed issues before opening new ones in the future. Thanks!

Hmm .. you've just closed the ticket without provide answers on any of my questions  :(
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)

Summary.
Cant import json data into Mysql Database because of this issue

What you expected.

A proper import of course.


What happened instead.
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)

## Reproduction Steps

```
if args.input:
        with open(args.input, 'r') as f:
            for e in f:
                event = json.loads(e)
                misp.add_event(event)
```

## System Information
{
   "response":[
      {
         "Event":{
            "id":"1",
            "orgc_id":"1",
            "org_id":"1",
            "date":"2017-09-27",
            "threat_level_id":"4",
            "info":"ZeuS IP blocklist (Standard) feed",
            "published":true,
            "uuid":"59cb4f10-4190-4f76-988e-66a239b59801",
            "attribute_count":"108",
            "analysis":"2",
            "timestamp":"1506496272",
            "distribution":"3",
            "proposal_email_lock":false,
            "locked":false,
            "publish_timestamp":"1533556422",
            "sharing_group_id":"0",
            "disable_correlation":false,
            "extends_uuid":""
}}]}

This is running on up-to-date Ubuntu 20.04 and Python 3.8.5



Hi @aymenbnl,

This doesn't appear to be related to Requests from the info provided. We'd recommend posting general Python questions on [StackOverflow](https://stackoverflow.com/).
POST request is resent as GET when the status code is 301

Hello, 

I'd like to report a bug with `requests` behavior towards processing a 301 response. The 301 response to a POST request causes the user agent to automatically redirect the request to a given address, but it also changes the request type from POST to GET. The RFC 2616 mentions such behavior as shown below:
```
Note: When automatically redirecting a POST request after
      receiving a 301 status code, some existing HTTP/1.0 user agents
      will erroneously change it into a GET request.
```

Therefore, I think that `requests` isn't compliant with the RFC [(10.3.2 301 Moved Permanently)](https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html) in this matter.

## Expected Result

When the response to a POST request has a status code 301, the automatic redirect should also be a POST request.

## Actual Result

Having sent a POST request to a given endpoint and receiving a response with a status code 301, the automatic redirect request becomes a GET request. Example shown below:
```
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [29/Apr/2021 16:43:03] "POST /test_route HTTP/1.1" 301 -
127.0.0.1 - - [29/Apr/2021 16:43:03] "GET /test_route HTTP/1.1" 405 -
```

## Reproduction Steps

Create two scripts - a server and a client.

`test_server.py`
```
import flask

app = flask.Flask(__name__)

@app.route("/test_route", methods=["POST"])
def test_response():
    return flask.redirect("http://localhost:5000/test_route", code=301)

if __name__ == "__main__":
    app.run()
```

`test_request.py`:
```
import requests

if __name__ == "__main__":
    print(requests.request("POST", "http://localhost:5000/test_route"))
```

After creating them, start the `test_server.py`. When the server starts, run the `test_request.py`. In the server logs you'll see the aforementioned base request and the redirected request.

## System Information

The library is system-independent, the behavior occurs on any device, from Raspberry Pi with Raspbian Buster, to x64 Debian Bullseye (testing).




Hi @Maelstro,

This is expected behavior, the RFC you’re referencing has been obsolete for several years now.

Please make sure to search open and closed issues as we’ve requested before opening new ones in the future. You’ll be able to find this [answered](https://github.com/psf/requests/issues/5494#issuecomment-644314490) and the link to the relevant RFC there. Thanks!
idna-3 is not supported

idna-3 has been released on Jan 1st. However, requests still requires idna<3 preventing a lot of projects from upgrading.

## Expected Result

requests supporting idna-3.1

## Actual Result

requests force idna<3

## Reproduction Steps

Errr, install requests? ;-)

## System Information

```
$ python -m requests.help
{
  "chardet": {
    "version": "4.0.0"
  },
  "cryptography": {
    "version": "3.4.7"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.4"
  },
  "platform": {
    "release": "5.11.16-gentoo-dist",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "101010bf",
    "version": "20.0.1"
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.26.4"
  },
  "using_pyopenssl": true
}
```

Hi @mgorny, this is a duplicate of #5710. Please be sure to check open issues before posting new ones. Thanks!
Gzip decompression corrupts file

## Summary
I want to download a gzip file (`https://iptoasn.com/data/ip2asn-v4.tsv.gz`) and save it locally already decompressed.
According to the [docs](https://docs.python-requests.org/en/master/user/quickstart/#raw-response-content), downloading a compressed file should automatically decompress it. 

## Expected Result

The `ip2asn-v4.tsv` file correctly decompressed.

## Actual Result

The `ip2asn-v4.tsv` generated is corrupted. Below the content after opening it with excel:

![image](https://user-images.githubusercontent.com/37975547/116474115-60e38f00-a878-11eb-8a1b-816ff9b89e54.png)


## Reproduction Steps
This is the minimal code snippet to reproduce my use case:
```python
import requests
r = requests.get('https://iptoasn.com/data/ip2asn-v4.tsv.gz', stream=True)
with open('ip2asn-v4.tsv', 'wb') as fd:
    for chunk in r.iter_content(chunk_size=None):
    fd.write(chunk)
```

If you look at the headers on your response you'll see that there is no `Content-Encoding: gzip` which you seem to be expecting. Even if you forcibly send `Accept-Encoding: gzip` the URL doesn't reply with a `Content-Encoding`, so Requests won't automatically decode the gzipped file because the endpoint hasn't declared it (and it wouldn't make much sense for the endpoint to do so, it's already gzipped and gzipping again will make the file even larger!)

Instead you need to gzip-decompress the file contents yourself, I recommend `gzip.decompress()` or `gzip.GzipFile` for streaming.

In the future, this isn't the place for asking questions about how to use Requests, this is for tracking defects in Requests (which this is not). Try Stack Overflow next time.

You're right, thanks for the answer :)
Custom HTTP adapters via Session constructor

I often find myself having to do the following when using `Session` with custom retrying logic:
```
retry = Retry(3, backoff_factor=1.)
http_adapter = requests.adapters.HTTPAdapter(pool_connections=4, max_retries=retry)
session = requests.Session()
session.mount('https://', http_adapter)
session.mount('http://', http_adapter)
```

Having to mount the adapter myself for such a simple use-case seems superfluous, error-prone and not very user-friendly so I'm suggesting we open up the Session constructor a little bit:

```
retry = Retry(3, backoff_factor=1.)
http_adapter = requests.adapters.HTTPAdapter(pool_connections=4, max_retries=retry)
session = requests.Session(http_adapter=http_adapter)
```

This project is under a feature-freeze which means we're not accepting any changes to the API at this time.

Thanks for submitting this but we will not be accepting it
Add PATCH example to the Quickstart of documentation

This PR mades users understand that the library supports the patch method and the simple use of it quickly.

Hi @nateprewitt, this simple change will mean a whole lot for many new users to quickly use this library. I hope you have a good day! Thanks and regards. : D

We aren't accepting small changes to the README.

Hi @sethmlarson, it's not the README, it's the document, so I need to open another PR just for the document change?

Either way, we're not accepting small changes to any documentation, we don't have the reviewer time to review and merge them all.

But this very samll change on the quicklystart page makes the document completed and greatly improve people's time. Thanks and regards.
![image](https://user-images.githubusercontent.com/9413601/116642940-82409a00-a9a2-11eb-8f97-14eb8c41e6e1.png)
Replace getattr(...) is not None with hasattr(...)

Replace `getattr(body, 'tell', None) is not None` with `hasattr(body, 'tell')`

Hi @takos22,

This syntax is intentional. You can read the reasoning [here](https://hynek.me/articles/hasattr/). 

Ok my bad, didn't know about that issue with ``hasattr``. Thanks for the info and for the fast response.
当响应头中有空格时，response.headers 会被截断

Summary.

## 这是完整的响应数据包

![image](https://user-images.githubusercontent.com/26384334/114485840-315d4180-9c3f-11eb-8473-60bb509f2b1f.png)


## 这是response.headers获取的数据包

![image](https://user-images.githubusercontent.com/26384334/114485886-4e921000-9c3f-11eb-9328-fbd46367e9df.png)
Short-circuit raise_for_status code for statuses that will not raise …

…an exception

This short-circuit code will skip the steps for decoding response.reason if the status code is not an exception-raising status code.

Decoding the `reason` certainly isn't that expensive in reality. Why add this complexity here? What does this gain anybody?

Early exit from an exception-raising routine, when it can be quickly determined that no exception will be raised, would be considered by some to be less complexity, not more. At run-time it saves several additional if-tests for all successful statuses, and for those reading the code, it clearly shows that only status codes >= 400 are really of any interest when raising an exception.

(I could understand if there were some possible restructuring of HTTP status codes in the future, but this status code organization is pretty cast in stone now.)

Hi @ptmcg, thanks for the initiative here! Unfortunately, I think I'm in agreement with @sigmavirus24. This is an optimization that adds extra code clutter without providing a tangible user benefit.

I did some quick benchmarking and performance difference is measured in fractions of a _microsecond_. There's never going to be a situation where we accept latency involved with sending a request across a network but are concerned about sub-microsecond precision in post processing.
Grammar fix in description

Fixes https://github.com/psf/requests/issues/5785

Could not find the second error as described `This is also a problem on the pypi page under the project description.`

Thank you!

@cocobennett,
To answer this-
> Could not find the second error as described This is also a problem on the pypi page under the project description.

This would essentially be fixed by your patch as the `long_description` picked up by PyPI page to populate the description section is set to `readme` in `setup.py` as seen in this line: https://github.com/psf/requests/blob/c45a4dfe6bfc6017d4ea7e9f051d6cc30972b310/setup.py#L70
Cleanup redundant allow_redirects kwargs in api

Not a big problem, just remove the redundant param `allow_redirects` in API `get()` and `options()`. Cause the underlying calling `Session.request()` has param `allow_redirects` defaults to `True`.

Only leaves

```python
def head(url, **kwargs):
    kwargs.setdefault('allow_redirects', False)
    return request('head', url, **kwargs)
```

in `api.py` to make a contrast that http method `head()` is the only one disallows redirects by default.
quickstart.rst: r.json() can raise JSONDecodeError on Py3

% `python2 -c "import requests ; requests.get('https://github.com').json()"`
% `python3 -c "import requests ; requests.get('https://github.com').json()"`

Python | Simplejson is installed  | Simplejson is NOT installed
--- | --- | ---
Python 2 | simplejson.JSONDecodeError | ValueError
Python 3 | simplejson.JSONDecodeError | json.JSONDecodeError

Thank you! 🎉
Added explanation why the timeout is doubled.(#5773)

## issue

#5773

@sigmavirus24

I added what the user should do and made the phenomenon stand out.

If you have time, I'd be happy if you could see it


Hi,

I would like to tell what I  understand on what you  want to publish, and possibly,  suggest a different worlds to  be more accurate.

In Networking, we have a reference model set by ISO, called OSI Ref Model,  it is a 7 layers  reference model that, by ISO, could made any complaint system , interoperable.

Saying that,  HTTP/HTTPS is a High L4 Protocol, while a connection is High L3/ Low L4 Pair...based on this....a Read Timeout, really means that once  all higher security layer have been passed  a sessin with a server is established, and a request has been made....without any protocol response status within the time set as timeout, it should be consider as a read timeout, the connection timeout, means that, when the orignation point request or has a established connection with an endpoint does not response within the timeout defined.

HTTP Keepalive is a way to call  TCP Keealive mechanims to take care of that connection, but...as the RFC Standard is Out of this world, the endpoint commonly drip the connectn without notification...so a Relk Read Timeout...is now conerted as a Connection Timeout..

And  Finallly, just to be crystal....timeout is not being double by client....it is really set by Server side response.

Cheers

I will use it as a reference.

I'm really thankful to you

I made some changes so that some people could understand

The IETF Standard defines a 2 hours lapse in order to start sending TCP KA Packets, obviously, it is too long, so server managers decrease that value in order to allocate the available resource properly... I.E, Netflix have a 4 minutes timeout, instead of 2 hours.... so no matter if you have enabled HTTP KA.... the server side will drop the connection on its side after 4 minutes and when  the  client side tries to get  data on  a open soclet, it will fail, cais, that socket no logr exists...BTW, NF,  has a avascript code, thtat modifies the connectin time out to 2 min.

So, having the requests/urllib3 tieout parameter enabled, it wont make any diffrence, since  the server might  drop the soclet before the client reches its threshold

Wow!
I learned for the first time that the initial value of keepalive was 2 hours.

@cpyberry I don't understand the documentation you've added here any longer. It's vague, unclear, and roundabout. You've been given clear suggestions that you can simply apply and would get this PR merged. Please either update this or close it, as I will not accept these updates in their current state

@sigmavirus24 
First of all, I apologize for leaving this pull request for several months.

I want to commit the changes you have suggested, but my stupid additional commits have made it an "old suggestion".

Apparently, github can only commit latest suggestions.

Is it possible to make the same proposal again?

Next time I will not do stupid things.
#5773: Fixed the number of seconds specified for timeout being doubled

## issue

#5773

## changes

Create a subclass that divides the connect argument of TimeoutSauce by 2.

IMHO blindly halving the specified timeout is not a good strategy, specially if the underlying issue is still unknown.

Besides, there are a few issues with that implementation, from not handling a `None` value to shadowing the imported name

Thank you for looking at my code.

I forgot about handling None.
Dependency on cryptography package breaks upgrades

Unfortunately, the maintainers of the `pyca/cryptography` package have made Rust a hard dependency in newer releases. Since they cut short any discussion about this (not your problem, but a problem for the Python community in general), I was not able to explain to them that a dependency that is required by default and must be manually disabled can be considered a hard dependency.

As a result, updating any package that depends on `cryptography` breaks. Yours happens to be a widely used one.

It's not your fault, for which I am sorry. But you can solve this for your package by keeping your requirement on `cryptography` restricted to versions that do not require Rust.

## Expected Result

When I `pip install -U requests[security]`, the update works.

## Actual Result

I get errors that newer versions of the `cryptography` require a Rust toolchain.

## Reproduction Steps

see above.

## System Information

various

Relates to https://github.com/pyca/cryptography/issues/5810

No. In fact, I think we should raise the lower limit to the ones requiring the Rust toolchain. It's significantly more secure which is one of the tenants of this library - maximum security by default for users

Ok.

I disagree. Rust has the potential to introduce more security, but the same was said of Java. We all know how that ended. While Rust is following a different design, it'll take more time to prove its approach.

Well, I raised the issue to the PSF directly, so maybe we'll come back to that. Maybe not.

The problem is that Rust just doesn't support the same range of target platforms that C or C++ do. So this excludes a bunch of e.g. embedded platforms that are currently still viable targets. I don't think that's the right approach for Python as a whole. Hence getting the PSF involved.

But it may be sufficient for requests. At least I'm not going to argue about it here. I tried raising this, if it's rejected then so be it.
Considerations before posting a new  "Issue"

I really appreciate your hard word guys, but I would like to suggest you that tell your lib consumers that your lib is a state of art HTTPimplementantion, that depends on urllib3 and OS TCP/IP stack... which means... this library just consume  and uses another libraries, so this lib is a pretty and easy way to grab content from WebServers,but it depends onthe urllib3 and OS network features and capabilies...

For Everybody, I am not a Contributor, Developer or friend of the ones who really have done this Lib....I just saying what I think and what we all might take in mind before opening an Issue.

Respect you all

To my concerns, I will only say..  OMFG Thanks you all for your great and hard effort...keep it going!!!
Couldn't modify user-agent while sending multipart/form-data request

Here is my code:
```python
with requests.Session() as s:
  s.cookies.set("CASTGC", CASTGC)
  s.get(...) // to get target website cookie through redirecting to CAS service.
  files = {"Filedata": (
                    'upload.png',
                    tmp_file,
                    'image/png'
   )}
  res = s.post('my url', files=files, header={'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:80.0) Gecko/20100101 Firefox/80.0'})
  assert(res.request.body is None) // excepted ======xxxx blabla
```
It is ok after changing user-gent header to '1111', But I need to pass user-agent header to bypass the firewall.

Oops, I found it is caused by the 302 redirecting.
model.RequestEncodingMixin._encode_files does not work for 3 or 4 elements tuple

https://github.com/psf/requests/blob/913880c45a3a8c3bf6b298e9c38709cd95a9c97c/requests/models.py#L110-L118
Anotation for this function mentions "The tuples may be 2-tuples (filename, fileobj), 3-tuples (filename, fileobj, contentype) or 4-tuples (filename, fileobj, contentype, custom_headers).", but it does not work for 3 or 4 elements tuple.

It will throw exception in following line
https://github.com/psf/requests/blob/913880c45a3a8c3bf6b298e9c38709cd95a9c97c/requests/models.py#L141

My test code:
```python
import requests

with open("file.txt", "r") as f:
    requests.get("http://www.example.com", files=[("file.txt", f, "multipart/form-data")])
```

And exception:
```
...
...
    for (k, v) in files:
ValueError: too many values to unpack
```

would you comfire which one, the anotation or the code, is right? thx~

That is not the documentation or annotation you should be reading for that parameter. Please read the appropriate documentation
Invalid JSON serialization of data with NaNs when using `json` request argument

When passing data to `json` argument in a `requests.post()` (or comparable) request and data contains "nan" floats (`float("nan")`), the data payload will be invalid JSON (in strict sense). Example:

    >>> import requests
    >>> requests.__version__
    '2.25.1'
    >>> data = {"foo": float("nan")}
    >>> resp = requests.post("https://httpbin.org/post", json=data)
    >>> resp.json()["data"]
    '{"foo": NaN}'

While the `NaN` in this result (the JSON serialized data payload that is sent) is valid JavaScript, it is not valid JSON.

The reason is that stdlib's `json.dumps` has `allow_nan` option which is `True` by default and consequently generates invalid JSON by default:

    >>> json.dumps(data)
    '{"foo": NaN}'


## Expected Result

A ValueError should be raised (at least by default). E.g.:

    >>> json.dumps(data, allow_nan=False)
    ...
    ValueError: Out of range float values are not JSON compliant

## Actual Result

library silently sends invalid JSON




I am a first-time contributor and I would like to work on this issue and implement it if possible? Do let me know. Thank you!


Hello, I am not sure if I have fixed this with the PR #5810 , can someone check it out ?
Would anyone know why using requests with proxies through repl.it would cause this SSLError?

As the title stated, I'm attempting to ping google to check the quality of the proxy through an api in repl.it but when I try it gives this error: 

HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1125)')))

![image](https://user-images.githubusercontent.com/67934523/109589099-6ff3ce00-7acf-11eb-92a2-a8b91fc370c5.png)


This isn't a Q&A forum and is likely best asked to someone familiar with `repl.it`.
No Error is thrown when proxie url or key is wrong or the proxie is not reachable

Summary.

## Expected Result

When I make a get request with 

```proxies = {key: value}
request.get(url, proxies=proxies)
```
I would assume that an error is thrown if the key value pair is not correct or if the proxie connection was not possible.
So if i would write something like this 
```{"i am a wonderfull key": "i am a wonderfull value"}```,
i think an error should be the output.

Also i think there should be an output if the proxie connection was not possible, so if the key: value pair is right but the proxie is not reachable.

## Actual Result

Nothing, the request just works fine you dont get an error message.

## Reproduction Steps

```python
import requests

proxies = {"i am a wonderfull key": "i am a wonderfull value"}
request.get(url=example.url, proxies=proxies)
```

```python
import requests

proxies = {"http": "unreachable proxie"}
request.get(url=example.url, proxies=proxies)
```

## System Information


```
{
  "chardet": {
    "version": "3.0.2"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.7"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "1010104f"
  },
  "urllib3": {
    "version": "1.26.2"
  },
  "using_pyopenssl": false
}
```

Hi @Mettfisto,

It looks like you may have some confusion about how the proxies dict works in Requests.

> I would assume that an error is thrown if the key value pair is not correct or if the proxie connection was not possible.
So if i would write something like this
{"i am a wonderfull key": "i am a wonderfull value"},
i think an error should be the output.

There's no reason this needs to throw an error. We allow users to implement their own Adapters for arbitrary protocols and adding strict validation on "correct" schemes makes that workflow more difficult. If a key is provided that matches a parsed scheme, we will attempt to use it.

> Also i think there should be an output if the proxie connection was not possible, so if the key: value pair is right but the proxie is not reachable.

We do provide a `requests.exceptions.ProxyError` when the proxy cannot be reached. The examples you've provided though are not valid Python code. If you use them with correctly specified urls, you'll find they either (correctly) ignore the proxy because the schemes do not match, or throw an appropriate error.
Issue while issuing GET requests to Google owned services.

GET Requests to Google owned services don't work or work very very slowly(minutes). Example of urls having problems : Google.com Google.fr Youtube.fr Youtube.com Googleapis.com
I have no issue with any other website(I tried github.com, facebook.com and a bunch of others)
It worked an hour ago, it doesn't work anymore, I didn't change anything.
I can still access those urls fine on my browser.

## Expected Result

I expect to get a response, but the program seems to be loading(?) indefinitely

## Actual Result

It seems to be loading indefinitely. Rarely, I get a response, 99% of cases(Only 2 worked) it just doesn't do anything and the program doesn't stop or show anything

## Reproduction Steps

```python
import requests

a = requests.get("https://www.youtube.fr")
print(a)

```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.0"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "1010104f"
  },
  "urllib3": {
    "version": "1.26.3"
  },
  "using_pyopenssl": false
}```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


It fixed by itself... Not sure what it was since no error messages. I haven't restarted my computer or done anything.
Provide directions for cloning repository despite malformed commits

Issues #2690, #3008, #3088, #3805 #4254 all refer to the issue with commit 5e6ecdad9f69b1ff789a17733b8edc6fd7091bd8 that cause Git to error out when cloning the repository. As originally discussed in #2690, this issue cannot be avoided without rewriting history, so why don't we instead include some helpful directions at the bottom of the README?
Enhance RequestEncodingMixin._encode_params to support complex dict struct when content-type is x-www-form-urlencoded

Add an util's method in `utils.py` named `unfold_complex_data_key`, called by `RequestEncodingMixin._encode_params` to support complex dict struct when content-type is `x-www-form-urlencoded`.
The function unfold_complex_data_key just do the follow work:
Unfold complex dict which has deep level key or list to simple list[tuple]
when requests.post()'s pemeter data is
`{ "id": "857-37-9333", "label": "User", "count": 0, "properties": { "name": "Rich Hintz", "city": "New Edythstad", "gender": "male", "age": 24, "profile": [ "Zondy", "ZTESoft", "YunWen", "Ci123" ] } }`
encoding the date to follow format:
`[('id','857-37-9333'), ('label','User'), ('count',0), ('properties[name]','Rich Hintz'), ('properties[city]','New Edythstad') ('properties[gender]','male'), ('properties[age]',24), ('properties[profile][0]','Zondy'), ('properties[profile][1]','ZTESoft'), ('properties[profile][2]','YunWen'), ('properties[profile][3]','Ci123')]`

There's already a way to do this in the requests-toolbelt and this feature has been rejected before in this project
enhance RequestEncodingMixin._encode_params to support complex dict struct when content-type is x-www-form-urlencoded

Add an util's method in `utils.py`, named `unfold_complex_data_key`, called by `RequestEncodingMixin._encode_params` to support complex dict struct when content-type is `x-www-form-urlencoded`.
The function unfold_complex_data_key just do the follow work:
Unfold complex dict which has deep level key or list to simple list[tuple]
when requests.post()'s pemeter data is 
`
 {
           "id": "857-37-9333",
           "label": "User",
           "count": 0,
           "properties": {
               "name": "Rich Hintz",
               "city": "New Edythstad",
               "gender": "male",
               "age": 24,
               "profile": [
                   "Zondy",
                   "ZTESoft",
                   "YunWen",
                   "Ci123"
               ]
           }
}
`
encoding the date to follow format:
`
[
        ('id','857-37-9333'),
        ('label','User'),
        ('count',0),
        ('properties[name]','Rich Hintz'),
        ('properties[city]','New Edythstad')
        ('properties[gender]','male'),
        ('properties[age]',24),
        ('properties[profile][0]','Zondy'),
        ('properties[profile][1]','ZTESoft'),
        ('properties[profile][2]','YunWen'),
        ('properties[profile][3]','Ci123'),
]
`
Params list with one item

If parameter has list with only one item, then the list is converted to a string.

## Expected Result

`params={'fields': ['fieldname']}` should convert to `?fields[]=fieldname`

## Actual Result

`params={'fields': ['fieldname']}` converts to `?fields=fieldname`

## Reproduction Steps

```python
import requests
requests.request(method,
                             posixpath.join(self.base_url, url),
                             params={'fields': ['fieldname']},
                             data=payload,
                             headers=self.headers)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "4.0.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.1"
  },
  "platform": {
    "release": "4.19.121-linuxkit",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "1010104f"
  },
  "urllib3": {
    "version": "1.26.3"
  },
  "using_pyopenssl": false
}
```


Hi @iBobik,

The behavior you're seeing is what's intended. Our documentation shows how we handle parameter encoding for lists and it's identical regardless of the length of the list. We'll generate a key-value pair for every list element.

The proposed expectation is non-standard and violates ABNF for the [URI query component](https://tools.ietf.org/html/rfc3986#section-3.4), so we won't be changing anything at this time.
Fix: #4362 - Redirect resolved even though allow_redirects is set to False causing exception for unsupported connection adapter

Good evening

The problem was that when checking, we got data that passed the condition test: they contained `:` , and the request did not start with `http`. Example `0.0.0.0:8080`.

Done:
- [x] Fix #4362
- [x] Add and fix tests
- [x] make test
- [x] flake8 (diff)

Good evening, @kennethreitz42

Please review my edits when you have some free time

Hi @LuckyDenis, we're looking at this as a possible candidate for 2.27.0. There was some feedback left in July on the current proposal. Would you have time to rebase this change and address that?
HTTPDigestAuth uses quotes for algorithm and qop tokens of the Digest header

The `HTTPDigestAuth.build_digest_header()` method produces the Digest header witch contains quoted strings for `algorithm` and `qop` tokens. E.g.:
`
Digest username="admin", realm="server", nonce="QScBItGtnPq4Dz3v25Tht4SlctJnsR", uri="/api/v1/info", response="e0d12a4b85789351a847c773e6f4b30e", algorithm="MD5", qop="auth", nc=00000001, cnonce="0f905170a2cafe15"`

While according to [RFC 7616](https://tools.ietf.org/html/rfc7616) these tokens must not be quoted:
`
Digest username="admin", realm="server", nonce="QScBItGtnPq4Dz3v25Tht4SlctJnsR", uri="/api/v1/info", response="e0d12a4b85789351a847c773e6f4b30e", algorithm=MD5, qop=auth, nc=00000001, cnonce="0f905170a2cafe15"`

Below is the [corresponding part of the RFC](https://tools.ietf.org/html/rfc7616#section-3.4):

_For historical reasons, a sender MUST only generate the quoted string syntax for the following parameters: username, realm, nonce, uri, response, cnonce, and opaque._

_For historical reasons, a sender MUST NOT generate the quoted string syntax for the following parameters: algorithm, qop, and nc._

This can also be seen in [requests examples](https://tools.ietf.org/html/rfc7616#section-3.9) in the RFC.

Current behavior may cause problems with some servers. The following subclass can be used as a temporary workaround:

```
class FixedHTTPDigestAuth(HTTPDigestAuth):
    def build_digest_header(self, method, url):
        header = super().build_digest_header(method, url)
        invalid_parts = ('algorithm', 'qop')
        parts = header.split(', ')
        for i, part in enumerate(parts):
            if any(part.startswith(ip + '=') for ip in invalid_parts):
                parts[i] = part.replace('"', '')
        header = ', '.join(parts)
        return header
```
Loss of the cookie state when using max-age=0.

Summary.
Loss of the `cookie` state when receiving a response from the server containing the lifetime in the format `max-age=0`.

## Expected Result

```python
import requests
r = requests.get('http://127.0.0.1:5000/expire')
>>> r.headers
{'Set-Cookie': 'test=test; Path=/; Max-Age=0', 'Content-Type': 'text/html; charset=utf-8', 'Content-Length': '10', 'Server': 'Werkzeug/1.0.1 Python/3.8.7', 'Date': 'Fri, 05 Feb 2021 09:26:48 GMT'}
>>> r.cookies
<RequestsCookieJar[Cookie(version=0, name='test', value='test', port=None, port_specified=False, domain='127.0.0.1', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=False, expires=1612520808, discard=False, comment=None, comment_url=None, rest={}, rfc2109=False)]>
```

## Actual Result

```python
>>> import requests
>>> r = requests.get('http://127.0.0.1:5000/expire')
>>> r.headers
{'Set-Cookie': 'test=test; Path=/; Max-Age=0', 'Content-Type': 'text/html; charset=utf-8', 'Content-Length': '14', 'Server': 'Werkzeug/1.0.1 Python/3.8.7', 'Date': 'Fri, 05 Feb 2021 09:16:43 GMT'}
>>> r.cookies
<RequestsCookieJar[]>
```

## Reproduction Steps

app.py
```python
from flask import Flask, make_response


app = Flask(__name__)


@app.route('/expire')
def expire():
    return "Expired cookie", {'Set-Cookie': 'test=test; Path=/; Max-Age=0'}
```
Start app: `FLASK_APP=app.py flask run -p 5000`

test.py
```python
import requests


r = requests.get('http://127.0.0.1:5000/expire')
print('headers:', r.headers)
print()
print('cookies:', r.cookies)
```

output
```bash
headers: {'Set-Cookie': 'test=test; Path=/; Max-Age=0', 'Content-Type': 'text/html; charset=utf-8', 'Content-Length': '14', 'Server': 'Werkzeug/1.0.1 Python/3.8.7', 'Date': 'Fri, 05 Feb 2021 09:40:32 GMT'}

cookies: <RequestsCookieJar[]>
```

## System Information

    $ python -m requests.help

```bash
{
  "chardet": {
    "version": "4.0.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.7"
  },
  "platform": {
    "release": "5.8.0-41-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.26.3"
  },
  "using_pyopenssl": false
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Good evening.
The cookie state is deleted at this location.
https://github.com/python/cpython/blob/39aeb9ff9064808b08ec629403edbc36a232369b/Lib/http/cookiejar.py#L1548-L1561
Safe Option for the Character Endoing of GET Request Parameters

This issue is related to character encoding of the request parameters when sending a GET request. In my scenario, I need the percent character to not be encoded as %25. The `urllib.parse.urlencode` function has the `safe` input for this purpose. However, the `requests.get` function does not have such input. Therefore, I use `urllib.parse.urlencode` with `safe='%'` to get an ampersand-separated string of my request parameters and send that in for the `params` value of `requests.get`. Even with this trick, the response that I get is different than what I get with `urllib.request.Request`. The interesting thing is that when I print `response.url` (where `response` is the response object of `requests.get`), I see that the percent character is not encoded, but still the response is incorrect. It seems like encoding takes place somewhere else. I really hate abandoning the requests package because of this defect. I was wondering if there is any way this can be fixed in a future release.

## Expected Result

```
<result>
    <TotalMatches>1135</TotalMatches>
    <TotalPages>57</TotalPages>
    <PageNumber>1</PageNumber>
   <item>
       <mid>36145</mid>
       <merchantname> ...
```

## Actual Result

```
<result>
    <TotalMatches>0</TotalMatches>
    <TotalPages>0</TotalPages>
    <PageNumber>1</PageNumber>
</result> 
```

## Reproduction Steps

This is the URL I am hitting: 

`http://productsearch.linksynergy.com/productsearch?token=<token>&mid=36145&cat=Women%27s+Warehouse+Sale+-+Up+To+70%+Off&keyword=dress&pagenumber=1`

Unfortunately, I cannot share the token. The decoded value for the `cat` parameter is "Women's Warehouse Sale - Up To 70% Off". The percent sign after 70 is the one that is causing the issue.

```python
import requests

with requests.Session() as http_session:
     response = http_session.get('http://productsearch.linksynergy.com/productsearch', params='token=<token>&mid=36145&cat=Women%27s+Warehouse+Sale+-+Up+To+70%+Off&keyword=dress&pagenumber=1')
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "4.0.0"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.2"
  },
  "platform": {
    "release": "5.8.0-40-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.26.3"
  },
  "using_pyopenssl": false
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


In the future, please search **closed and** open issues before creating new ones that are duplicates.

There are many issues here that are closed with the same root issue (wanting greater control over query parameter encoding). The short answer is to use the prepared request flow.
Timeout not working

Summary.

## Expected Result
div with temp email and not loading after couple seconds of loading
<input class="emailbox-input opentip" data-original-title="Your 10 Minute Mail address" data-placement="bottom" id="mail" onclick="select(this);" readonly="" type="text" value=**""**/>
What you expected.

## Actual Result
blank

## Reproduction Steps
No paused between timeout



```python

import requests
with requests.Session() as s:
  post = s.get('https://temp-mail.org/en/10minutemail',headers=headers,timeout=20)
  data = bs(post.text,'html.parser')
  email = data.find('input',{'id':'mail'})
  print(email)

```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "3.2.1"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.0"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "1010108f",
    "version": "20.0.1"
  },
  "requests": {
    "version": "2.24.0"
  },
  "system_ssl": {
    "version": "1010107f"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": true
}

google callaborator
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
How to force session pool connections to reestablish?

I have an app (desktop) that use sessions for keepalive connections, there is a case when users restart internet connection or switch vpn on/off, the next 2-3 requests (no matter how much time passed after switching internet) made from session hang until read timeout but internet connection is already up (script that runs in parallel in separate thread that making same requests but without session is already started to receive responses). How can I force session to reeastablish all connections or any other workaround?

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
How to send multiple images to Fastapi using requests library

I need to hit the api with multiple images 
```
@app.post("/text")
def get_text(files: List[UploadFile] = File(...),frame_timestamps: Optional[List[int]] = Body([]),is_video:bool=False)
```
its working when I try uploading multiple images using /docs interface, I tried with one file its working fine here is the code for it 
```
import requests
import json
def get_text(image_path):
    #images={}
    url = 'http://address/text''
    try:
        with open(image_path, "rb") as im:
            image_data={"files":im}
            response=requests.post(url,files=image_data)
            return json.loads(response.text)
    except Exception as er:
        print("error occured")
        return "{} error occured".format(er)
```
when I tried adding one more image. to the image_data I am getting error . tried this 
```
image_data ={"files":[]}
for image in image_list:
    with open(image, "rb") as im:
        image_data['files'].append(im)
```
The error I am getting is 
<img width="1015" alt="Screenshot 2021-01-25 at 9 27 02 AM" src="https://user-images.githubusercontent.com/38287214/105663586-a2227680-5ef8-11eb-9438-fa3bc09c51de.png">


I finally found the solution 
Incase any one  needs  the solution here it is 
```
files = [
        ('files', ('image1', open('/Users/ai/image1.jpg','rb'), 'image/png')),
        ('files', ('image2', open('/Users/ai/image2.jpeg','rb'), 'image/png'))
        ]
```
you can use the below function for multiple files
```
import requests
import json

def get_text(image_list,url):
    try:
        image_data=[]
        for image in image_list:
            image_data.append(('files',(image.split('/')[-1],open(image,'rb'),'image/png')))#('files',(image_name,open image,type))
        response=requests.post(url,files=image_data)
        return json.loads(response.text)
    except Exception as er:
        print("error occured")
        return "{} error occured".format(er)
```

You can check docs [Here][1] 

Thanks..!


  [1]: https://requests.readthedocs.io/en/latest/user/advanced/#post-multiple-multipart-encoded-files
endless redirection because of rewriting query containing "[]".

It's probably a combination of python3 libraries, since I am not seeing this effect on my webbrowser nor cURL.
I tried disabling automatic redirection but my subsequent prepare_request calls ends up in the same rewriting.

## Expected Result

[] should not rewrite save characters %5B%5D I think because those are safe characters?. For redirection, It probably just should take the redirection from the server as granted anyways.

## Actual Result

[] in queries gets rewritten to %5B%5D. Server rewrites %5B%5D back to [] in Location response header in a redirect.

## Reproduction Steps

```python
import requests
session = requests.Session()
session.send(session.prepare_request(requests.Request(method='get', url='https://www.suruga-ya.jp/search?category=&search_word=&restrict[]=brand=ATARI')), allow_redirects=False)
# ( just some example URL with [] )
```

## System Information

    $ python -m requests.help
```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "3.3.1"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.1"
  },
  "platform": {
    "release": "20.3.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "1010109f",
    "version": "20.0.1"
  },
  "requests": {
    "version": "2.25.1"
  },
  "system_ssl": {
    "version": "1010109f"
  },
  "urllib3": {
    "version": "1.26.2"
  },
  "using_pyopenssl": true
}
```
(the server runs Drupal 8 with PHP/7.3.11 if that matters)

## Workaround

In urllib3.util.url I replaced
`QUERY_CHARS = FRAGMENT_CHARS = PATH_CHARS | {"?"}`
to
`QUERY_CHARS = FRAGMENT_CHARS = PATH_CHARS | {"?","[","]","(",")","=",":","_"}`
(I think there are more safe characters though)

urllib3 follows RFC 3986 on URLs. If you look at the [ABNF](https://tools.ietf.org/html/rfc3986#appendix-A) for query you'll see that `[` and `]` are not allowed (they are characters in `gen-delims` so not in `pchar`) so I would argue that the server is not handling your request correctly. 

Further, not escaping the Location header can lead to a host of security issues, which is frankly unacceptable as a solution

Thank you for the elaborate answer. Accepted.
I am a bit shocked however of the many client interpretations. It's probably all about compatibility with old/buggy sites.

Funny to see how `curl/7.64.1` gives `GET /test?&wow[]=test HTTP/1.1` and `GET /test?&wow=test%5B%5D HTTP/1.1` depending on position relative to the `=`. (Using -G and --data-urlencode) 
Chrome doesn't care and just passes along the URL.
Of course some characters are still percent-encoded.