Raibek/requests-github-issues huggingface dataset:
Urllib3 >1.26.x | Introduces default User-Agent header

With pyrequest pullrequest https://github.com/psf/requests/commit/03957eb1c2b9a1e5e6d61f5e930d7c5ed7cfe853 v2.25.0 urllib3 is updated to v1.26.0. with this version bump urllib3 has changed default behavior in regards to header "User-Agent" : https://pypi.org/project/urllib3/ 

urllib3 1.26.0
```
Added default User-Agent header to every request (Pull #1750)
Added urllib3.util.SKIP_HEADER for skipping User-Agent, Accept-Encoding, and Host headers from being automatically emitted with requests (Pull #2018)
```
Urllib3 now introduces default User-Agent header when no user-agent header is set. Urlib3 also introduces the function "urllib3.util.SKIP_HEADER" as a means to deactivate default user-agent header.

## Expected Result

Pyrequest should adopt easy means of deactivating "User-Agent" headers by utilizing the "urllib3.util.SKIP_HEADER" function

## Actual Result

Default "User-agent" header is enforced when no header= is specified.
by using headers={"User-Agent": none} no longer deactivate default "user-agent" while using headers={"User-Agent": ""} will deactivate "user-agent" header

## Reproduction Steps

```python
import requests

req = Request('POST', url=BASE_URL + endpoint, data=params).prepare()  
```
---> Deafult "user-agent" is enforced "User-Agent: python-urllib3/1.26.2"
```
req = Request('POST', url=BASE_URL + endpoint, headers={"User-Agent": None}, data=params).prepare()
```
 ---> Does not deactivate "User-Agent"
```
req = Request('POST', url=BASE_URL + endpoint, headers={"User-Agent": ""}, data=params).prepare()
```
Deactivates "User-Agent" succesfully

```

## System Information

    $ python -m requests.help

```
<paste here>
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Python 3.8, Docker


Hi @prilly-dev, thanks for bringing this to our attention. I was actually surprised to see Requests wasn't emitting a header here, but it appears this is specific to the Session workflow.

We'll probably need something that is tolerant to urllib3 < 1.26 not being installed.  I'm not immediately sure if that means conditionally relying on the urllib3 SKIP_HEADER util in compat or if there are other options. We'll take a look at getting a fix together.

For anyone still trying to work around this issue. The temporary solution is as follows:

```python
import requests
import urllib3

# Where user-agent is the header you want to exclude.
# You can find the full list of "skippable headers" at urllib3.util.SKIPPABLE_HEADERS
headers = {'user-agent': urllib3.util.SKIP_HEADER, ... }

req = Request('GET', someurl)
prep = req.prepare()
prep.headers = headers

s = requests.Session()
resp = s.send(prep)
```
a post request body has one key corresponds to multiple values。

requests version = 2.22.0
I had some trouble sending a post request。
headers = {'Content-Type': 'application/x-www-form-urlencoded'}，
the body_data has a key corresponds to multiple values，

![12231BC6-03B4-4D4D-AE56-36CD5F9D08B4](https://user-images.githubusercontent.com/48149977/100365628-b8b46a80-303a-11eb-833c-67971bc4a01c.png)

my scripts
![9207E0D4-4033-4101-BFF7-3E2D75FC4BB2](https://user-images.githubusercontent.com/48149977/100366431-c74f5180-303b-11eb-9784-ab30a2590ae6.png)

data = cargo_info.update(important_info)
requests.post(url, data=data, headers=headers)

the result  takes the cargo_info['cargoName'] as one good('["无烟煤", "铝合金"]')  not two.

and I don't know what to do。please help me。thinks

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Getting stuck even with timeout

I'm trying to request google search results and access the top few links. However, it always gets stuck on some sites. Ctrl-C is not in effect but Ctrl-Z does bring me to the terminal.

I have set a timeout for each request. 
I have read messages that DNS is one of the sources of blocking and I patched this with an HTTP adapter.
It seems that the requests library is using blocking TCP connection(not sure), where the program has no control over the timeout if the underlying implementation does not return.

In another scenario, the connection is perfectly slow or the page is unexpected too big such that this connection keeps forever. This case is obviously not a bug but a vulnerability that can be used to anti-spider.

I would like to set a hard limit for my connection, regardless of DNS, TCP handshaking time, TLS handshaking time, or uploading/downloading time, and at any time I would like to exit, I just enter Ctrl-C. This can be done by using requests in another thread(POSIX thread)/process, ungracefully.

My program just stuck again.

In the future, please search **closed and** open issues before creating new ones that are duplicates.
Add support for HTTPS proxies.

Starting with 1.26 urllib3 now supports HTTPS proxies. To enable the support two changes are needed:

* An additional proxy_kwargs argument that can be passed from the session. This dictionary will be used to pass any arguments needed to the underlying adapter. The parameter is optional.
* The use_forwarding_for_https mode requires us to send the absolute URI when enabled.

The resulting API is very similar except it takes an additional parameter when needed. An example:

```
    session = requests.Session()
    proxies = {"http": "https://proxy.example", "https": "https://proxy.example"}

    proxies_kwargs = {
        "proxy_ssl_context": proxy_ssl_context(),
    }

    response = session.get(
       "https://www.google.com", proxies=proxies, proxies_kwargs=proxies_kwargs
    )
```


Requests is under feature freeze which means we will not add any additional keyword arguments to the public API like this.

@sigmavirus24 thanks for the feedback! Is adding an additional keyword argument to the public API the only concern? I've been reviewing the options available and there's no easy way around this unless we modify the ``proxy_manager_for`` method in ``HTTPAdapter``

Could we add a proxy_kwargs attribute to HTTPAdapter? Rather than passing the ``proxy_kwargs`` dictionary through multiple top level APIs we could expose it like this:

```
s = requests.Session()
s.adapters['https://'].proxy_kwargs = { ... }
```
That minimizes some of the changes needed. Would that work?

Facebook needs these changes upstreamed so I would love it if we can find a path forward here. I can create our own ``HTTPAdapter`` internally but t it leaves the ``requests`` community without being able to benefit from the work I put in to expand the [HTTPS proxy support in urllib3.](https://github.com/urllib3/urllib3/pull/1923)




@nateprewitt Are we still planning on doing this for 2.26.0? It doesn't look like it's moved since being added to the milestone.

I popped it out of 2.26.0 so we can get a release unblocked. I'll follow up with @jalopezsilva on getting this in the next minor release.

That would be amazing @nateprewitt. Let me know which changes are required.
GHA: Tests fail with ubuntu-20.04

GitHub Actions fails when using the Ubuntu 20.04 image, and pass with 18.04.

## Expected Result

Tests pass on GitHub Actions with `ubuntu-20.04`

## Actual Result

Tests fail on GitHub Actions with `ubuntu-20.04` for CPython, pass on PyPy.

## Reproduction Steps

I noticed this annotation on [this recent build](https://github.com/hugovk/requests/actions/runs/368989395):

> Ubuntu-latest workflows will use Ubuntu-20.04 soon. For more details, see https://github.com/actions/virtual-environments/issues/1816

I ran a quick test with explicit `ubuntu-20.04` instead of `ubuntu-latest`:

* https://github.com/hugovk/requests/commit/43e1c9ee709d2efd18c9294b941babd816193f3d
* https://github.com/hugovk/requests/actions/runs/369014648

I've not looked into the failures in detail, one is `requests/adapters.py:514: SSLError`.

The linked https://github.com/actions/virtual-environments/issues/1816 says:

> **This change will be rolled out over a period of several weeks beginning on November, 30.** (December, 22 - January, 4 will be a migration break due to holidays)

It's also possible to explicitly pin to `ubuntu-18.04`, which passes:

* https://github.com/hugovk/requests/commit/69e5c04372eb6f6dc2d14bb8617d47df1ba1c199
* https://github.com/hugovk/requests/actions/runs/369033603


Good catch, @hugovk! It looks like our local httpbin server for tests is having trouble either with the newer version of openssl or configuration being vended in 20.04. I think the best bet is to pin us to ubuntu-18.04 for now. We'll gladly accept a PR for that or I'll work on getting it resolved by the end of the week.
Drop support for EOL Python 3.5

> Requests v2.25.x will be the last release series with support for Python 3.5.

https://requests.readthedocs.io/en/latest/community/updates/

Hey @hugovk,

Thanks for the PR! I think we're unlikely to release Requests 2.26.0 prior to urllib3 2.0's release. That may mean this PR sits unmerged for quite a while.

We also have a standard deprecation process for this that will be done at release time to clean up any remaining deprecation items. I think in general we're less likely to accept PRs with cleanup tasks like this.

Sure, feel free to close this if you like, there's not so much stuff in this one. 👍 

Resolving as this was handled with the final changes for 2.26.0. Thanks again for being proactive with this, @hugovk.
Improve the Session.request() documentation

This PR improves the documentation of the `Session.request()` method. Its `stream` parameter's description used to have remnants of the old `prefetch` parameter (from PR #1283), which are now rephrased slightly.

The new sentence is the same as that in the [Body Content Worklflow](https://requests.readthedocs.io/en/latest/user/advanced/#body-content-workflow) section of the documentation.

This is my first PR in this repo, so please let me know if I missed something.

Repeating documentation doesn't clarify it enough. I would say "open a stream to iterate over content manually"

Thanks for the PR, @pzahemszky. I believe we'll pass on this as it unfortunately doesn't improve clarity and lacks some of the nuance from the [full docs](https://docs.python-requests.org/en/master/user/advanced/#body-content-workflow). To keep doc strings succinct, we'll keep the current wording.
Dead link to "development philosophy" in the contribution guidelines

The [Contribution Guidelines](https://github.com/psf/requests/blob/master/.github/CONTRIBUTING.md) is referencing the "development philosophy" using a seemingly incorrect URL: https://requests.readthedocs.io/en/latest/dev/philosophy/.

Upon accessing that website, I get the following error message.

```

        \          SORRY            /
         \                         /
          \    This page does     /
           ]   not exist yet.    [    ,'|
           ]                     [   /  |
           ]___               ___[ ,'   |
           ]  ]\             /[  [ |:   |
           ]  ] \           / [  [ |:   |
           ]  ]  ]         [  [  [ |:   |
           ]  ]  ]__     __[  [  [ |:   |
           ]  ]  ] ]\ _ /[ [  [  [ |:   |
           ]  ]  ] ] (#) [ [  [  [ :===='
           ]  ]  ]_].nHn.[_[  [  [
           ]  ]  ]  HHHHH. [  [  [
           ]  ] /   `HH("N  \ [  [
           ]__]/     HHH  "  \[__[
           ]         NNN         [
           ]         N/"         [
           ]         N H         [
          /          N            \
         /           q,            \
        /                           \

```
How to requests without removing the ./ or ../ in Python 3

Hi, if you do this:

```python
requests.get('http://127.0.0.1/hello/../blah')
```

Your server will receive:

```
GET /blah HTTP/1.1
```

But what I want here is to send the request with the dot-slash:

```
GET hello/../blah HTTP/1.1
```

In python2, I can use this (it worked in my test):

```python
import urllib

u = urllib.quote('http://127.0.0.1/./foo/../bar', safe=':/.')
```

But in python 3, it does not work for some reasons:

```python
import urllib.parse

u = urllib.parse.quote('http://127.0.0.1/./foo/../bar', safe=':/.')
```

```
GET /bar HTTP/1.1
```

How can I solve this?

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

Hey, I didn't see any answer that can solve this question on StackOverflow. The solutions are only for python 2. That is why I am asking here

That's why you ask on StackOverflow or search open and closed issues before opening new ones. See also #5324 and #5289.
Requests 2.25.0 issue

## Summary.

Our CI suite just ran and failed with the latest release of requests 2.25.0. Nothing in our code has changed aside from the version of requests installed.

## What you expected.

Our CI tests to pass as normal.

## What happened instead.

You can see the full test run on https://github.com/panoramichq/panoramic-cli/runs/1387129299?check_suite_focus=true for more details, but the important part is we received the following import error:

```
________ ERROR collecting tests/panoramic/cli/test_supported_version.py ________
  ImportError while importing test module '/home/runner/work/panoramic-cli/panoramic-cli/tests/panoramic/cli/test_supported_version.py'.
  Hint: make sure your test modules/packages have valid Python names.
  Traceback:
  .tox/py36/lib/python3.6/site-packages/responses.py:27: in <module>
      from requests.packages.urllib3.connection import HTTPHeaderDict
  E   ImportError: cannot import name 'HTTPHeaderDict'
  
  During handling of the above exception, another exception occurred:
  /opt/hostedtoolcache/Python/3.6.12/x64/lib/python3.6/importlib/__init__.py:126: in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
  tests/panoramic/cli/test_supported_version.py:1: in <module>
      import responses
  .tox/py36/lib/python3.6/site-packages/responses.py:29: in <module>
      from urllib3.connection import HTTPHeaderDict
  E   ImportError: cannot import name 'HTTPHeaderDict'
```

Hi @mchristopher, this was a change in urllib3 that was reverted in 1.26.1. If you update urllib3 as well, you should stop seeing this exception.

That fixed it, but I'm not sure why pip didn't automatically pick up the correct dependencies. It may have been an issue with caches on the pypi side... 🤷 

Either way, fixed now. Thanks for the quick response!
Upgrade Urllib3 Version

I just encountered a problem, when I tried to run a regular script, because the dependencies for requests are a bit outdated. In your setup.py you still require urllib3 with a version limitation: 'urllib3>=1.21.1,<1.26,!=1.25.0,!=1.25.1'. 
My Problem was that the current available version 1.26 is obviously not valid and my pip command fails within the installation of the requests package.

## Expected Result

Installation of package completes as usual.

## Actual Result

ERROR: Could not find a version that satisfies the requirement urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1

## Reproduction Steps

```
   pip install urllib3

   pip install requests
```



This will be resolved tomorrow in #5651.

Requests 2.25.0 should now be released.
Move CI to Github Actions

Moving us to GH Actions since our Travis configs are in relative disarray and we haven't been running Mac/Windows builds for years at this point.

### Other changes in this PR:

**Pipenv**
Pipenv has caused a lot of friction with users trying to submit PR updates that need dependency changes, so this removes it. I've readded a `requirements-dev.txt` to store dev dependencies.

**Removal of pytest-xdist**
`pytest-xdist` causes threading issues on macOS for some versions of Python [StackOverflow issue](https://stackoverflow.com/questions/50168647/multiprocessing-causes-python-to-crash-and-gives-an-error-may-have-been-in-progr/52230415). The performance gain on tests is negligible, so I've removed it for config simplicity. If there's strong demand, we could attempt the bash updates or split out the macOS test command in the action definition.

**Adding Python 3.9**
Adding 3.9 to tests so we can update our trove classifiers before releasing 2.25.


TODO: We'll need to remove the Travis integration in Settings after this is merged.

> TODO: We'll need to remove the Travis integration in Settings after this is merged.

I used to be able to help with that but no longer 

> I used to be able to help with that but no longer

It looks like it's just a toggle in settings we can shut off/remove after merge. The maintainers group should have updated permissions now so we can do that when we're ready.
v2.25.0

This is getting staged for a release this week to stay in sync with urllib3. This release should also resolve the outstanding doc issues with Requests on pypi (#5501).

2.25.0 (2020-11-11)
------------------

**Improvements**

- Added support for NETRC environment variable. (#5643)

**Dependencies**

- Requests now supports urllib3 v1.26.

**Deprecations**

- Requests v2.25.x will be the last release series with support for Python 3.5.
- The `requests[security]` extra is officially deprecated and will be removed
  in Requests v2.26.0.

Just an FYI since we're removing the !=1.25 and !=1.25.1: both of those releases have been yanked from PyPI so shouldn't be an issue.

https://github.com/psf/requests/blob/master/requests/__init__.py#L63 ? ;)

@asavah Great catch!

:tada: Very exciting!
requests.post() does not work until reset modem

Hi everyone
I have an interesting problem. When i run my basic script with raspberry pi i get response but after a couple post,  my codes do not work anymore until i reset the modem or reset my phone if i connect with my phone network.

```
import requests
import json
import time
url = '...'


myobj = {"status":1,"makina_id":1,"ambalaj":{"boyut":0.5,"cins":"plastic"},"doluluk":50,"error":0}
x = requests.post(url,json.dumps(myobj))
print(x)
print(myobj)
```

****Output** is**

```
<Response [200]>

{'status': 1, 'makina_id': 1, 'ambalaj': {'boyut': 0.5, 'cins': 'plastic'}, 'doluluk': 50, 'error': 0}
```
when the problem occuring, there is not any output, any error or warnings.

I could not find what is the problem. Thank you for your helps.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. This is likely related to your network or timeout settings. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Document how to turn off Keep-Alive in 2.x

Ref. #5646

This documentation isn't entirely correct and that issue is already clsoed

@sigmavirus24 I don't know how to improve it. What is not correct here?

I edited the description to mention issue without closing it (for me).

> What is not correct here?

It's not the only way to disable keep-alive. Sessions are not the only way people use this and the solution can be more generic without being specific because people who use this library generally understand the concept of headers and what the Keep-Alive setting in the Connection header is, which is why this doesn't need to be documented as desperately as you think

@sigmavirus24 it is needs to be documented, because in the past there were multiple ways to do this, which don't work now https://stackoverflow.com/questions/21008953/python-requests-1-how-to-disable-keep-alive

Reworded and moved to the paragraph about sessions.

> @sigmavirus24 it is needs to be documented, because in the past there were multiple ways to do this, which don't work now https://stackoverflow.com/questions/21008953/python-requests-1-how-to-disable-keep-alive

Information will always be outdated on the internet. Your initial commit was wrong immediately and is still overly specific to _your_ usecase. Beyond which, you seem to have (potentially deliberately) misunderstood my characterization of the urgency with the necessity

I'm going to resolve this as there hasn't been any movement in the last year. 

Killing keep alive literally.
HEAD Request never returns

Summary.
When used `requests.head` with `url='https://mega.nz'` is stuck forever.
## Expected Result

Either a response or a Timeout error or any exception.

## Actual Result

The function never returns.

## Reproduction Steps

```python
import requests
r = requests.head("https://mega.nz")
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.5"
  },
  "platform": {
    "release": "4.19.104-microsoft-standard",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.24.0"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": false
}
```

I have also verified this on [Google Colab](https://colab.research.google.com)

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

@sigmavirus24  The purpose of filing the issue was that there should a timeout mechanism *inside requests library* to prevent these types of events from happening.

@cstayyab [Try reading the documentation before asking questions](https://requests.readthedocs.io/en/master/user/quickstart/#timeouts).
Unable to turn off Keep-Alive

Documentation doesn't specify how to turn off Keep-Alive
https://requests.readthedocs.io/en/latest/user/advanced/#keep-alive

In the future, please search **closed and** open issues before creating new ones that are duplicates.

I thought I searched. My bad. The duplicate is #4913. However, no answer was given and no docs updated.

Turning off keep-alive means closing the connection. So set a session default for the `Connection` header of `close`. The server will close the connection and respects what the server does.

@sigmavirus24 how does it look in code?

I try to wait until a container is up and turning off Keep-Alive is the last thing I left to try. I start container on Fedora with port mapping that has no listening server inside the container.
```
$ podman run -d -p 9990 -it bash
```

The code below should timeout in 2 seconds. Instead if hangs indefinitely. At least more than a few minutes. I can not close the connection, because it hangs at `s.head()`.
```python
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
from requests.packages.urllib3.util import Timeout

with requests.Session() as s:
    retry = Retry(connect=3, backoff_factor=0.5)
    adapter = HTTPAdapter(max_retries=retry)
    s.mount('http://', adapter)
    s.mount('https://', adapter)
    r = s.head('http://localhost:9990', timeout=Timeout(2), stream=False)
    print(r)
```

requests 2.24.0 on Python 3.9

`s.headers["Connection"] = "close"`
PyPI page formatting

Sorry if this is the wrong place to report but the 'Project Description' for your entry on the PyPI site looks all mixed up with formatting and raw HTML tags.
https://pypi.org/project/requests/



We're aware of the issue. Please read [closed issues](https://github.com/psf/requests/issues/5601) before opening new ones.

Thanks. Not easy if you close issues without fixing them.

@pedro-w There's nothing left for us to fix, it's fixed in master and on the next release to PyPI there will be no issue.
Session.send() does not check for environment variables like proxy

I was following the mechanism of the usage of the PreparedRequest class.
But the `send()` does not call the `merge_environment_settings()` of the session automatically.

## Expected Result

So it would be helpful to enable this via an option or at least update the documentation, that it is necessary to call the `merge_environment_settings()` manually.

## Actual Result

It results in a ConnectionError.

## Reproduction Steps

using Python 3.6:
```
>>> import requests
>>> req = requests.Request('GET', 'https://httpbin.org/get')
>>> r = req.prepare()
>>> r
<PreparedRequest [GET]>

>>> s = requests.Session()
>>> s.send(r)
<Response [200]>
```


## System Information

```
{
  "chardet": {
    "version": "3.0.4"
  }, 
  "cryptography": {
    "version": ""
  }, 
  "idna": {
    "version": "2.8"
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.5"
  }, 
  "platform": {
    "release": "3.10.0-1062.9.1.el7.x86_64", 
    "system": "Linux"
  }, 
  "pyOpenSSL": {
    "openssl_version": "", 
    "version": null
  }, 
  "requests": {
    "version": "2.21.0"
  }, 
  "system_ssl": {
    "version": "100020bf"
  }, 
  "urllib3": {
    "version": "1.24.1"
  }, 
  "using_pyopenssl": false
}
```


This is what `r = s.prepare(req)` is for and it's documented as such
Sending body via requests.post gives empty body in Node server,Why?


Python program Which sends Post Request
![Screenshot 2020-10-25 at 9 08 08 AM](https://user-images.githubusercontent.com/30069626/97098339-cb392e00-16a1-11eb-9d09-f6f4a77d02c6.png)

Logs on NodeJS Express server
![Screenshot 2020-10-25 at 9 10 15 AM](https://user-images.githubusercontent.com/30069626/97098345-e73ccf80-16a1-11eb-878e-94ffc30c8065.png)

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

Thanks! now it is solved.

@Ammyy9908 how did you solve it? I'm also experiencing the same (but on a Go backend).
Exceptionally awkward MaxRetryError exception handling

If one uses custom HTTPAdapter with urllib3 Retry() class the `requests.packages.urllib3.exceptions.MaxRetryError` is ~~wrapped in~~chained with several exceptions making it hard to catch the exception we are looking for - that is MaxRetryError.

Tested on python 3.8 with requests==2.24.0

## Expected Result

If I setup HTTPAdapter with max_retries I actually *want* to catch MaxRetryError exception - this is kinda the point of using Retry() class - to be able to act on specific problem.

## Actual Result

Instead I got `requests.exceptions.ConnectionError` exception. With several wrapped exceptions.

## Reproduction Steps

```python3
import requests
import urllib3.exceptions

from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

s = requests.Session()
retries = Retry(total=2, backoff_factor=1)
s.mount('http://', HTTPAdapter(max_retries=retries))

try:
    s.get("http://localhost")
except requests.packages.urllib3.exceptions.MaxRetryError:
    print("Exception I should actually catch")
except urllib3.exceptions.MaxRetryError:
    print("Eventually this one....")
except requests.packages.urllib3.exceptions.ConnectionError:
    print("This one is kinda the one I'd expect if MaxRetryError is not caught")
except urllib3.exceptions.ConnectionError:
    print("Or this one")
except requests.exceptions.ConnectionError:
    print("But instead I got this one. Sweet baby Jesus. Fair enough - I should have guessed to use this one if I did not want to catch MaxRetryError")
except Exception as ex:
    template = "An exception of type {0} occurred. Arguments:\n{1!r}"
    message = template.format(type(ex).__name__, ex.args)
    print(message)
    raise

```

## System Information

    $ python -m requests.help

```{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.6"
  },
  "platform": {
    "release": "5.5.0-2-amd64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.24.0"
  },
  "system_ssl": {
    "version": "1010108f"
  },
  "urllib3": {
    "version": "1.25.10"
  },
  "using_pyopenssl": false
}

```



Welp, this is actually python3 doing it's glorified exception chaining. No dumpster fire here :D
Ability to specify default timeout value in session object

> Requests is not accepting feature requests at this time.

I've just read [this section](https://requests.readthedocs.io/en/latest/dev/contributing/#feature-requests), and I believe my feature request is still worthwhile and simple, and I'm happy to write it myself. I'm just [getting early feedback](https://requests.readthedocs.io/en/latest/dev/contributing/#get-early-feedback).

I would like the ability to set a default timeout value for a session object, rather than passing in the same argument for every single call.

Existing usage:

```
s = requests.Session()
s.get(url_a, timeout=10)
s.get(url_b, timeout=10)
```

Proposed Usage:

```
s = requests.Session()
s.timeout = 10 # seconds
s.get(url_a)
s.get(url_b)
```

In the future, please search **closed and** open issues before creating new ones that are duplicates.
Prepared Post Requests Timeout after changing body

I am trying to do a POST request using the requests library. However, for this specific request, it is important that the POST data stay unaltered (e.g. do not get URL-encoded).

I have seen in the documentation under "advanced usage" that you can do PreparedRequests. So, I did as shown in the docs, however it does not yield results as expected.

## Expected Result

We should get a response to our request.

## Actual Result

Prepared POST requests with changed body timeout.

## Reproduction Steps

```
python
import requests
s = requests.session()
data = {"file": "../p1.png"}
req = requests.Request(method='POST', url="http://localhost:8000", data=data)
prep = s.prepare_request(req)
prep.body = unquote(prep.body) #from urllib.parse, however the behavior is reproducible with any string, e.g. prep.body = "1234"
r = s.send(prep, timeout=timeout)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.8"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.6"
  },
  "platform": {
    "release": "5.8.15-201.fc32.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010104f",
    "version": "19.0.0"
  },
  "requests": {
    "version": "2.24.0"
  },
  "system_ssl": {
    "version": "1010107f"
  },
  "urllib3": {
    "version": "1.25.7"
  },
  "using_pyopenssl": true
}
```

Tested on local php server. PHP 7.4.11 (cli) (built: Sep 29 2020 10:17:06) ( NTS )

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Upgraded some packages, but issue still persists:

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.9"
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.6"
  },
  "platform": {
    "release": "5.8.15-201.fc32.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010106f",
    "version": "19.1.0"
  },
  "requests": {
    "version": "2.24.0"
  },
  "system_ssl": {
    "version": "1010107f"
  },
  "urllib3": {
    "version": "1.25.11"
  },
  "using_pyopenssl": true
}
```

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

You also likely should introspect the generated headers for clues.
Ocassionally getting temporary failure in name resolution

I am getting name resolution error with requests occasionally, it's happening approx once in 10 requests.

My stackoverflow question https://stackoverflow.com/questions/64357458/failed-to-establish-a-new-connection-errno-3-temporary-failure-in-name-resol




@sigmavirus24 Can you help me this issue?

You asked on StackOverflow which is the correct place. For what it's worth, I maintain systems making thousands of requests per minute with requests that don't have this issue. And a DNS issue is firmly outside of our control given that we use the standard library to query DNS. 

@sigmavirus24 Thanks for the response. I will check my DNS server config.
Segmentation Fault - Core Dumped

Hi, since I'm not 100% sure whether it's a bug or not I'm putting it under "Request for Help".
I looked up what the error actually means and apparently my program attempted to access restricted part of RAM.
You go figure.

I had to use Python debugger to find out that the module throws seg fault on the method call.

Anyway, I'd appreciated any feedback on the below

Request I'm sending:

   ```
 data = {
            'first_name': self.first_name,
            'middle_name': self.middle_name,
            'last_name': self.last_name,
            'new_comment': self.comment
        }
        files = {'image_file1': open(self.file_path1, 'rb')}
        #print(data)
        #pdb.set_trace()
        response = requests.post('http://localhost/users/add', json=data, files=files)
        if response.status_code == 201:
            print('ADDED !')
        else:
            print(response)
```

output from debugger:

```
/home/mark/front_end_android/venv/lib/python3.7/site-packages/requests/api.py(119)post()
-> return request('post', url, data=data, json=json, **kwargs)

Segmentation fault (core dumped)
```

It's likely this is coming from a lower-level library that's using C but at first blush, you can't (and this is documented) combine `json=` and `files=`. So I'm sure the JSON library you're using is doing something to cause this with the open file you're using.

Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here. (Using Stackoverflow is also well documented in this issue template)
Session.close() does not close underlying sockets

It seems that Session.close() does not close the underlying TCP socket. That was unexpected and it wasn't clear to me in the docs. Is there any way to explicitly/forcefully close all underlying sockets in a threadsafe way?

## Expected Result

Using sockets:

```python
"""Demo for Canceling IO by Closing the Socket

"""

import socket
import time

from concurrent import futures

start_time = time.time()

sock = socket.socket()


def read():
    "Read data with 10 second delay."
    sock.connect(('httpbin.org', 80))
    sock.sendall(b'GET /delay/10 HTTP/1.0\r\n\r\n')
    while True:
        data = sock.recv(1024)
        if not data:
            break
        print(data.decode(), end='')


with futures.ThreadPoolExecutor() as pool:
    future = pool.submit(read)
    futures.wait([future], timeout=5)
    sock.close()  # <-- Interrupt sock.recv(1024) in Thread:read().

end_time = time.time()
print(f'Duration: {end_time - start_time:.3f}')

# Duration is ~5s as expected.
```
The duration is ~5s, as expected.

## Actual Result

```python
"""Demo for Canceling IO in Threads

"""

import time

from concurrent import futures

import requests

start_time = time.time()

with futures.ThreadPoolExecutor() as pool:
    with requests.Session() as session:
        request = pool.submit(session.get, 'http://httpbin.org/delay/10')
        done, not_done = futures.wait([request], timeout=5)
        # Session.__exit__() calls Session.close() which should close the
        # socket from the main thread and interrupt the read in the
        # Thread:session.get().

end_time = time.time()
print(f'Duration: {end_time - start_time:.3f}')

# Duration is ~10s as *unexpected*.
# Why does session.close() not interrupt the socket.recv()?
```
The duration is ~10s which was unexpected.

## Reproduction Steps

See code samples above.

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.9"
  },
  "platform": {
    "release": "19.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.24.0"
  },
  "system_ssl": {
    "version": "1010107f"
  },
  "urllib3": {
    "version": "1.25.10"
  },
  "using_pyopenssl": false
}
```

Running in Python 3.7.9 on Mac OSX.


You have to first return the socket to the connection pool by closing the response if you haven't already consumed all the content. Then you can close the Session. Responses hold onto their socket until either:

- All the content has been read into memory (default)
- The user has read all the content (stream=True)
- The user has `close()`'d it (stream=True, but content not consumed)

You'll also note that `s = Session(); s.get(url); s.close(); s.get(url)` works. Closing a session doesn't prevent future requests from being made, but it does prevent connection reuse
ModuleNotFoundError

I have tried multiple things from removing python 2.7 to installing last version. 
I tried in multiple ways installing using PIP.
I keep getting this 'ModuleNotFoundError'
pylint says that it is unable to import the package.
My error
`Exception has occurred: ModuleNotFoundError
No module named 'requests'
  File "file-hidden", line 1, in <module>
    import requests`


Also everything is satisfied if i try reinstalling in pip


I just realized that I am using the windows 10 version of python 3.8. Installing the normal version to see if that is the issue


Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
content-type: media subtypes of "text" using ISO-8859-1 as default charset may be inappropriate

https://github.com/psf/requests/blob/143150233162d609330941ec2aacde5ed4caa510/requests/utils.py#L497
From [RFC 2616](https://tools.ietf.org/html/rfc2616#section-3.7.1):  
   When no explicit charset parameter is provided by the sender, media subtypes of the "text"
   type are defined to have a default charset value of "ISO-8859-1" when received via HTTP. 

However, from [RFC7231](https://tools.ietf.org/html/rfc7231#appendix-B):
  The default charset of ISO-8859-1 for text media types has been removed;

Actually, if requests uses the default charset of ISO-8859-1, the property text of response will  always lose the efficiency of chardet and cause messy code problem.
I think these lines should be deleted.

It's a backwards incompatible change.

In the future, please search **closed and** open issues before creating new ones that are duplicates.
Added python 3.9 to CI, tox and setup.py

# Added Python 3.9 into CI, tox and setup.py

This PR is directly related to this issue #5620 

closes #5620 

**This PR is currently a draft**

I am creating the PR to run test suite/see how CI performs using python3.9 via Actions. I was unable to get a Python3.9 virtual environment working locally so this is my alternative to see if it works. (This is my first open-source contribution so if this approach is poor please go easy on me :) )

Looks like travis doesn't have 3.9 yet

> Looks like travis doesn't have 3.9 yet

I looked through the on-demand python versions for travis and they do have python-3.9-dev.tar.bz2. So the dev version. However I don't know how to target. Any ideas?

Here are the available versions https://docs.travis-ci.com/user/languages/python/#python-versions

The Python 3.9 job failed trying to build greenlet from source:
```
[pipenv.exceptions.InstallError]: Collecting greenlet==0.4.15
[pipenv.exceptions.InstallError]:   Using cached greenlet-0.4.15.tar.gz (59 kB)
[pipenv.exceptions.InstallError]: Building wheels for collected packages: greenlet
[pipenv.exceptions.InstallError]:   Building wheel for greenlet (setup.py): started
[pipenv.exceptions.InstallError]:   Building wheel for greenlet (setup.py): finished with status 'error'
```

* https://travis-ci.com/github/psf/requests/jobs/416898376

The latest greenlet 0.4.17 has a (manylinux) wheel for Python 3.9:

* https://pypi.org/project/greenlet/0.4.17/#files

How about bumping to that version?

> The latest greenlet 0.4.17 has a (manylinux) wheel for Python 3.9:
@hugovk In order to bump the greenlet version I just changed it in the pipfile.lock directly. This caused all of the builds to fail. Is this not how it's supposed to be done?


You'll have to look up how to bump a version in `Pipfile.lock`, it's generated and contains locked hashes.

Hi @hughesp8,

We ended up having to do a larger refactor to remove Pipfile and Travis (#5652) for our upcoming 2.25.0 release. I'm afraid that's going to make this PR obsolete, but I wanted to say thanks for contributing! Hopefully, we'll have other opportunities for you to work on in the future.
Is it possible to support ftp  transfer ?

I see 'class HTTPAdapter' in adapters.py，if it's possible to deal with ftp requests?
Looking forward to your reply ，thanks！

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Inconsistent host behaviour when request is redirected

Summary.

Hi

Im trying to implement a script to test a host header injection. It is quite simple, it sends a valid request to an endpoint, but modifies the Host header injecting a fake domain (evil.com)

When the request is not a redirection, this header is ignored, and the request reaches the original url, however, if the request ends in a redirection (302), and you enable redirections (allow_redirects=True), this redirection is done to the host specified in the header (evil.com)

To summarise:

if i send a request to test.com/blah with Host: evil.com and it returns a 200, the requests goes to test.com/blah

if i send a request to test.com/blah with Host: evil.com and it returns a 302 that redirects to test.com/redirected_site, then, i see the first request going to test.com/blah with a 302, but the redirection, instead of going to test.com/redirected_site, goes to evil.com/redirected_site

So it looks like this redirection is using the host header instead of taking the url generated by the 302 request

Thanks


This isn't behaviour in Requests. We don't look at the `Host` header at all while [handling redirects](https://github.com/psf/requests/blob/143150233162d609330941ec2aacde5ed4caa510/requests/sessions.py#L96).
Preparing request with Session obj

Hi,
this is actually a question more than an issue.

```
req = requests.Request(method=None, url=url, cookies=self.cookies)
prepped = self.session.prepare_request(req)
```

raises error because method is None, and this is actually because of `method.upper()`. [Code here](https://github.com/psf/requests/blob/143150233162d609330941ec2aacde5ed4caa510/requests/sessions.py#L457) -- specifically _AttributeError: 'NoneType' object has no attribute 'upper'_.
(suppose I have an object incapsulating a `session = requests.Session()`; this explains why `self.session` in the above snippet)

On the other hand, 
```
req = requests.Request(method=None, url=url, cookies=self.cookies)
prepped = req.prepare()
```

works fine. In this second case, the same `prepare()` method is called ([code here](https://github.com/psf/requests/blob/143150233162d609330941ec2aacde5ed4caa510/requests/models.py#L254)).

I don't understand if there is a reason for requests made with sessions not to have None method, or if this is a bug.

Thank you!


I'd say it's a bug that `req.prepare()` doesn't raise an exception for a `None` method. It shouldn't raise an `AttributeError` but we can't swap that out for something more reasonable at this point or break `req.prepare()`.
Issue with using http proxies with requests

Hello I have recently been attempting to use http proxies for something im making however when I use http proxies it instead uses my ip but when I use https proxies it actually uses the proxies ip. Slightly confused about this I do not know if there is something stopping this from working, thanks a lot for reading.
I checked this by requesting a IP checker API.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

If you open this again as an issue, you will be prevented from opening further issues on this repository
Requests library (python) with http proxies

Hello I have recently been attempting to use http proxies for something im making however when I use http proxies it instead uses my ip but when I use https proxies it actually uses the proxies ip. Slightly confused about this I do not know if there is something stopping this from working, thanks a lot for reading.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.