Raibek/requests-github-issues huggingface dataset:
v2.19.0

363 days later, final prep steps for release.

* Updated HISTORY.rst.
* Moved version pin for idna to allow recent 2.7 release.
* Updated version warning for urllib3 to allow 1.23.
* Updated metadata dunders in __version__.py.
* Remove Python 2.6 trove classifier.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4682?src=pr&el=h1) Report
> Merging [#4682](https://codecov.io/gh/requests/requests/pull/4682?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/7e297ed95bdbd1018657f5d6000379ecdfa54423?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `0%`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4682/graphs/tree.svg?height=150&token=mZGswIiXLa&width=650&src=pr)](https://codecov.io/gh/requests/requests/pull/4682?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4682   +/-   ##
=======================================
  Coverage   66.79%   66.79%           
=======================================
  Files          15       15           
  Lines        1563     1563           
=======================================
  Hits         1044     1044           
  Misses        519      519
```


| [Impacted Files](https://codecov.io/gh/requests/requests/pull/4682?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/\_\_version\_\_.py](https://codecov.io/gh/requests/requests/pull/4682/diff?src=pr&el=tree#diff-cmVxdWVzdHMvX192ZXJzaW9uX18ucHk=) | `0% <0%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4682?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4682?src=pr&el=footer). Last update [7e297ed...100dc40](https://codecov.io/gh/requests/requests/pull/4682?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
idna version pin does not allow version 2.7

`idna` have just released 2.7. `requests` is pinned to `idna>=2.5,<2.7`

https://github.com/requests/requests/blob/d1aeb7ea8cec833e61ea390a6c19ef8b7ac3e4e7/setup.py#L53

Can the pin be updated to allow v2.7?

Hi @jakul, this is addressed (generally, replace "urllib3" with "idna") in https://github.com/requests/requests/issues/4673#issuecomment-395069592. We'll update our version pins in the next release of Requests (which will likely be this week). Thanks for letting us know!

hitting it in openstack as well now.

http://logs.openstack.org/40/574140/2/check/requirements-tox-py27-check-uc/7a0ed61/job-output.txt.gz#_2018-06-11_17_06_11_715280

Alright, 2.19.0 is out the door! Closing this out since it should be resolved now.
The conflict between proxies and TLSv1.2

I want to use the HTTPS proxy, the first request is success, but all subsequent requests failed.
the crash log is:

> HTTPSConnectionPool(host='***', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'sslv3 alert unexpected message')],)",),))

I used Wireshark to view the raw packets, the first request all packets protocol is TLSv1.2, the subsequent requests failed at **Client Hello** which protocol is TLSv1. (When i using other language never appear TLSv1)
And i aslo set ssl_version to be ssl.PROTOCOL_TLSv1_2 with requests.adapters, but it doesn't work.

 Finally i check up urllib.

> request = urllib.request.Request(url)
    request.set_proxy('host:port', 'https')
    context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
    response = urllib.request.urlopen(request, context=context)
    response.read().decode('utf-8')

when i using TLSv1.2 the crash log is:

> <urlopen error [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:777)>

when i using TLSv1 the crash log is: (not every subsequent requests crash)

> <urlopen error [SSL: SSLV3_ALERT_UNEXPECTED_MESSAGE] sslv3 alert unexpected message (_ssl.c:777)>

Version list:

- python: 3.6.3
- requests: 2.18.4
- openssl: stable 1.0.2o

Closed due to inactivity.
idna had a release

we should upgrade our bounds to avoid people running into trouble.

https://github.com/kjd/idna/commit/f8389a87092e5a5eb2f92aabf4fbcef4fc9e9499

Hi @russellballestrini, thanks for letting us know! This is something we’ll handle in our release process for the next formal Requests release.

This breakes most installations of 2.18.4. We fixed it in our case by adding `idna==2.6` to our `requirements.txt` before `requests`, in case someone else has this problem.
In docs, use intersphinx to link to Python documentation

The intersphinx extension can generate automatic links to the documentation of objects in other projects. It was already used for urllib3. For complete details on intersphinx, see:

http://www.sphinx-doc.org/en/master/ext/intersphinx.html

# [Codecov](https://codecov.io/gh/requests/requests/pull/4678?src=pr&el=h1) Report
> Merging [#4678](https://codecov.io/gh/requests/requests/pull/4678?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/922aa406b53f14a67264e07d12fb042704be7374?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4678/graphs/tree.svg?height=150&width=650&token=mZGswIiXLa&src=pr)](https://codecov.io/gh/requests/requests/pull/4678?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4678   +/-   ##
=======================================
  Coverage   66.79%   66.79%           
=======================================
  Files          15       15           
  Lines        1563     1563           
=======================================
  Hits         1044     1044           
  Misses        519      519
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4678?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4678?src=pr&el=footer). Last update [922aa40...5aa6a9b](https://codecov.io/gh/requests/requests/pull/4678?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hey @jdufresne, thanks for helping simplify this. Would you mind rebasing this onto the current tip of master?

Rebased. Thanks

Rebased, thanks
Fix unicode auth url bug in py3

Just Fixed bug in [issue4649](https://github.com/requests/requests/issues/4649)

I just updated one file, but why the ci can't pass?

Hi @mecforlove, thanks for contributing to Requests! The issue here is that you can't call `encode` on all valid inputs for username and password, specifically bytes. We're going to need a solution that works for all the inputs we test, as well as some new/updated tests to ensure this doesn't end up regressed later.

Further, as explained in my last comment, we can not fix this in Requests 2. Requests 3 will be Python 3 only and will make this fixable.
Update Python 2 doc links to Python 3

The Python 3 docs are better maintained and are the future of Python development.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4676?src=pr&el=h1) Report
> Merging [#4676](https://codecov.io/gh/requests/requests/pull/4676?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/d98f881118993483e4e9e2be384286ac2f90bb0a?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4676/graphs/tree.svg?height=150&token=mZGswIiXLa&width=650&src=pr)](https://codecov.io/gh/requests/requests/pull/4676?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4676   +/-   ##
=======================================
  Coverage   66.62%   66.62%           
=======================================
  Files          15       15           
  Lines        1564     1564           
=======================================
  Hits         1042     1042           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4676?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4676?src=pr&el=footer). Last update [d98f881...35a1874](https://codecov.io/gh/requests/requests/pull/4676?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Don't warn about compatibility with urllib3 v1.23

Requests allows v1.23 in #4669, but the compatibility check at import
time still looks for 22. Bump the version to 23.

edit: I do wonder if these checks should just be removed since it's pinned in setup.py and pip doesn't let you install 23 anyway without #4669.

Signed-off-by: Jeremy Cline <jcline@redhat.com>

# [Codecov](https://codecov.io/gh/requests/requests/pull/4675?src=pr&el=h1) Report
> Merging [#4675](https://codecov.io/gh/requests/requests/pull/4675?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/d98f881118993483e4e9e2be384286ac2f90bb0a?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4675/graphs/tree.svg?width=650&height=150&token=mZGswIiXLa&src=pr)](https://codecov.io/gh/requests/requests/pull/4675?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4675   +/-   ##
=======================================
  Coverage   66.62%   66.62%           
=======================================
  Files          15       15           
  Lines        1564     1564           
=======================================
  Hits         1042     1042           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4675?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4675?src=pr&el=footer). Last update [d98f881...015347e](https://codecov.io/gh/requests/requests/pull/4675?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


FYI: similar patches at https://github.com/requests/requests/pull/4671 and https://github.com/requests/requests/pull/4674 were rejected.

Hey @jeremycline, thanks for putting this together. We’re aware the tip of master is inconsistent and merging #4669 was a bit premature. In order to prevent multiple follow up PRs for other things we missed, we (maybe just I) decided we’d just wait to do all of the upgrade steps at once with the 2.19 release as per our usual release process.

We’ve addressed this a few times in a few other PRs and issues (#4671, #4673, #4674).

Ah cool. I should have checked closed PRs, sorry for the noise.
fixed urllib3 version dependency issue

The `requests` setup.py file allows this version range for `urllib3` `>=1.21.1,<1.24`. However, the `__init__.py` has assertion statements that only allow up to (and including) version `1.22`. In this PR, I'm updating the `assert` statements in `__init__.py` to be inline with the `setup.py` `requires`

Hi @nmante, thanks for the contribution to Requests! We’re aware of the recent release of urllib3 and the current pin discrepancy at the tip of master. The current mismatch between the __init__.py doesn’t affect a current release but only the master branch which will be updated.

This issue has been addressed a few times in #4669, #4671, and #4673. It will be addressed with the release of Requests 2.19. 

@nateprewitt got it! I was working with release 2.18.4 which was causing issues because of the mismatch. 

Just to reiterate what’s in the other threads in case someone stumbles on this. The release of 2.18.4 has an intentional requirement for urllib3 < 1.23. If your environment modifies the dependency requirements after Requests is installed, we emit a warning stating the dependencies no longer match the installation requirements and things are no longer guaranteed to work as expected.

Everything should work fine with Requests 2.18.4 as long as the correct dependencies are installed. Thanks again @nmante for checking in!
remove references to 2.6 support in documentation

This will remove our "official" support for 2.6 from the documentation only. We'll keep the work done to remove Python 2.6 code from the Requests code base (#4118) in the Requests3 branch so we don't forcibly break users installs with the 2.19.0 release.

@sigmavirus24 I looked at moving to urllib3's `secure` options, but wasn't immediately able to get `        'security': ['urllib3[secure]>=1.23']` working in our setup.py. That's likely the last step to close out #4659 if we're ok with urllib3 managing the security dependencies. I'll try to look closer later this week.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4672?src=pr&el=h1) Report
> Merging [#4672](https://codecov.io/gh/requests/requests/pull/4672?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/16fd7e03a980bea2dfa0f64c80bbda7cd7834d0f?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4672/graphs/tree.svg?height=150&src=pr&token=mZGswIiXLa&width=650)](https://codecov.io/gh/requests/requests/pull/4672?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4672   +/-   ##
=======================================
  Coverage   66.62%   66.62%           
=======================================
  Files          15       15           
  Lines        1564     1564           
=======================================
  Hits         1042     1042           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4672?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4672?src=pr&el=footer). Last update [16fd7e0...de64f0f](https://codecov.io/gh/requests/requests/pull/4672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Allow urllib 1.23 (part 2)

Follow-up of https://github.com/requests/requests/pull/4669

This eliminates the following warning:
```
$ python -c 'import requests'
/usr/lib/python3.6/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.23) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
```

Hmm, thanks @yan12125. This is why we usually do all of these steps at once. I may have jumped the gun this morning by merging th first PR. Rather than merge multiple follow ups, I think we’ll wait until we’re ready to release, likely next week to get everything wrapped up. I’ll try to go through this tonight and get things tidied with the urllib3 upgrade.

No problem and thanks for the awesome work! Indeed most likely there will be more follow-ups - I just tried to run tests locally and some of them failed with urllib3 1.23. Not sure why Travis CI and AppVeyor didn't catch them, though.

when would this get released to pypi?

@maxnbk: According to this comment: https://github.com/requests/requests/issues/4673#issuecomment-395069592 - a week or so
Add current request url tip when ConnectionError

I got an error about "ConnectionError" when I using thirdparty sdk to request server.
The thirdparty sdk is using requests to request server.
`
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))
`

I spent a long time looking for the error.
I print the request url from sdk source code, finally know why I get the error.
The sdk server domain has changed "https", but sdk source code still use "http".
Error message never tell me current url is "http" or "https".

If I can get the request url from error message, I will know how to fix that as soon as.

Hi @ChandlerBent, thanks for the suggestion! There are a host of reasons a server might close the connection prematurely, many of which are not specific to the scheme choice. We are raising this exception from an underlying library (urllib3, which raises it from the standard lib) and don't modify its contents in Requests.

My hunch is that you don't have `requests[security]` installed and which is possibly why you're having issues connecting. It's hard to tell without the information requested when opening these issues. I'm not sure that adding the url into the exception message would provide an obvious answer in the common case. Many times people are specifically requesting an https url in this case, rather than receiving a redirect.

Given that there's answers to this readily available in the closed issues section or with a quick google search for the exception message, I'm not sure there's much more to do here in Requests.
Allow urllib3 1.23

Changelog: https://github.com/urllib3/urllib3/blob/1.23/CHANGES.rst#123-2018-06-04

# [Codecov](https://codecov.io/gh/requests/requests/pull/4669?src=pr&el=h1) Report
> Merging [#4669](https://codecov.io/gh/requests/requests/pull/4669?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/00fd4c8eb4ac0fd7b8f8d76bbf15ab06351c052c?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4669/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4669?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4669   +/-   ##
=======================================
  Coverage   66.62%   66.62%           
=======================================
  Files          15       15           
  Lines        1564     1564           
=======================================
  Hits         1042     1042           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4669?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4669?src=pr&el=footer). Last update [00fd4c8...ac944b7](https://codecov.io/gh/requests/requests/pull/4669?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Can you please bump `requests` with this fix so we can fix the current version conflicts?

There are a lot of other libs out there that don't cap `urllib3` meaning they are failing due to this cap

Hi @bar, I’ve just commented on a PR to address this. We usually won’t modify this until we are ready for a release. It shouldn’t break anything, but simply emits a warning.

We’ll have a release out in the next week or two once urllib3’s new version has had a chance to settle. I’ll try to get master a bit cleaner this evening. 

Lots of warning emails send me here. I'm going to edit the local code now because I don't like the coming thousands of emails.
'requests' with proxy not working

Here are my test code:
` import requests
    url = 'https://api.ipify.org/'
    proxyapi = 'http://ip.11jsq.com/index.php/api/entry?method=proxyServer.generate_api_url&packid=1&fa=0&fetch_key=&qty=1&time=1&pro=&city=&port=1&format=txt&ss=1&css=&dt=1'
    proxy = {'http' : 'http://{}'.format(requests.get(proxyapi).text)}
    print ('Downloading with fresh proxy.', proxy)
    resp = requests.get(url, proxies = proxy_new)
    print ('Fresh proxy response status.', resp.status_code)
    print (resp.text)
    #terminal output
    Downloading with fresh proxy. {'http': 'http://49.84.152.176:30311'}
    Fresh proxy response status. 200
    222.68.154.34#my public ip address`

with no error message and seems that the requests lib never apply this proxy settings. The proxyapi is valid for I've checked the proxy in my web browser, and by visiting https://api.ipify.org/, it returns the desired proxy server's ip address.
    
I am using python 3.6.4 and requests 2.18.4.


Found that the reason is about the proxy protocol,  visit https websites with https proxy.
Best practices around logging and error handling of HTTP request failures

This is a request for best practices, or possibly an improvement of the documentation. I'm aware that I was referred to the [StackOverflow tag](https://stackoverflow.com/questions/tagged/python-requests), but anything around best practices seems quite likely to be "closed as subjective" there.

I'm wondering what the best practices are around requests and urllib3 for logging HTTP request failures.

Concretely, I'm making calls to a 3rdparty API with requests in background tasks (non-UI). Of course I leave `verify` on and I *want* all API connections to fail if the certificate isn't valid.

urllib3 seems to have the policy of logging on WARN or even ERROR when something fails, right before raising the exception. My monitoring systems generally notify me when WARNINGs or ERRORs occur, but this immediate logging means I'm losing the distinction between a caught exception and an uncaught exception.

Does that mean that it is considered "okay" and a safe practice to silence the `urllib3.connections` logger all the way? It just feels wrong...

```
'loggers': {
    'urllib3.connection': {
        'level': 'CRITICAL',
    },
},
```

Is there a design decision to never log anything on ERROR that doesn't also raise an exception? (I can see how this probably should be asked on the urllib3 project.)

How do other requests users who run tasks in the background handle this balance?

---

On a related note, I think the documentation of how to handle these (specific) exceptions could be improved. To handle the certificate issue, here's what I managed to do:

```
except requests.exceptions.SSLError as sslexc:   # NOT urllib3.exceptions.SSLError!!!
    if isinstance(sslexc.message, MaxRetryError):
        maxretryexc = sslexc.message
        if isinstance(maxretryexc.reason, urllib3.exceptions.SSLError):   # NOT requests.exceptions.SSLError!!!
            innersslexc = maxretryexc.reason
            if isinstance(innersslexc.message, urllib3.packages.ssl_match_hostname._implementation.CertificateError):   # NOT ssl.CertificateError!
                certexc = innersslexc.message
                msg = certexc.message
                logger.info('Certificate validation failed for customer %s: %s' % (customer, msg))
    raise
```

Is there a better way to do this? I found [Quickstart: Errors and Exceptions](http://docs.python-requests.org/en/master/user/quickstart/#errors-and-exceptions), but not much else, and it doesn't really deal with how to handle fine-grained exceptions.



Closed due to inactivity.
ConnectionError

i get this error every time i try to extract tweets. Can anyone please help:

ConnectionError: HTTPSConnectionPool(host='stream.twitter.com', port=443): Max retries exceeded with url: /1.1/statuses/filter.json?delimited=length (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x00000205A1A33DA0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
requests follow 302 redirection but didn't change the 'Host' header accordingly.

when using requests with sites that returns a 302 Moved Temporarily, and in the response header there will be a 'Location' header given the target redirect domain.
Now if you also defined the 'Host' header in a headers parameter to be  some fixed domain, the redirect won't be correctly performed.

for example:
## Expected Result
using requests to access https://login.website.com, while customizing headers= {'Host':'login.website.com'}
website returns 302 with 'Location' header > 'https://someothersubdomain.website.com/1.html'
at this moment I expected it redirects correctly to https://someothersubdomain.website.com

## Actual Result
it follows redirect to "https://login.website.com/1.html" rather than the correct someothersubdomain.website.com
inspecting the second http requests automatically sent by 'requests' we can see that the host header is incorrectly persisted to 'login.website.com'

## Reproduction Steps
like I described above.

```python
import requests
url = 'some url that redirect to another subdomain'
headers= {'Host':'login.website.com'}   
res = requests.get(url,headers=headers)
```

## System Information

    $ python -m requests.help

{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.8.1"
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.1"
  },
  "pyOpenSSL": {
    "openssl_version": "100020bf",
    "version": "17.0.0"
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": true
}




Hi there @1600 

I went looking in our documentation for where we have this clearly explained and am having trouble finding it right now. The short answer is:

There are headers (`Host`) included that if the user specifies, Requests does not alter. This means if you want the right thing to be done with your request involving the `Host` header, you have to take full control over the redirect process yourself. We will not change that out without your consent.

This is not a bug, as not stepping on users' toes is a intentional design choice.

Cheers,
Ian

I think maybe someone wants to be redirected to the same domain, but most of the time they don't.

@1600 not so much a feature as much as "We handle that for you, so when you want to specify it we don't know exactly what you want us to do" because there are a few behaviours people expect. So we make the decision of not forcing one and giving everyone the ability to do a little more work to get exactly what they expect.
requests package issue in pip10 version

When I try to uninstall requests package using pip10, below error I am getting.

Cannot uninstall 'requests'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.



That's interesting given that requests uses [setuptools](https://github.com/requests/requests/blob/00fd4c8eb4ac0fd7b8f8d76bbf15ab06351c052c/setup.py#L10) and has for years now. It sounds like the version of requests you have must be incredibly old. 

Yes, It works in the latest version of requests==2.18.4.
KeyError: 'csrf' or 'csrftoken' on python3 with osx

hey folks,

tying to run this command with python3:

#update csrf token for the first time
		sess.headers.update({'X-CSRFToken' : sess.get('https://www.instagram.com/accounts/login').cookies.get_dict()['csrf']})

or

#update csrf token for the first time
		sess.headers.update({'X-CSRFToken' : sess.get('https://www.instagram.com/accounts/login').cookies.get_dict()['csrftoken']})

i changed it many times but it wont work... even not with url 'www.instagram.com'

Result:
[*] Loading proxies
[*] Proxy: <Proxy IE 0.19s [HTTP: Transparent, HTTPS] 54.36.31.204:3128>
[*] 43113 passwords loaded successfully
Traceback (most recent call last):
  File "hackinsta.py", line 178, in <module>
    bruteforce.login()
  File "hackinsta.py", line 128, in login
    sess.headers.update({'X-CSRFToken' : sess.get('https://www.instagram.com/accounts/login').cookies.get_dict()['csrf']})
KeyError: 'csrf'

how can i fix that?

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

![Screenshot_2020-06-15-21-55-17-097_com termux](https://user-images.githubusercontent.com/66963067/84682209-f15ffc00-af52-11ea-9d3c-78b7bded9f57.jpg)
How to solve this..?? Pls help me

> hey folks,
> 
> tying to run this command with python3:
> 
> #update csrf token for the first time
> sess.headers.update({'X-CSRFToken' : sess.get('[https://www.instagram.com/accounts/login').cookies.get_dict()['csrf']}](https://www.instagram.com/accounts/login').cookies.get_dict()%5B'csrf'%5D%7D))
> 
> or
> 
> #update csrf token for the first time
> sess.headers.update({'X-CSRFToken' : sess.get('[https://www.instagram.com/accounts/login').cookies.get_dict()['csrftoken']}](https://www.instagram.com/accounts/login').cookies.get_dict()%5B'csrftoken'%5D%7D))
> 
> i changed it many times but it wont work... even not with url 'www.instagram.com'
> 
> Result:
> [_] Loading proxies [_] Proxy: <Proxy IE 0.19s [HTTP: Transparent, HTTPS] 54.36.31.204:3128>
> [*] 43113 passwords loaded successfully
> Traceback (most recent call last):
> File "hackinsta.py", line 178, in 
> bruteforce.login()
> File "hackinsta.py", line 128, in login
> sess.headers.update({'X-CSRFToken' : sess.get('[https://www.instagram.com/accounts/login').cookies.get_dict()['csrf']}](https://www.instagram.com/accounts/login').cookies.get_dict()%5B'csrf'%5D%7D))
> KeyError: 'csrf'
> 
> how can i fix that?



> ![Screenshot_2020-06-15-21-55-17-097_com termux](https://user-images.githubusercontent.com/66963067/84682209-f15ffc00-af52-11ea-9d3c-78b7bded9f57.jpg)
> How to solve this..?? Pls help me

Hey, send me your Instagram ID I'll explain it their 

Please find another place to have this discussion.
Response.content minor perf improvement

Python 2
```console
$ python -m timeit -s "blist = [b'foo', b'bar'];" "bytes().join(blist)"
1000000 loops, best of 3: 0.243 usec per loop
$ python -m timeit -s "blist = [b'foo', b'bar'];" "b''.join(blist)"
10000000 loops, best of 3: 0.125 usec per loop
```

Python 3
```console
$ python -m timeit -s "blist = [b'foo', b'bar'];" "bytes().join(blist)"
1000000 loops, best of 3: 0.285 usec per loop
$ python -m timeit -s "blist = [b'foo', b'bar'];" "b''.join(blist)"
10000000 loops, best of 3: 0.161 usec per loop
```

# [Codecov](https://codecov.io/gh/requests/requests/pull/4661?src=pr&el=h1) Report
> Merging [#4661](https://codecov.io/gh/requests/requests/pull/4661?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/fe8290b3d0ca21dd8ae5471a59a8bf13d3dcb31e?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4661/graphs/tree.svg?src=pr&token=mZGswIiXLa&width=650&height=150)](https://codecov.io/gh/requests/requests/pull/4661?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4661   +/-   ##
=======================================
  Coverage   66.62%   66.62%           
=======================================
  Files          15       15           
  Lines        1564     1564           
=======================================
  Hits         1042     1042           
  Misses        522      522
```


| [Impacted Files](https://codecov.io/gh/requests/requests/pull/4661?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/models.py](https://codecov.io/gh/requests/requests/pull/4661/diff?src=pr&el=tree#diff-cmVxdWVzdHMvbW9kZWxzLnB5) | `77.4% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4661?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4661?src=pr&el=footer). Last update [fe8290b...c1b828b](https://codecov.io/gh/requests/requests/pull/4661?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks for catching this @ofek!

If others are curious as to why this is happening it's because the syntactic sugar for `b''` built into the language is faster than creating an object via `bytes()`. The same actually holds true for `[]` instead of `list()`, `{}` instead of `dict()`, and the list goes on.
Multipart PUT?

Does `requests` currently support Multipart-encoded files in `PUT` requests?

It seems that both
http://docs.python-requests.org/en/master/user/quickstart/#post-a-multipart-encoded-file
and
https://toolbelt.readthedocs.io/en/latest/uploading-data.html
only work with `POST`. For `PUT` requests, I keep getting error "413 Request Entity Too Large" from my Apache httpd.

Should that be considered a bug, improvement suggestion, or there is some fundamental limitation why `PUT` shouldn't upload large files? (yes, my `GET` would stream the huge file unmodified)

Original StackOverflow question:
https://stackoverflow.com/questions/50533901/multipart-put-with-requests

> I keep getting error "413 Request Entity Too Large" from my Apache httpd.

The documentation might be misleading, but we do support `multipart/form-data` content in a `PUT`. `POST` requests are merely the 99% use-case. It seems your Apache server, however, will not accept the request you're sending for literally the reason it is telling you.

The only potential "bug" with requests might be that your Apache server is expecting us to use the `100-continue` workflow which we cannot support. There are other issues about this and so this is at best a duplicate of those.

Any further information should continue to be added on StackOverflow.
KeyError with Instabot on python 2.7.15 for mac

hey guys,

i'm really trying to fix that problem and did research on internet but im not able to finde the  selution.

can someone help me? *please *freshman

MacBook-Air:instabot.py-master2 XXX$ python example.py
23.05.2018_16:30 Instabot v1.2.0 started at 23.05.2018 16:30:

23.05.2018_16:30 Trying to login as XXXX...

Traceback (most recent call last):
  File "example.py", line 52, in <module>
    unfollow_whitelist=['example_user_1', 'example_user_2'])
  File "/Users/XXX/Desktop/instabot.py-master2/src/instabot.py", line 232, in __init__
    self.login()
  File "/Users/XXX/Desktop/instabot.py-master2/src/instabot.py", line 283, in login
    self.s.headers.update({'X-CSRFToken': r.cookies['csrftoken']})
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/cookies.py", line 329, in __getitem__
    return self._find_no_duplicates(name)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/cookies.py", line 400, in _find_no_duplicates
    raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
KeyError: "name='csrftoken', domain=None, path=None"

solution here: https://github.com/instabot-py/instabot.py/issues/1287

CLOSED
Requests to groupon 403 response

`import requests as re
headers = {
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:60.0) Gecko/20100101 Firefox/60.0',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Language': 'en-US,en;q=0.5',
    'Accept-Encoding': 'gzip, deflate, br',
    'Connection': 'keep-alive',
}


r = re.Session()
print r.get('https://www.groupon.com/signup?ref=https%3A%2F%2Fwww.groupon.com%2Fmyaccount', headers=headers)`


I receive 403 response. How to pass it

Hi @GinVlad, this looks like a request for help. As per our issue tracker guidelines, requests for help should be directed to [StackOverflow](https://stackoverflow.com). You can use the `python-requests` tag to better surface the question. Thanks!
Content-Length is not correct while sending post request

Hello~
I find the "REAL" CL, by 'tcpdump' the server host, is not according to the 'body data'.
The type of data is serialized protobuf string, and I have specified it in headers as "'Content-type': 'application/x-protobuf'".

Known point:
1. Content-Length(ab. CL) will be replaced even if it is specified in "headers".
2. CL can be set arbitrarily by Session.

After times of failed trying, I send the same PB string by 'httplib' instead, which works well.
Comparing the data dumped by tcpdumps, it seems like that data server received has been modified by 'requests' , while 'httplib' did not, even if the string dumped is not human readable.

So I found I did it in a wrong way, the point is how to send bytes string by 'requests' but how to set CL.
Right? 

I tired another way to send data in post: 
```
...
files = {
        'field_A': (
            'fileName_A', 
            open('./pbFile.data', 'rb'),
            'application/x-protobuf', 
            )
        }
response = requests.post(url, headers = headers, files=files)
```
It does not work either.

Please provide us with the output of

```
python -m requests.help
```

If that is unavailable on your version of Requests please provide some basic information about your system (Python version, operating system, etc).

We also need a complete example to reproduce with

Sorry, I missed the env, and can't reproduce again......
Could not fetch URL https://pypi.python.org/simple/spacy/: There was a problem confirming the ssl certificate: [Errno 1] _ssl.c:499: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version - skipping

Please refer to our [StackOverflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.

I'm using Stanford library in my python app.
Now, I want to install Spacy library to implement some text parsing logic.

When i'm trying to install Spacy library with python 2.7 version and pip 9.0.1 version. getting following error

```
C:\Users\Anil Jagtap>pip install -U spacy
Collecting spacy
c:\python27\lib\site-packages\pip\_vendor\requests\packages\urllib3\util\ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
c:\python27\lib\site-packages\pip\_vendor\requests\packages\urllib3\util\ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Could not fetch URL https://pypi.python.org/simple/spacy/: There was a problem confirming the ssl certificate: [Errno 1] _ssl.c:499: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version - skipping
  Could not find a version that satisfies the requirement spacy (from versions: )
No matching distribution found for spacy
c:\python27\lib\site-packages\pip\_vendor\requests\packages\urllib3\util\ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
```

How can I install Spacy library? Please help me. I'm facing this problem last 2 days.

We ask you to search StackOverflow first because these problems are often already there.

To push you in the right direction here's what I've found: https://stackoverflow.com/a/23093834/1953283
verify in Session object ignored when REQUESTS_CA_BUNDLE is set

Summary.

When REQUESTS_CA_BUNDLE has been set as an environment variable, the verify property in a requests.Session() will always be ignored. 

## Expected Result
requests.Session always uses REQUESTS_CA_BUNDLE unless a certificate bundle has been assigned 

## Actual Result
requests.Session always uses REQUESTS_CA_BUNDLE and overides any certificate bundle set on the Session object.

## Reproduction Steps
```REQUESTS_CA_BUNDLE="/lib/python3.6/site-packages/pip/_vendor/certifi/cacert.pem"```
```python
import requests
s = requests.Session()
s.verify = "/non_existing_bundle"
s.get("https://google.com")
```
Expected result:
```OSError: Could not find a suitable TLS CA certificate bundle, invalid path: /non_existing_bundle```

Actual result:
```<Response [200]>```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": ""
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.4"
  },
  "platform": {
    "release": "3.10.0-693.11.6.el7.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": false
}
```

Not sure if this is a bug, but I now need to unset the environment variable first before I can use a different certificate bundle in a session object or use the following workaround.

sessions.py:
```
            #if verify is True or verify is None:
            if verify is True or verify is None and not isinstance(self.verify, str):
                verify = (os.environ.get('REQUESTS_CA_BUNDLE') or
                          os.environ.get('CURL_CA_BUNDLE'))
```


In the future, please search **closed and** open issues before creating new ones that are duplicates.

Duplicate of #3829
bring Pipfile.lock up to date with current constraints

The Pipfile has strayed a bit from our current build constraints and contains information that's no longer consistent with Warehouse. Updating should help with installation and prevent problems like pypa/packaging-problems#147 from reoccurring when users pick Requests as a model for their packages.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4648?src=pr&el=h1) Report
> Merging [#4648](https://codecov.io/gh/requests/requests/pull/4648?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/f3fc892b901abe2c3605f2669af4f6469b896e88?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4648/graphs/tree.svg?token=mZGswIiXLa&width=650&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4648?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4648   +/-   ##
=======================================
  Coverage   66.53%   66.53%           
=======================================
  Files          15       15           
  Lines        1560     1560           
=======================================
  Hits         1038     1038           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4648?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4648?src=pr&el=footer). Last update [f3fc892...c6db879](https://codecov.io/gh/requests/requests/pull/4648?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@nateprewitt if I remember correctly, you're a Pipenv maintainer and the semi-official position is that the project is geared towards applications, not libraries. If that's correct, do you recall the reasoning for adding it to Requests? I'm wondering if we need the hassle of constantly updating the lockfile when we're just a library.

It's just as valid for libraries as `requirements.txt` is. 

So Pipenv is pbr + requirements.txt for libraries then?

@sigmavirus24 I think the reason the term “application dependencies” was chosen is because Pipfile is not (currently) a replacement for setup.py. The Pipfile/Pipfile.lock can be used in a library similar to how requirements.txt is used but won’t be used when installing from Warehouse.

I agree having to update this is tedious and creates noisy PRs. We should avoid updating it if we don’t have to but I also know Kenneth wants Requests to be an example of good practice. With our current lockfile, we’re not adhereing to that tenant. If we can release a stable lockfile and avoid pushing broken lockfiles in the future this should be a much less common occurrence.