Raibek/requests-github-issues huggingface dataset:
Fix a typo in utils

Signed-off-by: Xuanwo <xuanwo.cn@gmail.com>

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3958?src=pr&el=h1) Report
> Merging [#3958](https://codecov.io/gh/kennethreitz/requests/pull/3958?src=pr&el=desc) into [master](https://codecov.io/gh/kennethreitz/requests/commit/f061fb7e28a480e911cc49de7d41f9ec0ff5215b?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/kennethreitz/requests/pull/3958/graphs/tree.svg?height=150&width=650&src=pr&token=mZGswIiXLa)](https://codecov.io/gh/kennethreitz/requests/pull/3958?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3958   +/-   ##
=======================================
  Coverage   89.61%   89.61%           
=======================================
  Files          15       15           
  Lines        1946     1946           
=======================================
  Hits         1744     1744           
  Misses        202      202
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3958?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/utils.py](https://codecov.io/gh/kennethreitz/requests/pull/3958?src=pr&el=tree#diff-cmVxdWVzdHMvdXRpbHMucHk=) | `85.02% <ø> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3958?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3958?src=pr&el=footer). Last update [f061fb7...1cd173a](https://codecov.io/gh/kennethreitz/requests/pull/3958?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@Lukasa Done!
comma in case: join multiple Set-Cookie Headers

Many web-apps send cookie in multiple Set-Cookie headers, one header for one cookie. Requests joins this headers in one big header separate by comma. 

From same webapp to requests.get
```
Set-Cookie: ASP.NET_SessionId=token1; path=/; HttpOnly
Set-Cookie: Cookie1=token2; path=/ecp
Set-Cookie: X-BEResource=WIN-RBFR0BDA7V7.testlab.net~1; path=/ecp/15.0.516.30; HttpOnly
Set-Cookie: X-BackEndCookie=token3; expires=Mon, 03-Apr-2017 12:25:07 GMT; path=/ecp; HttpOnly
```
In requests:
resp.headers  this look like
```
Set-Cookie: ASP.NET_SessionId=token1; path=/; HttpOnly,    
  Cookie1=token2; path=/ecp, 
  X-BEResource=WIN-RBFR0BDA7V7.testlab.net~1; path=/ecp/15.0.516.30; HttpOnly, 
  X-BackEndCookie=token3; expires=Thu, 06-Apr-2017 08:27:22 GMT; path=/ecp; HttpOnly
```
Firefox, IE, Chrome invalid interpreted this cookies because separater is comma but in  field EXPIRES  comma separate day and date. 


Principal problem in modification traffic. In my opinion then server send multiple headers lib must return multiple headers

Requests makes no promises about leaving headers in the form they are received from the server. Right now, I highly recommend you use the cookiejar Requests uses to store cookies rather than extracting them from the headers: we correctly insert them in there.

Longer term, we should probably add a fix to exclude the `Set-Cookie` header from joining in this way.

This has bitten us and cost us quite a few hours. I'd like to vote to leave `Set-Cookie` intact. Why does requests fiddle with the headers anyways? Is there some good reason for it? IMHO it only complicates things for everybody. What am I missing?
Thanks for your good work!

Hi, there is a way to parse headers correctly with https://github.com/Ousret/kiss-headers `requests` won't be able to change their representation for headers any time soon.

In order to access the unmerged header lines, use the underlying `urllib3` response object ([HTTPResponse](https://urllib3.readthedocs.io/en/latest/reference/urllib3.response.html#urllib3.response.HTTPResponse)):

~~~py
resp = requests.get('https://myurl.com')
print(resp.raw.headers.items())
~~~

Yields (for example):

~~~
[('Cache-Control', 'no-store, no-cache, must-revalidate'),
 ('Cache-Control', 'post-check=0, pre-check=0'),
 ('Cache-Control', 'no-cache="set-cookie"'),
 ('Content-Type', 'text/html; charset=utf-8'),
 ('Date', 'Tue, 22 Feb 2022 08:12:47 GMT'),
 ('Expires', 'Mon, 26 Jul 1997 05:00:00 GMT'),
 ('Last-Modified', 'Tue, 22 Feb 2022 08:12:47 GMT'),
 ('Pragma', 'no-cache'),
 ('Server', 'Apache/2.4.27 (Amazon) PHP/5.6.35'),
 ('Set-Cookie', 'PHPSESSID=q0br22iudsfjjkshfakl54076m1; path=/; HttpOnly'),
 ('Set-Cookie', 'lan=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; Max-Age=0; path=/; domain=mydomain.com'),
 ...]
~~~

In my case, I had to do `response.raw.headers`, adding `.items()` at the end returns an empty `HTTPHeaderDictItemView`.

I'm closing this with [the above solution](https://github.com/psf/requests/issues/3957#issuecomment-1047539652) because it's unlikely we'll be changing our Response object anytime soon.
content-length is missing from requests

Can reproduce with the below:

```
import requests
r = requests.get('https://downloads.atlassian.com/software/confluence/downloads/atlassian-confluence-6.0.5.zip', stream=True)
print r.headers.get('content-length')
None
```

This results in None - I'm  using content-length for a download progress bar and can't get it with requests ;(

If we use something else like urllib it returns it:

```
import urllib
r = urllib.urlopen(url).info()
print r
Content-Type: application/zip
Content-Length: 541131731
```

This used to work a while back, not sure if anything changed.  Reproduced with Python 2.7.13 & 3.6.0 with requests 2.13.0.

If you look at the headers that *are* returned, you will notice

```
Transfer-Encoding: chunked
```

If you compare the headers that are sent, however, using `httpbin`, you might notice that Requests also sends `Accept-Encoding: gzip, deflate`. If you remove that header, e.g., 

```py
r = requests.get(url, stream=True, headers={'Accept-Encoding': None})
```

Then you get your `Content-Length` header. This has been a header that's been set for quite a long time (I think since around when 1.0 was released) so you must have last used this years ago. I hope this helps.

Cheers!
Ian

@sigmavirus24 you're the best, thanks - I totally missed this. 👍 

@sigmavirus24 sorry for mention. I still have this issue even when using your suggest.

for example when I want to get content-length from this image link

https://kenhsinhvien.vn/topic/conan-chap-699-tieng-viet.371791/ there is no result

Could you please give some advice?

@quyleanh it's possible that server doesn't send the content length ever. Some servers prefer not to calculate that if possible. Regardless, this isn't a question and answer support forum and you'll have better luck on stack overflow

@sigmavirus24  thank you for replying. Actually I have already asked on [stackoverflow](https://stackoverflow.com/questions/62496190/).
The thing is although I cannot get the `Content-Length` with `get` method, I still can get with `head` method. So the server sent the content length, right? Then what is the problem?
You can try with the link I send above.

You need to add Content-Encoding param , in my case only with this works, and content-length is not zero

`r = requests.get(url, stream=True, headers={'Accept-Encoding': None, 'Content-Encoding':'gzip' })`
Small help in docs to indicate client side authentication

See https://github.com/kennethreitz/requests/issues/3951

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3952?src=pr&el=h1) Report
> Merging [#3952](https://codecov.io/gh/kennethreitz/requests/pull/3952?src=pr&el=desc) into [master](https://codecov.io/gh/kennethreitz/requests/commit/993cbf57419fca7562ff0990df1bd32f748a9bfe?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/kennethreitz/requests/pull/3952/graphs/tree.svg?width=650&src=pr&token=mZGswIiXLa&height=150)](https://codecov.io/gh/kennethreitz/requests/pull/3952?src=pr&el=tree)

```diff
@@          Coverage Diff           @@
##           master   #3952   +/-   ##
======================================
  Coverage    89.6%   89.6%           
======================================
  Files          15      15           
  Lines        1944    1944           
======================================
  Hits         1742    1742           
  Misses        202     202
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3952?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/sessions.py](https://codecov.io/gh/kennethreitz/requests/pull/3952?src=pr&el=tree#diff-cmVxdWVzdHMvc2Vzc2lvbnMucHk=) | `92.93% <ø> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3952?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3952?src=pr&el=footer). Last update [993cbf5...a63ae02](https://codecov.io/gh/kennethreitz/requests/pull/3952?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Request signing and access to SSLContext

I've just finished porting a large project from httplib to requests, it talks to over 50 APIs with pretty much every weird and obtuse spec you could think of.

The only one I got slightly stuck on was the OpenStack Identity, Azure Management (classic) APIs and Docker when TLS is enabled. All require certificate based authentication.

In `httplib` we would use the std library and access SSLContext to wrap the connection with a signed payload. I spent a few hours researching the correct way to do this in `requests`, found that I could access them via `urllib3` and create an ssl context through a utility method https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/util/ssl_.py#L321-L324 but didn't want to couple myself directly to the internals of a 3rd party package. 

The `PoolManager` initializer has a kwarg for both `key_file` and `cert_file`, it doesn't expose those in the Session API in any way that I could see without having to write a custom adapter like this:

```python
class SignedHTTPSAdapter(HTTPAdapter):
    def __init__(self, cert_file, key_file):
        self.cert_file = cert_file
        self.key_file = key_file
        super(SignedHTTPSAdapter, self).__init__()

    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(
            num_pools=connections, maxsize=maxsize,
            block=block,
            cert_file=self.cert_file,
            key_file=self.key_file)

session = requests.Session()
session.mount('https://', SignedHTTPSAdapter(cert_file, key_file))
```

I'd like to help the next person and save them the 3 hours it took me to figure this out by either making it easier, documenting the solution or the scenario. 

Happy to raise PR for either 

1. Lift access to set the `cert_file` and `key_file` higher up in the API so it doesn't require a custom adapter. 
2. Leave it as is, document this in the advanced section, happy to contribute docs on how to make a signed request.
3. Nothing needs to change and this issue can be a reference for the next person!

Did you see the `cert` argument in the docs? It should cover what you need. 

Cert argument on the session initializer? http://docs.python-requests.org/en/master/api/#requests.Session.cert

Ah ok it can be a tuple to provide cert_file and key looking here
https://github.com/kennethreitz/requests/blob/993cbf57419fca7562ff0990df1bd32f748a9bfe/requests/adapters.py#L247

Do you think this justifies a documentation example? 

It can also be provided per call to requests.*. This is documented in [the docs](http://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification), but could be expanded on I think.

opened a PR, it was probably just me missing the obvious. but hopefully this helps the next person!

Improvement in documentation clarity is always helpful!
Header payload

I'm trying to post image to a server like this:
```
headers = {
	'Accept': '*/*',
	'Content-Type': 'image/png'
}

files = { None: open('2.png', 'rb')}
response = request.post('http://server', headers = headers, files = files)	
```

but in debug tool of request I see that:
```
### --9eddf6ca1dde46278024b688838539ad
Content-Disposition: form-data; filename="2.png"

PNG
binary data....
```
But server not accept this image. 
if I upload image from safari browser it worked the request looks like:
```
PNG
binary data....
```
without `Content-Disposition: form-data; filename="2.png"`
How can I remove this meta info in request via requests?  

You want to change to this:

```python
headers = {
	'Accept': '*/*',
	'Content-Type': 'image/png'
}

files = { None: open('2.png', 'rb')}
response = request.post('http://server', data = open('2.png', 'rb'), headers = headers)
```
Redirection behaviour not consistent between Python2 & 3

Hi there,
consider this minimal test:

```
from requests import exceptions, Session

url="http://ansgar-skoda.tumblr.com/post/96703389502"

try:
    s = Session()
    r = s.get(url)
except exceptions.TooManyRedirects as t:
    print('ERROR: {}'.format(t))
    r = s.get(url, allow_redirects=False)

print('========')
print(r.status_code)
print(r.headers)
```

Calling the link in a browser or via `curl -L` will follow the redirects and return a HTTP 200 after the redirection. That's what I expect.

The code above acts according to expectation when executed with Python2 but not with Python3. 
With Python3 you'll get stuck in a redirection loop that gets broken by the TooManyRedirection exception.

In both cases I installed requests 2.13.0 via pip.

I originally posted that on [Stackoverflow](http://stackoverflow.com/questions/43155127/how-to-correctly-setup-redirection-with-python-requests-for-some-corner-cases).


Hey @BastiTee, thanks for taking the time to open this. The reason you're seeing this work in Python 2 vs Python 3 is the character encoding used. It also appears to be a dupe of #3888 but that puts more weight towards addressing this.

Since Python 2 has bytes as the default representation for strings, we're receiving the redirect url which ends with `gewÃ¤sch-und#_=_` as the already `latin-1` encoded byte sequence `gew\xc3\xa4sch-und#_=_`. This allows the redirect to succeed because that's the correct encoding for the address.

In Python 3, it stores the strings as `utf-8` by default, which when decoded back to bytes ends with the utf-8 equivalent byte sequence `gew\xc3\x83\xc2\xa4sch-und#_=_`. This is causing the redirect to continuously loop because the utf-8 encoding isn't correct.

For the time being, you should be able to manually encode the redirect headers containing non-ascii characters as `latin-1` bytes in Python 3. That's obviously not ideal, but seems to be the only thing we can suggest at present.

Yeah, I'm closing this as a dupe. The suggested fix on #3888 should be the direction we go. 

Okay, thanks for suggesting a workaround. For so. else here's a workaround using the information above:
```
import requests
from requests import exceptions

url="http://ansgar-skoda.tumblr.com/post/96703389502"

try:
    r = requests.get(url)
except exceptions.TooManyRedirects as t:
    # see https://github.com/kennethreitz/requests/issues/3949
    r = requests.get(url, allow_redirects=False)
    latin1_location = r.headers['Location'].encode('latin1')
    r = requests.get(latin1_location)
    
print('========')
print(r.status_code)
print(r.headers)
```
`datetime.datetime() is not JSON serializable` error 

If I do a `request()` with JSON containing unserializable types, an exception is raised from the `json` module.

e.g. `requests.post('http://example.com', json={'time': datetime.datetime.now()})`

https://github.com/kennethreitz/requests/issues/1380 looks vaguely related, but doesn't directly cover this.

https://docs.python.org/2/library/json.html#json.JSONEncoder seems pretty well suited for this case; if it was possible to configure the json encoder class that Requests uses, then users could supply an implementation that handles their special cases.

Should be possible to either plumb the encoder through from the `request` call, or make it package-level config in some way.  Would you consider a patch to make the JSONEncoder that `request` uses configurable?

(The solution of just massaging the data is viable in many cases, but for complex dicts it is going to be quite painful.)

So, we absolutely can do that, but ultimately I'm not sure that the value proposition is worth it. In 99% of use-cases the JSON argument literally only does `json.dumps` and then sets the `Content-Type` header to `application/json`. If you're not setting the JSON field *and* some other fields, you can totally replicate that behaviour yourself in two lines of code.

We added it only because the simple use of JSON was a neat shortcut that most users would find helpful. For more advanced uses, we strongly recommend just rolling your own.

Is that likely to work out for you?

Yeah sounds reasonable, that's probably a line more code on my side to just
dump the json myself, and keeps the library conceptually simpler.

:+1:

On Fri, Mar 31, 2017 at 12:24 AM, Cory Benfield <notifications@github.com>
wrote:

> So, we absolutely can do that, but ultimately I'm not sure that the value
> proposition is worth it. In 99% of use-cases the JSON argument literally
> only does json.dumps and then sets the Content-Type header to
> application/json. If you're not setting the JSON field *and* some other
> fields, you can totally replicate that behaviour yourself in two lines of
> code.
>
> We added it only because the simple use of JSON was a neat shortcut that
> most users would find helpful. For more advanced uses, we strongly
> recommend just rolling your own.
>
> Is that likely to work out for you?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/kennethreitz/requests/issues/3947#issuecomment-290637261>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ARTsilP7WEVWcFaatRHlciSUbnMNIe6Iks5rrKoigaJpZM4MvKsR>
> .
>
requests takes a half-second to import

I noticed it was taking a half-second just to run `--help` for my app, and narrowed it down to requests 2.12.0 taking a long time to load.

I'm on Python 2.7.13, Fedora 25.

Here is the 2.12 series:
```
$ pip install requests==2.12.0
$ time python -c 'import requests'
real	0m0.487s
user	0m0.445s
sys	0m0.040s
$ strace -e open python -c 'import requests' 2>&1 | wc -l
6836
```
0.487 seconds to import, almost 7000 open() calls.

Compared to 2.11.1:
```
$ pip install requests==2.11.1
$ time python -c 'import requests'
real	0m0.085s
user	0m0.066s
sys	0m0.019s
$ strace -e open python -c 'import requests' 2>&1 | wc -l
2321
```
under a tenth of a second to import, and 66% fewer open() calls.

Can you show us the output of `pip freeze`? Does this happen when the *only* change you make is what Requests version is installed?

Oh, never mind, I know what this is: this is due to the idna library we vendored in that release. Does this slowness apply up to the current 2.13 release?

Thanks for the pointer at 2.13. I tried that one and the same problem occurred. Then I realized that it only happens when I use the `--system-site-packages` argument to `virtualenv`. I'll need to do more digging to find out what's going on here.

Oh that's very interesting. It's possible that the delay you're seeing is actually to do with PyOpenSSL, which has in some older versions had very long import times.
response.iter_lines(decode_unicode=True) sometimes returns bytestrings

```python
response = requests.get('https://s3-us-west-2.amazonaws.com/org.mozilla.crash-stats.symbols-public/v1/wntdll.pdb/D74F79EB1F8D4A45ABCD2F476CCABACC2/wntdll.sym', stream=True)
for line in response.iter_lines(decode_unicode=True):
    print(type(line), repr(line[:50]))
    break

response = requests.get('https://s3-us-west-2.amazonaws.com/org.mozilla.crash-stats.symbols-public/v1/ntdll.pdb/120028FA453F4CD5A6A404EC37396A582/ntdll.sym', stream=True)
for line in response.iter_lines(decode_unicode=True):
    print(type(line), repr(line[:50]))
    break
```
When I run this, I get this output:
```
▶ python3.5 test.py
<class 'str'> 'MODULE windows x86 D74F79EB1F8D4A45ABCD2F476CCABAC'
<class 'bytes'> b'MODULE windows x86 120028FA453F4CD5A6A404EC37396A5'
```

Here's what I get with python 2.7 (with `from __future__ import print_function`)
```
▶ python test.py
<type 'unicode'> u'MODULE windows x86 D74F79EB1F8D4A45ABCD2F476CCABAC'
<type 'str'> 'MODULE windows x86 120028FA453F4CD5A6A404EC37396A5'
```


Actually, I think it's `iter_content`, not `iter_lines` specifically. 
```python
response = requests.get('https://s3-us-west-2.amazonaws.com/org.mozilla.crash-stats.symbols-public/v1/wntdll.pdb/D74F79EB1F8D4A45ABCD2F476CCABACC2/wntdll.sym', stream=True)
for chunk in response.iter_content(100, decode_unicode=True):
    print(type(chunk), repr(chunk[:50]))
    break

response = requests.get('https://s3-us-west-2.amazonaws.com/org.mozilla.crash-stats.symbols-public/v1/ntdll.pdb/120028FA453F4CD5A6A404EC37396A582/ntdll.sym', stream=True)
for chunk in response.iter_content(100, decode_unicode=True):
    print(type(chunk), repr(chunk[:50]))
    break
```
Yields
```
▶ python3.5 test.py
<class 'str'> 'MODULE windows x86 D74F79EB1F8D4A45ABCD2F476CCABAC'
<class 'bytes'> b'MODULE windows x86 120028FA453F4CD5A6A404EC37396A5'
```

Err... Nevermind. It was my wrongful assumption that both files were UTF-8 encoded and that that's exposed in some header. They might very well be UTF-8 but S3's download headers doesn't not expose that. 

I can manually set the response encoding, before I consume the stream, with 

```python
response = requests.get('https://s3-us-west-2.amazonaws.com/org.mozilla.crash-stats.symbols-public/v1/ntdll.pdb/120028FA453F4CD5A6A404EC37396A582/ntdll.sym', stream=True)
response.encoding = 'utf-8'
for chunk in response.iter_content(100, decode_unicode=True):
    print(type(chunk), repr(chunk[:50]))
    break
```
Now it works.
HTTPAdapter.send() timeout defaults

This PR allows to use urllib3 ``Timeout`` objects as ``timeout`` argument in ``HTTPAdapter.send()``.

Besides it omits the ``timeout`` argument when calling ``conn.urlopen()`` if ``HTTPAdapter.send()`` is called without a timeout. This allows setting a default timeout at connection pool level:
`adapter.poolmanager.connection_pool_kw['timeout'] = urllib3.Timeout(...)`

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3943?src=pr&el=h1) Report
> Merging [#3943](https://codecov.io/gh/kennethreitz/requests/pull/3943?src=pr&el=desc) into [master](https://codecov.io/gh/kennethreitz/requests/commit/993cbf57419fca7562ff0990df1bd32f748a9bfe?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `83.33%`.

[![Impacted file tree graph](https://codecov.io/gh/kennethreitz/requests/pull/3943/graphs/tree.svg?width=650&src=pr&token=mZGswIiXLa&height=150)](https://codecov.io/gh/kennethreitz/requests/pull/3943?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3943      +/-   ##
==========================================
- Coverage    89.6%   89.58%   -0.02%     
==========================================
  Files          15       15              
  Lines        1944     1950       +6     
==========================================
+ Hits         1742     1747       +5     
- Misses        202      203       +1
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3943?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/adapters.py](https://codecov.io/gh/kennethreitz/requests/pull/3943?src=pr&el=tree#diff-cmVxdWVzdHMvYWRhcHRlcnMucHk=) | `92.55% <83.33%> (-0.27%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3943?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3943?src=pr&el=footer). Last update [993cbf5...1be6a17](https://codecov.io/gh/kennethreitz/requests/pull/3943?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

I've added copies of the existing timeout tests using Timeout() objects instead of timeout tuples.

Great, thanks so much!
requests.put redirect from http to https doesn't forward JSON payload

Python version - 2.7.12
Requests version - 2.13.0

I was incorrectly hitting an internal server with an put http request versus an https and the json=<blah> blob doesn't appear to be passed through. The headers and params keys do appear to be passed through.



Hey @colekas-csc, this is intended functionality in Requests 2.13.0. When Requests receives a redirect, it strips the payload to prevent unintentional transfer of information to the wrong server. We're a tad overzealous with this right now which is why this behavior will be modified in 3.0.0, but for now we must maintain this for backwards compatibility.

Thanks for taking the time to check in on this though!
request with allow_redirect use the wrong Host in Headers ( setting the Host in headers)

send a request with allow_redirect=true and set the Host,  but auto redirect the 302 response will use the wrong Host in Headers
```
s = requests.session()
url = 'https://sso.jeffxue.com/login'
headers.update({'Host', 'sso.jeffxue.com'})
datas = {'username': 'xxxx', 'password': 'xxxxx', service: 'web'}
cookies = {'xxxx': 'xxxxx'}
r = s.post(url, data=datas, headers=headers, cookies=cookies, allow_redirects=True, timeout=10)
```


but after checking the history, found it use the wrong Host like:

first request use the right host, because i set the Host in headers 
```
history request url: https://sso.jeffxue.com/login
history request headers: {'Content-Length': '52', 'Accept-Encoding': 'gzip, deflate', 'Connection': 'keep-alive', 'Accept': '*/*', 'User-Agent': 'python-requests/2.9.1', 'Host': u'sso.jeffxue.com', 'Cookie': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', 'Content-Type': 'application/x-www-form-urlencoded'}
history response status_code: 302
history response headers: {'Content-Length': '0', 'Set-Cookie': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxx', 'Business-Status': '200', 'Expires': 'Wed, 31 Dec 1969 23:59:59 GMT', 'Connection': 'keep-alive', 'Location':
'https://corp.jeffxue.com/index?ticket=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx&client=web', 'Pragma': 'no-cache', 'Cache-Control': 'no-store', 'Date': 'Mon, 27 Mar 2017 07:58:02 GMT'}
```

first 302 request will redirect to the diff host(corp.jeffxue.com), but it will use the sso.jeffxue.com in headers 
```
history request url: https://corp.jeffxue.com/index?ticket=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx&client=web
history request headers: {'Accept-Encoding': 'gzip, deflate', 'Host': u'sso.jeffxue.com', 'Accept': '*/*', 'User-Agent': 'python-requests/2.9.1', 'Connection': 'keep-alive', 'Cookie': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', 'Content-Type': 'application/x-www-form-urlencoded'}
history response status_code: 302
history response headers: {'Content-Length': '0', 'Set-Cookie': 'xxxxxxxxxxxxxxxxxxxxxxxxx', 'Business-Status': '200', 'Connection': 'keep-alive', 'Location': 'http://sso.jeffxue.com/index?fileTicket=xxxxxxxxxxxxxxxxxxxxxxxx&client=web', 'Date': 'Mon, 27 Mar 2017 07:58:02 GMT'}
```

Yes, this is intended behaviour.

Requests assumes that if you set the host header yourself then you know better than us, and that you want us to use what you set. You will get the behaviour you want by just not setting the host header.

Other headers you should not set are: Content-Length, Accept-Encoding, Connection. We will manage all of those for you. You'll find if you just unset the Host header that everything works fine. :smile:
Disabling SSL verification at application level 

Is there a way to disable SSL verification at my application level? 

I'm using the requests module which allows be to disable it per request level(by setting verify to false) but I wanted to way to disable for all requests. I'm running my tests against my local emulator, so I don't need that check and hence want to disable it at my application(test) level.

I'm looking for something like process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0" as used in Node.js

I'm on Python 2.7

Thanks for reaching out!

There is no way to do this, no. This is quite deliberate: such a setting is what I'd call an "attractive nuisance". Put another way, it's a setting that users are likely to attempt to use rather than solving the problem they have. Worse, it's a setting that is easily enabled and then forgotten, causing users to put themselves at enormous risk unexpectedly and silently.

Generally I'd recommend resolving the actual problem by locating all the relevant root certificates you need and bundling them. That way you get the security benefits of HTTPS *and* a functioning application, rather than sacrificing the former for the latter.

Hope that helps!

@Lukasa Thanks for your comments!

I was able to resolve this by explicitly specifying the certificate like below:
os.environ["REQUESTS_CA_BUNDLE"] = "F:/emulator_certificate.cer"

I wanted to avoid checking-in this certificate in my file system for running my automated tests. Is there a way in which Python can look into my Windows Certification Manager store to see if this certificate is installed and use it rather than me having to explicitly export it and provide it as above?


So you can use [wincertstore](https://pypi.python.org/pypi/wincertstore). Otherwise, we need to wait for [PEP 543](https://www.python.org/dev/peps/pep-0543/).

@Lukasa Thanks for the suggestion!
I went ahead and used wincertstore and it works without me having to extract and specify the certificate.
Prevent unicode escape in json

``` requests.post(url, json=payload)``` always transforms non-ascii chars in the forms of unicode escape , i.e., \uXXXX since the parameter ensure_ascii in```json.dump() ```  is True in default. It is inconvenient for global/non-ascii users, since the receiver have to de-escape it again. It will be better if requests can optionally prevent unicode escape in json. Any ideas?

JSON with escaped characters is supposed to be identical to JSON without escaped characters. There is no *general* reason to provide an escape hatch to avoid encoding, as it will only bust services that are not properly handling the JSON.

That said, the solution to fixing this is very easy:

```python
import requests
import json

r = requests.post(url, data=json.dumps(your_json, ensure_ascii=False), headers={'Content-Type': 'application/json'})
```
Requests and Let's Encrypt Certificates

While using Requests, I noticed strange thing: Requests doesn't accept Let's Encrypt Certificates. Only with `verify=False` library doesn't raise an exception (tested on my https Tornado server). Maybe add Let's Encrypt Authority X3 to Requests' CA?


works fine for me!

tested against httpbin.org

Httpbin use lets encrypt?

I have issue like https://github.com/kennethreitz/requests/issues/2955 

@MonsterDeveloper If you have an issue like #2955, have you tried examining the solution provided there? Can you tell me what server you're having trouble with? Can you tell us what version of Python and OpenSSL you're using? The reason is that Requests *does* trust LE certificates.

@MonsterDeveloper I also cannot reproduce this, but there's also too many versions of Python, OpenSSL, and Requests to try to guess at your current problematic configuration. Please supply as much information as possible.

Python: 3.5
Requests: latest!
PyOPENSSL: latest
OpenSSL: latest
OS: Ubuntu 16.04

And what URL?

Also...can you tell us what you mean by latest? There are different values of latest depending on where you're getting those packages from.

I got these packages from PyPi

You definitely didn't get OpenSSL from PyPI. So can you please provide the actual version numbers in all of those cases?

@MonsterDeveloper it would also be helpful to say what site you are trying to access. One of the most common problems is that a site is misconfigured and doesn't serve its intermediate certificate (as described in #2955).

There's been very little detail provided to help us debug this issue. Since the discussion stopped months ago, I'm closing this. We can reopen if necessary.
verify=/path/to/root/certs ignored

I use:
    requests.get(URL,verify=/path/to/root/certs
in /path/to/root/certs are some certificates and hashes:
    lrwxrwxrwx 1 root root   10 Mär 22 14:43 3104f96d.0 -> cert-2.pem
    lrwxrwxrwx 1 root root   10 Mär 22 14:43 33815e15.0 -> cert-1.pem
    lrwxrwxrwx 1 root root   10 Mär 22 14:43 738ed572.0 -> cert-2.pem
    lrwxrwxrwx 1 root root   10 Mär 22 14:43 ae8153b9.0 -> cert-1.pem
    -rw-r--r-- 1 root root 2760 Mär 22 14:43 cert-1.pem
    -rw-r--r-- 1 root root 2151 Mär 22 14:43 cert-2.pem
Certificate verification fails with:
    ERROR [Errno 336265218] _ssl.c:351: error:140B0002:SSL 
    routines:SSL_CTX_use_PrivateKey_file:system lib
When I install the CA certs in the system verification works when "requests" falls back to verify. I try this on Debian 8 with "requests" 2.4.3-6.

Requests 2.4 does not support openssl certificate directories: only single files. You'd need to cat all those files into a single file and use that. That limitation was lifted in 2.9. Please upgrade Requests. :smile:

Thanks, pip install requests --upgrade solved the Issue.
ConnectionError not handled by requests

**ConnectionError not handled by requests
('Connection aborted.', gaierror(-2, 'Name or service not known'))**

This happened when I tried to load an image from an old tumblr url:

```
url = 'http://41.media.tumblr.com/facde6697dff1c78fcfadbc653a8a295/tumblr_mlc3qpepqL1son6j8o1_1280.jpg'
r = requests.get(url, stream = True)

if r.status_code != requests.codes.ok:
    print('not an exception')
```

I expected that r.status_code == requests.codes.ok returned False instead of throwing the exception.



Why? There is no response to get there. It doesn't 404, it's returning NXDOMAIN on the DNS lookup. This is entirely expected behaviour.
idea - `Response.response_flow` or `Response.response_chain` convenience method

We consume a lot of URL shorteners and redirects. Unlike many requests users, we care about the entire journey -- not just the destination.  The current API makes it slightly inconvenient to iterate over the responses in order when needed. 

I often end up using a function that does the following, and wanted to suggest handling this upstream with something like...

    @property
    def response_chain(self):
        '''iterate over the responses in order, from the original request to ultimate response'''
        if self.history:
            for response in self.history:
                yield response
        if self.response:
            yield self.response



So, my initial inclination is to say that this isn't likely to get in. Our general rule is that a feature request must meet at least two of the following criteria:

1. Be generally useful: that is, be valuable for a large number of our users. 
2. Be complex or subtle to implement, in such a way that will make it difficult for other users. 
3. Rely on private parts of the codebase that we may change or break otherwise. 

This is none of those things as far as I can see: it's simple to implement, moderately widely useful but not anything like 50% of users, and relies entirely on published parts of the API. So I'm inclined to say there isn't much value add in bringing that code in. Does that make sense?

I'm 100% fine with it not being implemented upstream.  I think maybe 10% of users would find this useful.  I've just done this enough times to think it was worth discussing.

Agreed, you should always feel tee to propose new feature ideas. It's just worth knowing what our policy is on them. 😁

Thanks for the suggestion!
"stati" should be "statuses"

"stati" is not a word. Plural of "status" is "statuses" (`REDIRECT_STATI` etc.).

It's a purely internal name that is not intended to be used by others, so what it's called really doesn't matter. It could be called `MY_FAVOURITE_NUMBERS` and that would be fine. However, if it helps, I'll let you know that the person who wrote the original code knew that "stati" is not a word: it was intended as a joke. 😉
Drop 2.6 support

Our main dependent that still supports 2.6, pip, has just dropped support (pypa/pip#4343). We are now free to do the same.

let's do this in 3.0. 

@kennethreitz perfectly reasonable to do it with 3.0

✨🍰✨

i think this is a bit of a political statement, but a good one. 

With #4118 merged into the 3.0.0 branch, I think we can consider this resolved. Python 2.6 support will be removed in that release. Thanks @hobarrera!

✨🍰✨
Raise IOError when providing an invalid path to a CA bundle or …

Use IOError whenever invalid paths are used for ssl certificates (#3926)

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3927?src=pr&el=h1) Report
> Merging [#3927](https://codecov.io/gh/kennethreitz/requests/pull/3927?src=pr&el=desc) into [master](https://codecov.io/gh/kennethreitz/requests/commit/b60c5a5ed46d75c56e36db5a58a98a50d880141d?src=pr&el=desc) will **increase** coverage by `0.27%`.
> The diff coverage is `100%`.


```diff
@@            Coverage Diff            @@
##           master   #3927      +/-   ##
=========================================
+ Coverage   89.32%   89.6%   +0.27%     
=========================================
  Files          15      15              
  Lines        1940    1944       +4     
=========================================
+ Hits         1733    1742       +9     
+ Misses        207     202       -5
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/sessions.py](https://codecov.io/gh/kennethreitz/requests/compare/b60c5a5ed46d75c56e36db5a58a98a50d880141d...4207867aaf917ed8d223064cc5c163dfe027dc00?src=pr&el=tree#diff-cmVxdWVzdHMvc2Vzc2lvbnMucHk=) | `92.93% <ø> (ø)` | :arrow_up: |
| [requests/api.py](https://codecov.io/gh/kennethreitz/requests/compare/b60c5a5ed46d75c56e36db5a58a98a50d880141d...4207867aaf917ed8d223064cc5c163dfe027dc00?src=pr&el=tree#diff-cmVxdWVzdHMvYXBpLnB5) | `81.81% <ø> (ø)` | :arrow_up: |
| [requests/adapters.py](https://codecov.io/gh/kennethreitz/requests/compare/b60c5a5ed46d75c56e36db5a58a98a50d880141d...4207867aaf917ed8d223064cc5c163dfe027dc00?src=pr&el=tree#diff-cmVxdWVzdHMvYWRhcHRlcnMucHk=) | `92.82% <100%> (+2.57%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3927?src=pr&el=footer). Last update [b60c5a5...4207867](https://codecov.io/gh/kennethreitz/requests/compare/b60c5a5ed46d75c56e36db5a58a98a50d880141d...4207867aaf917ed8d223064cc5c163dfe027dc00?src=pr&el=footer&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Thanks for the review. In addition I also ported the definitions you provided for the `verify` parameter to the other methods that were getting this argument for consistency

I'm happy to help :) Just by curiosity, is it a good practice to squash the commits into one?

It depends very much on the project. For my part, I generally allow people to work the way they feel most comfortable unless the number of commits vastly outweighs the complexity of the change. Other projects may ask you to do so: it's polite to do what they want. But if no-one asks, then you're totally safe. :smile:

Ok good to know, thanks
SSLError raised when path to CA_BUNDLE is not valid could be more meaningful

Hi there,
First of all thanks for the requests library, it's just awesome!
In one of my project I was pulling 'verify' from a config file and passing it along every requests as an argument. Instead of receiving a boolean, it turned out that I was actually passing verify='true' to the requests methods, which makes requests think that I was providing a path to a ca bundle. 
Although I was obviously misusing the api, it took me some time to debug it and I believe that the error message could have been more meaningful.

In fact the following:
```
requests.get('https://github.com/', verify='true')
```
raises
```
/.../requests/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    495         except (_SSLError, _HTTPError) as e:
    496             if isinstance(e, _SSLError):
--> 497                 raise SSLError(e, request=request)
    498             elif isinstance(e, ReadTimeoutError):
    499                 raise ReadTimeout(e, request=request)

SSLError: [Errno 2] No such file or directory
```
I'll do a PR and make it raise an IOError (with the path being printed) in case the `verify` param does not point to a valid file/dir. As minor as it is, I hope you'll find it valuable.

So, before you leap into this (which is a good idea), be aware that you need to be able to distinguish this case from when either of the paths given to the `cert` parameter are bad.

Thanks for the quick feedback, I'll take that into account
Streaming uploads for files

It would be really nice if there was a way to do a multipart encoded file uploads without having to load all the files into memory before sending the request so that it could be streamed.

I'm pulling a large file from one place (using `requests.get(stream=True)`) and have to send it somewhere else as part of a multpart post, but I'm running out of memory.

Thanks for this! Handily, there is exactly such a thing: check out the requests-toolbelt's [`StreamingMultipartEncoder`](https://toolbelt.readthedocs.io/en/latest/uploading-data.html#streaming-multipart-data-encoder).
Rebase #3745 and add some tests

2 years after the original #2431, and after #3745
I encountered the strange behavior of `r.iter_lines()`, found this and tried to complete it.
Rebased once more and added more tests using the breakdown of @ianepperson https://github.com/kennethreitz/requests/pull/2431#issuecomment-72333964

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3923?src=pr&el=h1) Report
> Merging [#3923](https://codecov.io/gh/kennethreitz/requests/pull/3923?src=pr&el=desc) into [proposed/3.0.0](https://codecov.io/gh/kennethreitz/requests/commit/84dc6b66daf3d864737a61bacf86a91b68408656?src=pr&el=desc) will **increase** coverage by `0.12%`.
> The diff coverage is `100%`.


```diff
@@                Coverage Diff                 @@
##           proposed/3.0.0    #3923      +/-   ##
==================================================
+ Coverage           89.32%   89.44%   +0.12%     
==================================================
  Files                  15       15              
  Lines                1929     1932       +3     
==================================================
+ Hits                 1723     1728       +5     
+ Misses                206      204       -2
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3923?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/models.py](https://codecov.io/gh/kennethreitz/requests/compare/84dc6b66daf3d864737a61bacf86a91b68408656...d491e9f9b28815343d4114be51832960907b12d8?src=pr&el=tree#diff-cmVxdWVzdHMvbW9kZWxzLnB5) | `94.22% <100%> (+0.48%)` | :white_check_mark: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3923?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3923?src=pr&el=footer). Last update [84dc6b6...d491e9f](https://codecov.io/gh/kennethreitz/requests/compare/84dc6b66daf3d864737a61bacf86a91b68408656...d491e9f9b28815343d4114be51832960907b12d8?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

I'll take a look

I attempted a fix; though I'm not sure it's correct
Maybe use `self.encoding` or `self.apparent_encoding`?

When `decode_unicode` is False we don't deal with the encoding, so I'm not sure it makes sense to assume utf-8 and to encode only the last byte
Change apostrophe placement in advanced.rst

See Rule2a on http://www.grammarbook.com/punctuation/apostro.asp

Yup, this looks right. Thanks for the patch! :sparkles:

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3922?src=pr&el=h1) Report
> Merging [#3922](https://codecov.io/gh/kennethreitz/requests/pull/3922?src=pr&el=desc) into [master](https://codecov.io/gh/kennethreitz/requests/commit/29085e7d95915df95848af44f09a894e8b684134?src=pr&el=desc) will **decrease** coverage by `1.44%`.
> The diff coverage is `n/a`.


```diff
@@            Coverage Diff             @@
##           master    #3922      +/-   ##
==========================================
- Coverage   89.32%   87.88%   -1.45%     
==========================================
  Files          15       15              
  Lines        1940     1940              
==========================================
- Hits         1733     1705      -28     
- Misses        207      235      +28
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3922?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/compat.py](https://codecov.io/gh/kennethreitz/requests/compare/29085e7d95915df95848af44f09a894e8b684134...e0ba82949b82aa387c5fe519f6717007d682fa0d?src=pr&el=tree#diff-cmVxdWVzdHMvY29tcGF0LnB5) | `64.86% <0%> (-35.14%)` | :x: |
| [requests/__init__.py](https://codecov.io/gh/kennethreitz/requests/compare/29085e7d95915df95848af44f09a894e8b684134...e0ba82949b82aa387c5fe519f6717007d682fa0d?src=pr&el=tree#diff-cmVxdWVzdHMvX19pbml0X18ucHk=) | `83.33% <0%> (-10%)` | :x: |
| [requests/_internal_utils.py](https://codecov.io/gh/kennethreitz/requests/compare/29085e7d95915df95848af44f09a894e8b684134...e0ba82949b82aa387c5fe519f6717007d682fa0d?src=pr&el=tree#diff-cmVxdWVzdHMvX2ludGVybmFsX3V0aWxzLnB5) | `93.75% <0%> (-6.25%)` | :x: |
| [requests/models.py](https://codecov.io/gh/kennethreitz/requests/compare/29085e7d95915df95848af44f09a894e8b684134...e0ba82949b82aa387c5fe519f6717007d682fa0d?src=pr&el=tree#diff-cmVxdWVzdHMvbW9kZWxzLnB5) | `91.15% <0%> (-2.27%)` | :x: |
| [requests/adapters.py](https://codecov.io/gh/kennethreitz/requests/compare/29085e7d95915df95848af44f09a894e8b684134...e0ba82949b82aa387c5fe519f6717007d682fa0d?src=pr&el=tree#diff-cmVxdWVzdHMvYWRhcHRlcnMucHk=) | `89.75% <0%> (-0.49%)` | :x: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3922?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3922?src=pr&el=footer). Last update [29085e7...e0ba829](https://codecov.io/gh/kennethreitz/requests/compare/29085e7d95915df95848af44f09a894e8b684134...e0ba82949b82aa387c5fe519f6717007d682fa0d?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

These CI failures aren't your fault. @kennethreitz, looks like our CI setup has busted again: want to take a look?

Yep, this is a bug in the last release of pipenv. There's a patch underway, thanks @Lukasa!