Raibek/requests-github-issues huggingface dataset:
Add header name to InvalidHeader exception message

requests.get('http://example.com', headers={'foo': 1})
requests.exceptions.InvalidHeader: Header value 1 must be of type str or bytes, not <class 'int'>

It would be good to add the name of the bad header to make it easier
to track this down in large bodies of code. Something like:

requests.exceptions.InvalidHeader: Header foo value 1 must be of type str or bytes, not <class 'int'>

Thanks.

Summary.

## Expected Result

What you expected.

## Actual Result

What happened instead.

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help

```
<paste here>
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).e

Agreed, this seems like a useful enhancement. 

I'd like to help on this one if that's alright, I can have a pull request in shortly.


Hi: Please be my guest.

mgasvoda writes:
 > I'd like to help on this one if that's alright, I can have a pull request in shortly.
 > 
 > -- 
 > You are receiving this because you authored the thread.
 > Reply to this email directly or view it on GitHub:
 > https://github.com/requests/requests/issues/4239#issuecomment-321900563

-- 
Les Smithson
Suggestion: Improve the raise for status to include response body if there is one

Looking for more details on the HTTPError exception. The API I am working with sends back a 400 but also has a text/json body in it. It would be nice if the raise_for_status() function included the response text if there was any.

## Expected Result

That the body would be included

## Actual Result

No response body

## Reproduction Steps
N/A Feature Request! Current workaround:
```python
import requests
from requests.exceptions import HTTPError
def expanded_raise_for_status(res):
    """
    Take a "requests" response object and expand the raise_for_status method to return more helpful errors
    @param res:
    @return: None
    """
    try:
        res.raise_for_status()
    except HTTPError as e:
        if res.text:
            raise HTTPError('{} Error Message: {}'.format(str(e.message), res.text))
        else:
           raise e
    return
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.0.3"
  },
  "idna": {
    "version": ""
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.13"
  },
  "platform": {
    "release": "16.7.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "1010006f",
    "version": "16.2.0"
  },
  "requests": {
    "version": "2.18.3"
  },
  "system_ssl": {
    "version": "100020cf"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": true
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Added a pull with an implementation

Let's close this to focus on the pull itself: it's always nicer to discuss an idea with actual code, rather than in the abstract. :smile: Closing to focus on #4234.
Posting Multiselect Forms

Please help me. I am trying to automate a post from a form. The multiselect portion is throwing me off. the below doesnt work. quite sure the way i am posting the mutliselect portion ('session_id') is wrong 

## Expected Result

Expected this to work

![expect](https://user-images.githubusercontent.com/5037305/29161100-7afff488-7de7-11e7-8f21-fc407e1cf9e0.PNG)

    payload2 = {'from_date': '2017-08-17', 'fac_id': '10', 
                'par_fac_id': '1', 'book_purpose': '0', 'session_id': ['50']}
    p2 = s.post(new_url, data=payload2)

session_id comes from a multiselect form.

## Actual Result

The request command was not accepted by the server


i found the answer!

Sorry about that please close.

payload2 = {'from_date': '2017-08-17', 'fac_id': '10', 
            'par_fac_id': '1', 'book_purpose': '0', 'session_id[]': ['50']}

Well done finding the answer!

The [toolbelt](/requests/toolbelt) also provides a function to do this for you. Glad you found the answer on your own though. 🎉 

Thanks all. Somehow it wasn't that easy to find
Drop Python 3.3 support.

At this point, urllib3 is dropping 3.3 support. I think we should do so as well: 3.3 is long past support and never had the usage that 2.6 did, so we should probably abandon it. As a bonus reason, our CI is breaking on 3.3, so we should probably take the easy route out of that problem.

Nice :)

Don't forget to update the docs!

Good call, updated.

This is also currently blocking other builds from succeeding #4234 because of dependency issues in 3.3 specifically!
Please bump certifi to 2017.7.27.1

Cannot build requests via pip/buildout/easy_install due to old specified certifi

## Expected Result

Buildout succeeds

## Actual Result

```
Got pykerberos 1.1.14.
Picked: pykerberos = 1.1.14
Getting required 'certifi>=2017.4.17'
  required by requests 2.18.3.
We have no distributions for certifi that satisfies 'certifi>=2017.4.17'.
```


## Reproduction Steps

Build requests via pip/pypi fetches

See: https://pypi.python.org/pypi/certifi

This bug report makes no sense. Our certifi requirement specifies only a *minimum* version. You should be able to install alongside any newer certifi without difficulty.
Please support idna 2.6

Today I updated idna to 2.6 then I tried to install requests 2.18.3 and got an
error that dna must be <2.6.

Would be nice it you support the actual idna package,
thanks.

System: python 3.6.2 64 bit on windows7 64 bit

*Please* check recent issues. You'll find that two have been opened before you, both saying that we have updated the pin in the current master and will be releasing an update next week.

Oops sorry, I didn't realize it (I use seldomly the github issues pages)
idna dependency

Hi,

Could we get a new release of requests w/ an updated idna dependency.

Thanks a lot,

Carlos

This is already updated in master. A new release will be shipped next week. 

It seems that we get into an invalid state when something that has an unbounded-upper requirement of `idna` is listed first to `pip install`.  Example:
```
$ pip install cryptography requests
Collecting cryptography
  Using cached cryptography-2.0.3-cp35-cp35m-manylinux1_x86_64.whl
Collecting requests
  Using cached requests-2.18.3-py2.py3-none-any.whl
...
Collecting idna>=2.1 (from cryptography)
  Using cached idna-2.6-py2.py3-none-any.whl
...
Successfully installed asn1crypto-0.22.0 certifi-2017.7.27.1 cffi-1.10.0 chardet-3.0.4 cryptography-2.0.3 idna-2.6 pycparser-2.18 requests-2.18.3 six-1.10.0 urllib3-1.22
````
Here, `idna-2.6` is installed.

On the other hand,
```
$ pip install requests cryptography
Collecting requests
  Using cached requests-2.18.3-py2.py3-none-any.whl
Collecting cryptography
  Using cached cryptography-2.0.3-cp35-cp35m-manylinux1_x86_64.whl
...
Collecting idna<2.6,>=2.5 (from requests)
  Using cached idna-2.5-py2.py3-none-any.whl
...
Successfully installed asn1crypto-0.22.0 certifi-2017.7.27.1 cffi-1.10.0 chardet-3.0.4 cryptography-2.0.3 idna-2.5 pycparser-2.18 requests-2.18.3 six-1.10.0 urllib3-1.22
```

Is this a bug in `pip` or desired functionality?

Disregard my last comment, pip lacks a proper dependency resolver https://github.com/pypa/pip/issues/988 which explains the behavior.
support idna 2.6

idna 2.6 has been released, so the constraint in setup.py is now blowing up:

```
pkg_resources.ContextualVersionConflict: (idna 2.6 (.../ve/lib/python2.7/site-packages), Requirement.parse('idna<2.6,>=2.5'), set(['requests']))
```


Hang on, why is that constraint exploding? 2.5 should satisfy the requirement.

My guess something is installing it earlier in the build chain, and pip isn't clever enough to downgrade to 2.5 once 2.6 is installed.

Is there a problem with 2.6 that makes it backwards incompatible?

We don't know at this time: we began with very tight pins because we only unvendored the package a few months ago and so didn't want to run the risk of accidental breakage. We need to get a feel for what the actual compatibility structure is here.

In the short term, you can resolve your problem by using `requirements.txt` or similar to pin the `idna` version. In the longer term, we can aim to get a Requests patch that widens the pin.

Thanks for the quick turnaround!

Is there any plan for releasing a new version with this fix? That would be really helpful.

Yes, next week. 

The same is starting to happen with idna 2.7 ~~because cryptography requires idna 2.7~~
I still don't know why pip is resolving idna 2.7, cryptography is not requiring 2.7

I have a similar issue but with version 2.10:

```
error: idna 2.10 is installed but idna<2.9,>=2.5 is required by {'requests'}
```

> I have a similar issue but with version 2.10:
> 
> ```
> error: idna 2.10 is installed but idna<2.9,>=2.5 is required by {'requests'}
> ```

Me too, getting the same error while deploying my project on scrapy cloud. Have you found a solution yet?
Error:
requests 2.25.1 has requirement idna<3,>=2.5, but you have idna 2.1.
Warning: Pip checks failed, please fix the conflicts.
{"message": "Dependencies check exit code: 1", "details": "Pip checks failed, please fix the conflicts", "error": "requirements_error"}
{"status": "error", "message": "Requirements error"}
Traceback format_exc failing with ConnectionError

Hi,

I tried to use a script using requests (v2.18.1) on a remote website without having any internet connectivity. Of course, this resulted in a long stack traces due to the absence of internet connectivity, as expected, which ended up with

```
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='updates.weboob.org', port=443): Max retries exceeded with url: /1.3/main/modules.list (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe5f7474be0>: Failed to establish a new connection: [Errno -2] Name or service not known',))
```

Problem is that I catch any such exception and pass them to `traceback.format_exc` in order to have a nice error formatting in my logs.

## Expected Result

Using something like
```
try:
    my_code_calling_requests()
except Exception as e:
            log.error(("Unknown exception raised: %s" % traceback.format_exc(e)))
```

I would have expected a nicely formatted exception in my logs.

## Actual Result

Actually, it raised another exception, which I am not 100% sure to understand :/ Full traceback is:

```
Traceback (most recent call last):
  File "MY_CODE", line 363, in <module>
    traceback.format_exc(e)),
  File "/usr/lib/python3.6/traceback.py", line 163, in format_exc
    return "".join(format_exception(*sys.exc_info(), limit=limit, chain=chain))
  File "/usr/lib/python3.6/traceback.py", line 117, in format_exception
    type(value), value, tb, limit=limit).format(chain=chain))
  File "/usr/lib/python3.6/traceback.py", line 486, in __init__
    _seen=_seen)
  File "/usr/lib/python3.6/traceback.py", line 486, in __init__
    _seen=_seen)
  File "/usr/lib/python3.6/traceback.py", line 486, in __init__
    _seen=_seen)
  File "/usr/lib/python3.6/traceback.py", line 497, in __init__
    capture_locals=capture_locals)
  File "/usr/lib/python3.6/traceback.py", line 332, in extract
    if limit >= 0:
TypeError: '>=' not supported between instances of 'ConnectionError' and 'int'
```

## System Information

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.9"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.1"
  },
  "platform": {
    "release": "4.11.7-1-ARCH",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010006f",
    "version": "17.0.0"
  },
  "requests": {
    "version": "2.18.1"
  },
  "system_ssl": {
    "version": "1010006f"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": true
}
```

Thanks!

The first argument of `format_exc` is not the exception to print, it's a limit on how many lines of trace back to print. You may want to reread the documentation for the function on python.org. The correct call is just `format_exc`.

Oops, truly sorry about this :/ I took it from a snippet and should have checked the snippet thoroughly :/
multiline header should be supported!

Summary.
2.11 version introduced this change:
- Reject header values that contain leading whitespace or newline characters to
reduce risk of header smuggling.

BUT, as per standard https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 It whould be possible to have multiline headers.

f.ex. I need to attach a certificate via request header (currently I just mock the header validation method).

## Expected Result
support multiline header.

## Actual Result
requests.exceptions.InvalidHeader: Invalid return character or leading space in header: X-CERTIFICATE

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Multiline headers *should not* be supported. RFC 2616 is not the HTTP specification anymore: the relevant one is RFC 7230. 

[Here](https://tools.ietf.org/html/rfc7230#section-3.2.4) is the relevant part:

> Historically, HTTP header field values could be extended over multiple lines by preceding each extra line with at least one space or horizontal tab (obs-fold).  This specification deprecates such line folding except within the message/http media type (Section 8.3.1).  A sender MUST NOT generate a message that includes line folding (i.e., that has any field-value that contains a match to the obs-fold rule) unless the message is intended for packaging within the message/http media type.

There is no plan to add this support. Even if we did, the new line characters are not semantic in the header and get stripped for processing, so they're ill-suited to your use case. You need to come up with a different method of sending that cert.
Incorrect default value of Session attributes in documentation

See for instance `trust_env`, that is `True` by default, but `None` in the documentation:
http://docs.python-requests.org/en/latest/api/#requests.Session.trust_env

In that case, True/None are opposite, and I understood my own issue only after looking at the code :/

Same for all attributes actually, that are all `None` in doc.

I totally get that it's complicated for sphinx to generate instance attribute (since the value is assigned from inside `__init__`), but at least would be nice to have like `max_redirects` the default value in the docstring:

![image](https://user-images.githubusercontent.com/1050156/28797272-99d8b8c6-75f5-11e7-81fa-59e1298f4e40.png)

I could probably try to PR some of these strings if it helps and if you confirm the "This defaults to XXXX" pattern in the docstring.

Thanks! :-)

Hrm, Sphinx autodoc should be getting those rights they're very simple. Rather than change the text, which will end up out of sync with the code, it may be worth digging in to why Sphinx is failing to detect this. 

Defaults are no longer displayed in the documentation, closing this issue.
SSLError has been changed to ConnectionError in requests 2.18.2?

In  requests-2.18.1 and earlier:

`requests.get('https://testssl-expire-r2i2.disig.sk/index.en.html')`

Used to raise `SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)`

But as of requests-2.18.2 (the latest version), the same URL is returning `ConnectionError: HTTPSConnectionPool(host='testssl-expire-r2i2.disig.sk', port=443): Max retries exceeded with url: /index.en.html (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))`

I just need a confirmation that this change is intentional and if it is going to remain so (or not).

Thanks.

Thanks for this! This is a duplicate of #4902, so I'll refer you to that issue. 

Sorry, I meant #4209.
SSLError -> ConnectionError in 2.18.2: intended?

My dev builds started failing after 2.18.2 was released (they purposefully run on the latest releases to flush out problems). They work on 2.18.1.

It looks like the cause is a minor breaking change: in at least my particular situation, an SSLError has changed to ConnectionError.

I can update my tests to look for ConnectionError instead, but I figured I'd check first to see if this behavior is intended.

## Expected Result

Getting `https://wrong.host.badssl.com` raises SSLError.

## Actual Result

Getting `https://wrong.host.badssl.com` raises ConnectionError, mentioning SSLError as the cause.

## Reproduction Steps

```
# starting with 2.18.1...
$ python -c 'import requests; requests.get("https://wrong.host.badssl.com")'
Traceback (most recent call last):
...
requests.exceptions.SSLError: hostname 'wrong.host.badssl.com' doesn't match either of '*.badssl.com', 'badssl.com'

$ pip install --upgrade requests
Collecting requests
...
Successfully installed requests-2.18.2 urllib3-1.22

$ python -c 'import requests; requests.get("https://wrong.host.badssl.com")'
Traceback (most recent call last):
...
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='wrong.host.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'wrong.host.badssl.com' doesn't match either of '*.badssl.com', 'badssl.com'",),))
```

In case it's helpful, here's an example build: https://travis-ci.org/simon-weber/gmusicapi/jobs/248191438.

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  }, 
  "cryptography": {
    "version": "1.4"
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.11"
  }, 
  "platform": {
    "release": "4.4.2ph+", 
    "system": "Linux"
  }, 
  "pyOpenSSL": {
    "openssl_version": "1000204f", 
    "version": "16.0.0"
  }, 
  "requests": {
    "version": "2.18.2"
  }, 
  "system_ssl": {
    "version": "1000207f"
  }, 
  "urllib3": {
    "version": "1.22"
  }, 
  "using_pyopenssl": true
}
```

Hrm, this breaking change happened because we made urllib3 retry on `SSLError`. In this case I think we need to add some custom code to Requests to make this work a bit better, namely by throwing SSLError again. Good catch!

Thanks!
`requests.exceptions.*` do not inherit from Python's own specific exceptions.

Several `Exceptions` in https://github.com/requests/requests/blob/master/requests/exceptions.py inherit (indirectly) from `IOError`, but not from other Python exception. Two I can easily see are `ConnectionError` (see https://docs.python.org/3/library/exceptions.html#ConnectionError) and `TimeoutError`. 

It might make sense to inherit from those too. If you think it doesn't, just close the Issue.

I don't think it does at this time; while we're keeping compatibility across Python 2 and 3 it is probably not worth attempting to build complex exception hierarchies on different versions. For now, I think it's fine as-is. 

Thanks for the suggestion!
some docs on request redirect detection

This is just a small docstring that I wanted to suggest for inclusion. I should have offered it in my earlier commit.

The purpose of the docstring: we had some unit tests to ensure that `get_redirect_target` was being called the appropriate number of times. They broke because I calculated the number incorrectly.  It took a while to re-familiarize myself with the code and figure out what was going on.

this looks solid to me!
Requests not working with Mac osx 10.11 Python-3 

Environment is Max OS-X 10.11, fully updated.
Python 3.6.2
Requests:
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
urllib3<1.23,>=1.21.1 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
chardet<3.1.0,>=3.0.2 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
 idna<2.6,>=2.5 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages 
certifi>=2017.4.17 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages

Any get requests called from Python3 fails:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
More detail in attached file.
[Python3Requests.txt](https://github.com/requests/requests/files/1177297/Python3Requests.txt)



This seems to be a problem with your Python installation. It appears that this would happen with any TLS connection. How did you get your Python 3?

Hello Lukasa,
Installed a package from Python.org
https://www.python.org/downloads/release/python-362/
The problem seems particular to OSX 10-11. Works OK on 10.10, same package install for Python 3.
I have to use Python 3 as I'm also using PyQt5 which only works on P3


Ok, I think this makes this a bug with your Python. You may need to report this to the python-dev team at bugs.python.org.

Ok Thanks!

On Wed, Jul 26, 2017 at 11:24 AM, Ian Stapleton Cordasco <
notifications@github.com> wrote:

> Closed #4205 <https://github.com/requests/requests/issues/4205>.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/requests/requests/issues/4205#event-1180424873>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AdD_Tv-xo5zxoW4LUCX40J3xv5q5Twmeks5sR4RVgaJpZM4OkKD_>
> .
>
Correct urllib3 compatibility comment

So that it's in line with the code. 

I saw the warning raised recently and had to dive into the code for exact compatibility, but I stopped after reading the comment rather than the code. 

Maybe a better solution would be to surface the supported versions in the warning? However, I see it was changed recently in https://github.com/requests/requests/pull/4144/, any specific reason for removing that?

The primary reason is just trying to reduce the maintenance area of this code: it's painful to need to update in lots of places. Ideally we'd have a nice refactor that will work here to minimise the number of places we need to write our version constraints: if someone wants to do that, it might be a worthwhile project.
requests is asking for urllib3 1.21.1, when 1.22 is on pypi

Summary.

## Expected Result

requests is asking for urllib3 1.21.1, when 1.22 is on pypi. If pip/setuptools pulls the latest version of urllib3, you will not be able to run setup.py here.

## Actual Result

```'
Error: Couldn't find a distribution for 'urllib3<1.22,>=1.21.1'
```

## Reproduction Steps

* install latest urllib3
* run setup.py

## System Information



A new release is coming tomorrow to resolve this.
request not return in time

I'm using request('POST', url, data=data) to access a 4 layer SLB. I found that some of requests delayed for about 2s. 

The tcp packets (dumped using tcpdump) shows that the packets are really received from SLB, but request doesn't returned to the caller. Does anyone have any idea about it?

BTW: my requests lib version is 2.12.4

   

@zjbzju could you provide more information, including the operating system, whether or not TLS is involved, what versions of openssl you have installed and are using, what you mean by `4 layer SLB`, and perhaps even the `tcpdump` itself.

Closing since no new information has been supplied to provide additional assistance.
Feature request: timeout on full request completion

A simple way to place a timeout on the entire request duration would be very useful. I see this was discussed previously but workarounds involved using additional libraries, signals, etc.

Can this not be fixed by simply modifying `Response.iter_content()` and checking the time after each chunk is read, raising an exception after some "complete_timeout" is surpassed?

I've used a function like  this to do something similar by hijacking the `stream` parameter but it would be nice if this could be built right into `Session` and/or `Response`.

```python
from time import time
import requests

class CompleteTimeout(requests.exceptions.Timeout):
    pass

def request_with_complete_timeout(method, url, complete_timeout, **kwargs):
    """
    Call requests.request() but raise an Exception if the total time exceeds
    complete_timeout seconds.
    """
    kwargs['timeout'] = complete_timeout
    kwargs['stream'] = True
    timeout_time = time() + complete_timeout
    response = requests.request(method, url, **kwargs)
    content_barr = bytearray()
    for chnk in response.iter_content():
        if time() > timeout_time:
            raise CompleteTimeout(
                "Request exceeded complete timeout of {} seconds: {}"
                .format(complete_timeout, url))
        content_barr.extend(chnk)
    response._content = bytes(content_barr)
    return response
```

> Can this not be fixed by simply modifying Response.iter_content() and checking the time after each chunk is read, raising an exception after some "complete_timeout" is surpassed?

Not really. This won't prevent `iter_content` potentially blocking well past any such timeout, because the socket reads may take a very long time, as may the buffering until the `iter_content` read size. This also ignores all problems with such a timeout while reading the response. 

Ultimately this is a problem of library structure: the way we use the underlying libraries makes expressing his semantic very difficult. We acknowledge it's a problem and hope to remove httplib in order to take steps towards fixing it. However, until that time we're stuck with the semantic we have now. 

Thank you. I figured it would have been already added if it were as simple as what I had suggested.
Cookies lost after redirect

After redirect, the JSESSIONID cookie is lost.

## Expected Result
Status code: 200
Cookies: 
**JSESSIONID=91D71FBD2086F8FD1968001E6EECC9E5 for jizdenky.regiojet.cz,** 
currency=CZK for jizdenky.regiojet.cz
locale=cs for jizdenky.regiojet.cz

Status code: 302
Cookies: 
**JSESSIONID=91D71FBD2086F8FD1968001E6EECC9E5 for jizdenky.regiojet.cz,** 
currency=CZK for jizdenky.regiojet.cz
locale=cs for jizdenky.regiojet.cz

## Actual Result

Status code: 200
Cookies: 
currency=CZK for jizdenky.regiojet.cz
locale=cs for jizdenky.regiojet.cz

Status code: 302
Cookies: 
**JSESSIONID=91D71FBD2086F8FD1968001E6EECC9E5 for jizdenky.regiojet.cz,** 
currency=CZK for jizdenky.regiojet.cz
locale=cs for jizdenky.regiojet.cz


## Reproduction Steps

```python
import requests

        # Without redirect I get session id cookie
        with requests.Session() as s:
            s = s.get("https://jizdenky.regiojet.cz/Login", allow_redirects=False)
            print("Status code: {0}".format(s.status_code))
            print("Cookies: {0}".format(s.cookies))

        # With redirect I don't get session id cookie
        with requests.Session() as s:
            s = s.get("https://jizdenky.regiojet.cz/Login")
            print("Status code: {0}".format(s.status_code))
            print("Cookies: {0}".format(s.cookies))
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.8.1"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.1"
  },
  "platform": {
    "release": "16.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "100020bf",
    "version": "17.0.0"
  },
  "requests": {
    "version": "2.18.1"
  },
  "system_ssl": {
    "version": "100020cf"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": true
}

```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

@HynekBlaha why did you close this?

My mistake. Didn't read doc enough. 
requests.cookies return cookies of the last response. 
Cookie was already used in the second request header. Everything is ok.
new relase of requests since urllib3 dep was bumped to 1.22

Could we get a new release of requests that uses urllib3 >=1.22 as dependency. Latest released version of requests has urllib3<1.22,>=1.21.1

Thanks

Carlos

See #4196. A new release is planned.

@Lukasa Is there a time frame on that release? Just looking for a ballpark, if possible.

@dmtucker Yeah, there is: early next week, realistically. We try to release on Tuesdays or Wednesdays to allow ops teams time to roll out releases (for boring compliance reasons some places have to take manual steps to handle updates of critical dependencies like us), so expect next Tuesday.

Perfect, thank you!
compat.py uses try/except to prevent linter errors

flake8 testing of https://github.com/requests/requests on Python 3.6.2

$ __time flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics__
```
./requests/compat.py:53:11: F821 undefined name 'unicode'
    str = unicode
          ^

./requests/compat.py:54:18: F821 undefined name 'basestring'
    basestring = basestring
                 ^

./requests/compat.py:55:27: F821 undefined name 'long'
    numeric_types = (int, long, float)
                          ^

./requests/compat.py:56:27: F821 undefined name 'long'
    integer_types = (int, long)
                          ^

./requests/models.py:357:19: F821 undefined name 'unicode'
            url = unicode(url) if is_py2 else str(url)
                  ^
```

Thanks for this! However, I'm not concerned about the linter errors here because any moderately intelligent static analysis tool could tell you that they're not a concern. So I don't think this patch provides any particular value to the project. Sorry!

Understood.  Is there a better linter that you would recommend for Python projects?

Not particularly: this linter is fine. It's just ill-suited to this kind of file.

As a more general note, you'll find you may encounter hostility when you open PRs that do nothing more than make linter changes to files. If the project isn't using the linter in CI, that's usually a signal that they aren't interested in having their project linted. :smile:

Agreed.  It is always difficult to know who are those who know linting is possible but don't want to do it and those who don't even know that it is possible and also don't know of the lintable problems that lurk in their code.

Thanks for the awesome work -- this project is essential to tons of developers!
requests 2.18.1 with proxies throw SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

Hi,
When I use requests 2.18.1 with proxies, it throw SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

## Scenario 1

Debian GNU/Linux 8
Python 3.6.1
OpenSSL 1.0.1t 

pip list:

certifi (2017.4.17)
cryptography (2.0)
ndg-httpsclient (0.4.2)
pip (9.0.1)
pyasn1 (0.2.3)
pycparser (2.18)
pyldap (2.4.35.1)
pyOpenSSL (17.1.0)
PySocks (1.6.7)
requests (2.18.1)
requests-toolbelt (0.8.0)
setuptools (36.0.1)
six (1.10.0)
urllib3 (1.21.1)

My code:
```
import requests
proxies={'http':'socks5://127.0.0.1:8388', 'https': 'socks5://127.0.0.1:8388'}
r=requests.get('https://twitter.com',proxies=proxies)
```

Error message:
```
  File "/venv/lib/python3.6/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)

```

When I degrade requests=2.11.1
The error gone.

## Scenario 2

MacOS Sierra
Python 3.6.1
OpenSSL 1.0.2l 

pip list:
The same as above

It also return the same error message 

And also when I degrade requests=2.11.1
The error gone.

I want to know what the reason is, help me~ thanks

This is almost certainly because we removed 3DES from our default cipher suite list. Try following the steps [in this blog post](https://lukasa.co.uk/2017/02/Configuring_TLS_With_Requests/) to see if that resolves your issue.

Thanks, it resolved my problem
Add requirements.txt to MANIFEST.in

Three tests [1] rely on `requirements.txt` being available. If the file is not present, the following error is observed in those tests:

```
  >       with open('requirements.txt') as f:
  E       IOError: [Errno 2] No such file or directory: 'requirements.txt'
```

The latest (2.18.1) source distribution on PyPI does not include requirements.txt, which precludes successfully passing the tests.

This change adds the required file to the source distribution. An alternative is to use a different file already included in the sdist.

Additionally, skipping tests if a required resource (library, file, etc) is not available would be a user experience improvement.

[1] test_POSTBIN_GET_POST_FILES, test_POSTBIN_GET_POST_FILES_WITH_DATA, test_conflicting_post_params,
Sometimes it happens, What can I sove this? Thanks a lot.

HTTPConnectionPool(host='xxx', port=xxx): Max retries exceeded with url: /Upload/Processing.aspx (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f79f9c9c1d0>: Failed to establish a new connection: [Errno 113] No route to host',))
Traceback (most recent call last):
  File "/data/apps/browser_9090/UuwiseUtil.py", line 131, in uploadPic
    resp = requests.post("http://" + self.UPLOAD_SERVER + "/Upload/Processing.aspx", data=data , files=files, headers=headers)
  File "/home/python27/lib/python2.7/site-packages/requests/api.py", line 112, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/home/python27/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/python27/lib/python2.7/site-packages/requests/sessions.py", line 502, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/python27/lib/python2.7/site-packages/requests/sessions.py", line 612, in send
    r = adapter.send(request, **kwargs)
  File "/home/python27/lib/python2.7/site-packages/requests/adapters.py", line 504, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='xxx', port=xxx): Max retries exceeded with url: /Upload/Processing.aspx (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f79f9c9c1d0>: Failed to establish a new connection: [Errno 113] No route to host',))


This error is systemic: it means that when we attempted to connect to the IP address returned by DNS, the operating system told us it didn't know how to get there. This is not really a thing we can fix: when you get this error, you should just retry.

Thanks for your answer.

It just happens sometimes.
And sometimes It throws exceptions below, Whether it is related to HTTP persistent connection？

ConnectionError: HTTPConnectionPool(host='xxx', port=xxx): Max retries exceeded with url: xxx (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2f225a1b50>: Failed to establish a new connection: [Errno 111] Connection refused',))

Thanks~


It's not related to the persistent connections: that's caused by the remote peer refusing a TCP connection when we attempt to connect to it.

OK, I got it , thanks a lot.
Allow feeding certificates via buffer (instead of path)

Summary.

* I am trying to make a certificate-based request to an external URL using requests.get. I am trying to feed it the output of a call from the crypto library but it only allows paths.

What you expected.

* I should be able to feed `cert=(certificate, private_key)` as part of a request with byte strings instead of file paths (of course in addition to)
* This allows one to possibly avoid storing private keys at the path level and instead import a secured container (JKS for example) containing the key

What happened instead.

The request call seems to only take a file path and results in an error unable to find the file:

```OSError: Could not find the TLS certificate file, invalid path: b'-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----\n```

## Reproduction Steps

```python
import requests
import crypto

# Omitting argparse, file.read() commands for brevity in this example
p12 = crypto.load_pkcs12(keystoreFile, keystorePassphrase)

# These following commands output the contents of the loaded keystore
# In order to work around the error, they need to be redirected to new files (certificate.file, private_key.file
# These new file paths needs to be provided to the `requests.get` call
cert = crypto.dump_certificate(crypto.FILETYPE_PEM, p12.get_certificate())
caPem = crypto.dump_privatekey(crypto.FILETYPE_PEM, p12.get_privatekey())

resp = requests.get(APIURL, cert=(cert, caPem), verify=False)
```

## System Information

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "1.9"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.1"
  },
  "platform": {
    "release": "16.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "1010006f",
    "version": "17.1.0"
  },
  "requests": {
    "version": "2.18.1"
  },
  "system_ssl": {
    "version": "100020cf"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": true
}
```

So this is a sufficiently unusual use-case that we recommend solving this by passing a [`SSLContext` directly](https://lukasa.co.uk/2017/02/Configuring_TLS_With_Requests/). This can be configured with the appropriate client certs loaded from the underlying library as appropriate.

Does that sufficiently address your use case?

Hmm, passing certificates as env variables might also be a good use case here, it is often used in containerized environments.

@deejay1 Given that it's about three lines of product code to write it for your own apps, I'm not sure it's worth adding it to the core library. Again, I think the `SSLContext` is the best approach.
Allow hooking (de)compression

I have searched the repo for strings "gzip", "zlib" and have not found this packages imported. It seems that (de)compression is done implicitly somewhere. It poses some problems, for example if I want to add support of some compression/decompression algo like brotli.

## Expected Result

There should be a dict mapping compression algo name from http headers to (de)compresssion function.

## Actual Result

I have not found such one.

I don't think there is any need for this. I think it'll be sufficient to wrap the `read` function in the response object to achieve this goal.
Fix help ssl check

This will resolve the test failures discovered in #4182. We were retrieving the `OPENSSL_VERSION_NUMBER` from the system `ssl` module in all cases before now. This attribute doesn't exist in Python 2.6 causing new tests for the `info()` method to fail. This patch will return an empty value for Python 2.6 since we don't seem to have a way to reliably check the openssl version.

@Lukasa @sigmavirus24, I've pushed this branch up to the requests repo directly, so feel free to chop this up or do anything you feel is necessary to get #4182 moving. I don't know that the provided test is especially useful, but it does show that the fix here works.

Also, does this need a changelog entry?

@alexwlchan, we historically haven't done changelog entries with PRs in Requests, but I know @Lukasa has chosen to ask for that in other projects. Perhaps it's worth asking if we want to consider moving to that here, or continue to compile them once before a release.
Add idna version info to requests.help

Resolves #4180.

I’d try to fix the failing tests, but I can’t see the Jenkins instance so I don’t know what’s gone wrong. 😕 

(My two new tests seem like obvious culprits, but they’re passing locally.)

Hey @alexwlchan, the two new tests are failing but I don't believe it's related to your patch. Jenkins is complaining on Python 2.6 that `AttributeError: 'module' object has no attribute 'OPENSSL_VERSION_NUMBER'`.

The `info()` function was never tested before, and `OPENSSL_VERSION_NUMBER` doesn't appear to exist on the system [`ssl`](https://github.com/python/cpython/blob/2.6/Lib/ssl.py) module for Python 2.6. We'll need to fix that to get the tests passing. If you feel like throwing together another PR to fix that, it would be greatly appreciated. Otherwise, I'll put something together this evening.

> If you feel like throwing together another PR to fix that, it would be greatly appreciated.

It’s five past midnight here, so not right now. Maybe in the morning. 😉 

No problem at all, @alexwlchan :) I've opened #4183 to address the test issue. Once we get another maintainer to green light that, we should be set here.

Okay, rebased against master, so (fingers crossed) this time the tests will pass.

Well this test output seems extremely problematic. @kennethreitz any reason all the tests are failing to build now?

OK, it looks like the builds are working again. @alexwlchan, would you mind rebasing this onto the current master branch?

> would you mind rebasing this onto the current master branch?

Done.

Sorry for the CI issue, I didn't see this notification. 

Let me know if it happens again!

@kennethreitz, no worries 😄!
RFE: partial_updates 

http://docs.python-requests.org/en/master/api/  already have it 

Hi @sergiomb2,

Thanks for opening an issue. There's very little detail here about what issues or enhancements you are seeing or would like to see. As such, I'm closing this until that detail presents itself in one way or another.

Cheers,
Ian

OK , link  [1] fooled me because only show (...) HTTP request types: PUT, DELETE, HEAD and OPTIONS , should also refer to patch method ... 

[1]
http://docs.python-requests.org/en/master/user/quickstart/#make-a-request


You can use any method verb you want with requests.

http://docs.python-requests.org/en/master/user/advanced/#custom-verbs

On Sat, 1 Jul. 2017, 2:24 am Sérgio Basto, <notifications@github.com> wrote:

> OK , link [1] fooled me because only show (...) HTTP request types: PUT,
> DELETE, HEAD and OPTIONS , should also refer to patch method ...
>
> [1]
> http://docs.python-requests.org/en/master/user/quickstart/#make-a-request
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/requests/requests/issues/4181#issuecomment-312312277>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AGk_7QdoW49OCASc0hnOFaFDdOlfKTG-ks5sJSFBgaJpZM4OKxFC>
> .
>
sessions.py merge_setting(...)

Hello, my head HTTP request is sent, although is ordered, but this is out of order, the anti spider detection target site to request the head not in the same order will ban off request. After finally put the code here watching, actually can bypass anti crawler mechanism.

 你好，我发送http请求时的头部，虽然是有序的，但是这里是顺序被打乱，目标网站的反爬虫检测到请求头的顺序不一样就会ban掉请求. 最后把这里的代码注视之后，居然可以绕过反爬虫机制。

   # merged_setting = dict_class(to_key_val_list(session_setting))
   # merged_setting.update(to_key_val_list(request_setting))


Requests does not provide any ability to mandate ordering of HTTP headers. If that's a thing you need to do you'll need to drop down to a lower-level library.
add missing packages when parse document


Thanks for this! However, there is no need for it: when you install the Requests package by pip or setup.py those packages will be installed for you. 

Ah, I see. it looks my requests version is too old. I use `requests==2.13.0`.

maybe we should add some note in document to notice user ?


What would that look like?

I think maybe we can add it in [release-and-version-history](http://docs.python-requests.org/en/master/community/updates/#release-and-version-history) ?

parse document should be no error. if user didn't install `requests` and they just need document. 
they should know which package depends on. I still think a better way is to create a `requirements.txt` for document. this is a normal way.e.g:[scrapy](https://github.com/scrapy/scrapy/tree/master/docs)

of course, different community,different rule. respect community leaders choice.




It is considered a bad idea to create `requirements.txt` for this. `requirements.txt` is really considered to be used only for a *specific installation*, not for a general statement of what is needed. `setup.py` is pretty much the standard way to do this.

If you want to build the docs then you *have* to install Requests, as we use Sphinx autodoc to build our API documentation, and that only works if you've installed the package.
Pass pool_kwargs rather than updating connection_pool_kw

This addresses an issue where making HTTPS through proxies used the
default urllib3 connection pool settings.

fixes #3633

This is only a problem in the proposed/3.0.0 branch so even though this breaks the API that's okay, right?

Yup, no problem at all.

I have made it so.

Failing tests. =)

Oh yeah, that's Python 2.6 because I used a dictionary comprehension in the tests. I can switch it to not use a comprehension, I just thought 2.6 was getting dropped in 3.0.0 and couldn't resist the opportunity to use them :smile:

heh, right now we can't remove them from the test matrix, so you might have to just tolerate not having it. :grin:

It's left me with a heavy heart, but the dictionary comprehension has been excised and tests pass on Python 2.6 again.
How to respond to Form Submit click?

How to respond to Form on HTML click and requests.post triggered as a response? Most examples showing it 

    r = requests.post(...)

but how to bind this to respond to Button on Form in HTML clicked?

     <form method="post" class="navbar-form navbar-right" action="/shorten_url">
          <div class="form-group">
            <input type="text" id="url" name="url" class="form-control input-lg chat-input" placeholder="URL" />
                   <button type="submit" class="btn btn-lg btn-info">Submit</button>
           </div>
      </form>

Hey @scheung38, thanks for taking the time to submit this question. As a future reference, we usually prefer to have these kinds of questions posted on https://stackoverflow.com rather than the issue tracker for the project.

Could you briefly clarify what you're looking to do here. Are you trying to submit the form to the server, or trigger another request after the form has been submitted?

If it's the former, you can find the "action" a submit button triggers by looking at the form `action` attribute. In this case it is calling the relative URL path `/shorten_url`. While these are often URL triggers, you may occasionally find calls to javascript functions which Requests does not support.

In this specific example, you'll want to provide a POST request to the full URL (e.g. `http://yourdomain.com/shorten_url`) with a body containing `url=http://theurltoshorten.com`.

```python
r = requests.post('http://yourdomain.com/shorten_url', data={'url': 'http://theurltoshorten.com'})
```

I'm going to close this out for now, but let us know if this isn't what you were asking.