Raibek/requests-github-issues huggingface dataset:
revert removal of request param from resolve_redirects

This will fix the build issue on the proposed/3.0.0 branch.

When the `resp` and `req` parameters were condensed into `response`, it had some unintentional effects on responses modified by hooks. The `req` parameter was the request prior to being sent and hooks being run. Since `handle_401` resends the request again with the Authorization header, using `response.request` continues to pass it on with every redirect.

Reverting this change is the simplest approach since it's mostly cosmetic. We could alternatively look into a more explicit removal of Authorization headers on redirects.

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3869?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`proposed/3.0.0@85a29f1`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `100%`.


```diff
@@                Coverage Diff                @@
##             proposed/3.0.0    #3869   +/-   ##
=================================================
  Coverage                  ?   89.18%           
=================================================
  Files                     ?       15           
  Lines                     ?     1904           
  Branches                  ?        0           
=================================================
  Hits                      ?     1698           
  Misses                    ?      206           
  Partials                  ?        0
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3869?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/sessions.py](https://codecov.io/gh/kennethreitz/requests/compare/85a29f1da0d0ccd79e865a7172ea39a2a7667312...16bbc7478fb9115ed6f52cf5832fd191f13addef?src=pr&el=tree#diff-cmVxdWVzdHMvc2Vzc2lvbnMucHk=) | `92.91% <100%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3869?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3869?src=pr&el=footer). Last update [85a29f1...16bbc74](https://codecov.io/gh/kennethreitz/requests/compare/85a29f1da0d0ccd79e865a7172ea39a2a7667312...16bbc7478fb9115ed6f52cf5832fd191f13addef?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

This is an annoyance, but looks right. Thanks @nateprewitt!
Make Response.raise_for_status() return the response object if the response is successful

This allows for chaining method calls in cases where we want to raise for bad codes but use the response otherwise,
e.g. `requests.get(URL).raise_for_status().json()['value']`

@Lukasa added a changelog entry to 3.0-HISTORY. Also the Travis build fails, but it doesn't appear like it's because of my change... not sure how to investigate...

Yeah, I don't think it is: the 3.0.0 build has a breakage on it that was caused by one of the recent merges. I'll chase it on Monday. In the meantime we'll let this sit, if that's ok?

@Lukasa of course. No rush. Thanks a lot!

Ok, this branch should be fixed. 

@Lukasa will rebase and push again.

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3868?src=pr&el=h1) Report
> Merging [#3868](https://codecov.io/gh/kennethreitz/requests/pull/3868?src=pr&el=desc) into [proposed/3.0.0](https://codecov.io/gh/kennethreitz/requests/commit/b26f4c7f58180ee2d7c52687a9caafc7cc626418?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.


```diff
@@                Coverage Diff                 @@
##           proposed/3.0.0    #3868      +/-   ##
==================================================
+ Coverage           89.18%   89.18%   +<.01%     
==================================================
  Files                  15       15              
  Lines                1904     1905       +1     
==================================================
+ Hits                 1698     1699       +1     
  Misses                206      206
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3868?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/models.py](https://codecov.io/gh/kennethreitz/requests/compare/b26f4c7f58180ee2d7c52687a9caafc7cc626418...efcbe93075c8674657e45005c5d59af1d94fc600?src=pr&el=tree#diff-cmVxdWVzdHMvbW9kZWxzLnB5) | `93.77% <100%> (+0.01%)` | :white_check_mark: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3868?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3868?src=pr&el=footer). Last update [b26f4c7...efcbe93](https://codecov.io/gh/kennethreitz/requests/compare/b26f4c7f58180ee2d7c52687a9caafc7cc626418...efcbe93075c8674657e45005c5d59af1d94fc600?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Allow use of 'no_proxy' in the proxies argument

Add the ability to add 'no_proxy' and a value to the 'proxies'
dictionary argument.

https://github.com/kennethreitz/requests/issues/2817

Closes gh-2817

Had done this previously as: https://github.com/kennethreitz/requests/pull/3561

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3865?src=pr&el=h1) Report
> Merging [#3865](https://codecov.io/gh/kennethreitz/requests/pull/3865?src=pr&el=desc) into [master](https://codecov.io/gh/kennethreitz/requests/commit/ad65b0cb19124b5ae4dd01bf19d82c16ffb2485d?src=pr&el=desc) will **increase** coverage by `0.1%`.


```diff
@@            Coverage Diff            @@
##           master    #3865     +/-   ##
=========================================
+ Coverage   89.05%   89.16%   +0.1%     
=========================================
  Files          15       15             
  Lines        1873     1892     +19     
=========================================
+ Hits         1668     1687     +19     
  Misses        205      205
```


| [Impacted Files](https://codecov.io/gh/kennethreitz/requests/pull/3865?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/sessions.py](https://codecov.io/gh/kennethreitz/requests/compare/ad65b0cb19124b5ae4dd01bf19d82c16ffb2485d...85400d8d6751071ef78f042d1efa72bdcf76cc0e?src=pr&el=tree#diff-cmVxdWVzdHMvc2Vzc2lvbnMucHk=) | `92.91% <100%> (+0.1%)` | :white_check_mark: |
| [requests/utils.py](https://codecov.io/gh/kennethreitz/requests/compare/ad65b0cb19124b5ae4dd01bf19d82c16ffb2485d...85400d8d6751071ef78f042d1efa72bdcf76cc0e?src=pr&el=tree#diff-cmVxdWVzdHMvdXRpbHMucHk=) | `84.82% <91.3%> (+0.64%)` | :white_check_mark: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3865?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3865?src=pr&el=footer). Last update [ad65b0c...85400d8](https://codecov.io/gh/kennethreitz/requests/compare/ad65b0cb19124b5ae4dd01bf19d82c16ffb2485d...85400d8d6751071ef78f042d1efa72bdcf76cc0e?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

✨🍰✨
gevent + requests blocks when using socks

I use python(2.7.6),gevnet(1.1.2),requests(2.11.1) to make http requests concurrent, and it works good. But when I add socks proxy to requests, it blocks.

This is my code:

```
import time
import requests
import logging
import click

import gevent
from gevent import monkey
monkey.patch_all()

FORMAT = '%(asctime)-15s %(message)s'
logging.basicConfig(format=FORMAT)
logger = logging.getLogger('test')

#socks proxy
user = MY_SOCKS_PROXY_USERNAME
password = MY_SOCKS_PROXY_PASSWORD
host = MY_SOCKS_PROXY_HOST
port = MY_SOCKS_PROXY_PORT
proxies = {
        'http': 'socks5://{0}:{1}@{2}:{3}'.format(user, password, host, port),
        'https': 'socks5://{0}:{1}@{2}:{3}'.format(user, password, host, port),
}
url = 'https://www.youtube.com/user/NBA'


def fetch_url(i,with_proxy):
    while True:
        logger.warning('thread %s fetch url'%i)
        try:
            if with_proxy:
                res = requests.get(url,proxies=proxies, timeout=5)
            else:
                res = requests.get(url, timeout=5)
        except Exception as e:
            logger.error(str(e))
            continue

        logger.warning(res.status_code)

def do_other_thing():
    while True:
        logger.warning('do other thing...')
        time.sleep(1)

@click.command()
@click.option('--with_proxy/--without_proxy',help='if use proxy', default=True)
def run(with_proxy):
    if with_proxy:
        logger.warning('with proxy......')
    else:
        logger.warning('without proxy......')
    ts = []
    ts.append(gevent.spawn(do_other_thing))
    for i in xrange(3):
        ts.append(gevent.spawn(fetch_url,i,with_proxy))
    gevent.joinall(ts)

if __name__=='__main__':
    run()
```

These pictures show the result.

[run with proxy](https://i.stack.imgur.com/RfTbM.png)
[run without proxy](https://i.stack.imgur.com/9UVVM.png)

>With proxy, do_other_thing will blocks before fetch_url done.
>Without proxy， it works good. (timeout error occurs, because of the GFW)

Can anyone help me solving this problem? Thanks very much!

My best guess is that you are monkeypatching too late. When using gevent you must monkeypatch *first*, before importing anything else. Doing it later runs the risk of breaking anything that has done a `from` import from the things that gevent monkeypatches.

Can you try moving your import and monkeypatch to the very top of your file before doing anything else?

Yeah,  ' moving your import and monkeypatch to the very top of your file ' solved my problem! Thank you very much!

This suggestion also solves my project's problem.
 I hava 3 files called A,B,C, and run sequence is A->B->C.  I use gevent in B, so I add monkeypatch in B, but i blocks with requests+socks. Now I add monkeypatch at the very top of A, it works great now! Hope this will help others!

Thank you again!

Happy to help!
File names stripped of leading dots

Currently using the following to upload a file to a webserver:
```
files = {'files': ('.png', open(file_path, 'rb'))}
res = requests.post(url, files=files)
```
The reason I rename the file to just ".png" is because with an empty base name the server will randomly give the file a name.
However, and I'm not sure if requests is the one to blame here, the file ends up being uploaded with the name "png" instead, as if the leading "." was stripped from the name.

Is this something requests is intentionally doing or should I look elsewhere for the cause?

This is not something requests is doing. A quick Wireshark dump of your sample code against `http://httpbin.org/post` shows that the request looks like this:

```
POST /post HTTP/1.1
Host: httpbin.org
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.13.0
Content-Length: 655
Content-Type: multipart/form-data; boundary=746c7fc2a98549969068cf0f6860f3d7

--746c7fc2a98549969068cf0f6860f3d7
Content-Disposition: form-data; name="files"; filename=".png"
<...file data ...>
```

Note that the filename still has the leading dot. =) So this isn't Requests doing something, it has to be elsewhere.

Alright, thanks for checking 👍
why I

Hi, execuse me to trouble you!
I want to use requests with gevent to make a concurrent spider. The offical docs says  

> If you are concerned about the use of blocking IO, there are lots of projects out there that combine Requests with one of Python's asynchronicity frameworks. Two excellent examples are grequests and requests-futures.

But after read [this question in stackoverflow](http://stackoverflow.com/questions/9501663/how-enable-requests-async-mode), i am confused. Because I run the code in my machine,  but it seems requests does better than urllib(python 2.7.10,gevent 1.1.2,requests 2.11.1).

the code:
```
import sys

import gevent
from gevent import monkey

monkey.patch_all()

import requests
import urllib2

def worker(url, use_urllib2=False):
    if use_urllib2:
        content = urllib2.urlopen(url).read().lower()
    else:
        content = requests.get(url).content.lower()
    title = content.split('<title>')[1].split('</title>')[0].strip()

urls = ['http://www.baidu.com']*15

def by_requests():
    jobs = [gevent.spawn(worker, url) for url in urls]
    gevent.joinall(jobs)

def by_urllib2():
    jobs = [gevent.spawn(worker, url, True) for url in urls]
    gevent.joinall(jobs)

if __name__=='__main__':
    from timeit import Timer
    t = Timer(stmt="by_requests()", setup="from __main__ import by_requests")  
    print 'by requests: %s seconds'%t.timeit(number=3)
    t = Timer(stmt="by_urllib2()", setup="from __main__ import by_urllib2")  
    print 'by urllib2: %s seconds'%t.timeit(number=3)
    sys.exit(0)
```

the result:
```
➜  test_requests python test04.py
by requests: 0.178946971893 seconds
by urllib2: 1.94786500931 seconds
➜  test_requests python test04.py
by requests: 0.233242034912 seconds
by urllib2: 1.55849504471 seconds
➜  test_requests python test04.py
by requests: 0.52639913559 seconds
by urllib2: 3.95958900452 seconds
➜  test_requests python test04.py
by requests: 0.56934094429 seconds
by urllib2: 4.00116705894 seconds
```

My question:
1. The result seems that now gevent + requests is asynchronous. But many people in the website say I should use grequests. Is that right?
2. Why the result I use requests same to I use grequests?

Hope you can answer my question . Thanks very much.



Requests can work with gevent directly without any trouble: this is the nature of gevent. grequests provides some specific additional utilities for managing Requests with gevent. It's a nice addition, but not required.

OK！Thank you very much！
Say that we use a dictionary of strings.

Resolves #3856.

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3857?src=pr&el=h1) Report
> Merging [#3857](https://codecov.io/gh/kennethreitz/requests/pull/3857?src=pr&el=desc) into [master](https://codecov.io/gh/kennethreitz/requests/commit/5c68b8f2279be82a672b908d0017d60a8ce6efcd?src=pr&el=desc) will **not impact** coverage.


```diff
@@           Coverage Diff           @@
##           master    #3857   +/-   ##
=======================================
  Coverage   89.05%   89.05%           
=======================================
  Files          15       15           
  Lines        1873     1873           
=======================================
  Hits         1668     1668           
  Misses        205      205
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3857?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3857?src=pr&el=footer). Last update [5c68b8f...08ce652](https://codecov.io/gh/kennethreitz/requests/compare/5c68b8f2279be82a672b908d0017d60a8ce6efcd...08ce652b8b42c64ad904133ceb3e4f71df90c03c?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Support urllib3 Retries at Individual Request Level

Right now, requests supports urllib3 retries at the pool level, allowing retries to occur fairly globally. It seems as if urllib3 supports retries at a more granular level as well, for individual requests:

```
>>> r = http.request(
...     'GET',
...     'http://httpbin.org/redirect/3',
...     retries=urllib3.Retries(
...         redirect=2, raise_on_redirect=False))
>>> r.status
302
```

It would be nice if requests had support for this level of granularity, as some of our cases require retries for only specific endpoints on a single API.

Apologies if this is the wrong place for this, wasn't sure where to place feature requests.

Thanks!
Tim

Thanks for this request!

Unfortunately, we're unlikely to support this: it's fundamentally an additional complexity that we don't think is justified in the API. At this point our top-level API is largely frozen, and we're very unlikely to widen it or add new features.

However, it's worth noting that the Transport Adapter in use for a given request is chosen on a longest-prefix-match on the *entire* URL. That means that you can register different adapters for different paths, as well as different hosts, which should give you the flexibility you need.

I hope that helps!
Do not convert /o\ into /O\

There is code that already check that a guy raising (at least) his right arm does not get big-headed (a.k.a uppercased). Just add the same for the other, poor guy complaining about an error.

Yes, sorry, I just (w)hacked it in 3 minutes without proper testing. Shame on me. Let me see...

Cool, looks good now, thanks! :sparkles:

\o/
Missing response headers when malformed header is part of the response

Using Python 3.5, requests 2.13.0

```
>>> import requests
>>> r = requests.get('https://online.chasecanada.ca/ChaseCanada_Consumer/Login.do')
>>> print(r)
<Response [200]>
>>> print(r.headers)
{'Date': 'Sat, 04 Feb 2017 19:52:14 GMT'}
>>> 
```

I'm pretty sure the problem is caused by an invalid HTTP header returned by the server:

    HTTP/1.1 200 OK
    Date: Sat, 04 Feb 2017 19:16:34 GMT
    My Param: None
    [...]

It directly follows the `Date` response header, which is returned fine, but since no other response headers is returned, I think this broken header is breaking the HTTP response headers parser.

Hum. Not sure this is a `requests` issue, since the `headers` dict in the `urllib3.response.HTTPResponse` object also only has the `Date` header...
Maybe there is a workaround somehow..? 

As I answered on the urllib3 issue you opened, there is no workaround until urllib3 is no longer reliant on `http.client`. Cheers!
Unsupported authorization method

Hello,

My code:
```
headers = {'Authorization': 'Bearer '+access_token}
payload = {'grant_type':'refresh_token','refresh_token':refresh_token,'client_id': client_id, 'client_secret': client_secret}
resp = requests.post(url,headers=headers,data=payload).json()
```

I am trying to get refresh token as mentioned in the [docs](https://build.envato.com/api/#oauth) for Envato API.

When I run this code I get the following response:
**{'error': 'invalid_request', 'error_description': 'Unsupported authorization method: '}**

Can anyone point out as to what I am missing here?

As per the docs I need to make a POST request to get the refresh token.
```
POST https://api.envato.com/token
     grant_type=refresh_token&
     refresh_token=[REFRESH_TOKEN]&
     client_id=[CLIENT_ID]&
     client_secret=[CLIENT_SECRET]
```

Requests: v2.13.0
Python: v3.5.3

So, this is not really a Requests problem: the problem is with the specific data you need to send in the OAuth flow.

However, can I suggest you try removing the `Authorization` header? This uses the `token` endpoint, which some of the other requests do, and none of the others provide an `Authorization` header. I suggest trying that.

I totally agree. Thank you so much!
A little suggestion about 'LookupDict' in 'requests/requests/structures.py' 

Dear Sir,
I like your 'requests' Python library very much and I'm reading the source code these days. I found a problem that is tricky. The 'LookupDict' class definition is:
`
```
class LookupDict(dict):

    def __init__(self, name=None):
        self.name = name
        super(LookupDict, self).__init__()
    def __repr__(self):
        return '<lookup \'%s\'>' % (self.name)

    def __getitem__(self, key):
        # We allow fall-through here, so values default to None

        return self.__dict__.get(key, None)

    def get(self, key, default=None):
        return self.__dict__.get(key, default)
```
`

It inherits the 'dict' object but does not use any of the 'dict' methed. It only use the class's \__dict__ and it is ok to replace the 'dict' with 'object'. 


`LookupDict` absolutely includes all of the dict methods. In particular, it allows the use of `__setitem__` and other things from the dict class. It specifically does only one thing, which is override `__getitem__` to allow returning `None`.

So it's not ok to replace the dict with `object` in this case, even though the requests codebase doesn't use that functionality.

Thanks for reporting this issue though, and I hope you keep reading through the codebase!

Thanks a lot for your reply. I'll read the code with your clue.
requests messes up a ZIP stream (?)

There's an interesting bug that I've run into involving reading [a zipfile](https://data.cityofnewyork.us/dataset/Broadband-Data-Dig-Datasets/ft4n-yqee) off the web.

The details as in the comments to [this answer on StackOverflow](http://stackoverflow.com/a/42016924/1993206):

This fails with a `BadZipFileError`: 

```
z = zipfile.ZipFile(io.BytesIO(requests.get("https://data.cityofnewyork.us/dataset/Broadband-Data-Dig-Datasets/ft4n-yqee").content))
```

But writing `requests.get("https://data.cityofnewyork.us/dataset/Broadband-Data-Dig-Datasets/ft4n-yqee").content` to a file and *then* opening it with `ZipFile` works fine. So this works:

```
with open("temp.zip", "wb") as fp:
    fp.write(requests.get("https://data.cityofnewyork.us/dataset/Broadband-Data-Dig-Datasets/ft4n-yqee").content)

with open("temp.zip", "rb") as fp:
    zipcontent = zipfile.ZipFile(fp)
    
```

This is on Python 3.5.2 and Requests 2.12.4.

That doesn't work for me at all. =)

An FYI: the response I get from that website is HTML, always. By my reckoning the download link is `https://data.cityofnewyork.us/download/ft4n-yqee/application%2Fzip`, and that link does appear to work correctly in the code sample. =)

Yes, my bad for posting the wrong link! What platform are you on?

Mac. =) Python 3.5.2.
`except` without exception class specified makes IDNA-encoding non-threadsafe.

Hello,

We have a python program with resident process running in separate thread. We make requests in said thread and when master thread spots a timeout, a custom exception is dispatched to child thread via OS signal. We use Requests `2.12.4`.

However, that exception is caught not by our handler (in child thread main loop), but by `except:` directive in `alabel` function deep in `packages/idna/core.py`: https://github.com/kennethreitz/requests/blob/master/requests/packages/idna/core.py#L264 because that handler is an unconditional "catch-all" `except:` directive. That leads to very cryptic behaviour like child thread not stopping and trying to access urls with unnecessary IDNA-encoding (all our URLs are in plain ascii).

Expected behaviour for that handler is not to catch everything, but to expect a specific exception class(es) and work just with them.

Yup, that looks wrong, thanks for reporting it! The `idna` code is vendored, without edits, from [this repo](https://github.com/kjd/idna). I recommend you open an issue there, which when fixed will make its way into Requests.

The changes to `idna` repo landed about two weeks ago: https://github.com/kjd/idna/pull/39

When can we expect the chages to be reflected in `requests`?

The changes will land when we prepare our next minor release: those releases are when we update our vendored dependencies.
PUT (requests.put) specifying the source file location for zip file error 10054

With using trying an zip file I get the following, any clues: (requests-2.13.0)

```
with open('default.zip', 'rb') as data:
    requests.put(url, data=data)

```
Output:

```
C:\svn\libraries\cpp>python req_put.py default.zip

Traceback (most recent call last):
  File "req_put.py", line 102, in <module>
    resp = requests.put(uri,  headers=headers, data=data) # data=data , params=payload)
  File "C:\Python27\lib\site-packages\requests\api.py", line 124, in put
    return request('put', url, data=data, **kwargs)
  File "C:\Python27\lib\site-packages\requests\api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests\adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host'))

```


The server will have sent a 4XX response, which means it doesn't like your upload. Unfortunately Requests can't handle this yet, because httplib doesn't make it easy. We're working on a substantial refactor to fix this, but it'll be a long time coming. In the meantime, I suggest you try to investigate what format the server is actually expecting. 

@Lukasa Many thanks. Seeing that is really seems you know what is going on here 3 qs:

1. What other Python cURL lib can I use here that you might know of that works?
2. Why would requests.put() work perfectly not specifying the data param, hence not specifying the source location of file?
3. Not being sure how cURL does this, is it not possible to also add the cURL `-T, --upload-file <file>` like option in the requests library?

1. pycurl
2. HTTP is a very general protocol. PUT does not inherently have anything to do with uploading files. In fact, it's perfectly allowed to send a PUT without any body. This is why Requests allows it.
3. `curl -T` does exactly what you just did. The only difference between Requests and curl is in one case: if you do `curl -T <filename> http://host` (that is, with no path), then curl will send a PUT to `/filename`. To do the same thing with Requests, you'd need to do:

    ```python
    url = "http://example.com/"
    with open('default.zip', 'rb') as data:
        requests.put(url + "default.zip", data=data)
    ```

what should I do for the tar.gz file in a PUT request using pycurl module @Lukasa 

I figure out the solution.
Update Copyright in LICENSE to 2017

Branch name was meant to be 'Happy New Year' but apparently typing the word 'Year' isn't my strongest skill. 

# [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3841?src=pr&el=h1) Report
> Merging [#3841](https://codecov.io/gh/kennethreitz/requests/pull/3841?src=pr&el=desc) into [master](https://codecov.io/gh/kennethreitz/requests/commit/e84086ac86075321aa479a24f6c83dd3ead40212?src=pr&el=desc) will **not impact** coverage.


```diff
@@           Coverage Diff           @@
##           master    #3841   +/-   ##
=======================================
  Coverage   89.05%   89.05%           
=======================================
  Files          15       15           
  Lines        1873     1873           
=======================================
  Hits         1668     1668           
  Misses        205      205
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3841?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kennethreitz/requests/pull/3841?src=pr&el=footer). Last update [e84086a...1a4ba6c](https://codecov.io/gh/kennethreitz/requests/compare/e84086ac86075321aa479a24f6c83dd3ead40212...1a4ba6c833f16283103d4eac488bd46640280bdf?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Thanks @TetraEtc!
multipart/form-data not encoding correctly on recent versions

I have a request that works in older version of the `requests` library but not in more recent versions.  The trouble point is the `model` key in the files dictionary.  In version `2.11.1`, the model key is correctly recognized by the remote endpoint.  In newer versions, the `model` key appears to be omitted or malformed in some way.

I don't believe the API endpoint is the issue.  I can execute a `curl` command and it will work as expected.

A sample (functioning) curl request:

```bash
curl -F data_file=@my_file.mp3 -F model=en-US "https://api.speechmatics.com/v1.0/user/1/jobs/?auth_token=ABC"
```

### Request

```python
import requests


files = {
    'data_file': open(filename, 'rb'),
    'model': ('', 'en-US'),
}
response = requests.post(
    'https://api.speechmatics.com/v1.0/user/1/jobs/?auth_token=ABC',
    files=files)
print(response.content)  # b'{\n  "code": 400, \n  "error": "No language selected"\n}'
```

### Versions tested:
*All versions installed through `pip` and are being used with `python 3.5`.*

`2.11.1` - Works.
`2.12.4` - **Fails.**
`2.13.0` - **Fails.**

Yeah, this behaviour changed. In particular, the lower-level urllib3 library wanted to allow the possibility of setting an *empty* filename (as opposed to not providing the filename at all). You can get the correct behaviour on all platforms by changing `'model': ('', 'en-US'),` to `'model': (None, 'en-US'),`

@Lukasa Thanks!
ignoring MTU and MSS in TCP handshake

Hi
I was using requests flawlessly till yesterday I realised I cannot send large packets to https server getting following error
`requests/adapters.py", line 473, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))
`
Turns out my ISP started filtering ICMP fargmentation needed packets and drops large packets. Otherwise packets would have been fragmented and retransmited. Shouldn't requests fragment packets with respenct to MTU or MSS to prevent this from happening?
Is there a work around to this problem?

Requests does not control the way packets are emitted. The socket API for TCP does not expose or allow us to control packet boundaries: the TCP layer in the kernel manages all of this. Any problem you're seeing would be encountered by other programs on your system. 

You'll need to investigate your system's configuration.
pin pipenv until kennethreitz/pipenv#90 is resolved

Fixing build for now until we can properly parse all dependencies in pipenv.

## [Current coverage](https://codecov.io/gh/kennethreitz/requests/pull/3836?src=pr) is 89.09% (diff: 100%)
> Merging [#3836](https://codecov.io/gh/kennethreitz/requests/pull/3836?src=pr) into [master](https://codecov.io/gh/kennethreitz/requests/branch/master?src=pr) will increase coverage by **0.05%**

```diff
@@             master      #3836   diff @@
==========================================
  Files            15         15          
  Lines          1870       1870          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits           1665       1666     +1   
+ Misses          205        204     -1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [23057db...808e4c6](https://codecov.io/gh/kennethreitz/requests/compare/23057dbc22d08ee831198f287d11d750f200e092...808e4c62dc54ec7a10197038d56fb70f738b19cf?src=pr)

Thanks @nateprewitt!
Only send HTTPDigestAuth on 4xx challenges

Resolves: #3772

@Lukasa I believe there were issues with ONLY doing 401 as noted in #3772. The related RFCs allow for the WWW-Authenticate header in a non-401 response which the client SHOULD respond to if received. I think the conclusion we came to is to support 4XX response codes.

I believe @nateprewitt is correct. It seemed to me that the conclusion of the discussion was to support 4XX response codes.

Done!

Might I also suggest renaming the hook to something like handle_4xx_auth_challenge?

> Might I also suggest renaming the hook to something like handle_4xx_auth_challenge?

I think I'd rather not. Most auth plugins use the same method name for consistency. But I'm not firmly set in that opinion.

Hrm. @mmedal, can you rebase on top of the current master? We seem to be having some build problems and I want to check whether they're ones we've seen before and fixed, or new ones.

I triggered a new build (my branch is up to date), but it looks like you're still having build problems.

@Lukasa this is a bug in pipenv (kennethreitz/pipenv#90) that may not have an immediate solution. It may be best to pin Requests' dependency for a bit. `pipenv==3.1.9` should work for now.

@nateprewitt I'll merge a PR that does that. 😁

On it!

Retriggering build.

## [Current coverage](https://codecov.io/gh/kennethreitz/requests/pull/3835?src=pr) is 89.05% (diff: 100%)
> Merging [#3835](https://codecov.io/gh/kennethreitz/requests/pull/3835?src=pr) into [master](https://codecov.io/gh/kennethreitz/requests/branch/master?src=pr) will increase coverage by **0.01%**

```diff
@@             master      #3835   diff @@
==========================================
  Files            15         15          
  Lines          1870       1873     +3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits           1665       1668     +3   
  Misses          205        205          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [d3c97dd...8a58427](https://codecov.io/gh/kennethreitz/requests/compare/d3c97dda1101cc579e4300212929abb9cb43b331...8a58427d8aa73d6af81b6e5f6fd934b3386ef3de?src=pr)
Strict-Transport-Security Header not complet submittet

I'm trying to get the strict-transport-security header in python3 with requests.
My code looks like the following:
`import requests
r = requests.get("https://%s" % domain")
print (r.headers)`

My problem is that the strict-transport-security header does not contain all data. The real Header looks like ```max-age=31536010; includeSubDomains; preload```, but the header by r.headers['strict-transport-security'] only contains ```max-age=31536010; includeSubDomains```, not the preload which is needed by my application.


Tested on FreeBSD 11.0 and Mint Rosa and Arch 4.8.13

Requests does not amend the headers in any way, so you get what we received. Are there any proxies or other intermediaries which may be amending headers?
fixing codecov with pipenv

codecov broke with the introduction of pipenv because the dependency isn't installed outside of the virtualenv. Running it inside of pipenv fixes this.

## [Current coverage](https://codecov.io/gh/kennethreitz/requests/pull/3832?src=pr) is 89.03% (diff: 100%)


> No coverage report found for **master** at b92058b.

> Powered by [Codecov](https://codecov.io?src=pr). Last update [b92058b...795106c](https://codecov.io/gh/kennethreitz/requests/compare/b92058b6fe68a3b114b872d1dba072c115af413b...795106c1885f19c84cefe037914a12d37ddd6248?src=pr)

LGTM, thanks @nateprewitt!
Fixing Pipfile source

This fixes the issue that was encountered this morning during the 2.13.0 release.

The source specified in the original Pipfile was for an incorrect pypi endpoint. This didn't matter until pipenv 2.6 was released, specifically [cb22a12](https://github.com/kennethreitz/pipenv/commit/cb22a129eae8c8e800e603c38bf1fe04d420fbde), which started using the provided `source` value.

@kennethreitz I used `pipenv lock` with pipenv 3.0.0 to generate the new lock file which I'm assuming is what we want here.

While we're here, it looks like this value is still used in pipenv [here](https://github.com/kennethreitz/pipenv/blob/3fa204b734f67522d7858ab47b1d10263a76dc90/pipenv/_pipfile/api.py#L73) , but I'm not sure if that code is actively being used. I can open a quick PR updating it over there too if needed.

ah yes, this is the issue.
Blacklist known bad version of pipenv

We pinned pipenv to release v2.13.0 but in reality, we could have just
blacklisted the known bad version. For future us, we now blacklist it.

I think it's more than just 2.8.0, I can't get a working build on travis after 2.5.0. It looks like something happened there that's breaking the build, checking it out now.

Odd, 0.2.7 worked just fine locally

We can blacklist everything between 0.2.5 and 0.2.8 though
Release v2.13.0

This includes updates to:

- when we load idna
- urllib3 (updated to version 1.20)
- idna (updated to version 2.2)
- release publishing machinery

It looks like `init` in the Makefile may need `pipenv shell` run to initialize the virtualenv?

Maybe? pipenv 0.1.14 worked, 0.2.9 doesn't. The changelog is not immediately forthcoming as to what the specific problem is.

For now we'll just pin it so the builds keep working: @kennethreitz should feel free to remove the pin and investigate when he has some time.

Just tested with pipenv 0.2.7 locally and that seems to work, so I'm going to say it's just 0.2.8 that's busted. We can blacklist it after this is merged and 2.13.0 is released


BAM.
Access to the server certificate

Hi,
I checked documentation and the stackoverflow questions (like [this](https://stackoverflow.com/questions/16903528/how-to-get-response-ssl-certificate-from-requests-in-python)) and from what I see, it is not possible to get the server certificate information with requests.

* Is there any hack to get this info?
* Is there any plan to add this info to the answer? If no, consider this as a feature request :+1: 

> Is there any hack to get this info?

Yes. If you stream the response (that is, set `stream=True`), then when you get the response back you can go digging for the socket. Depending on the Python version it'll be in slightly different places: on Python 2 it'll be at `response.raw._fp.fp._sock`, on Python 3 it'll be `response.raw._fp.fp.raw._sock`. At this point, for SSL connections, you can call `getpeercert()`.

> Is there any plan to add this info to the answer?

There is not. =)

Ultimately, this is a feature that's of minimal utility to most people. It comes too late to make doing any kind of security validation worthwhile, and if the goal is simply to determine the certificate for a remote peer it is almost always going to be better to use the ssl and socket modules directly, or even the OpenSSL command line tool.

So the number of people for whom this feature would provide a benefit is ultimately too low to justify adding it. Sorry!

Thanks for your quick answer.

As I could not find any answer to that question, some other people may find this hack interesting. Is there a place where I could add this for documentation? (Like a  "ugly hack that works" in the documentation?)

I noticed that the cert is not available if verify is False, is there any way of still accessing this information even without verifying the certificate?

I would vastly prefer not to add this hack to the documentation, primarily because we don't promise it'll continue to work.

> I noticed that the cert is not available if verify is False, is there any way of still accessing this information even without verifying the certificate?

I'm afraid not. This is a product of the way OpenSSL works: `verify=False` turns into `CERT_NONE`, which causes OpenSSL to not store the certificate if the remote peer presents it.
Prepare changelog for 2.12.5

Getting ready to ship the last patch release in the somewhat accursed 2.12 release series.

## [Current coverage](https://codecov.io/gh/kennethreitz/requests/pull/3823?src=pr) is 89.12% (diff: 100%)
> Merging [#3823](https://codecov.io/gh/kennethreitz/requests/pull/3823?src=pr) into [master](https://codecov.io/gh/kennethreitz/requests/branch/master?src=pr) will decrease coverage by **0.05%**

```diff
@@             master      #3823   diff @@
==========================================
  Files            15         15          
  Lines          1857       1857          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits           1656       1655     -1   
- Misses          201        202     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [160d364...219d9a7](https://codecov.io/gh/kennethreitz/requests/compare/160d36450350cbb8948657a60aa4f0cbe18c601e...219d9a7297efc5673a8c77400204130d66bca311?src=pr)
"import requests" breaks OpenSSL when used in mod_python

When creating a mod_python (Python 2.7, mod_python 3.3.1) page handler that just imports the requests module (version 2.12.3), it breaks pyOpenSSL (pyOpenSSL-16.2.0, cryptography-1.5.3, idna-2.1 in my case). For example creating certificates like this is not working anymore:

```
from OpenSSL import crypto
pkey = crypto.PKey()
pkey.generate_key(crypto.TYPE_RSA, 2048)
```

An exception like this is raised:  Error ([('rsa routines', 'RSA_BUILTIN_KEYGEN', 'BN lib')])

```
  File "key_mgmt.py", line 224, in _generate_key
    pkey.generate_key(crypto.TYPE_RSA, 2048)

  File "lib/python/pyOpenSSL-16.2.0-py2.7.egg/OpenSSL/crypto.py", line 258, in generate_key
    _openssl_assert(result == 1)

  File "lib/python/pyOpenSSL-16.2.0-py2.7.egg/OpenSSL/_util.py", line 61, in openssl_assert
    exception_from_error_queue(error)

  File "lib/python/pyOpenSSL-16.2.0-py2.7.egg/OpenSSL/_util.py", line 48, in exception_from_error_queue
    raise exception_type(errors)
```

When removing this from requests __init__.py, it works again:

```
try:
    from .packages.urllib3.contrib import pyopenssl as pyopenssl
    pyopenssl.inject_into_urllib3()
except ImportError:
    pass
```

I tried to remove only the `pyopenssl.inject_into_urllib3()` call, but that did not solve the issue. I had to remove the import which then fixed the issue. So it seems that it is somehow related to that module directly or some dependency.

The same code works when running as dedicated python script. Maybe it's related to the mod_python importer or something.

Any ideas?

This is definitely related to mod_python. The cryptography library which PyOpenSSL uses has had problems with mod_python in the past due to mod_python's unusual method of running Python. I suggest you open this issue on the pyca/cryptography repository: they should be best placed to fix this issue.
Import error with idna version

Hello,

I recently upgraded to 2.12 and I started to see this import error:

> (Pdb) request('get', url, params=params, **kwargs)
> *** ImportError: No module named uts46data

Looking in the changes I saw that it was recently added idna. 
My projects are using import hooks and inside of it the hook use requests.

The code of idna is doing an import inside the method uts46_remap, which is breaking my code.
Moving the import uts46_remap outside fixes this issue.



Closed for reasons discussed in #3819.
Proxy and SSL : the CONNECT request fails on some proxy (cntlm) as the request is HTTP 1.0 instead of 1.1

It looks like requests (or urllib3 ?) is creating a CONNECT request that fails with some proxies when trying to initiate the SSL tunnel:

INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): api.hipchat.com
send: 'CONNECT api.hipchat.com:443 HTTP/1.0\r\n'
send: '\r\n'

as we can, it generates an HTTP/1.0 header which seems to be inaccurate. I could not find any definition of the CONNECT command in the HTTP/1.0 RFC, and seems to have been really defined in HTTP 1.1 RFC.

By manually telneting to my cntlm, i could make the CONNECT work by replacing 1.0 by 1.1.

This makes the requests library fail to work on my side where i have to use CNTLM because our usual proxy is using Kerberos/CNTLM for authentication. 

Is there any way to have this reviewed ?

best,

Unfortunately, this is not something we can easily change in urllib3 at the moment, because this CONNECT is actually created by httplib. Simply changing to HTTP/1.1 is not sufficient because that CONNECT request is then not HTTP/1.1 compliant (it has no `Host` header field). We'd need to change both.

Ultimately, the upcoming urllib3 v2 release will resolve this issue by removing our dependence on httplib. Until then, the only way to fix this is to monkeypatch or override httplib's `tunnel()` logic.

Found some additional information here : http://stackoverflow.com/questions/1841730/how-can-urllib2-httplib-talk-http-1-1-for-https-connections-via-a-squid-proxy/1841740#1841740
Add validation of HTTP URL

I'd like to verify that a URL will be valid for a HTTP request. The main motivation is telling the user in the GUI that the typed URL cannot be used.

Currently, I'm using `requests.models.parse_url`, but this is obviously not enough because `PreparedRequest.prepare_url` performs additional checks (e.g. `host.encode('idna')`). It would be great if the same checks were exposed as a public API.

The simplest thing to do is to use the [PreparedRequest flow](http://docs.python-requests.org/en/master/user/advanced/#prepared-requests): specifically the second example, which goes via a Session.

You can catch exceptions raised in this process and use those to determine the validity of the URL.

You could also use rfc3986 which has some validation code built-in.

Thanks @Lukasa.