Raibek/requests-github-issues huggingface dataset:
SSLError: [Errno 185090050] _ssl.c:336: error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib

I am seeing this when compiling with requests in Python 2.7.10 with py2exe.

Everything goes just fine till I test the exe it generates.

My setup.py I used to create it is this.

``` Python
from distutils.core import setup
import py2exe


setup(console=['Decorater.py'])
```


This is almost certainly a py2exe problem. It seems like py2exe is not handling the ssl module correctly. 


I think that missing 3 dependencies could cause this as well although it still happens even when I just solved the "Insecure Platform Warning".


looking closer it seems to be similar to the one like this that is closed.

here is some Code that I do not know would work in the module that I make executable.
[personal login data to the account the bot uses stripped from code.]

https://i.gyazo.com/20b51bd6ac81a8f613f047146b910287.png
seems like the below does not work fully atm. But that is when I do not run it with py2exe. Testign it with now.

``` Python
import sys
from requests.certs import where

where()
os.environ['REQUESTS_CA_BUNDLE'] = os.path.join(os.path.dirname(sys.executable), "cacert.pem")
```


Fixed it with the code above but only works in py2exe. Closing this.


The only issues in py2exe are the Platform ones which I have not found out a fix on.
Although I tried the deps on the other issue that was closed.

https://github.com/kennethreitz/requests/issues/2837


@AraHaan this is absolutely not a bug in requests. Please look into using [data files](http://www.py2exe.org/index.cgi/data_files) with py2exe and [StackOverflow](https://stackoverflow.com) for further help with py2exe.


I noticed that when I find the warnings in the files it says from that other issue when I comment them out I do not see them anymore. Now it works just fine.
Posting files with unicode file names

Hi all. I'm having issues with posting files with UTF-8 file names. 
I've thoroughly read these 2 issues: 
https://github.com/kennethreitz/requests/issues/2505
https://github.com/mitsuhiko/flask/issues/1384

Please also refer to my question on SO:
http://stackoverflow.com/questions/33717690/python-requests-post-with-unicode-filenames

Unlike the first issue I linked to, my filename doesn't end with '\n'.

I noticed that the file ends up in request.form and not in request.files (just like described in the issues I linked to), must be because of the '_'  in 'Content-Disposition: form-data; name="file0"; filename_=utf-8''%D7%90.txt'  just after 'filename'.


Sorry, you've said you're "having trouble", but haven't actually specified what's happening that you aren't expecting. Could you elaborate?


I'm sorry it was not clear but the problem is that the server doesn't
receive the file (request.files is empty).
On Nov 15, 2015 12:23, "Cory Benfield" notifications@github.com wrote:

> Sorry, you've said you're "having trouble", but haven't actually specified
> what's happening that you aren't expecting. Could you elaborate?
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2883#issuecomment-156799188
> .


So, I believe this is fundamentally the same problem as shown as #2313. We're encoding the file using RFC 2231, which is a more-than-10-year-old specification, and your server appears not to be following it. To resolve this problem, either set the filename to a bytestring (or one that does not contain unicode data) or get the server fixed. =)
Please add a Session.close()

Is there a reason why the .close() has yet to be implemented, the equivalent of the .close() from the original requests library? To ensure the session is closed, without leaving any open/dead/hanging/zombie sockets, would a Session.close() be the way to avoid having socket nightmares on a multi-threaded crawler utilizing requesocks.Session()?

Trying to do:

```
sesh = requesocks.Session()

# do stuff

sesh.close()
```

From requests:

```
     def close(self):
        """Closes all adapters and as such the session"""
        for v in self.adapters.values():
            v.close()
```

Thanks!


Uh...this is the repository for the original requests library. We _do_ have a close method.
cache Response.text

`Response.text` is a pure function, seems pointless doing the same conversions in repeated calls. 

Cached

```
In [4]: %timeit any(word in resp.text for word in words)
The slowest run took 6.20 times longer than the fastest. This could mean that an intermediate result is being cached 
100000 loops, best of 3: 17.4 µs per loop
```

Not cached

```
In [5]: %timeit any(word in resp.text_old for word in words)                                                                            
10000 loops, best of 3: 64.4 µs per loop
```

`len(words) == 3` in these examples, in a real world use case it could be much longer.


Thanks for this! However, `Response.text` is _not_ a pure function, and cannot be safely cached. This is actually explicitly called out [in the docs](http://docs.python-requests.org/en/latest/user/quickstart/#response-content):

> If you change the encoding, Requests will use the new value of `r.encoding` whenever you call `r.text`.

This is intentional and desirable. For this reason, we want to prevent text from being cached. 

Sorry we can't accept your contribution! =(
Redundant import in docs?

I am talking about this doc.
https://github.com/kennethreitz/requests/blob/master/docs/user/quickstart.rst

Here:

``` python
>>> import json
>>> url = 'https://api.github.com/some/endpoint'
>>> payload = {'some': 'data'}

>>> r = requests.post(url, json=payload)
```

json parameter is optional.
Is there any reason to import json module?
Thank you.


@kiddick You're entirely right: that import of the `json` module is from an old version of the documentation. Would you like to open a pull request to remove it?


@Lukasa Thanks for reply. Yep I'll open pull request!


Thanks for fixing this @kiddick
urlib3.HTTPResponse.stream would confuse gzip content originally not encoded utf8

Hi, i was using requests for get gziped web page which originally encoded in gbk.

as requests would try their best decoding gziped pages but as originally not by utf-8 would make the result weried and hard to debug :( 


@yupbank Sorry, I didn't fully understand this message. Can you post some code that demonstrates the problem?


basically 
i figured the correct way:

```
zlib.decompress(requests.get('http://zhongyanghuachengzs.fang.com/', stream=True).raw.read(),  
         zlib.MAX_WBITS|32).decode('gbk'))
```

as the page is encoded in gbk and  gziped.

also due to the fact that requests would auto decompress gzip, this would give weird results.

```
requests.get('http://zhongyanghuachengzs.fang.com/')
```


I don't believe this is a urllib3 bug: nothing unexpected is happening here.

As you've correctly spotted, requests is transparently decompressing the gzip. This is working perfectly, nothing unexpected is happening. However, I suspect you're then using `response.text`, which is trying to decode the response body. Because there's no header explaining what encoding was used, requests falls back on the spec which claims that text/html should be served in ISO-8859-1. This is wrong for this page, and so it breaks the rendering.

All you need to do is:

```
requests.get('http://zhongyanghuachengzs.fang.com/').content.decode('gbk')
```

That'll work perfectly. =)


thanks a lot, i should follow the document carefully :)
GitHub release page lists the "latest release" as v2.5.1

On https://github.com/kennethreitz/requests/releases , the "latest release" label is next to v2.5.1, whereas the newest release is v2.8.1, which is hidden under the "Show 8 newer tags" link.

If newer release entries are missing because it's a hassle to transpose the release notes into each, perhaps just linking to the relevant entry on http://docs.python-requests.org/en/latest/community/updates/ would be sufficient? (Though it seems the permalinks on that page are just generic, eg `#id1` and not `#v2-8-1` etc). Or even leaving the notes section blank would be fine.


Yeah, it's mostly annoying to transpose the notes. I'll do it now anyway. =)


@Lukasa let's just remove them. This is a hassle and we've had .... 4 bugs about this? It's worthless and it doesn't provide real value.


Removing them works for me; at least GitHub then defaults to just showing the tags, which will always be up to date :-)


Releases deleted.


:heart: @sigmavirus24 

Thanks for the report @edmorley!


Many thanks :-)
unnest deep loop

Flat is better than nested. Instead of nesting `if` statements, check for the opposite early in the loop and `continue` if the opposite is the case.


Good idea @vienno!

If we're going to un-nest this, let's un-nest it the whole way. Want to break out the next nested couplet as well and use the same logic?


How about now @Lukasa ? I rewrote the third `if` statement as well, for consistency.


Thanks, much better! I have one small note, inline on the diff. =)


Of course! It's been updated.


Looks good to me @vienno! Would you like to add yourself to AUTHORS.rst?


Sure!


Thanks so much @vienno! :sparkles: :cake: :sparkles:
How to deal with using requests and urllib3 together

Ran into an interesting issue in the past week while testing things out with SaltStack.  ( SaltStack is a remote execution engine / cfg mgmt tool ).  SaltStack uses requests internally.

However, one of the 'modules' you can use is python-etcd.  python-etcd uses urllib3.  But, because of request's wrapping of urllib3 exceptions, some of the python-etcd code bombs because it tries to catch urllib3 exceptions via fully-qualified names.

An example of this happening can be easily demo'd with:

```
#!/usr/bin/python

#import requests
import etcd

c = etcd.Client('127.0.0.1', 4001)
c.read("/blah", wait=True, timeout=2)
```

Without Requests, you get etcd.EtcdConnectionFailed.  With it, you get urllib3.exceptions.ReadTimeoutError.  python-etcd tries to catch urllib3 exceptions, but because of request's wrapping, it fails to catch them.

Is there a way to work around this properly ( besides going to the python-etcd author and asking him to change everything to use requests instead of urllib3 )?


How are you installing requests and urllib3?


Via RPM ( both from RHEL's official repos for RHEL6 )

```
[root@alpha site-packages]# rpm -qa | grep urllib3
python-urllib3-1.10.2-1.el6.noarch
[root@alpha site-packages]# rpm -qa | grep requests
python-requests-2.6.0-3.el6.noarch
```


So this is a version of requests that had some magic handling for downstream redistributors that remove parts of requests. This should be fixed in newer versions of requests but I have no control over RHEL's repos.


@sigmavirus24 No problem - the pip versions should be good though?


@sjmh should be!


@sigmavirus24 Alrighty - thanks, just confirmed those versions work! Do you know if there's any way to get around the issue with those RPM versions?  If not, is there a minimum version that this issue would be fixed in ( Looks like newest from pip is 2.8.1 ) or is this a general thing for all downstream redistributors in any version?

Thanks for the help.


I forget in which version we fixed it upstream but I'm tempted to say 2.7.0.
Can't pass boolean parameter in body of POST request when using multipart encoding

Here's a short example of what I'm experiencing:

``` python
params = {
"someBoolean": True
}

response = requests.post(endpoint, data=params, files={'input_file', open(fileobj, 'rb')})
```

The issue is that requests encodes the `someBoolean` parameter as the string value `"True"` rather than the boolean literal `true` value.


The `data` argument does not encode as JSON: it encodes as form-encoded data. This has no `true` literal. If you want to encode json, you'll want to use the `json` argument, but you cannot combine that argument with the `files` argument. Instead, try this:

``` python
import json
response = requests.post(endpoint, data=json.dumps(params), files={'input_file', open(fileobj, 'rb')})
```


@Lukasa that won't work. We expect `data` to be a dictionary (or list of tuples with length two) and `files` to be the same. That said, without a description of what the endpoint you're talking to is actually expecting, we can't provide more help than we already have.

This is probably a problem better suited for [StackOverflow](https://stackoverflow.com/questions/tagged/python-requests).
Fix breakage introduced by 8f591682

Turns out we broke chunked uploads in 8f591682e6a901baf0cc8a0393b5930252f0318a. I discovered this while investigating #2215.

While we were here, I added a test that would at least have caught errors as egregious as this one.


To note: this breakage did not make it into any shipped version of requests, so we don't need to schedule a patch release for this.


Why is the build failing?


Good question. The test behaves fine on my
Mac, but it's a tricky one because chunked encoding and WSGI don't necessarily play nice. 

> On 5 Nov 2015, at 19:02, Ian Cordasco notifications@github.com wrote:
> 
> Why is the build failing?
> 
> —
> Reply to this email directly or view it on GitHub.


Hmm, so this failure is intermittent on my machine.


(my machine == Linux box I found)


I wonder if this is actually a problem with the WSGI server being used by pytest-httpbin. Specifically, I think it might be related to the speed of the server.


Specifically, I think what's happening here is that the chunked-encoding header is being stripped, and so the server is sending the 200 right away and then killing the connection. This works _sometimes_ because the socket sends happen before the connection is closed, but sometimes the server closes the connection before we can get there.

This is pretty annoying, because it means it's likely very hard to find a good test-case here.


More importantly, it means we can't really have tests _at all_ for chunked transfer encoding.


I honestly don't know what to do about this. To get a test that confirms we don't break this we need either to change the WSGI server used by pytest-httpbin (the only options are ones that are _really_ heavyweight and probably not suitable) or start writing new tests that don't use a httpbin at all.

The second is plenty do-able, it just requires some py.test test fixtures and some socket work. But it makes me a bit nervous given @kennethreitz's attitude to changing our test layout in the past: I'd want to move all the tests to a `test/` subdirectory so that I can add the necessary `conf.py` file, and also start writing new tests that explicitly use sockets rather than a web server.

@kennethreitz, do you have any objection to me adding some more thorough testing around here? The reality is that our chunked upload logic is _entirely_ untested, and has been since it was originally written. That is a recipe for accidental breakage, and it makes me really nervous.


Ok, so I pushed a new commit that doesn't contain the test. This should pass, and the fix _is_ right, but I don't want us to lose track of the need for better testing here so I'm going to open a separate issue for that.


@sigmavirus24 This should be good to merge.


@Lukasa I'm :+1: but I think we should work on fixing our test suite. If Kenneth comes along and decimates it again, then we at least had test coverage for some period of time there.
Move test suite offline using pytest-httpbin

Resolves #2184, and should allow us to move back to Travis for our testing.


There _ought_ to be a lot of improvements to doing this. Tests should be a lot closer to reproducible, and the runtime should improve greatly. This is still not really how I want this test suite to look, but it's a _lot_ closer.


One request, then I'll press the big green button. :)

Alternatively, we can add a bug to track the work of adding that test.


I think it's better to add a bug than a note in the code: we'll just lose the note. ;)


Thanks @Lukasa!
Expose expires attribute of cookie in class RequestsCookieJar

Right now, the class `RequestsCookieJar` class methods only expose `name` and `value` attributes of its cookies. It will be useful to have a method that checks if any of the cookies in `RequestsCookieJar` are expired. 
Like `is_expired()` would return True if any cookies are expired otherwise false


The `RequestsCookieJar` is a subclass of the standard library's [`cookiejar`](https://docs.python.org/2/library/cookielib.html#cookiejar-and-filecookiejar-objects), which means that it behaves exactly like that class. In particular, you can get the actual cookies themselves by iterating over it. This means the method you want is one line of code:

``` python
any_expired = any(cookie.is_expired() for cookie in session.cookies)
```

We're generally reluctant to add methods that are as simple as that one, given that there are a number of other constructions users might want (a method to say whether _all_ the cookies have expired, for example). Restricting the surface area of our APIs is extremely important to us.

One final thing: what's your use case? From where I'm sat it's hard to see why this would be useful: the cookie jar will remove expired cookies when they would have gotten used anyway, so I'm not entirely sure what advantage you receive by making this change.
Session level default timeout

Best I can tell, today there is no way to set a default timeout at the Session level. That is, I want to create a session and give it a default timeout which should be used on any request() for the session if no timeout is given on the actual **kwargs of the request invocation.

Example:

``` python
session = requests.Session()
session.default_timeout = 9
...
# default timeout of 9 is used
session.get('http://thehost/good-stuff') 

# passed value of 3 is used
session.get('http://thehost/good-stuff', timeout=3)
```

This would be very helpful for consumers wanting to pool request sessions above the requests lib layer using common kwargs for all requests in the session(s).


Hi @bodenr thanks for using requests and stopping by with this feature request.

The library strives to make very specific divides in what is appropriate to add where. Timeout values being specified on a session do not make very much sense to the project although it's plausible for you to add some code to do this (like [pip](/pypa/pip)) has. 

In the future please search for closed and open issues discussing your topics. A quick search immediately brought up https://github.com/kennethreitz/requests/issues/2011 which was closed over a year ago. I'm sure there are others, but you can find them too with a quick search.
Added Travis support with tox

- Added `.travis.yml`
  I think it is useful for every contributors to have a online tool for test.
  I added Travis' badge in `README.rst``
  Maybe you want to add Coveralls support in future ?
- Added `tox.ini`
  I implement tox usage for simplify tests across Python versions.
  Maybe you want to add documentation tests in future ?

There are two errors (don't come from my commit) and the build is failing...

```
____________ RequestsTestCase.test_BASICAUTH_TUPLE_HTTP_200_OK_GET _____________
self = <test_requests.RequestsTestCase testMethod=test_BASICAUTH_TUPLE_HTTP_200_OK_GET>
    def test_BASICAUTH_TUPLE_HTTP_200_OK_GET(self):
        auth = ('user', 'pass')
        url = httpbin('basic-auth', 'user', 'pass')

        r = requests.get(url, auth=auth)
        assert r.status_code == 200

        r = requests.get(url)
>       assert r.status_code == 401
E       AssertionError: assert 200 == 401
E        +  where 200 = <Response [200]>.status_code
test_requests.py:317: AssertionError
_________ RequestsTestCase.test_auth_is_stripped_on_redirect_off_host __________
self = <test_requests.RequestsTestCase testMethod=test_auth_is_stripped_on_redirect_off_host>
    def test_auth_is_stripped_on_redirect_off_host(self):
        r = requests.get(
            httpbin('redirect-to'),
            params={'url': 'http://www.google.co.uk'},
            auth=('user', 'pass'),
        )
        assert r.history[0].request.headers['Authorization']
>       assert not r.request.headers.get('Authorization', '')
E       AssertionError: assert not 'Basic dXNlcjpwYXNz'
E        +  where 'Basic dXNlcjpwYXNz' = <bound method CaseInsensitiveDict.get of {'Authorization': 'Basic dXNlcjpwYXNz...ing': 'gzip, deflate', 'User-Agent': 'python-requests/2.8.1', 'Accept': '*/*'}>('Authorization', '')
E        +    where <bound method CaseInsensitiveDict.get of {'Authorization': 'Basic dXNlcjpwYXNz...ing': 'gzip, deflate', 'User-Agent': 'python-requests/2.8.1', 'Accept': '*/*'}> = {'Authorization': 'Basic dXNlcjpwYXNz', 'Connection': 'keep-alive', 'Accept-Encoding': 'gzip, deflate', 'User-Agent': 'python-requests/2.8.1', 'Accept': '*/*'}.get
E        +      where {'Authorization': 'Basic dXNlcjpwYXNz', 'Connection': 'keep-alive', 'Accept-Encoding': 'gzip, deflate', 'User-Agent': 'python-requests/2.8.1', 'Accept': '*/*'} = <PreparedRequest [GET]>.headers
E        +        where <PreparedRequest [GET]> = <Response [200]>.request
```


Thanks @ZuluPro but we already use an external Jenkins server that we maintain. Travis proved to be unstable for our needs.

Cheers!
Bad default parameters to iter_content

Apologies if this is a duplicate or not a bug – I ran into this recently and figured I should report it.

`iter_content` takes a `chunk_size` argument, which defaults to 1. Various code snippets on the internet call `request.iter_content()`. This is incredibly inefficient, because it is using a buffer size of 1 _byte_. I think `requests` should either a) set `chunk_size` to something more reasonable, like 2048 or 4096, or b) make `chunk_size` a required argument. (Solution a is more palatable since it's backwards compatible.)


See https://github.com/kennethreitz/requests/issues/844 please


Yeah, this is an outstanding problem that we haven't gotten around to resolving yet. However, it's a dupe of #844.
Problem with verbose logging

I am trying to enable verbose logging with my request string and I am getting the following error:
"request() got an unexpected keyword argument config"

I have tried searching but have not found a fix for this and I followed the following directions:

> > > my_config = {'verbose': sys.stderr}
> > > requests.get('http://httpbin.org/headers', config=my_config)
> > > 2011-08-17T03:04:23.380175   GET   http://httpbin.org/headers
> > > <Response [200]>

Thanks


Please consult our documentation: The section immediately above [the Licensing section](http://docs.python-requests.org/en/latest/api/?highlight=log#licensing) explains how to enable verbose logging in versions of requests 1.0.0 and greater.


Also in the future, please ask all questions on [StackOverflow](https://stackoverflow.com/questions/tagged/python-requests). This is a defect tracker, not a support forum.


Thanks and noted for future questions!
Documentation Problem?

When I run the following code from requests documentation, I get a 401 response.

import requests
r = requests.get('https://api.github.com', auth=('user', 'pass'))

Given that this is the quick start code for requests, is there something that needs to be updated?


@djchou In this context, `user` and `pass` are placeholder strings to be replaced by your own Github username and password.


Yes, I replaced them with my user pass and still had the same result. 

> On Oct 27, 2015, at 5:53 AM, Cory Benfield notifications@github.com wrote:
> 
> @djchou In this context, user and pass are placeholder strings to be replaced by your own Github username and password.
> 
> —
> Reply to this email directly or view it on GitHub.


@djchou What is the result of:

``` python
import requests
r = requests.get('https://api.github.com', auth=('user', 'pass'))
print(r.content)
```


ah, its the two factor auth that is causing it to fail.

'{"message":"Must specify two-factor authentication OTP
code.","documentation_url":"https://developer.github.com/v3/auth#working-with-two-factor-authentication"}'

On Tue, Oct 27, 2015 at 6:07 AM, Cory Benfield notifications@github.com
wrote:

> @djchou https://github.com/djchou What is the result of:
> 
> import requests
> r = requests.get('https://api.github.com', auth=('user', 'pass'))print(r.content)
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/kennethreitz/requests/issues/2851#issuecomment-151489049
> .


Yeah, that's what I thought too.

I don't think it's worth gumming up the docs with this: it's extraneous detail that won't help much.
urllib3 SSL errors in 2.8.x

We recently tried to upgrade to requests 2.8.1 and have been touched by this urllib3 issue: https://github.com/shazow/urllib3/issues/717

Downgrading to 2.7.0 resolves it.


@flupke The urllib3 and requests projects work together closely: we're entirely aware of the bug that you're encountering, but we won't be able to fix it until a new urllib3 release comes out.
Add 511 Network Authentication Required status code for completeness

Based on http://tools.ietf.org/html/rfc6585, looks like all the other status codes mentioned in that document are included except for 511. Any specific reason/concern?

I'd be happy to open a pull request with the following additional line if it looks reasonable to you.

```
511: ('network_authentication_required', 'network_auth', 'network_authentication'),
```


Please do @annp89. I think it was just an error code that we happened to miss. I don't think there was any reason or concern we had with adding it.

Cheers!


Awesome! Will do shortly :)


Closed by #2849
HTTP PROXY password contains '#' failed to connect

``` python
import requests

PROXIES = {
    'http':'http://{usr}:{pwd}@proxy:port'.format(usr = 'XXX',pwd = 'XXX#XXX'),
}

url = 'http://www.sina.com'
r = requests.get(url, proxies = PROXIES)
print r.text
```

with error :
Traceback (most recent call last):
  File "autohome.py", line 11, in <module>
    r = requests.get(url, proxies = PROXIES)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\api.py",
 line 55, in get
    return request('get', url, *_kwargs)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\api.py",
 line 44, in request
    return session.request(method=method, url=url, *_kwargs)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\sessions
.py", line 383, in request
    resp = self.send(prep, *_send_kwargs)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\sessions
.py", line 486, in send
    r = adapter.send(request, *_kwargs)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\adapters
.py", line 305, in send
    conn = self.get_connection(request.url, proxies)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\adapters
.py", line 215, in get_connection
    block=self._pool_block)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\packages
\urllib3\poolmanager.py", line 258, in proxy_from_url
    return ProxyManager(proxy_url=url, **kw)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\packages
\urllib3\poolmanager.py", line 207, in __init__
    proxy = parse_url(proxy_url)
  File "C:\Python27\lib\site-packages\requests-2.2.1-py2.7.egg\requests\packages
\urllib3\util.py", line 397, in parse_url
    raise LocationParseError("Failed to parse: %s" % url)
requests.packages.urllib3.exceptions.LocationParseError: Failed to parse: Failed
 to parse: XXX:XXX

the password after '#' missed and can not connect to the proxy server


Since the proxy URL needs to be a valid URL, you'll need to URL encode any special characters in the credentials yourself before you put them in an URL - otherwise, the `#` will be interpreted as a fragment identifier, in this case by `urllib3`'s [`parse_url()`](https://github.com/kennethreitz/requests/blob/v2.2.1/requests/packages/urllib3/util.py#L335-L417).

Try `XXX%23XXX` for your password (`%23` is the URL encoded form of `#`). Or programmatically:

```
import urllib
encoded_pwd = urllib.quote_plus(pwd)
```


```
import urllib
encoded_pwd = urllib.quote_plus(pwd)
```

it worked! Thank you
_encode_params return bytes that break urlencode

Simple example:

```
import requests


r = requests.get('http://python.org', params=b'test=foo')


print(r.status_code)
```

Trace:

```
Traceback (most recent call last):
... skip ...
    url = requote_uri(urlunparse([scheme, netloc, path, None, query, fragment]))
  File "C:\Python35\lib\urllib\parse.py", line 383, in urlunparse
    _coerce_args(*components))
  File "C:\Python35\lib\urllib\parse.py", line 111, in _coerce_args
    raise TypeError("Cannot mix str and non-str arguments")
TypeError: Cannot mix str and non-str arguments
```

I know, that this one is a bad use case, but API allows us to do so.

Should we do something?

P.S. Python 3.4/3.5 on win 8.1 x64


I think what we need to do here is to change `encode_params` to make sure that it calls `to_native_string` on its argument before returning it. This seems like a bug we can fix pretty swiftly, and is well suited to a new contributor, so I'll mark this as `Contributor Friendly`.


This fix seems completely broken. It prevents me from uploading binary data through PUT.


@untitaker please open a new issue.


See #2030


Could you release a quick 2.9.1 with this fix plz ?

I was waiting the 2.9.0 for the REQUEST_CA_BUNDLE env var but cannot use it for the moment due to this bug ;-(


@touilleMan We're planning to release 2.9.1 on Monday.


Awesome, you guys rock ! ^^


Looking forward to the new release. This bug breaks everything, since we use Chinese...
Timeout used by `resolve_redirects` generator can now depend on URL

`timeout` parameter given to the `Session::resolve_redirects` member method
can be a `callable` object taking the URL in parameter.

It allows you to specify different timeout values for URLs encountered
by the generator.


How is it expected that you'd use this? By setting `allow_redirects=False` and calling `resolve_redirects` manually?


Yes exactly!


Ok. =)

So, generally speaking, we don't allow the parameters to `resolve_redirects` to be different to those that you can pass to `requests.*`: it just leads to too much confusion. In this case, it should be safe enough to step through `resolve_redirects` one URL at a time and check the `Location` header as you go, using the information there to determine what timeout you'd like to apply.

Given that we are in feature freeze, I don't think this meets the bar for a merge into the library. Sorry! =(
PEP8 compliance

Hello,

I've just forked requests and tried flake8 on it.

1       E122 continuation line missing indentation or outdented
1       E127 continuation line over-indented for visual indent
1       E129 visually indented line with same indent as next logical line
1       E401 multiple imports on one line
1       E713 test for membership should be 'not in'
1       E714 test for object identity should be 'is not'
1       F402 import '_' from line 13 shadowed by loop variable
1       W391 blank line at end of file
2       E121 continuation line under-indented for hanging indent
2       E225 missing whitespace around operator
2       F841 local variable 'url' is assigned to but never used
3       E125 continuation line with same indent as next logical line
3       E221 multiple spaces before operator
3       E301 expected 1 blank line, found 0
4       E126 continuation line over-indented for hanging indent
10      E303 too many blank lines (3)
10      W291 trailing whitespace
11      E131 continuation line unaligned for hanging indent
16      F821 undefined name 'unicode'
20      E251 unexpected spaces around keyword / parameter equals
20      E302 expected 2 blank lines, found 1
39      E128 continuation line under-indented for visual indent
65      E262 inline comment should start with '# '
84      F401 'urllib3' imported but unused
85      E261 at least two spaces before inline comment
111     E265 block comment should start with '# '
257     E501 line too long (80 > 79 characters)
2651    E231 missing whitespace after ','

Is this relevant? I think it should be available some linting check before doing setup.py install


Please check old issues before opening new ones.
- https://github.com/kennethreitz/requests/issues/1899
- https://github.com/kennethreitz/requests/pull/2298
- https://github.com/kennethreitz/requests/pull/1964

This project does not follow PEP-0008.
test_connection_error_invalid_domain and test_connection_error_invalid_port fail with proxies

testing should handle environments with proxies consciously, otherwise these tests fail:

``` ======================================================================
ERROR: test_connection_error_invalid_domain (__main__.RequestsTestCase)
Connecting to an unknown domain should raise a ConnectionError
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_requests.py", line 322, in test_connection_error_invalid_domain
    requests.get("http://doesnotexist.google.com")
  File "/usr/lib/python2.7/site-packages/_pytest/python.py", line 1307, in __exit__
    pytest.fail("DID NOT RAISE")
  File "/usr/lib/python2.7/site-packages/_pytest/runner.py", line 478, in fail
    raise Failed(msg=msg, pytrace=pytrace)
Failed: DID NOT RAISE

======================================================================
ERROR: test_connection_error_invalid_port (__main__.RequestsTestCase)
Connecting to an invalid port should raise a ConnectionError
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_requests.py", line 327, in test_connection_error_invalid_port
    requests.get("http://httpbin.org:1", timeout=1)
  File "/usr/lib/python2.7/site-packages/_pytest/python.py", line 1307, in __exit__
    pytest.fail("DID NOT RAISE")
  File "/usr/lib/python2.7/site-packages/_pytest/runner.py", line 478, in fail
    raise Failed(msg=msg, pytrace=pytrace)
Failed: DID NOT RAISE
```

Environment: openSUSE 13.2/x86_64, squid 3 proxy


Yeah, this is probably true. I suspect we'll fix it by moving to an entirely offline test suite though.


Neither of these tests exist anymore, so I think this can be closed out.
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 52-55: ordinal not in range(256)

Requests 2.8.1

Python 3.5.0 (default, Sep 20 2015, 11:56:03) 
[GCC 5.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import requests
r = requests.post('https://2ch.hk/makaba/posting.fcgi', data='json=1&task=post&board=test&thread=80980710&comment=тест')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.5/site-packages/requests/api.py", line 109, in post
    return request('post', url, data=data, json=json, *_kwargs)
  File "/usr/lib/python3.5/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, *_kwargs)
  File "/usr/lib/python3.5/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, *_send_kwargs)
  File "/usr/lib/python3.5/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, *_kwargs)
  File "/usr/lib/python3.5/site-packages/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/usr/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/usr/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 353, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.5/http/client.py", line 1083, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1127, in _send_request
    body = body.encode('iso-8859-1')
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 52-55: ordinal not in range(256)


Please do not attempt to send body data in unicode strings. You need to encode that before you send it: `data='json=1&task=post&board=test&thread=80980710&comment=тест'.encode('utf-8')`
Insecure Platform Warning

I am contributing to a project that uses requests library (version 2.7.0) and even though the requests return outputs without any issue, I see the following warnings being printed:
/Users/ue/.virtualenvs/imgurpython/lib/python2.7/site-packages/requests-2.7.0-py2.7.egg/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail.

Any help on how to fix this?


@ueg1990 This is occurring because you're using this on an old version of Python which does not have a fully-featured `ssl` module. You can fix it either by upgrading to 2.7.9 or later, or by installing `pyopenssl`, `ndg-httpsclient`, and `pyasn1`.
Use session.verify

session.verify is currently ignored. Didn't have time for a unit test, but this is one solution.


@causton81 What evidence do you have that `Session.verify` is ignored? This works fine for me:

``` python
>>> import requests
>>> s = requests.Session()
>>> s.verify = False
>>> s.get('https://kennethreitz.org/')
/Users/cory/.pyenv/versions/2.7.10/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:789: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
/Users/cory/.pyenv/versions/2.7.10/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:789: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
<Response [200]>
```


Sorry for the lack of details. The problem is when either `REQUESTS_CA_BUNDLE` or `CURL_CA_BUNDLE` is set in the environment and session.trust_env==True, then session.verify is ignored.  I added a unit test to demonstrate. Just for completeness I missed this for awhile because I was testing with Python 2.7.3 which ignores cert issues. I noticed it when testing with Python 2.7.9 and env variable REQUESTS_CA_BUNDLE.


Ah, yes, I see the problem now.

We have a similar issue with proxy settings, where proxies set in the environment can override proxies set on the session. I'm beginning to wonder if we can fix two problems in one go here, by taking the `trust_env` block of `merge_environment_settings` and moving it below the `merge_setting` lines in that function.

@causton81 This is definitely a real bug, but we need a bit of time to work out what the fix is and what branch it'll go on to (while this is strictly a bug fix, it's been in the product long enough that it's also an API compatibility break and needs to be dealt with cautiously).

@sigmavirus24 What do you think about my proposed rearrangement of `merge_environment_settings`?


@Lukasa the rearrangement is something we had discussed for 3.0.0 because it's fundamentally backwards incompatible. I'm in favor of it though because I think the order of precendence should be (in order of least to most important):
- Environment
- Session settings
- Per-call settings

(For those features to which this applies)


Ok, let's get this done for 3.0.0.


Ok, I'm closing this in favour of #2839.
Response.text as byte string

Hey,

we are missing informations how to stop the requests library from guessing the content type of response.text.

http://docs.python-requests.org/en/latest/api/?highlight=text#requests.Response.text

For our use case we need the original byte string. The current docs don't explain how to get it.
Please improve the docs.

Best regards
René


The docs absolutely do say how to get the original byte string, just a few lines up from where you linked: http://docs.python-requests.org/en/latest/api/?highlight=text#requests.Response.content

It's `response.content`.
Not marked as python3.4 compatible in PyPi

Hi there,
 http://docs.python-requests.org/en/latest/ says there is support for python 2.6-3.4, but PyPi lists only 2.7.  I discovered this when i used the tool `caniusepython3` and saw requests on the list of packages holding me back.


Yeah, all our trove classifiers seem to be missing on PyPI. Looks like it was a while since it was registered. Resolved now. =)

Thanks for the report!
custom retries doesn't work while using POST method

I had very confused

```
#!/usr/bin/env python
import logging

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3 import Retry

logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger('requests')
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

with requests.Session() as s:
    retry = Retry(status_forcelist=[503])
    adapter = HTTPAdapter(max_retries=retry)
    s.mount('http://', adapter)

    try:
        # r = s.get('http://httpbin.org/status/503')
        r = s.post('http://httpbin.org/status/503')
        r.raise_for_status()
    except requests.HTTPError:
        print 'BANG!'
        print "where are my retries???"
    except requests.exceptions.RetryError:
        print 'computation does not rich this exception'
```

When `session.get()` all works fine.


You need to specify the methods via the `method_whitelist` parameter to the Retry class. 


thx, forgive my inattention


No worries @tribals. If you read my blog post about retries with requests I probably failed to mention this. Cheers!


if it's [this post](http://www.coglib.com/~icordasc/blog/2014/12/retries-in-requests.html), then yep, i've read it
Use buffering for HTTP responses on chunked requests

For non-chunked requests, the request is performed using the 'urlopen'
method that underlying uses buffering for the HTTP responses (for Python
2.7+ versions). For chunked requests though, the request is made via a
different code path and so the 'getresponse' method is called without
using buffering. So, the response is consumed per single byte via a
recv() call on the underlying socket.

This patch, fixes that issue to mimic the non-chucked request behavior.


This seems like a sensible change to me, I'm :+1:. @sigmavirus24?


What's the benefit?


Without `buffering`, `httplib` will issue a lot of 1-byte reads in order to satisfy the constraints of the `read` call. That's...not good.


Sorry, I was reading the commit message/PR message backwards. LGTM :+1: 


Thanks for this @dblia! :cake: :sparkles: :cake:


Thanks for the immediate response @Lukasa and @sigmavirus24!
PUT send is doing Quotation on json payload

Hi,
i need to send via PUT an octetstream/json element via put. requests is quoting the full string. Example:

```
    testdata = '[{"cid": "1","uuid": "sdasd"}]'        
    response = requests.put(requestUrl, auth=self.auth, json=testdata)
```

can be seen in Wireshare like this:
        "[{\"cid\": \"1\",\"uuid\": \"sdasd\"}]"

if i use POST it is unquoted.
My remote REST Interface is not able to handle the quoted strings. I wasn't able to find any option where i can turn of the quotation.


If you read the [documentation](http://docs.python-requests.org/en/latest/user/quickstart/?highlight=json#more-complicated-post-requests) closely, you'll see your code should look like:

``` py
    testdata = [{"cid": "1","uuid": "sdasd"}]
    response = requests.put(requestUrl, auth=self.auth, json=testdata)
```


the response is within a class which contains the auth definitin like:

self.auth = HTTPBasicAuth(username, password)

and i use this later on.
And how does the auth content influence the payload?


@czerwe 
1. I don't understand your question
2. The issue tracker is not a support forum. Please use [StackOverflow](https://stackoverflow.com/questions/tagged/python-requests)


I am sorry about missleaded:
if found out that this seems to work:
        testdata = [{"cid": 1, "uuid":2}]
        response = requests.put(requestUrl, auth=self.auth, headers={"Content-Type": 'application/json'}, data=json.dumps(testdata))

using jsons doing quotation. From my point it's a missbehavior of the json handler in the session. 
Nevertheless sorry.


It is not an error in the Json handler. The json handler does not accept a _string_, it accepts the Python object directly. The difference between what you provided and what @sigmavirus24 gave is this:

``` python
testdata = '[{"cid": "1","uuid": "sdasd"}]'
testdata = [{"cid": "1","uuid": "sdasd"}]
```


I was already wondering.
Your right. I tried so many things that some error output from the response miss leaded me. 

Thank you both. Next time i will ask in StackOverflow first.
Sorry.
Fixed usage example in sessions.py

The usage example for sessions.py's Session class is not correct in my opinion. The get method in fact returns a `Response <Response>` object, the notation "200" may be misleading the user into thinking only the http statuscode will be returned. Furthermore the output you get on console if typing the example in is different from the example and similar to my change.

The two usage examples in models.py are also similar to my change.

Thank you for considering.


Thanks for this @daniel-mueller!
Bad examples of PUSH and PUT in Quickstart

The quickstart guide had these examples:

```
>>> r = requests.post("http://httpbin.org/post")
>>> r = requests.put("http://httpbin.org/put")
```

With PUSH and PUT you almost always want to include some data. Without it they become fairly meaningless.


That's certainly true, though with the quickstart guide we're generally trying to err on the side of not providing _too_ much extraneous information. Nevertheless, if you'd like to open a pull request giving both of those some data (a simple bytestring will do), I'd be happy to merge that. =)


@Lukasa I have opened a pull request for this.


Thanks @shagunsodhani. I got busy and didn't notice this.
Add warnings about text/binary mode files.

Resolves #2725.

@sigmavirus24 I'm wondering if we should add a warning to `super_len` for whenever we decide on the length of the file using `os.fstat`, if the file has not been opened in binary mode. It should be easy enough to do. Thoughts?


@Lukasa do we want to add that warning in `super_len` and then make it a hard exception in 3.0.0?


@sigmavirus24 Yeah, probably.


&hearts; thanks brosephs @sigmavirus24 @Lukasa 


@Lukasa do you want me to send that separately or do you want to do that as part of this PR?


Ah, I forgot about that. Let's do it here.


Ok, I added a warning that should fire once the first time this happens (no need to spam the crap out of people with it). It's strictly a subclass of `DeprecationWarning`, which I think is ok too.


Excellent work @Lukasa . Thanks!