Raibek/requests-github-issues huggingface dataset:
Added the ability to pass matrix parameters for rest requests.

Unfortunately according to wc3 (http://www.w3.org/DesignIssues/Axioms.html#matrix) matrix parameters to http requests has been unsupported since 2001. The company I work for recently tried integrating a REST interface with third party software and their REST request required HTTP matrix parameters. Which up until this morning I had never heard of. I have added a quick patch into requests to support this.

Information on how matrix parameters are used and created is found at this link:
http://www.w3.org/DesignIssues/MatrixURIs.html


Hi @jase1987! Thanks so much for this pull request. :cake:

Unfortunately, I think it's really very unlikely that we'll accept this. This has nothing to do with the PR itself: the code is in great shape. However, I don't think we want the feature.

Matrix parameters are really very, very infrequently used. The [URI RFC](http://pretty-rfc.herokuapp.com/RFC3986) does not include them, On top of that, the article you linked to doesn't just say they stopped being supported in '01: they hadn't been supported up until then _either_. I haven't done exhaustive research here, but it's quite possible they were never part of the URI (or URL) standards.

You've also only tackled a subset of the problem. Insanely, matrix parameters can apply to any (and all) portions of the path element, e.g. `http://example.com/res/categories;name=foo/objects;name=green/?page=1`. This is very difficult to cleanly represent in Requests' standard functional API.

Given that they're not part of the standard, are very infrequently used, and are difficult to do 'properly', I think we won't want to put this in mainline Requests. With that said, I encourage you to maintain a downstream fork of Requests that includes them if that would be useful to you.

Also, I'm very sorry that you have to interact with such an unhelpful REST API. =)

I'm going to leave this open until @kennethreitz takes a look, but I'd be surprised if he wanted to add support for matrix parameters.

Again, thanks so much for the work!


The example you give @Lukasa looks like this is part of RFC 6570 and is supported in [uritemplate.py](/sigmavirus24/uritemplate).

If they can come up with templates for the URIs they're handling, they can pass the construction off to my library and then pass the string returned directly into requests. Unfortunately the library isn't well-documented at the moment, but it is well-tested and it follows the RFC to the 't'. @jase1987 you should look into "Level 3" expansions for semi-colon prefixed paths. It should fit your needs well. An example can be found in my tests [here](https://github.com/sigmavirus24/uritemplate/blob/master/test_uritemplate.py#L131). The template which is the key for the dictionary and the expected values are what should be of interest to you.


In other news, I'm very +1 on **closing** this.
Exception links

Sphinx has a neat cross-referencing feature where if you include the tilde
character in front of a :py: class, it'll link to the full object but provide
only the last part of class name in the text. For more info see
http://sphinx-doc.org/domains.html#cross-referencing-syntax


I'm +0 on this, actually. I like having clearer class names, but having the full class name makes it way more obvious how to import the class. If it were me, I'd probably leave it as is, but @kennethreitz will probably disagree with me. =)


Whatever you want to do is fine


@Lukasa is correct :)

Thanks @kevinburke!
Easier way to disable cookie persistence

I was looking for a way to temporary disable cookie persistence. 
When I use requests to access an URL **cookies are automatically sent back to the server** (in the following example the requested URL set some cookie values and then redirect to another URL that display the stored cookie)

```
>>> import requests
>>> response = requests.get("http://httpbin.org/cookies/set?k1=v1&k2=v2")
>>> response.content
'{\n  "cookies": {\n    "k2": "v2",\n    "k1": "v1"\n  }\n}'
```

Is it possible to temporary disable cookie handling in the same way you set Chrome or Firefox to not accept cookies?

[I made this question on SO](http://stackoverflow.com/questions/17037668/how-to-disable-cookie-handling-with-the-python-requests-library?answertab=votes#tab-top) and it seems that the only way to go is patching the `requests.cookies.RequestsCookieJar.update`. I think that should be an easier way to do this in requests, Maybe passing parameter to the `get` method?


Hi @esaurito, thanks for raising this issue!

By design Requests' primary API supports the 90% use-case, which involves persisting cookies. We are very unlikely to add a parameter to the main methods to allow you not to persist cookies.

If you absolutely don't want to, my recommendation is that you use a session along with a subclass of the `RequestsCookieJar`. I haven't tried the following, but it ought to work:

``` python
import requests
from requests.cookies import RequestsCookieJar

class BlackHoleCookieJar(RequestsCookieJar):
    """
    This class's attitude to cookies can be described in three words: nom nom nom.
    """
    def set(self, name, value, **kwargs):
        pass

    def update(self, other):
        pass

s = requests.Session()
s.cookies = BlackHoleCookieJar()
```

Subsequent requests through that session should throw cookies away. Bear in mind that the `BlackHoleCookieJar` above is for example purposes only: in real code there are a few other methods you'd want to override, like `.copy()`.

Is this helpful?


Thank you very much for the explanation and the sample code. At first I thought that cookie persistence were available only for sessions. I'll go with the BlackHoleCookieJar.
Assorted docs updates

This should resolve #1390, #1399 and #1383.

```
            DOCUMENT
                          ,,
                         ';;
                          ''
            ____          ||
           ;    \         ||
            \,---'-,-,    ||
            /     (  o)   ||
          (o )__,--'-' \  ||
,,,,       ;'uuuuu''   ) ;;
\   \      \ )      ) /\//
 '--'       \'nnnnn' /  \
   \\      //'------'    \
    \\    //  \           \
     \\  //    )           )
      \\//     |           |
       \\     /            |

        ALL THE THINGS!!!!
```


:sparkles: :cake: :sparkles: :cake: :sparkles: :cake: :sparkles:
Integration of Github with Window Azure.

Please suggest me to how we integrate a Github(Source code) with window azure .


Hi there!

Sorry, but Requests' issue pages are not really the place for this kind of discussion. If you need help, I suggest trying an IRC channel or asking Stack Overflow. In the meantime, [this article](http://www.windowsazure.com/en-us/develop/net/common-tasks/publishing-with-git/) seems like a good place to start.
cookies.morsel_to_cookie morsel['expires'] can't be strtime, and morsel['max-age'] convert to expires problem repair

eg. view yahoo.com will return:

```
Set-Cookie  B=9a3jpi98r1dd8&b=3&s=fb; expires=Sun, 07-Jun-2015 16:15:36 GMT; path=/; domain=.yahoo.com
```

expires is str time, but cookielib.Cookie.expires only int
py code:

```
    str_cookie = "B=9a3jpi98r1dd8&b=3&s=fb; expires=Sun, 07-Jun-2015 16:15:36 GMT; path=/; domain=.yahoo.com"
    C = Cookie.SimpleCookie(str_cookie)
    for morsel in C.values():
        cookie = requests.cookies.morsel_to_cookie(morsel)
        print cookie
```

1.raise TypeError:
TypeError: create_cookie() got unexpected keyword arguments: ['path_specified', 'domain_specified', 'port_specified', 'domain_initial_dot']

so we should remove these param from morsel_to_cookie(morsel):create_cookie(....)

2.where  repair 1 after, will raise:
  File "/usr/lib/python2.7/cookielib.py", line 739, in **init**
    if expires is not None: expires = int(expires)
ValueError: invalid literal for int() with base 10: 'Sun, 07-Jun-2015 16:15:36 GMT'


Hi @wasw100, thanks for raising this pull request!

Unfortunately, it's not clear to me whether we want to continue supporting adding Morsels to `RequestsCookieJars`. Can I get a call on this @kennethreitz?


Hi @Lukasa , thanks for your reply.

If we handler response cookie by ourself, Morsels to RequestCookieJar is usefull.
I use these methods in other projectes.


I agree that it's useful, I just wonder how useful it is. Requests does not include plenty of useful code on the grounds that it complicates the API and increases the amount of code we need to maintain. I'm on the fence about this: I don't mind keeping it and I don't mind throwing it away. In situations like that, I'd usually throw it away.

That's why I'm interested to see what @kennethreitz thinks.


This seems like a great addition. Many thanks!


:cookie: get it? :)


This few days I can not surf the Internet. Thanks.
Can no longer override response.raw with a file-like object

`Response.raw` is stated to be a 'file-like' object ([comment here](https://github.com/kennethreitz/requests/blob/v1.2.1/requests/models.py#L474)) however a change between 1.2.0 and 1.2.1 means that a new parameter (`decode_content`) was added.

The change was made in the following commit: https://github.com/kennethreitz/requests/commit/59f916ca4a6a8e55528988cfc1bee445e71c4602

This is no longer compatible with 'file-like' objects, as it relies on the `urllib3.response` [method signature](https://github.com/kennethreitz/requests/blob/master/requests/packages/urllib3/response.py#L129).

This breaks libraries which override `response.raw`, such as the tests in `github3.py` (see this issue on github3.py: https://github.com/sigmavirus24/github3.py/issues/112)


Hi @carlio, thanks for raising this issue!

This has been raised before.  I'm increasingly bothered by it, but the keyword argument is really useful. For the moment, downstream libraries like github3.py can wrap their file-like objects in classes that accept-and-drop this keyword argument. That's unsatisfying, but in the interim is probably the way to go.


Ah, I didn't think to look through the closed issues, oops. 

If this will be the decision, it'd be good to change the comment on `raw` just to avoid confusion!

I'll go create a pull request for `github3.py`'s tests to use your suggestion.
Is this a bug?

MacOSX + Python 2.7.5 + latest requests Traceback :

```
r = session_var.post(url, data=payload, verify=False)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 377, in post
    return self.request('POST', url, data=data, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 438, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 292, in send
    timeout=timeout
  File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 423, in urlopen
    conn = self._get_conn(timeout=pool_timeout)
  File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 234, in _get_conn
    if conn and is_connection_dropped(conn):
  File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/util.py", line 256, in is_connection_dropped
    return select([sock], [], [], 0.0)[0]
TypeError: argument must be an int, or have a fileno() method.
```

Perhaps the exception raised should be a bit more easy to read? Or am I missing something obvious. 


If it is a bug, it should be raised against urllib3. The last line is in urllib3, not requests. Thanks!
fix - AttributeError: 'CaseInsensitiveDict' object has no attribute 'has_key'

fixes https://github.com/kennethreitz/requests/issues/1402


I'm -0. I don't think it should be "fixed" because we should be promoting future-proof behaviour anyway but that as it is, I'm not one for breaking what is effectively part of the API.


`has_key` should be used by no one :)


Haha, the BDFL has spoken :) :+1: 


Keep the PRs coming! :cake:

## 

Kenneth Reitz

On Jun 2, 2013, at 2:01 AM, Cal Leeming notifications@github.com wrote:

Haha, the BDFL has spoken :) [image: :+1:]

—
Reply to this email directly or view it on
GitHubhttps://github.com/kennethreitz/requests/pull/1403#issuecomment-18802730
.
AttributeError: 'CaseInsensitiveDict' object has no attribute 'has_key'

In requests <= 1.0, you could do;

```
r.headers.has_key('content-type')
```

Now this results in:

```
  File "adapters.py", line 260, in _download_url
    if r.headers.has_key('content-length'):

AttributeError: 'CaseInsensitiveDict' object has no attribute 'has_key'
```

You could do;

```
r.headers.keys().count('content-length')
```

Or

```
headers = dict(**r.headers)
```

But it would be nice to just be able to do .has_key().


Why not just `key in r.headers`?


Hmm, I didn't know you could use that syntax on dict, and it appears to be the standard now.
http://stackoverflow.com/questions/1323410/has-key-or-in

Also seems it was removed in python 3;
http://docs.python.org/3.1/whatsnew/3.0.html#builtins

In future I will be using the 'in' statement, however, this would break a lot of backwards compatibility for users upgrading to the next version of requests, and there was no deprecation warning on this.

Thoughts?


Yeah it's a pain for some people relying on it, but anyone writing future (python 3) proof code, should have been using it. Clearly I'm a bit unsympathetic but I'm not against reintroducing it because it is sort of part of the API.


No problem, either way thank you for the heads up on the 'in' statement :)
Retrieve environment proxies using standard library functions

I ran into a problem earlier today when trying to debug an issue with migrating some code from urllib2 to requests. To compare the requests being made, I fired up my trusty web debugging proxy (Charles), but the requests from requests (ahem) weren't being sent through the proxy.

After digging around in the code, I noticed that requests was only checking the OS environment variables for proxy settings, not the extra OS-specific locations handled by urllib2's ProxyHandler (the System Configuration framework on OS X and the Windows registry). Since the requests code appeared to be duplicating some functionality already in urllib, I made a quick patch to use the standard library functions instead.

It works as expected on Python 2 and 3 using OS X; since it's using urllib under the hood I assume it's safe, but I'll test on Windows/Linux when I'm having less of a nightmare day. However, I'm unsure how to write a test that covers this, since it depends on the current environment's configuration. Any advice would be appreciated!


Hi James, thanks for opening this pull request!

I'm in favour of the idea., My current worry is that `urllib.proxy_bypass()` appears to be totally undocumented. Am I going mad, or is that right?


@jam _bump_. :smile:


honestly, i'm afraid to change this behavior. it could break lots of people's stuff


I lied.


:100:
Add documentation about "non-blocking"

Please explain in the documentation how you can use the library in non-blocking mode.

If this is not supported, please state this in the documentation.

I think this search should succeed:

http://docs.python-requests.org/en/latest/search/?q=non-blocking

Thank you very much


Hi there, and thanks for the suggestion!

I think that's a good idea. The short answer is that Requests does not support non-blocking IO with its default transport adapter. You can get very close by using `Response.iter_content()` with a small chunk size, but if your network glitches that will still block. I'll add this information to the docs.


This should be resolved by #1413: let me know if it isn't. =)
`pip install requests` results in installing v0.14.2

While installing [httpie](httpie.org), pip grabs requests as a dependency, and it should be grabbing the most recent version, but instead, I end up with v0.14.2, which is incompatible with httpie. Attempting to install from a mirror has the same result. Installing from source code works without issue, however.

Have Python 2.7.5 installed via Homebrew on OS X 10.8.3


Which version of pip?


pip 1.3.1 from /usr/local/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg (python 2.7)


I cannot reproduce. Anyone else?


Nope, not me.


Are you maybe using a mirror of PyPI instead of pypi.python.org? If so, they could have fallen out of sync with PyPI and be installing the wrong thing. Also are you sure that you're installing a recent version of httpie?


As far as I know, I'm not using any mirrors. Unless I'm mistaken, you need to explicitly set `pip` to use mirrors, yes? When I opened this, I had installed httpie 0.5.1, just ran `pip install --upgrade httpie` and ended up with 0.6.0, which is current accoring to [PyPI's page for httpie](https://pypi.python.org/pypi/httpie/0.6.0).

If no one can reproduce, it's likely some kind of config error on my box; I'll poke at it some more on my own.


You could have the mirror option set in your pip rc file. With that and the fact that b.pypi.python.org is grossly broken, I still think it's possible your using a mirror without realizing it. Can you try specifying with -r where you'd like to install from? If you specify pypi.python.org or crate.io and it still doesn't work, this will be one of the first things I attack after I get internet or find a suitable place to hack. 


Not sure what path I'd use for httpie's requirements file; there doesn't seem to be one I can spot on the PyPI page for the package. However, on a whim, I went ahead & just tried `pip install --upgrade httpie` again, and now it's working like a charm.

@sigmavirus24, I suspect you're right, that I was ending up at a mirror that was out-of-date, and now I'm either pulling from PyPI directly, or that mirror or mirrors have caught up.

Gonna close this out, as it appears to have resolved itself independently. Thanks all for your suggestions/help!


The b mirror I mentioned above was shutdown and redirected to PyPI proper. My best guess is that you were hitting that.
Support HTTP pipelining


Hi there! Thanks for opening this issue.

Unfortunately, Requests is a long way from supporting HTTP pipelining. Our connections are handled by [urllib3](https://github.com/shazow/urllib3), which currently does not support HTTP pipelining. There is an open issue for it (shazow/urllib3#52), but it has not been under active development for some time.

In the event that shazow/urllib3#52 gets resolved, we might be able to go ahead and support pipelining in Requests. On balance, though, I'd have to say that we're a long way from that unless you're prepared to do the pipelining work in urllib3 yourself. It's not highly likely that someone else will do it in the near future: we've had very little demand for it, and the core devs don't miss it either. =)


@shazow and I have explored adding this several times. Every time, our conclusion is that it's simply too much work with little performance benefit.

Waiting for SPDY now :)


what about this? https://gist.github.com/coady/f9e1be438ba8551dabad
Default timeout with retries

At the moment if I specify a timeout on a request, it does not attempt to retry.

It would be nice to be able to specify a default timeout that adhered to the default retry.

Here is the workaround I used for now;

```

requests.adapters.DEFAULT_RETRIES = 3

def fetch_page(url, timeout=3):
    c = 0
    last_exc = None
    while True:
        if c >= requests.adapters.DEFAULT_RETRIES:
            raise last_exc[0], last_exc[1], last_exc[2]
        try:
            return requests.get(url, timeout=timeout)
        except requests.Timeout, e:
            c+=1
            last_exc = sys.exc_info()
            logging.info("request timeout, retrying.. (%s/%s)" % ( c, 
                requests.adapters.DEFAULT_RETRIES))
        except Exception, e:
            raise

```


The intended API for retries is to use a different transport adapter:

``` python
import requests
from requests.adapters import HTTPAdapter

s = requests.Session()
s.mount('http://', HTTPAdapter(max_retries=3))
s.mount('https://', HTTPAdapter(max_retries=3))
```

You can then set the timeout on each request and this should work.

Does this look like what you want?


Ah that makes sense.. I couldn't figure this out from the docs, so perhaps the docs could be improved.. Although I'm not too sure how it would be explained

Thanks for the quick reply!


I'll have a think about how best to document this, but I might just end up writing a blog post about it and calling it a day.

Thanks for opening the issue, let me know if you need anything else! =)
Update quickstart.rst

fix misleading info about raise_for_status method


Hi @revolunet, thanks for the fix!

Can I suggest that, instead of using the phrase "status between 400 and 600", you use the phrase "a 4XX client error or 5XX server error response"? It makes it clearer that these errors are categories, not a continuum of numbers.


thanks @Lukasa you're perfectly right. i'll close this one and issue another PR
Broken tarballs of requests-1.2.1 and requests-1.2.2 on PyPI

Tarballs of last 2 releases of requests on PyPI are broken:
https://pypi.python.org/packages/source/r/requests/requests-1.2.1.tar.gz
https://pypi.python.org/packages/source/r/requests/requests-1.2.2.tar.gz
They contain test_requests.py incompatible with Python 3 (in test_content_length_override function).
This function is absent in test_requests.py in tags in repository:
https://github.com/kennethreitz/requests/blob/v1.2.1/test_requests.py
https://github.com/kennethreitz/requests/blob/v1.2.2/test_requests.py

Difference between 1.2.2 tag and 1.2.2 tarball:

```
--- 1.2.2-tag/requests/models.py
+++ 1.2.2-tarball/requests/models.py
@@ -399,7 +399,13 @@

         self.body = body

-    def prepare_content_length(self, body):
+    def prepare_content_length(self, body, force=False):
+
+        # Skip this if it's already set, you crazy crazy person.
+        if 'Content-Length' in self.headers:
+            if not force:
+                return
+
         if hasattr(body, 'seek') and hasattr(body, 'tell'):
             body.seek(0, 2)
             self.headers['Content-Length'] = str(body.tell())
@@ -431,7 +437,7 @@
             self.__dict__.update(r.__dict__)

             # Recompute Content-Length
-            self.prepare_content_length(self.body)
+            self.prepare_content_length(self.body, force=True)

     def prepare_cookies(self, cookies):
         """Prepares the given HTTP cookie data."""
--- 1.2.2-tag/test_requests.py
+++ 1.2.2-tarball/test_requests.py
@@ -702,5 +702,10 @@
         self.assertEqual(frozenset(cid), keyset)


+    def test_content_length_override(self):
+        r = requests.Request('POST', httpbin('post'), data='ohai', headers={'f-length': 1})
+        print r.prepare().headers
+
+
 if __name__ == '__main__':
     unittest.main()
```

I suggest to create correct tarballs and re-upload them to PyPI


@Arfrever Unfortunately we rushed the last two releases out due to a pair of nasty bugs, one affecting everyone on 3.3.2 and one affecting anyone using OAuth (both bad). Presumably @kennethreitz was midway through working on something when he published the last two releases, and my badgering him to release fast caused him to lose track of it.

However, the library itself functions fine, only the tests fail to run. Is it sufficient for you to simply change the tests locally?


Looks passing to me?

https://travis-ci.org/kennethreitz/requests


The Github repository is fine: these changes are in your local repository. =)


oh damn it


this is why i want a git-based pypi :)


v1.2.3 released, which fixes this.
Unicode error when installing with PowerShell on Windows

With Python 3.3 on Windows 8 I get the follow error when installing Requests:

```
Downloading/unpacking requests
  Running setup.py egg_info for package requests

Installing collected packages: requests
  Running setup.py install for requests

      File "C:\Users\***\Documents\***\Lib\site-packages\requests\packages\urllib3\contrib\ntlmpool.py", line 38
        """
         ^
    SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 130-132: truncated \uXXXX escape

Successfully installed requests
Cleaning up...
```


Thanks for raising this issue!

Happily, this doesn't actually immediately affect your install of Requests. You won't be able to use anything that involves NTLM authentication, but hopefully that won't immediately be a problem for you.

As for the actual issue, the bug is in shazow/urllib3, so I have to ask you to open an issue over there. The exactly issue is that `urllib3/contrib/ntlmpool.py`, there's a docstring that includes the string "DOMAIN\username". Python3 thinks that's a unicode code point literal, but it isn't one it recognises. You can probably just open a Pull Request that changes it and be fine.


It's actually fixed over there. We just need to pull it in.
default adapters insist on exact matching against lower-case 'http' and 'https'

The URL specification (RFC 2396) recommends that resilient platforms support upper-case letters in scheme specifiers.

At the moment `requests.sessions.Session.get_adapter(self,url)` and `requests.adapters.HTTPAdapter.cert_verify(self,conn,url,verify,cert)` do a `url.startswith()` that match against 'http' and/or 'https' (lower case only), not admitting for the possibility of an upper-case or mixed-case scheme in the url being examined.

In these cases, can we do a url.lower.startswith() instead of simply url.startswith() to allow for mixed case schemes and still be able to effectively match against 'http' and 'https'?

Created pull request #1385 for this issue.


Closing this to centralize the discussion on #1385.
Cope with mixed-case URL schemes (like 'HTTP') by lower-ifying 'url' string before calling startswith() on it

The URL specification (RFC 2396) recommends that resilient platforms support upper-case letters in scheme specifiers.

At the moment `requests.sessions.Session.get_adapter(self,url)` and `requests.adapters.HTTPAdapter.cert_verify(self,conn,url,verify,cert)` do a `url.startswith()` that match against 'http' and/or 'https' (lower case only), not admitting for the possibility of an upper-case or mixed-case scheme in the url being examined.

In these cases, can we do a url.lower.startswith() instead of simply url.startswith() to allow for mixed case schemes and still be able to effectively match against 'http' and 'https'?


You might be more familiar with ruby, but your line should read: `url.lower().startswith(...)` for it to be valid Python. :-P

On a separate note, please add a test to make sure this doesn't regress.

Once you've fixed this and added a test, :+1:.


You're right -- apologies for requesting a pull of broken code.


@ViktorHaag You don't need to close this. =) You can push more commits to the same branch and they all get added to the same Pull Request.


Ah; again, apologies; I did not know this.


@ViktorHaag I like that you're being polite, but you don't need to apologise for not knowing things. =) We were all new to this at some point.


OK -- I've added a test to attempt verification of these changes, and also added another change to adapters.py that lower-ifies urls passed down into urllib3 so that its poolmanager can handle using the scheme as a dict key without hiccuping over mixed-case schemes.


Cool, looks good to me. =) :+1:
sphinx template issue

There is message in documentation's sidebar: "You are currently looking at the documentation of the development release." It is the same for all documentation versions.
For example, version 1.0.3 (http://docs.python-requests.org/en/v1.0.3/) displays the same message about development release.


Thanks for raising this issue!

The text is hardcoded into the template in `docs/_templates/sidebarintro.html`. Feel free to open a Pull Request that removes it. (If you don't want to, I'll do it at some stage, just not right now. =) )


This should be resolved by #1413. Thanks so much for pointing it out!
HTTPConnection strict deprecated since 3.2

When using requests [as shown here](https://gist.github.com/tasuk/5639256), I get _"DeprecationWarning: the 'strict' argument isn't supported anymore"_ because of [this bit of code](https://github.com/kennethreitz/requests/blob/v1.2.2/requests/packages/urllib3/connectionpool.py#L203-L205).

Please also see:
http://docs.python.org/3.2/library/http.client.html#http.client.HTTPConnection
http://docs.python.org/3.3/library/http.client.html#http.client.HTTPConnection
http://docs.python.org/3.4/library/http.client.html#http.client.HTTPConnection


This is the wrong place to send this pull request. Please raise it over at shazow/urllib3.


:open_mouth: 


This has been fixed yet on [shazow/urllib3 - issue #342](https://github.com/shazow/urllib3/issues/342) three weeks ago, but the warning is visible.

Please upgrade your copy of urllib3 to the latest version 1.8


@hynekcer we do not upgrade urllib3 until we are about to cut a release.
Don't force chunked transfer on 0-length file-like object.

Only switch to chunked if we don't know the length of a file like object. This fixes the case of trying to transfer a 0-length file - chunked upload was being forced. Services like S3 that disallow chunked upload will fail.


This looks good to me. +1


This looks fine. But for future reference you could also have done `if length is not False` because `True`, `False` and `None` are all alike in that you should compare them using is if you're not just looking for a truthy value.

:+1: though


:cake:
Python 3.3.2: 'MockRequest' object has no attribute 'origin_req_host'

I recently upgraded to Python 3.3.2 (from Arch Linux packages).  With it, a script of mine broke.  The script worked fine with 3.3.1 and still works fine with 2.7.5.  Full traceback (`s = requests.Session(); starturl = 'http://example.com'` (okay, not exactly that, but you don’t need nor want to know the actual URL))

``` pytb
  File "req.py", line 43, in <module>
    orig = s.get(starturl).text
  File "/usr/lib/python3.3/site-packages/requests/sessions.py", line 369, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3.3/site-packages/requests/sessions.py", line 354, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.3/site-packages/requests/sessions.py", line 460, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.3/site-packages/requests/adapters.py", line 256, in send
    r = self.build_response(request, resp)
  File "/usr/lib/python3.3/site-packages/requests/adapters.py", line 125, in build_response
    extract_cookies_to_jar(response.cookies, req, resp)
  File "/usr/lib/python3.3/site-packages/requests/cookies.py", line 105, in extract_cookies_to_jar
    jar.extract_cookies(res, req)
  File "/usr/lib/python3.3/http/cookiejar.py", line 1647, in extract_cookies
    if self._policy.set_ok(cookie, request):
  File "/usr/lib/python3.3/http/cookiejar.py", line 931, in set_ok
    if not fn(cookie, request):
  File "/usr/lib/python3.3/http/cookiejar.py", line 952, in set_ok_verifiability
    if request.unverifiable and is_third_party(request):
  File "/usr/lib/python3.3/http/cookiejar.py", line 707, in is_third_party
    if not domain_match(req_host, reach(request.origin_req_host)):
AttributeError: 'MockRequest' object has no attribute 'origin_req_host'
```


Yes, we spotted this and rushed out a Requests point release to fix it. Please upgrade Requests to 1.2.1 or later. Thanks for reporting this! :smile_cat:
Avoid two requests on a TypeError

Rather than test behavior via try: except, inspect python version
number.  

NOTE: This may not be a huge issue in real life. 

I noticed it using wsgi-intercept where an intercepted app raised a
TypeError (which got bubbled up the stack to this catch). If the purpose of this code is as the comments say, checking sys.version_info is probably a more efficient and predictable.  But maybe more is happening here than I've investigated?

anyway, this is a friendly testless PR, feel free to disregard ;)


I prefer the old method, but thanks!


Just a side-note, two requests are not made in this code at all. The request has been placed already, this tries to read the response one way and if that throws an exception, reads it the default way. It really isn't necessary to make this change. :)


that's not the behavior I was seeing on the server side.  will put together
a test.

On Wed, May 22, 2013 at 4:05 PM, Ian Cordasco notifications@github.comwrote:

> Just a side-note, two requests are not made in this code at all. The
> request has been placed already, this tries to read the response one way
> and if that throws an exception, reads it the default way. It really isn't
> necessary to make this change. :)
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/kennethreitz/requests/pull/1378#issuecomment-18307870
> .

## 

<=>
david "whit" morriss

 "If you don't know where you are,
  you don't know anything at all"

  Dr. Edgar Spencer, Ph.D., 1995
test_unicode_multipart_post_fieldnames() fails sometimes

```
self = <test_requests.RequestsTestCase testMethod=test_unicode_multipart_post_fieldnames>

    def test_unicode_multipart_post_fieldnames(self):
        r = requests.Request(method='POST',
                             url=httpbin('post'),
                             data={'stuff'.encode('utf-8'): 'elixr'},
                             files={'file': ('test_requests.py',
                                             open(__file__, 'rb'))})
        prep = r.prepare()
        self.assertTrue(b'name="stuff"' in prep.body)
>       self.assertFalse(b'name="b\'stuff\'"' in prep.body)
E       AssertionError: True is not false

test_requests.py:356: AssertionError
```


Yeah, that does look wrong.


In my environment, this is reproducible when `__file__` is test_requests.pyc.
Replacing `__file__` with `os.path.splitext(__file__)[0] + '.py'` works for me.


@papeye: Oh yeah, my fix is totally wrong. Do you want to open a Pull Request with your fix in it? =)


@Lukasa OK, I will.  Sorry for misleading you by my comment.


@papaeye: Not at all! I mislead myself by being stupid. =D
Only pass unicode multipart fieldnames to urllib3.

`filepost.encode_multipart_formdata` says all fieldnames should be unicode and currently that is not the case - https://github.com/shazow/urllib3/blob/master/urllib3/filepost.py#L55 - since requests encodes to bytes here - https://github.com/kennethreitz/requests/blob/master/requests/models.py#L108.

Discovered after plenty of fun in https://github.com/requests/requests-oauthlib/pull/43. This should fix https://github.com/kennethreitz/requests/issues/1371.

Tested in 2.6, 2.7 and 3.3.

cc @Lukasa, @michaelhelmick, @sigmavirus24


Excellent. :+1:
Python 3 encoded form parameters in post request with files

I believe due to pull request #1279 the field parameters (along with values) are encoded to bytes in python3 which makes parameters in post requests that have use encode_multipart_formdata from urllib3/filepost.py show up as <code>"b'stuff'"</code> instead of <code>"stuff"</code> as reported by httpbin.org/post. I've written a quick gist which shows what I'm talking about. It's a modified version of one of the tests in test_requests.py, https://gist.github.com/trblae/5617488.

I've traced the cause of this issue to the dependent package in urllib3/filepost.py. As you can see at line 80 form-data is built via a string format with the field name. It seems in python 3 this produces the bytes representation of the fields. This only happens in specific cases with files or more generally when encode_multipart_formdata is used.

I'm not exactly familiar enough with the differences between python2 and python3 strings to really know how to tackle this issue in order to solve it for both python2 and python3. This is kind of my first rodeo so please let me know if I'm doing something wrong or how I can help.


Firstly, thanks for opening this issue! This exact bug was found about 8 hours before you opened this, in requests/requests-oauthlib#43. The problem is that all field and file names passed to `encode_multipart_formdata` should be Unicode. As you identified, #1279 caused all of these field and file names to be bytes, not unicode.

@ib-lundgren has proposed a fix across in requests-oauthlib, which involves changing L108 as follows:

```
- (field.encode('utf-8') if isinstance(field, str) else field,
+ (field.decode('utf-8') if isinstance(field, bytes) else field,
```

If you would like to fix this yourself, you can write a test and apply the fix, then send a Pull Request. Alternatively,@ib-lundgren will probably do that at some point soon. =)


Looks like pull request #1373 makes it work. Thanks.
Cannot fetch "http://www.google.com/"?

```
import requests
r = requests.get('http://www.google.com/')

Traceback (most recent call last):
  File "<pyshell#6>", line 1, in <module>
    r = requests.get('http://www.google.com/')
  File "C:\Python33\lib\site-packages\requests\api.py", line 55, in get
    return request('get', url, **kwargs)
  File "C:\Python33\lib\site-packages\requests\api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python33\lib\site-packages\requests\sessions.py", line 354, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python33\lib\site-packages\requests\sessions.py", line 460, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python33\lib\site-packages\requests\adapters.py", line 256, in send
    r = self.build_response(request, resp)
  File "C:\Python33\lib\site-packages\requests\adapters.py", line 125, in build_response
    extract_cookies_to_jar(response.cookies, req, resp)
  File "C:\Python33\lib\site-packages\requests\cookies.py", line 105, in extract_cookies_to_jar
    jar.extract_cookies(res, req)
  File "C:\Python33\lib\http\cookiejar.py", line 1647, in extract_cookies
    if self._policy.set_ok(cookie, request):
  File "C:\Python33\lib\http\cookiejar.py", line 931, in set_ok
    if not fn(cookie, request):
  File "C:\Python33\lib\http\cookiejar.py", line 952, in set_ok_verifiability
    if request.unverifiable and is_third_party(request):
  File "C:\Python33\lib\http\cookiejar.py", line 707, in is_third_party
    if not domain_match(req_host, reach(request.origin_req_host)):
AttributeError: 'MockRequest' object has no attribute 'origin_req_host'
```

Using official CPython 3.3.2 binary on Windows 7 x64.


@jellyflower Thanks for reporting this bug. It's actually one we already knew about: see requests/requests-oauthlib#43.

I wonder if 3.3.2 has introduced a bug in Requests. @sigmavirus24, I'm leaving this for you. =)


Yeah was just sending the pr


Yup, I can repro this in OS X on 3.3.2 as well.
Content-Type header in requests is duplicated when POSTing bytes

Using python 3.3 on Mac OS:

```
import requests
requests.post("http://localhost:8080/nonexistent",
              headers = {'Content-Type': 'text/plain;charset=UTF-8'},
              data=b'foo')
```

results in the following request (captured with tcpdump):

```
POST /nonexistent HTTP/1.1
Host: localhost:8080
Accept-Encoding: identity
Accept-Encoding: gzip, deflate, compress
Content-Type: application/x-www-form-urlencoded
Content-Type: text/plain;charset=UTF-8
Accept: */*
Content-Length: 3
User-Agent: python-requests/1.2.0 CPython/3.3.1 Darwin/12.3.0

foo
```

Note that the specified content-type header is present, but another "Content-Type: application/x-www-form-urlencoded" is added. As far as I can tell, the extra Content-Type header is added at the end of prepare_body in models.py due to  `'content-type' in self.headers` not recognizing the existing content-type header in the `self.headers` dict (the key is, at that time, `b'Content-type`, and it seems it's this str <-> bytes comparison that fails).

It is possible to work around this by wrapping the bytes in a BytesIO object (and setting it's len property), but it still seems like a bug?


It is definitely a bug, and we have a PR open that will fix it: #1338. In the meantime, the easiest way to fix it is to make the key of your user-specified Content Type a bytestring: e.g. `b'Content-Type'`.


Thank you for the quick response! (I guess it's ok to close this issue? )
Uploading Files

I'm trying to upload files to server, let's take this simple page as example:
http://fiberdk.horus.feralhosting.com/

this page will get .torrent file and it will fix it if it's broke, I would do something like this:

url = 'http://fiberdk.horus.feralhosting.com/'
files = {'torrent': open('torrent.torrent', 'rb')}
r = requests.post(url, files=files)

but this didn't work for me, and all i can get from googling is the same results.


In that sense did it not work for you? It seems to work just fine for me. Can you tell me how it failed to work?


well if I do "r.content" it would be the same as [ r = requests.get(url) ] 's content, so nothing is happening
- I tested it on other pages without any luck, i'm giving this page as example, the other pages i'm working with should return some text if you upload something, and if i check " r.text " I won't get that text like "You Uploaded something" or "Your upload didn't work", so basically i'm not posting anything :(


Can you provide requests.**version** for me?


'0.14.2'


Hmm. Are you bound to 0.14.2, or are you able to upgrade? We haven't really supported 0.14.2 for a while...


W00t! I just realised that i'm using this too old version, i'll upgrade and test again. 


well, i'm now using version 1.2.0 and still facing the same problem here's my test:

In [1]: import requests

In [2]: files = {'torrent': open('torrent.torrent', 'rb')}

In [3]: url = 'http://fiberdk.horus.feralhosting.com'

In [4]: post = requests.post(url, files)

In [5]: get = requests.get(url)

In [6]: post.content == get.content
Out[6]: True


In line 4, can you change it to read:

``` python
post = requests.post(url, files=files)
```

and try again?


YESSS, I was missing (files=files),  my bad :-1: 

I really appreciate your help Lukasa :+1: 


Always happy to help. Thanks for raising the issue, let me know if you encounter any more problems!
Make HTTPResponse support `io`

Right now, the following does not work:

```
>>> r = requests.get('http://www.google.com',stream=True)
>>> io.BufferedReader(r.raw).read()
AttributeError: 'HTTPResponse' object has no attribute 'readable'
```

That is, the `requests.packages.urllib3.response.HTTPResponse` object does not respect the `io` API.  The following _does_ work, however:

```
>>> r = requests.get('http://www.google.com',stream=True)
>>> r.raw.readable=lambda:True
>>> r.raw.closed=False
>>> io.BufferedReader(r.raw).read()
```

Suggesting that the main thing that is neeeded is a `readable` method and a `closed` attribute.  (I think `flush` and `close` are probably also necessary, and maybe also `writable` and `seekable`).

Is there a reason this is not possible, or would it be fine to just add the relevant methods to `HTTPResponse`?


I suggest you open this on shazow/urllib3. We do not make any changes to urllib3, we simply vendor the excellent package to prevent users having to download additional dependencies.


Ahh, my mistake - sorry for the noise!


No worries!


@sigmavirus24 - this has now been added in shazow/urllib3#187.  Is there a regular "timetable" for when you pull in a new `urllib3`?  Or is it just whenever they have a release?


@eteq: It's mostly 'whenever'. I'll bear in mind that this is a high priority for you though, and try to bug Kenneth to include an update in our next release. =)


Ok, great, thanks!
Fix for #1362

- Replaced usage of `urlparse` in `PreparedRequest.prepare_url` with `urllib3` equivalent.
- Modified IDNA encoding section so that it only encodes the host portion of the URL


Can you squash these commits? The first (dd41e22) is unnecessary and a bit misleading. The only relevant commits here are 38d4ad6 and 25c1f75. Otherwise :+1:.

If you need help with squashing, let me know.


@sigmavirus24 does 6e76ab7 do the job?  I'm relatively new to git so it wouldn't surprise me if I mucked something up.


@dave-shawley it most certainly does! Thank you kind sir!

:+1: for @kennethreitz when he comes around these parts again


Very nice! :+1:


:cake:


Congrats @dave-shawley :cake:
prepare_url fails for long username and passwords

`PreparedRequest.prepare_url` incorrectly applies IDNA encoding to the URL's `netloc`.  It should only be encoding the hostname portion of the URL.  The `netloc` is described in [RFC2396](http://tools.ietf.org/html/rfc2396#section-3.2.2) as:

```
<userinfo>@<host>:<port>
```

IDNA should only be applied to the `host` portion of the URL.

I encountered this when integrating with an API that uses UUIDs as the username and password.  This results in a `netloc` that is far larger than a 64 character DNS label.  The result is the following stack trace:

``` python
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/unittest.py", line 279, in run
    testMethod()
  File "/Users/daves/Source/requests/test_requests.py", line 528, in test_uuid_user_and_password_components
    prepped = r.prepare()
  File "/Users/daves/Source/requests/requests/models.py", line 221, in prepare
    p.prepare_url(self.url, self.params)
  File "/Users/daves/Source/requests/requests/models.py", line 298, in prepare_url
    raise InvalidURL('URL has an invalid label.')
InvalidURL: URL has an invalid label.
```

The fix is pretty simple - split the `netloc` into its components, IDNA encode the `host` portion, and recombine.


Because you're awesome you've already opened a Pull Request for this (#1363). =D I'm going to close the issue to centralise the discussion there.

Thanks so much for pointing this out! =D