Raibek/requests-github-issues huggingface dataset:
Make alternate json libraries a hook to replace builtin json.

Currently, in the `_compat.py` module, if you have simplejson installed, it will use that over the standard built-in json.

This works fine in Python 2, since simplejson is typically faster than the built in json package, and it is more 'correct' in handling JSON.

However, in Python 3, simplejson is slower than the built in json package especially for smaller json payloads, which is probably a typical use case with the requests library. Even though simplejson handles various JSON strings better than the built in JSON, those cases are not typical.

I would like to see a hook that allows a user to opt to use a alternate JSON library so that a user can specify if they want to use rapidjson, ujson, simpleson for the `response.json()` decoding.


The 3.0 release of Requests will remove the use of simplejson in favour of the stdlib module. Until that time, however, we cannot safely remove the option to use simplejson, as it would potentially represent a subtle backward-incompatible dependency change.

I don't really see any reason to allow more general hooking of JSON libraries: ultimately, additional JSON features are not really a thing I think we need to support.

Thanks for your report!

What's the breaking change?

@SharpEdgeMarshall replacing the dependency on simplejson in https://github.com/psf/requests/blob/bbc3d43522bcb1363050557655508d66bfebcfcf/requests/compat.py#L28

I still think you should be able to replace the JSON library used.
Unable to trigger a Jenkins build remotely with a file parameter using python requests

I was trying to trigger a Jenkins build remotely with a file parameter using python requests

## Expected Result

I was expecting the remote job to be triggered with the contents of file.log saved in ${WORKSPACE}/files/abc.zip

## Actual Result
This code triggers a build. 

## What happened instead.
But the file /some/file/path/file.log is not posted to the Jenkins job.
The contents are not available in ${WORKSPACE}/files/abc.zip


## Reproduction Steps

```
python
import requests
url = 'myurl/buildWithParameters'
data = {'Jenkins-Crumb': '8f4d0320c0982763098571789b2b65ec', 'token': 'mytoken'}
filepath='/some/file/path/file.log'
files = {'file': ('files/abc.zip', open(filepath, 'rb')) }
r = requests.post(url, data=data, files=files)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.3"
  },
  "cryptography": {
    "version": "1.9"
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.10"
  },
  "platform": {
    "release": "16.6.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "1010006f",
    "version": "17.0.0"
  },
  "requests": {
    "version": "2.17.3"
  },
  "system_ssl": {
    "version": "9081df"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": true
}

```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Do you have some indication of what the format of a successful request should be? It's hard to debug this without knowing that success looks like. 

@Lukasa No sorry, I don't have format of a successful request. 

In that case we can't really help you: if we don't know what you're trying to produce, we can't tell you how to use Requests to do it. Sorry!
figure out sticker delivery

- https://www.blackbox.cool
- https://maillift.com/

sir please describe the issue, i would like to work on it..

@techytushar, thanks for the interest in helping with Requests! This was a TODO for Kenneth to determine the best way for distributing Requests' stickers. I believe he settled on a solution [here](https://gumroad.com/l/requests-stickers) at gumroad.

@kennethreitz, should we close this close out at this point, or did you have more outstanding tasks?

not figured out yet, just ordering is solved. planning on having my wife stuff envelopes at this point, but we’ll see. 

FWIW, I used to be a partner in a company that sold limited art editions online.  The big game-changer for us in fulfillment was getting a thermal label printer.  The thermal labels (or paper rolls) cost a bit more than inkjet/laser ones, but we could very quickly print them on-demand, one at a time.  All the layouts/etc were just pdfs i generated in python from plaintext documents.  We bought the first one new, but bought a second used off ebay for super cheap.
can't POST

Trying to use python requests to http post.
r = requests.post(address, data=payload)
r.url only shows the url, not the payload.
My php page is returning an error, but I can't see the full url passed to it.
How do I inspect the WHOLE url posted to the web server to debug it?

Whereas
r = requests.get(...)
r.url returns the entire url including the payload.

TIA.

In requests, `data` *always* adds data to the request body, and `params` always adds data to the request URL. If you want data in the URL you *must* use `params`, and if you want it in the body you *must* use `data`. This is unrelated to the request verb you're using.

Many thanks - resolved.
Content-Length header issue

I have used remove property for the header property Content-Length as I dont want to have the response from the backend truncated. But curious enough when I call the API, sometimes the content-length property is removed and sometimes it is not. Why is this behavior observed. I included the remove property line after the call is made to backend and we get the response.

Can you show me some sample code and what you expect to see, vs what you actually see?

This hasn't been updated in well over a month. Without further detail, there's nothing we can do. I'm closing this for now but if you have the extra detail that can help us help you, then we'll reopen it.
requests 2.18.0  installation with easy-install 

Summary.
For legacy (easy-install / buildout) installations the egg and / or source for requests 2.18.0 is not being pick up. Not sure if this an issue with pypi since the source is listed

e.g.

easy_install-2.7 -U requests==2.18.0
Searching for requests==2.18.0
Reading https://pypi.python.org/simple/requests/
No local packages or working download links found for requests==2.18.0
error: Could not find suitable distribution for Requirement.parse('requests==2.18.0')

Thanks

Carlos


This seems like it's an issue with PyPI caching. @dstufft?

pip works fine

This isn't an issue with Requests. The source distribution is on PyPI: https://pypi.org/project/requests/2.18.0/#files

I genuinely think this is an issue with setuptools and this hasn't ben updated in over a month. I'm closing this.
Making requests to URLs with username and password does not respect no_proxy evironment settings

## Summary

When determining whether proxies should be bypassed or not, netloc is considered, however it contains the username and password separated by a colon, thus proxy settings cannot be applied, as the either username or host[:port] is checked among the no_proxy hosts.

## Expected Result

Proxy settings are respected

## Actual Result

Following errors are thrown:
```
OSError: Tunnel connection failed: 504 Gateway Timeout
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=8080): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 504 Gateway Timeout',)))

```
## Reproduction Steps

```python
import requests
import os

os.environ['https_proxy'] = #insert your proxy here
os.environ['no_proxy'] = 'localhost,127.0.0.1'

requests.get('https://username:password@localhost:8080/')
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.3"
  },
  "cryptography": {
    "version": ""
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.0"
  },
  "platform": {
    "release": "3.16.0-4-amd64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.17.3"
  },
  "system_ssl": {
    "version": "1000114f"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": false
}
```

## Workaround
proxies kwarg can be used to bypass this issue, e.g. by setting {'https': None}. Should be used with caution.

### Problem in my opinion:
should_bypass_proxies(url, no_proxy) function in utils.py:

network location is used, which matches the following pattern: [user[:password]@]host[:port]

- ip is determined based on ':' position, which is not valid in this case, as ip will be the username: `ip = netloc.split(':')[0]`
- when checking if proxy should be applied, two things are checked: `if netloc.endswith(host) or netloc.split(':')[0].endswith(host):`, neither of which will be true, as host may not contain port, and username surely does not end with the host.

I propose to use `domain = urlparse(url).hostname` instead of `netloc = urlparse(url).netloc` when checking against the no_proxy environment variable.

Yup, this seems incorrect to me. I'd welcome a patch to resolve this problem.
with requests.get(url, stream=True) as r AttributeError('__exit__',)

I read the python-requests doc,[http://docs.python-requests.org/en/master/user/advanced/#body-content-workflow](url),I run the example`with requests.get('http://httpbin.org/get', stream=True) as r:
    pass`,I get the exception 'AttributeError('__exit__',)',is the example left closing clause?

Sorry, no: that feature is going to be released today.
Multiple values in a single param as list

I don't know if this has already been discussed earlier.

### What happens now

`GET "https://graph.facebook.com/v2.6/USER_ID?fields=first_name,last_name,gender&access_token=PAGE_ACCESS_TOKEN"`

is done with the help using requests by

`r = requests.get('https://graph.facebook.com/v2.6/' + USER_ID, params={
                'fields': 'first_name,last_name,gender',
                'access_token': PAGE_ACCESS_TOKEN )})`

### Preferred way of doing it

`r = requests.get('https://graph.facebook.com/v2.6/' + USER_ID, params={
                'fields': [ 'first_name', 'last_name', 'gender'],
                'access_token': PAGE_ACCESS_TOKEN )})`

I feel this will be a more pythonic way to do it if that is even a word. When I was working on it I found the existing solution not so user friendly. I would love to take this up and contribute.

Thanks for your enthusiasm! Unfortunately there is no standard for handling a list of values, and multiple possible ways to represent them. For this reason Requests largely remains neutral on them: if you pass us a list, we'll just encode them all separately. Given that we have a pre-existing behaviour for this syntax, I don't think we can accept this kind of change.
how to post multi same name k-v to website?

like this:

![image](https://user-images.githubusercontent.com/15062548/27015666-cf744fca-4f46-11e7-8068-a9fc2c0fa917.png)

this post data have 3 key name 'tags[]' and value is not same, how to use requests post data like this?

find answer in https://stackoverflow.com/questions/26444557/how-to-submit-multiple-files-with-the-same-post-name-with-requests

```
data = {'raw','this is a test', 'title':'test test test', 'tags[]': ['test', 'submit', 'tool']}
requests.post('example.com', data)
```
wish add it to docs
Cannot find reference 'urllib3' in packages.py

**Summary**.

Can not find urllib3 when using latest `PyCharm` and `requests` from versions 2.17.1 to 2.17.3

**What you expected.**

Expected `urllib3` to be found on `packages.py`

**What happened instead.**

![image](https://user-images.githubusercontent.com/26851112/26999613-a60370ce-4d63-11e7-8e2a-9f5261b7e667.png)

**Steps to reproduce**

+ Open (or create) any project in PyCharm
+ You can use the systems Python or a virtualenv
+ Install requests 2.17.3 (by using pip on terminal or by going to project interpreter settings)
+ Try to import urllib3
  + `from requests.packages.urllib3.exceptions import InsecureRequestWarning`

Maybe it has something to do with db076c098e962b5096f6b0c77d69b80396228464?

**System Information**

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "implementation": {
    "name": "CPython",
    "version": "3.5.3"
  },
  "platform": {
    "release": "4.4.0-79-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.17.3"
  },
  "system_ssl": {
    "version": "1000207f"
  },
  "urllib3": {
    "version": "1.21.1"
  },
  "using_pyopenssl": false
}
```



Hrm. So PyCharm is wrong here: this path does exist. But it's added dynamically, which may cause PyCharm some problems. I recommend opening a bug report with PyCharm. 

Copy the urllib3 folder into the package. because it doesn't install it dependency itself.

try installing urllib3 and then.
from urllib3.exceptions import InsecureRequestWarning
http response is not decoded when encoding is missing from the response header, even if decode_unicode is set

Platform: Win7 Pro SP1
Python: 3.6.1 64 bit
package:
requests==2.17.3
requests-file==1.4.2
requests-ftp==0.3.1
urllib3==1.21.1

When I try to request [http://www.google.com/finance/historical?q=NASDAQ%3AAAPL&startdate=Jun+1%2C+2017&enddate=Jun+2%2C+2017&output=csv](http://www.google.com/finance/historical?q=NASDAQ%3AAAPL&startdate=Jun+1%2C+2017&enddate=Jun+2%2C+2017&output=csv), the http response has no encoding info. However, the response is utf-8 encoded with utf-8 BOM.

Even if I set `decode_unicode=True` in `iter_content`, it does not try to decode the content and simply gives the raw bytes.

I wonder if this is a designed feature or could you add a feature to peek the few bytes in the beginning and try to guess the encoding from BOM when `decode_unicode` is set and the encoding is not available?

Thank you very much!

This behaviour of `iter_content` is unfortunately well-known: see #3574 for a PR that fixed this issue as reported in #3359 and #3481.  Unfortunately, to resolve it we have to turn this into an error condition, which is a breaking change and so this change is not making it into the v2 series: it'll land in v3.

To work around the problem, you can set the encoding yourself via `response.encoding`:

```python
response.encoding = 'utf-8'
```

Sorry we can't do more to help!
Urllib3 ConnectionError exception passing through requests API

I am using Requests Version: 2.11.1 catching all request exceptions like this:

except requests.exceptions.RequestException as e:
          logger.debug("%s", str(e))

However, this does not catch a NewConnectionError thrown by requests.packages.urllib3.connection.HTTPConnection

The full error statement is: 

ConnectionError: HTTPConnectionPool(host='REDACTED', port=27001): Max retries exceeded with url: /REDACTED (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x6bf5a50>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',))

Based off [https://github.com/requests/requests/issues/1572](https://github.com/requests/requests/issues/1572), it appears that all urllib3 exceptions should be caught by requests.exceptions.RequestException and that this falls into the category of an exception leak.

Full trace: 

Traceback (most recent call last):
 ...
 File "REDACTED/app/process.py", line 399, in REDACTED
   r = requests.post(url, data=json.dumps(payload), headers=headers)
 File "/usr/lib/python2.7/site-packages/requests/api.py", line 110, in post
   return request('post', url, data=data, json=json, **kwargs)
 File "/usr/lib/python2.7/site-packages/requests/api.py", line 56, in request
   return session.request(method=method, url=url, **kwargs)
 File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
   resp = self.send(prep, **send_kwargs)
 File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 596, in send
   r = adapter.send(request, **kwargs)
 File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 487, in send
   raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='REDACTED', port=27000): Max retries exceeded with url: REDACTED (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x6bf5a50>: Failed to establish a new connection: [Errno 111] ECONNREFUSED',))



Hey @aJetHorn, it looks like this is a `ConnectionError` wrapping a `NewConnectionError` in your traceback. The `ConnectionError` inherits from `RequestsException`, so it should be caught.  Do you have an example domain you're experiencing this on?

This is the repro I used to test this:

```python
import requests

try:
    requests.get('https://not_a_real_domain.com')
except requests.exceptions.RequestException as e:
    print('exception caught', e)
```

It is a slightly different underlying error message, but the same type of Exception object.

@nateprewitt I just realized that the environment where I was running this had not been updated to handle all exceptions (the try catch wasn't what I mistakenly stated above). Sorry about that.

No worries, glad you got it sorted 😊
Revert "Convert version compatibility checks to warnings"

Reverts requests/requests#4144

continuation of https://github.com/requests/requests/pull/4144#issuecomment-307426091

restarting for windows

So we definitely don't want a complete revert of this. I do think we we want a selective revert though. We absolutely want to be more resilient to versions we can't parse. I don't think we want to emit warnings though.

windows passed!

Alright, I'd like to wrap this up so these commits aren't stuck in limbo. @sigmavirus24, my take away from the discussion is you'd still like to see exceptions over the warnings but allow for more resilient checks. What do you see happening when we can't parse a dependency? Is that still an exception with the unparsed version included in the message? How would we allow something like `python-telegram-bot` to work if they force a vendored copy of urllib3 onto the path?

You also said you'd rather not use `assert` statements to the `python -O` concerns. I'm assuming you'd rather see this as a conditional check that raises the exception?

If that all sounds correct, and @Lukasa/@kennethreitz are on board, I'll put the follow-up together.

> How would we allow something like python-telegram-bot to work if they force a vendored copy of urllib3 onto the path?

Briefly responding to one question here: The point is to not allow that. I have to wonder if the exception shouldn't also include the location to the module so people can see "Hey, telegram bot is doing something real stupid and it's breaking requests."
No method to retrieve cookie object by name

I would like to be able to get a cookie object out of the cookie jar by name and inspect its attributes (e.g. domain, expires, secure flag). My use case is a test script that ensures my webserver is configuring cookies properly.

I expect to have some kind of function like `get` that returns the whole cookie object, not just the value, but no function seems to exist.

## Reproduction Steps

```python
import requests
response = requests.get('http://google.com')
response.cookies.get('NID')  # Only returns cookie value
for c in response.cookies:
    print(c)  # c is entire cookie object
```

If this is a desired feature, I'd be happy to code it up myself. My suggestion is a simple `get_cookie` function, but am open to other ideas.

You're right, this is not a feature that exists currently. I don't think we want it though: I'm reluctant to expand `RequestsCookieJar` too far, especially when we have some pretty mixed (ie. negative) feelings about cookielib. My inclination is that the best thing to do is for you to write a helper function that iterates over the cookiejar and does the search by name (this is how `RequestsCookieJar.get` works).

I agree, mostly because there's a possible future were were we remove Cookielib entirely from the equation, and want to keep it as abstract as possible. 

Thanks for the suggestion, though, @lordjabez! ✨🍰✨

A consideration for the future, however you decide to manage cookies, is the ability to inspect all cookie attributes, not just the value. This is a common use case for me writing automated tests to ensure web servers are doing the right things with their session cookies.

And thank _you_ for a fantastic library.
Windows Jenkins CI failing on master

As spotted in #4136 and elsewhere, the Windows Jenkins CI job is failing on master:
http://jenkins.kennethreitz.org:8080/job/requests/job/requests-windows/buildTimeTrend

...with:
```
j:\jenkins\workspace\requests\requests-windows>.env\Scripts\pip install -r requirements.txt 
Obtaining file:///J:/jenkins/workspace/requests/requests-windows (from -r requirements.txt (line 1))
    Complete output from command python setup.py egg_info:
    running egg_info
    ...
      File "j:\jenkins\workspace\requests\requests-windows\.env\lib\site-packages\setuptools\command\egg_info.py", line 470, in append
        if self._safe_path(path):
      File "j:\jenkins\workspace\requests\requests-windows\.env\lib\site-packages\setuptools\command\egg_info.py", line 503, in _safe_path
        if os.path.exists(u_path) or os.path.exists(utf8_path):
      File "j:\jenkins\workspace\requests\requests-windows\.env\lib\genericpath.py", line 26, in exists
        os.stat(path)
    TypeError: stat() argument 1 must be encoded string without null bytes, not unicode
```

In addition, the script that reports the result to GitHub appears to be broken, since the GitHub status for the windows job says "Epic Fail! No test results found.", rather than "Failed" or similar.

cc @kennethreitz

Closing this out since the Windows CI issues seem to have been resolved.
why don't wrapper urllib3 LocationValueError("No host specified.")

Summary.

## Expected Result
i can import exception from requests.

## Actual Result

when i catch LocationValueError, i need import LocationValueError from urllib3.exceptions, not from requests....


## System Information

    $ python v2.7.0

```
<paste here>
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Hey @dading, it looks like this may be a duplicate of #3220. Could you please provide some more information on when you're encountering this exception? Is it occurring with a specific domain, or is this failure periodic?

Could you also provide the version of Requests you're currently using? Thank you!

I am wrong, thanks
Log output for `make test-readme` failures is not clear

The first round of CI for #4137 failed in the `readme` Jenkins job:
http://jenkins.kennethreitz.org:8080/job/requests/job/requests-readme/67/

The console output for the failure was:
```
...
Installing collected packages: requests
  Found existing installation: requests 2.17.3
    Uninstalling requests-2.17.3:
      Successfully uninstalled requests-2.17.3
  Running setup.py develop for requests
Successfully installed requests
+ make test-readme
python setup.py check -r -s
running check
warning: Check: :128: (WARNING/2) Bullet list ends without a blank line; unexpected unindent.

warning: Check: Invalid markup which will not be rendered on PyPI.

error: Please correct your package.
Makefile:11: recipe for target 'test-readme' failed
make: *** [test-readme] Error 1
Build step 'Virtualenv Builder' marked build as failure
```
(http://jenkins.kennethreitz.org:8080/job/requests/job/requests-readme/67/console)

This confused me initially, since the log output doesn't mention a particular filename, so all I had to go on was the name of the test (`test-readme`) - but the PR only modified `AUTHORS.rst`, `HISTORY.rst` and `docs/user/advanced.rst` - and not `README.rst`.

Not being familiar with `setup.py check`, and the functionality `docutils` adds to it, it took me a while to figure out it was validating the value passed to `long_description` in setup.py's `setup()`, which is:

```python
    long_description=readme + '\n\n' + history,
```

...and therefore in my case the bad syntax was actually from `HISTORY.rst`.

Things that might make this more clear:
* The Makefile command being renamed to say `test-readme-history` or similar
* Leaving the makefile command named as-is, but in case of failure make it output "Validation of setup.py metadata failed! Check markup in X and Y"
* Using another way to validate the markup that scans individual files (rather than the concatenated text in setup.py) and can give specific filenames and line numbers

Many thanks :-)

Also, currently the `test-readme` make command calls `python setup.py check -r -s` which isn't clear for someone who isn't familiar with the check command. Perhaps `python setup.py check --restructuredtext --strict` would make the purpose clearer?
Allow Requests.Response to be used as a context manager

This saves having to wrap the call to requests with `contextlib.closing()`, allowing it to be used directly in a `with` statement, like so:

```python
with requests.get('http://httpbin.org/get', stream=True) as r:
    # Do things with the response here.
```

Fixes #4136.

FYI, @kennethreitz, as you can see here the master branch *is* failing on the Windows builder. @requests/core for the moment we can ignore that failing Windows test and continue to merge until @kennethreitz works out what's going on with the builder. ;)
Allow Requests.Response to be used as a context manager in `with` statements

The [docs for streaming responses](http://docs.python-requests.org/en/master/user/advanced/#body-content-workflow) currently state:

> If you find yourself partially reading request bodies (or not reading them at all) while using stream=True, you should consider using contextlib.closing ([documented here](http://docs.python.org/2/library/contextlib.html#contextlib.closing)), like this:
> ```python
> from contextlib import closing
> 
> with closing(requests.get('http://httpbin.org/get', stream=True)) as r:
>     # Do things with the response here.
> ```

It would be great to be able to omit `contextlib.closing()` entirely, and use the call to requests directly within a `with` statement, like so:

```python
with requests.get('http://httpbin.org/get', stream=True) as r:
    # Do things with the response here.
```

However that currently results in `AttributeError: __exit__`, since [`Requests.Response`](https://github.com/requests/requests/blob/v2.17.3/requests/models.py#L576-L942) doesn't have the `__enter__` and `__exit__` methods expected of a context manager:
https://docs.python.org/3/reference/datamodel.html#context-managers

I think it would be worthwhile to add them, to allow safe streaming without the extra boilerplate. (There's also prior art - `Requests.Session()` [can be used as a context-manager](https://github.com/requests/requests/blob/v2.17.3/requests/sessions.py#L389-L393).)
POST converts request data from UTF-8 to ISO-Latin

Posting a chunk of XML in utf-8 encoding arrives garbled (iso-latin) at the target server.
It should arrive as encoded.
## Reproduction Steps

```python
import requests

request = '''
<?xml version="1.0" encoding="UTF-8"?>
<Requ>
<GivenName>Özdemam</GivenName>
<FamilyNameGrünzl</FamilyName>
</Requ>
'''
print(request)
headers = {'Content-Type': 'text/xml; charset=utf-8', }
req = requests.Request('POST', 'http://192.168.1.5:8888/api',
                       headers=headers,
                       data=request)
prepped_requ = req.prepare()
s = requests.Session()
http_response = s.send(prepped_requ)
```

Running `nc -l 8888' will show the garbled Umlaut characters, whereas print or command-line curl will produce the expected result.

## System Information

Tested on CentOS7/ Python 3.4.6 (EPEL) and OSX 10.12 MacPorts Python 3.5.3

Seems to be similar to #3476, but is unresolved.

You haven't encoded it. ;)

On Python 3 the native string type is unicode: it has no encoding. We have to auto-encode the data, and in practice that's done by httplib with no reference to whatever you put in your header field. Try changing your code to this:

```python
import requests

request = '''
<?xml version="1.0" encoding="UTF-8"?>
<Requ>
<GivenName>Özdemam</GivenName>
<FamilyNameGrünzl</FamilyName>
</Requ>
'''.encode('utf-8')
headers = {'Content-Type': 'text/xml; charset=utf-8', }
req = requests.Request('POST', 'http://192.168.1.5:8888/api',
                       headers=headers,
                       data=request)
prepped_requ = req.prepare()
s = requests.Session()
http_response = s.send(prepped_requ)
```
Error when using pip install requests - SyntaxError in setup.py?

Hi,

I am not sure if I am just doing something stupid but I can't seem to install requests on my raspberry pi which is running raspbian.

## Expected Result

Install requests for python

## Actual Result

When trying to install i got this:

`Downloading/unpacking requests
  Downloading requests-2.17.3.tar.gz (123Kb): 123Kb downloaded
  Running setup.py egg_info for package requests
    Traceback (most recent call last):
      File "<string>", line 14, in <module>
      File "/home/pi/build/requests/setup.py", line 52, in <module>
        exec(f.read(), about)
      File "<string>", line 14
        __cake__ = u'\u2728 \U0001f370 \u2728'
                                             ^
    SyntaxError: invalid syntax
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 14, in <module>

  File "/home/pi/build/requests/setup.py", line 52, in <module>

    exec(f.read(), about)

  File "<string>", line 14

    __cake__ = u'\u2728 \U0001f370 \u2728'

                                         ^

SyntaxError: invalid syntax

----------------------------------------
Command python setup.py egg_info failed with error code 1 in /home/pi/build/requests
Storing complete log in /home/pi/.pip/pip.log`


## Reproduction Steps

pip-3.2 install requests

## System Information

I am trying to install this on a Raspberry Pi 2 B+ running Rasbian.
python version 3.2.3




Requests does not support Python 3.2, and has not for more than a year.
digest auth with netrc

According to the documentation:

netrc Authentication

If no authentication method is given with the auth argument, Requests will attempt to get the authentication credentials for the URL's hostname from the user's netrc file. The netrc file overrides raw HTTP authentication headers set with headers=.

If credentials for the hostname are found, the request is sent with HTTP Basic Auth.

How can i use then Digest with ~/.netrc ?

At this time Requests does not enable the use of Digest Auth with `~/.netrc`. This would be a feature worth having, though it's a difficult one to add because right now Requests does not have any kind of multi-auth support. The toolbelt does though.

@PhilipStefanov "the toolbelt" = https://github.com/requests/toolbelt. Feel free to move this issue over there. 😃
chardet version check in >=2.16.0 is breaking our tox tests

The latest update (2.16.0 and 2.17.3) to version check chardet has broken one of our sphinx docs tox tests on travis.

```
  File "/home/travis/build/cas--/Deluge/.tox/docs/local/lib/python2.7/site-packages/requests/__init__.py", line 66, in <module>

    major, minor, patch = chardet.__version__.split('.')[:3]

AttributeError: type object '__version__' has no attribute 'split'
```

I have no idea how/why this is occurring but it's your new code in `__init__.py` that is breaking when version checking chardet.

Commit that introduced the issue: 2b34880e2fa3fe6006bf64517e036f5ed74478d6

Full details of OS and versions: https://travis-ci.org/cas--/Deluge/jobs/239401586

Is this reproducible for you outside of Travis CI? I cannot reproduce it.

@cas-- It looks like this is directly related to the [`conf.py`](https://github.com/deluge-torrent/deluge/blob/6743b0c813769bd1fd6e3b0c96e2f037adc78dd0/docs/source/conf.py) file in Deluge's `docs/source` directory. It's replacing `chardet` with a Mock object which has an attr `__version__` of type `type`. Are modern versions of `chardet` using C extensions? I can't seem to find anything that points to that, so I'm not sure why this extension is being mocked out.

So sorry, recently been dealing lots of failing tests due to module changes and completely forgot about mock in docs test!

> Are modern versions of chardet using C extensions?

To answer the question: No.

@sigmavirus24 thanks! 😊

@cas-- No need to apologize, glad you got it worked out.
Add support for MessagePack


Hi there @suzaku!

Thanks for sending this pull request! We're so glad you've chosen to contribute to Requests and we appreciate all the time and effort you've put in to get to the point of sending this PR.

Sadly, however, it's highly unlikely that we will accept this pull request. Here's why:

Requests focuses on satisfying the needs of 98% of its users. As it stands, roughly that percentage of our users will utilize and take advantage of the `json` keyword. Sadly, we've not received nearly as much feedback about providing the same functionality for `msgpack`.

I'll leave this open for other core reviewers to comment on, but I wanted to set your expectations appropriately.

Cheers,
Ian

Ian is correct. This would make a great extension module, however!

Thanks again @suzaku 

@sigmavirus24    Thanks. I will handle msgpack in my application then.
@kennethreitz Good idea.
iter_lines gives different results depending on chunk_size 

When using iter_lines with chunk_size other than None, it can generate an extra blank line if the end of a chunk lands on a delimiter. 

## Reproduction Steps

```python
import StringIO
import requests

def getTestFile():
    f = StringIO.StringIO()
    f.write('a-b')
    f.seek(0)
    return f

r = requests.Response()
r.raw = getTestFile()
print [l for l in r.iter_lines(delimiter='-')]

r = requests.Response()
r.raw = getTestFile()
print [l for l in r.iter_lines(delimiter='-', chunk_size=2)]
```

## Expected Result
['a', 'b']
['a', 'b']

## Actual Result
['a', 'b']
['a', '', 'b']

## Proposed Fix
I modified iter_lines as follows, and the above test passes:

```python
    def iter_lines(self, chunk_size=ITER_CHUNK_SIZE, decode_unicode=None, delimiter=None):
        """Iterates over the response data, one line at a time.  When
        stream=True is set on the request, this avoids reading the
        content at once into memory for large responses.

        .. note:: This method is not reentrant safe.
        """

        pending = None

        for chunk in self.iter_content(chunk_size=chunk_size, decode_unicode=decode_unicode):

            if pending is not None:
                chunk = pending + chunk

            if delimiter:
                lines = chunk.split(delimiter)
            else:
                lines = chunk.splitlines()

            if lines and lines[-1] and chunk and lines[-1][-1] == chunk[-1]:
                pending = lines.pop()
            elif lines and lines[-1] == '':
                pending = lines.pop()
            else:
                pending = None

            for line in lines:
                yield line

        if pending is not None:
            yield pending
```

## System Information

    $ python -m requests.help
```
{
  "chardet": {
    "version": "3.0.3"
  }, 
  "cryptography": {
    "version": ""
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.13"
  }, 
  "platform": {
    "release": "16.6.0", 
    "system": "Darwin"
  }, 
  "pyOpenSSL": {
    "openssl_version": "", 
    "version": null
  }, 
  "requests": {
    "version": "2.17.3"
  }, 
  "system_ssl": {
    "version": "100020bf"
  }, 
  "urllib3": {
    "version": "1.21.1"
  }, 
  "using_pyopenssl": false
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Hey @a-p-f, thanks for putting together such a detailed issue. This is actually a duplicate of #3980 and addressed by #3984. This is a breaking change though, so it won't be available until 3.0.0 is released.

I'm going to close this out as a dupe, but please let us know if you have any further questions.

Sorry about that. I searched the issues for 'iter_lines', but I had the 'is:open' filter on so it didn't pop up.

No worries! 😄
Bring 3.0 up-to-date with master.

This was a gnarly merge, and the odds of it going right first time are pretty low, so let's have the CI prove or disprove the success of it for me.

@Lukasa you're merging this into master?

Other than definitely not wanting to merge this into master, this looks like it's right. It would be nice to see a smaller diff against proposed/3.0.0 though.

Okay, I updated the branch for you

FYI, the windows tests won't finish because they're blocked. I pinged @kennethreitz about this in IRC.

Ok, the Windows CI won't get working any time soon as @kennethreitz is on the road. I suggest we move on without it to unblock #4118.

@Lukasa, we have a [successful run](http://jenkins.kennethreitz.org:8080/job/requests/job/requests-windows/lastSuccessfulBuild/) of master on Windows from yesterday. Since nothing has changed on master, and I don't see anything directly related to Windows in the 3.0.0 merge, I think it's probably OK to merge this.


Kenneth restarted the builder, so we should get a result shortly.

windows had a failure!

It did, that's true, but that failure seems to be related to having had something wacky happen with the environment build. I'm restarting the builder.

Yeah, I don't know that I trust the output of this Windows builder. It seems to not really be working properly.

@kennethreitz What are we doing about this Windows builder?

@Lukasa not sure why the build is hanging on Windows. I'll take a peek around. It runs fine on master. 

Looks like we're good to go here.

I've rebased #4118 onto this locally, should I push that update?

@hobarrera Go for it
TypeError when using a custom HTTPSConnectionPool

An issue #4010 was closed, but this error
```python
TypeError: _new_pool() got an unexpected keyword argument 'request_context'
```
still appears in requests 2.17.3. Details are in the issue #4010.

@amatyukhin0 Did you read [this comment](https://github.com/requests/requests/issues/4010#issuecomment-300352930)? I believe it explains your concern.

@Lukasa Sorry, i was inattentive, it works after the fix.
Drop python2.6 support

Drop support for python2.6. Fixes #3928

Note that this targets the 3.0.0 branch, not master.

Not entirely sure why CI is failing -- is this branch not updated for it?
```
+ make ci
make: *** No rule to make target 'ci'.  Stop.
```

@hobarrera No, it looks like it isn't. I'll try to resolve that problem today.

Thanks!

Hey @hobarrera, it looks like 2.6 is still in the tox.ini file. If we can pull that out, I think this should be good to go.

Tests are "passing". Just 2.6 failed, as expected.

Whoops! There we go. Jenkins is still configured to run 2.6 though.

@kennethreitz, are the versions running on Jenkins hardcoded for the server? Do we have any way to delegate this to something like tox?

@nateprewitt they are currently pinned

"work around" is a bit tricky here, because they're required builds. I think we need a way to turn them off for this branch.

The only workarounds I can think of are extremely hacky things, like raising a "SkipTest" exception before most imports run, or things like that.

@Lukasa 😕 well that's unfortunate. I talked briefly with Kenneth last night and it seemed like there was some difficulty differing build requirements between branches. We'll probably need to investigate configuration options for [Jenkins with tox](https://tox.readthedocs.io/en/latest/example/jenkins.html), or sit on this branch until we're ready to shut off 2.6 tests across the board.

@hobarrera this is definitely an infrastructure issue on our end. I'd *really* like to avoid making any code changes in Requests' tests to deal with a CI server.

so, i have a solution to this — we just need a separate job for the 3.0.0 branch, which removes python 2.6. I'll see if I can get to this today. 

I guess I'll just leave this here until Jenkins is ready? I don't think there's anything else I can do. 🙂 

The "workaround"' here is to allow administrators to merge anyway. I'll turn that on now. 

there, that was easy. 

Thanks!
Add editorconfig for easier styling


I don't think this is needed.

Thanks for opening your first pull request, @ishvad! We appreciate your effort in trying to improve the Requests project.

We try to keep the code inside the Requests repository to minimum essentials. While this will likely be useful for some developers, I don't know if this qualifies as essential. This is something that is likely best left to the user to decide.

You could consider publishing this in a [gist](https://gist.github.com/) for others to reference. Feel free to leave a link to it here if you wish.
Catch error if multiprocessing is not available or is unable to determine the number of CPUs

resolves #4113 

FYI @kennethreitz, it seems only one check is running here: I have no idea why.

add to whitelist

(Taking a shot in the dark)

There we go, spelling was never my forte (but the comment box here has spell check :)

no rush from me, I'm stuck with 2.15.1 until chardet/chardet#127 is sorted.

@sigmavirus24 was a configuration issue, i fixed the problem. Sorry for the issues!
Installation fails on platforms without multiprocessing

On the latest version of requests in pypi, requests requires the multiprocessing module to install correctly, even though it is not used in the installation, only in the unit tests. This means it is no longer possible to install requests on platforms that lack multiprocessing support (such as Android or IoT devices)

## Expected Result

Requests is installed and available for use

## Actual Result

Installation fails with the following:

```
Downloading/unpacking requests
  Downloading requests-2.17.3.tar.gz (123Kb): 123Kb downloaded
  Running setup.py egg_info for package requests
    Traceback (most recent call last):
      File "<string>", line 14, in <module>
      File "/tmp/build/requests/setup.py", line 11, in <module>
        from multiprocessing import cpu_count
    ImportError: No module named multiprocessing
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 14, in <module>

  File "/tmp/build/requests/setup.py", line 11, in <module>

    from multiprocessing import cpu_count

ImportError: No module named multiprocessing

----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in /sdcard/com.hipipal.qpyplus/.pip/pip.log
```

## Reproduction Steps

From a platform without multiprocessing:

```
pip install requests
```

## System Information

For obvious reasons, requests.help is not available, but I've tried to piece together the information that it would produce:

- platform.system()   Linux
- platform.release()   3.14.29
- implementation     CPython
- implementation_version   2.7.2
- ssl.OPENSSL_VERSION_NUMBER   268443727L
- requests_version   2.17.3


Appears to be since e6e3589, I'll see if I can do a fix for it

Thanks for this report. It should be entirely possible to pull the multiprocessing import into a tighter scope.
Removed Myself from PyPi (security concerns, and otherwise)

![letter](https://d3vv6lp55qjaqc.cloudfront.net/items/39202X1U0d430o2O3G1X/Screen%20Shot%202017-05-30%20at%2012.10.22%20AM.png?X-CloudApp-Visitor-Id=2577&v=0505af2c)

Posting here as a matter of record.
Concerns about 3.0.0 Plans (cryptography)

As @Lukasa and I discussed at PyCon, `twisted[tls]`, as it stands, would be a dependency of Requests for version 3.0.0. 

I have major concerns for this, as I struggled a lot to get `crypography` to install properly on Linux this weekend. 

We need to ensure that if we plan to to do this, before we put all the work in, that zero installation issues will be had by any of our users. 

One of the reasons we are popular and well-loved is the fact that we have no installation pains, compared to some other libraries. I think we may need to contribute to `cryptography` (potentially by just making suggestions, or otherwise) in order for this to be a tenable plan for us.

-----------------

Correct me, of course, if I'm unfounded in any of my concern @Lukasa. 

Any thoughts?

/cc @alex (potentially)

So, the biggest thing we're planning to improve this is that we're looking into shipping Linux wheels. /cc @reaperhulk

The major blocker for doing that is making pyOpenSSL able to find certificates (not relevant for requests, because `certifi`, but still needed for other consumers). Once that happens we can ship wheels which will help anywhere that can use `manylinux1` wheels.

We already have wheels on macOS and Windows, so both of those should be good.

fantastic news!

So, yes: in the current climate `twisted[tls]` would be mandatory. That may not remain true: there are some rumblings in python-dev about the possibility of backporting the `SSLObject` class from the standard library to 2.7: if that happened, Twisted could be updated to provide TLS using the standard library instead of or as well as with PyOpenSSL, which would resolve our concerns.

Longer term, we'd like to use platform-native TLS implementations where possible, which requires a bunch of other code.

As a general note, I don't think we're likely to be able to do anything that @alex and @reaperhulk can't. They've been worrying about this problem a lot more than we have.

As @alex said we're planning on shipping a `manylinux1` wheel soon-ish. This will make it an easy install under these conditions:

* A sufficiently new `pip` (8.1+)
* The linux distribution is manylinux1 compatible (excludes Alpine linux and any other distribution using musl libc)

So it's a big improvement but there will still be users not covered.

And just for fun, here's the statistics on pip major version for the last month out of PyPI:
![image](https://cloud.githubusercontent.com/assets/161495/26584611/0634c676-450f-11e7-948e-d3f3c614aaa6.png)


PEP for the `ssl` module backport: https://github.com/python/peps/blob/master/pep-0546.txt

Concerns resolved :)
Since 2.17.1, urllib3 is no longer in requests.packages

The newly released v2.17.1 of the requests package introduced an issue in that `urllib3` is no longer in `requests.packages`:

```
python -c "import zhmcclient; print('Import: ok')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "zhmcclient/__init__.py", line 30, in <module>
    from ._session import *       # noqa: F401
  File "zhmcclient/_session.py", line 29, in <module>
    from requests.packages import urllib3
ImportError: cannot import name urllib3
```

Version 2.17.0 and before worked fine.

We circumvented this issue by fixing the version of requests to 2.17.0, for now.

## Expected Result

`urllib3` is again in `requests.packages`.

## Actual Result

`urllib3` is no longer in `requests.packages`.

## Reproduction Steps

```python
from requests.packages import urllib3
```

## System Information

This could be reproduced on Python 2.7, 3.4, 3.5, 3.6 and on all OS platforms that we test on (Ubuntu, OS-X and Windows).

Closing for #4104