Raibek/requests-github-issues huggingface dataset:
should_bypass_proxies not thread-safe

`requests.utils.should_bypass_proxies` temporary modifies the environment, calls into httplib and then restores the environment. However, it takes no locks while doing so, so two threads both using requests could lead to a permanent change to the environment. Even with a lock, any other thread that depends on the environment may observe the change (e.g. it may launch curl in a subprocess, and curl will then pick up this `no_proxy` setting).

## Expected Result

The environment is not modified by requests.

## Actual Result

I haven't observed this bug in practice - it's a theoretical race condition I can see in the code. In the reproduction code below, I believe the following sequence could occur:

1. Thread 1 enters the set_environ context manager, sets `old_value = None`, `os.environ['no_proxy'] = 'xyz'`.
2. Thread 2 enters set_environ, sets `old_value = 'xyz'`, `os.environ['no_proxy'] = 'xyz'.
3. Thread 1 exits, deletes `os.environ['no_proxy']`.
4. Thread 1 exits, sets `os.environ['no_proxy'] = 'xyz'`.

Now after all the requests have been finished, the environment has been modified.

## Reproduction Steps

I think this code can in theory trigger the race condition, although I haven't observed it. It needs an HTTP server on localhost:8080 (or just edit the URL)

```python
#!/usr/bin/env python3

import threading
import os

import requests

def my_thread():
    session = requests.Session()
    for i in range(100):
        with session.get('http://localhost:8080/', proxies={'no_proxy': 'xyz'}) as resp:
            pass

threads = [threading.Thread(target=my_thread) for _ in range(16)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()
print('no_proxy:', os.environ.get('no_proxy', 'not set'))
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.6"
  },
  "platform": {
    "release": "4.15.0-45-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "1010007f"
  },
  "urllib3": {
    "version": "1.24.1"
  },
  "using_pyopenssl": false
}
```
Redirect with empty location header not followed

I am working with an external service whose authentication pattern is to set a cookie authentication token and return a 302 redirect to the same location (the location header is an empty string). This redirect is not being followed, I believe due to [this line](https://github.com/kennethreitz/requests/blob/v2.21.0/requests/sessions.py#L151).

This forces code like the following:

```python
import requests

with requests.session() as session:
    session.cert = (cert, key)
    resp = session.get(url)

    # Extra code to handle redirect with empty location
    if resp.status_code == 302 and resp.headers["location"] == "":
        resp = session.get(url)
```

## System Information

Requests v2.21.0
Python 2.7.15
Windows Server 2012 R2

According to [RFC 7231](https://tools.ietf.org/html/rfc7231#section-7.1.2) the `Location` header is not allowed to be empty. Do most browser handle this as you describe?

Given we haven't heard back on this in quite a while, I'm going to resolve. The current behavior of the service is objectively incorrect and I'm insistent to start adding behavior where we're guessing at the right thing to do. The best solution for this case will be to handle the redirect manually.
[fix] fix failing doctests in requests/*

## Why this PR
Running the command:
`
$ py.test --doctest-module requests
`
gives failing doctests. This PR aims to fix these failing doctests.

Issue Link: https://github.com/kennethreitz/requests/issues/4986

## How to test
`
$ py.test --doctest-module requests
`
All the 11 doctests should pass.


fantastic
Need a clarification on adapters.py

Can someone please tell me what is the use of the below lines in **adapters.py**?
Does it actually check for a proxy and uses it in get_connection?

```
        if proxy:
            proxy = prepend_scheme_if_needed(proxy, 'http')
            proxy_url = parse_url(proxy)
            if not proxy_url.host:
                raise InvalidProxyURL("Please check proxy URL. It is malformed"
                                      " and could be missing the host.")
            proxy_manager = self.proxy_manager_for(proxy)
            conn = proxy_manager.connection_from_url(url)
```
My system has no proxy enabled, I confirmed the same with `netsh winhttp show proxy` 
Output:
`Current WinHTTP proxy settings: Direct access (no proxy server)`

Not sure why the control is getting into `if proxy:` after `proxy = select_proxy(url, proxies)`
Requests returns 200 for a url that is invalid

Summary.

## Expected Result
requests should return 404 for an invalid url

What you expected.
status_code should be 404

## Actual Result
status_code was 200

What happened instead.
status_code was 200

## Reproduction Steps

```python
import requests
# this is a VALID url
req = requests.get("https://talkpython.fm/episodes/transcript/96/exploring-awesome-python")
print(req.status_code)
```
200 is displayed


```python
import requests

# this is a BAD url
req = requests.get("https://talkpython.fm/episodes/transcript/this_ur_is_just-WRONG")
print(req.status_code)
```
200 is displayed SHOULD display 404


## System Information
Windows 7, Pro 64-Bit
conda 4.6.2

    $ python -m requests.help

```
(py36sci) C:\>python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.5"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.6"
  },
  "platform": {
    "release": "7",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "1010101f",
    "version": "19.0.0"
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "100020ff"
  },
  "urllib3": {
    "version": "1.24.1"
  },
  "using_pyopenssl": true
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

(py36sci) C:\>python
Python 3.6.6 | packaged by conda-forge | (default, Jul 26 2018, 11:48:23) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.


Hi @ReddyKilowatt,

I'm not able to reproduce this bug:  
```
$ try requests==2.21.0
==> Use python python3.6
[*] Downloading packages: requests==2.21.0
Python 3.6.7 (default, Oct 22 2018, 11:32:17) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> valid_request = requests.get("https://talkpython.fm/episodes/transcript/96/exploring-awesome-python")
>>> print(valid_request.status_code)
200
>>> invalid_request = requests.get("https://talkpython.fm/episodes/transcript/this_ur_is_just-WRONG")
>>> print(invalid_request.status_code)
404
```

Can you double check it please? Thanks!

I've talked to the site maintainer and he confirmed it was a server issue and quickly fixed it.
I'm now seeing the expected 404 value.
I think there is a bug with supporting multi-part POST with multiple files, and one JSON body

Summary.

## Expected Result

I expect that I can expand this code which works for ONE zip "files" file, and JSON message body, to be expandable to 2 or more ZIP files  along with the JSON body:

```
    multipart_form_data_object_orig = {
        'files': (args.files[0], open(args.files[0], 'rb'), "application/json"),
        'message': (None, open(args.message, 'rb'), 'application/json')
    }
    response = requests.post(args.url + ':' + str(args.port) + '/' + args.endpoint, files=multipart_form_data_object_orig,
                             proxies=proxies)
```

## Actual Result

When I add a second element `'files': (args.files[1], open(args.files[1], 'rb'), "application/json"),`, one overwrites the other.

In Curl, we can do -F "files=@file.zip,file2.zip" and both are properly passed

What happened instead.

## Reproduction Steps

```python
import requests
##args handling stuff..

    multipart_form_data_object_orig = {
        'files': (args.files[0], open(args.files[0], 'rb'), "application/json"),
        'files': (args.files[1], open(args.files[1], 'rb'), "application/json"),
        'message': (None, open(args.message, 'rb'), 'application/json')
    }
    response = requests.post(args.url + ':' + str(args.port) + '/' + args.endpoint, files=multipart_form_data_object_orig)

```

The JAVA API is only seeing one file in the files array, while Curl's request shows 2 files.
## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.15"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "100020ff"
  },
  "urllib3": {
    "version": "1.24.1"
  },
  "using_pyopenssl": false
}
```

There is a comperhensive question on StackOverflow I posted with all the variations to requests I tried to do in order to reproduce the CURL request:
https://stackoverflow.com/questions/54656686/python-requests-emulate-a-curl-post-sending-multipart-request-with-1-or-more-fil

Including the comparison of what httpbin shows for CURL, and for Python - they are DIFFERENT, and I was unable to find a way to make them identical, hence I believe this must be a bug in Requests, since this is a rarely used capability in APIs...

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here. Please also inspect your `multipart_form_data_object_orig` dictionary and read up on how dictionaries in Python work.
iter_content returned size is far bigger than chunk_size

I expect `iter_content` to return me `chunk_size` number of bytes, but it never does. It returns me a far larger and random number of bytes. In contrast, `urllib.request` returns me exactly the number of bytes I request using its `.read(chunk_size)` method.

It is not acceptable for the returned chunk to use up to 5x more memory as evidenced below. Imagine this happening in say 64 threads at once. It should return exactly the number of bytes I asked for (or less at the end).

## Expected Result

```python
>>> import requests
>>> requests.__version__
'2.21.0'
>>> req = requests.get('http://docs.python-requests.org/en/latest/user/advanced/', stream=True)
>>> it = req.iter_content(8192)
>>> len(next(it))
8192
>>> len(next(it))
8192
>>> len(next(it))
8192
```

## Actual Result

```python
>>> import requests
>>> requests.__version__
'2.21.0'
>>> req = requests.get('http://docs.python-requests.org/en/latest/user/advanced/', stream=True)
>>> it = req.iter_content(8192)
>>> len(next(it))
3345
>>> len(next(it))
48672
>>> len(next(it))
47182
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.2"
  },
  "platform": {
    "release": "4.18.0-15-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "1010007f"
  },
  "urllib3": {
    "version": "1.24.1"
  },
  "using_pyopenssl": false
}
```

The docs mention that the returned item length may be different because of decoding, so I believe this is expected behavior.

> The chunk size is the number of bytes it should read into memory. This is not necessarily the length of each item returned as decoding can take place.

(See [docs for Response.iter_content](http://docs.python-requests.org/en/master/api/#requests.Response.iter_content))

EDIT: @kenneth-reitz can you confirm this?

It may be documented and expected, but then the chunk_size loses any value for the user. urllib has no such problem. It works perfectly all the time for `read()` and returns exactly the number of bytes that were asked for. Please consider using an internal buffer or such for the extra bytes that the user hasn't asked for. Returning fewer bytes than chunk_size is even worse! I thought `requests` was supposed to present a clean and friendly interface to the user. For a GET request, the user may not care about how many bytes are internally read behind the scenes, as long as it's somewhat close to chunk_size.

this is expected behavior, if what you're saying is correct @rHuggler 

Does the problem go away if you don't use compression?
QUESTION: How does content length of chunked encoded data is decided?

I am trying to use [treq](https://github.com/twisted/treq/) for downloading a file from https://speed.hetzner.de/100MB.bin (this link is just for testing). Now if I use requests to send a get the file like

```python
import requests
url = 'https://speed.hetzner.de/100MB.bin'
r = requests.get(url)
```

I get the content and checking the headers
```python
>>> r.headers
{'Server': 'nginx', 'Date': 'Mon, 11 Feb 2019 13:51:08 GMT', 'Content-Type': 'application/octet-stream', 'Content-Length': '104857600', 'Last-Modified': 'Tue, 08 Oct 2013 11:48:13 GMT', 'Connection': 'keep-alive', 'ETag': '"5253f0fd-6400000"', 'Strict-Transport-Security': 'max-age=15768000; includeSubDomains', 'Accept-Ranges': 'bytes'}
```
We can see the content length i.e 104857600 but If I do the same thing with treq and then try to find the response `Content-Length` then it returns `unknow size`. 

I have talked on treq mailing list and they told me
```
Do you have Chunked encoding
https://en.wikipedia.org/wiki/Chunked_transfer_encoding
In this case, the content length is not advertised in the response headers.
```
So I just wanted to know how does `requests` shows the content length? Is there any special function in requests that finds out the content length of such kind of files?

Thanks


We don't. You seem to be encountering a different path on the server than `treq` for unknown reasons. If the server sent back headers we present them, otherwise we do not magically replace or misrepresent what the server sent.

@sigmavirus24 I apologize if it a dumb question but I tried requests with the following URL: https://heyguys.cc/ and I got the `content-length: 1615` but I don't see any content-length header when I am seeing the request in the raw headers of Firefox developer tool. Do you mind telling me why that could be?

Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Accessing response.content twice removes forgets read error

I had a hard debugging time today because an error in the response stream is only reported when accessing `response.content` for the first time.

This is especially irritating when running code in a debugger.

## Expected Result

If accessing `response.content` the first time raises an exception I would expect that accessing `response.content` again would also raise an exception (ideally the same). 

## Actual Result

Instead after raising on the first get, getting `response.content` again returns an empty string.

## Reproduction Steps

Here is a patch with a new test case for this: [error_replay_test.diff.gz](https://github.com/requests/requests/files/2838360/error_replay_test.diff.gz).

Basically, it boils down to this:

```python
import requests

response = requests.post("http://connreset.biz/get/incomplete/chunked", stream=True)
try:
    response.content
except Exception:
    # Error handling code, may try something else or fall through
    pass

content = response.content  # empty string
```

Output of my test case:

```
$ pipenv run py.test tests/test_lowlevel.py -q --tb=short -k retain
F                                                            [100%]
============================= FAILURES =============================
_______________ test_response_content_retains_error ________________
tests/test_lowlevel.py:343: in test_response_content_retains_error
    assert False, "error response has content: {0!r}".format(content)
E   AssertionError: error response has content: ''
E   assert False
1 failed, 15 deselected in 0.60 seconds
```

## System Information

    $ python -m requests.help

*Edit*: Oops, I used `pipenv run python -m requests.help` which actually called into system python 2.7. Here comes the real data:

```
$ pipenv run python3 -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.8+"
  },
  "platform": {
    "release": "4.15.0-43-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "1000207f"
  },
  "urllib3": {
    "version": "1.24"
  },
  "using_pyopenssl": false
}
```

Thanks for looking into this!


It looks like we never closed the issue after the PR was merged. I'm reverting #5087 because it doesn't cover all of our read APIs (`content`, `iter_content`, `iter_lines`, `text`, `__iter__`) uniformly.

I took a quick pass but the edge cases the tests are missing aren't something I can get resolved today. We can reopen a new PR and revisit options later if anyone wants.
phython error

Summary.

## Expected Result
output should run without any error or exception

What you expected.

## Actual Result

Can please resolve this issue??
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:





  self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x02D87DB0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:



This is not a Requests bug.
problem in python

Can please resolve this issue??
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:





  self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x02D87DB0>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:
Requests.get producing different JSON than curl

When making a get request requests.get and curl, the returned JSON object is identical in every way, except with requests.get leading digits in an "id":# kv pair are omitted.

## Expected Result

cURL and requests.get returned JSON objects should be identical

## Actual Result

requests.get returns
`"id":12345`

Whereas curl returns
`"id":10000000000012345`

## Reproduction Steps

```python
import requests
HEADERS = {'Authorization': f'Bearer {args.api_key}'}
API_URL = f'https://canvas.instructure.com/api/v1'
STUDENT = requests.get(f'{API_URL}/courses/', params=None, headers=HEADERS)

studnetlst = STUDENT.json()
print(studnetlst)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.7.2"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "101000af"
  },
  "urllib3": {
    "version": "1.24.1"
  },
  "using_pyopenssl": false
}
```

More info: https://stackoverflow.com/questions/54527648/pythons-requests-get-producing-different-json-than-api-client

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Can you share the relevant bit from `STUDENT.content`?

`STUDENT.content`:

`b'[{"id":12345,"name":"Class1","account_id":188,"uuid":"itsadifferentsecret","start_at":"2019-01-13T17:33:04Z","grading_standard_id":null}]'`

the desired result should look like:

`b'[{"id":10000000000012345,"name":"Class1","account_id":188,"uuid":"itsadifferentsecret","start_at":"2019-01-13T17:33:04Z","grading_standard_id":null}]'`


So this shows that Canvas is responding with incorrect data. Literally the bytes we receive from the server do no thave the digits you're looking for and there's nothing Requests can do.

Often times, this is the result of the server introspecting user-agent headers so I suspect you try sending a different one.

Cheers,
Ian

But if I change the user-agent to the same as one that runs successfully I have the same problem. 

@sigmavirus24 does adding `'User-Agent': 'adifferentagent'` actually change the header? From captured frames in Wireshark it appears to, just checking my understanding.
docs: correct error in 'merge_environment_settings' usage

This resolves a very minor documentation issue for the proxies argument in 'merge_environment_settings' See #4959 

# [Codecov](https://codecov.io/gh/requests/requests/pull/4960?src=pr&el=h1) Report
> Merging [#4960](https://codecov.io/gh/requests/requests/pull/4960?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/64bde6582d9b49e9345d9b8df16aaa26dc372d13?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4960/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4960?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4960   +/-   ##
=======================================
  Coverage   66.89%   66.89%           
=======================================
  Files          15       15           
  Lines        1577     1577           
=======================================
  Hits         1055     1055           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4960?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4960?src=pr&el=footer). Last update [64bde65...827bbe2](https://codecov.io/gh/requests/requests/pull/4960?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Exception when calling 'merge_environment_settings' as documented

When calling `merge_environment_settings` as documented in http://docs.python-requests.org/en/master/user/advanced/#prepared-requests an exception occurs. There appears to be a conflict between the code and the documentation.

## Expected Result

Calling `merge_environment_settings` on a Session object with 'None' in all arguments except the first proceeds without error.

## Actual Result

An exception occurs: `'NoneType' object has no attribute 'setdefault'`. This is caused by https://github.com/requests/requests/blob/eaab47f033cae3bd443abb6707dfc30ae3482c36/requests/sessions.py#L702 when the 'proxies' argument is set to 'None'.

## Reproduction Steps

As-per the documentation example
```python
from requests import Request, Session

s = Session()
url = "https://google.com"
req = Request('GET', url)
prepped = s.prepare_request(req)
settings = s.merge_environment_settings(prepped.url, None, None, None, None)
```

## System Information
```
Python 3.5
Requests 2.9.1
Ubuntu Xenial
```

Yes, the documentation is out of date. I'd be happy to merge a pull request that updates it.
Delegate chunked encoding handling to urllib

This MR removes the custom handling of sending chunked HTTP requests from requests to delegate to the equivalent logic in urllib3 (https://github.com/urllib3/urllib3/blob/4325867d1ae0d139a11c8689c2d2a5ba2c666c83/src/urllib3/connectionpool.py#L351).

As far as I can tell this has a couple of advantages:
* Remove code that is duplicated between the two libraries
* Allow fixes for outstanding issues in this area to be made in urllib3 so that users of both libraries benefit (I noticed #4445 when having a quick trawl of the issues but I guess there are more)
* All requests are now logged by the urllib3 debug logger (which was the issue I was originally trying to diagnose)

Quick note on testing that I forgot to put in the description: I haven't added any specific tests to cover this feature, but I think that the code is covered by existing tests from what I can see - if there are extras that would be helpful I am happy to add them.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4958?src=pr&el=h1) Report
> Merging [#4958](https://codecov.io/gh/requests/requests/pull/4958?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/64bde6582d9b49e9345d9b8df16aaa26dc372d13?src=pr&el=desc) will **decrease** coverage by `0.23%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4958/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4958?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4958      +/-   ##
==========================================
- Coverage   66.89%   66.66%   -0.24%     
==========================================
  Files          15       15              
  Lines        1577     1554      -23     
==========================================
- Hits         1055     1036      -19     
+ Misses        522      518       -4
```


| [Impacted Files](https://codecov.io/gh/requests/requests/pull/4958?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/adapters.py](https://codecov.io/gh/requests/requests/pull/4958/diff?src=pr&el=tree#diff-cmVxdWVzdHMvYWRhcHRlcnMucHk=) | `68.84% <100%> (-1.43%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4958?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4958?src=pr&el=footer). Last update [64bde65...5ac5caa](https://codecov.io/gh/requests/requests/pull/4958?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
requests.post taking integer as string

There is something I am not able to understand using the **requests.post** method,

Having this set of parameters, the requests.post works perfect.

params = {
        "profiles":  123456789, 
        "date_start":  2019-01-01,
        "date_end": 2019-01-10,
        "fields": [
            "attachments",
            "author_id",
            "comments",
            "created_time",
            "id",
            "interactions",
            "likes",
            "message",
            "post_labels",
            "type",
            "insights_engagement",
            "insights_impressions", 
            "insights_reach", 
            "insights_saved", 
            "insights_video_views", 
            "url"
            ],   
        }

However, when I add a key as an **integer**, I am getting a response that is taken as a **string**

Here is the example adding the new key: 

params = {
        "profiles":  123456789, 
        "date_start":  2019-01-01,
        "date_end": 2019-01-10,
        "fields": [
            "attachments",
            "author_id",
            "comments",
            "created_time",
            "id",
            "interactions",
            "likes",
            "message",
            "post_labels",
            "type",
            "insights_engagement",
            "insights_impressions", 
            "insights_reach", 
            "insights_saved", 
            "insights_video_views", 
            "url"
            ],  
            "limit": 500
        }

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
[fix] Remove extra blank line

1. The removed blank line was added at commit 8a2014e26 by Kenneth Reitz at
2017-05-26 22:51:40.
The blank line under it was created at 2012, there is a 5 year gap
between them.

2. It has conflict with the style used on line 63.

So I assume it was misadded and removed it.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4954?src=pr&el=h1) Report
> Merging [#4954](https://codecov.io/gh/requests/requests/pull/4954?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/64bde6582d9b49e9345d9b8df16aaa26dc372d13?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4954/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4954?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4954   +/-   ##
=======================================
  Coverage   66.89%   66.89%           
=======================================
  Files          15       15           
  Lines        1577     1577           
=======================================
  Hits         1055     1055           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4954?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4954?src=pr&el=footer). Last update [64bde65...94936d0](https://codecov.io/gh/requests/requests/pull/4954?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hi there,

Thanks for sending a PR to requests. Unfortunately, we have no style guide that shows one thing to be right or the other to be wrong. As such, PRs that adjust style in one way or another are not accepted.

> Hi there,
> 
> Thanks for sending a PR to requests. Unfortunately, we have no style guide that shows one thing to be right or the other to be wrong. As such, PRs that adjust style in one way or another are not accepted.

I understand your principles, but style is very important element in Python. And `Requests` package is famous of its beautiful coding. If you review the `compat.py` file now, you can see the style conflict (blank line in import area) between its `is_py2` and `is_py3` import part(in line 47, there are 2 blank lines, and in line 64, same logic, only 1 blank line). I found it was caused by the action of 
 commit `7fd9267b3bab1d` didn't fully revert the change induced in `8a2014e265d6c`.

Even though this PR can't be accepted as an official PR. I think this issue should be solved in other functional PRs.

With all my respects.
Unable to call API on a system using unsigned SSL cert

I'm not able to call an API on a system using an unsigned SSL certificate

## Expected Result

JSON output data

## Actual Result

`requests.exceptions.SSLError: HTTPSConnectionPool(host='apitest.internal', port=443): Max retries exceeded with url: /login/oauth2/token (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:847)')`,))

## Reproduction Steps

```python
from oauthlib.oauth2 import BackendApplicationClient
from requests.auth import HTTPBasicAuth
from requests_oauthlib import OAuth2Session
import requests, json

def client_credentials(client_id, client_secret, token_uri):
    auth = HTTPBasicAuth(client_id, client_secret)
    client = BackendApplicationClient(client_id=client_id)
    oauth = OAuth2Session(client=client)
    token = oauth.fetch_token(token_url=token_uri, auth=auth)
    return token

storecreds=client_credentials(client_id, client_secret, token_uri)
print(storecreds)
```

## System Information

    $ python3 -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.7"
  },
  "platform": {
    "release": "4.15.0-43-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.20.0"
  },
  "system_ssl": {
    "version": "1010007f"
  },
  "urllib3": {
    "version": "1.23"
  },
  "using_pyopenssl": false
}
```

Same issue if I install pyOpenSSL and upgrade PIP to 18.0

`{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.5"
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.7"
  },
  "platform": {
    "release": "4.15.0-43-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010101f",
    "version": "19.0.0"
  },
  "requests": {
    "version": "2.20.0"
  },
  "system_ssl": {
    "version": "1010007f"
  },
  "urllib3": {
    "version": "1.23"
  },
  "using_pyopenssl": true
}`

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
About the difference of requests.get and requests.session().get

Hai ! In your `api.py` file, the function `get` declare the para `params`, and assign a default value to it, then we can use the `get` func like `requests.get(url, {"k1":"v1", "k2":"v2"})` without pointing out the key `params` of the second para. But in your `sessions.py` file, the method `get` of the class `Session` didn't declare the para `params`, so we must use the `Session.get` method like `s.get(url, params={"k1":"v1", "k2":"v2"})`, or it will raise a TypeError `TypeError: get() takes 2 positional arguments but 3 were given`. I checked all request methods defined under the class `Session` and all requests API in the `api.py` file, and found that only the `get` method has this little difference. It used to bother me. Could you please eliminate this little difference? :) Good Luck !

I don't think this warrants an update. Keyword parameters are better and as soon as we can get on Python 3 only I think we should switch everything except URL to be a keyword-only parameter. This just nudges people in the right direction sooner. We could, however, make `params` be a keyword argument in the functional API but that'd be backwards incompatible so we won't do that either.
Allow redirects not configurable in sessions

I have an application I need to reach behind a firewall, I've set up a tunnel to this application via SSH running on my machine which forwards all connections from `127.0.0.1:1234` to `192.168.1.10:80`. Whenever I make a request to my application, it redirects me to a login page first.

The issue with this is that all redirect URL's have the `192.168.1.10` hostname, which requests tries to follow and inevitably crashes (as this host does not exist on *my* network). 

I tried to set `allow_redirects` to `False` on my session by overriding the `request` method, but the session source hard-codes the `allow_redirects` option to `True` if it isn't explicitly set when calling `get`, `head` and `options` ([source](https://github.com/requests/requests/blob/master/requests/sessions.py#L545))

## Expected Result
I expected no redirect to be followed, but rather that that the request ends after the first response. I expected the `allow_redirects` option to be configurable in sessions like the `verify` flag is ([source](https://github.com/requests/requests/blob/master/requests/sessions.py#L625))

## Actual Result
The 302 was resolved, and attempted to redirect me to a nonexistent host, leading to a connection error exception.

## Reproduction Steps

```python
import requests


class Example(requests.Session):
    def __init__(self, allow_redirects=False, *args, **kwargs):
        super(Example, self).__init__(*args, **kwargs)
        self.allow_redirects = allow_redirects

    def request(self, *args, **kwargs):
        kwargs.setdefault('allow_redirects', self.allow_redirects)
        return super(Example, self).request(*args, **kwargs)


sess = Example()
sess.get('http://127.0.0.1:1234')  # ConnectionError ... 192.168.1.10 connection refused (redirected)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.0"
  },
  "platform": {
    "release": "10",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.21.0"
  },
  "system_ssl": {
    "version": "100020af"
  },
  "urllib3": {
    "version": "1.24.1"
  },
  "using_pyopenssl": false
}
```
chardet/urllib3 warnings

While debugging a bug in Ansible, I noticed these warnings:

```
RequestsDependencyWarning: urllib3 (1.24.1) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
```

I spent some time trying to get rid off them, but to no avail. What are the exact/supported dependencies?

Here is a list of installed packages:

```
# pip list
backports.ssl-match-hostname (3.5.0.1)
cached-property (1.5.1)
certifi (2018.11.29)
chardet (3.0.4)
configobj (4.7.2)
decorator (3.4.0)
docker (3.7.0)
docker-compose (1.23.2)
docker-pycreds (0.4.0)
dockerpty (0.4.1)
docopt (0.6.2)
enum34 (1.1.6)
functools32 (3.2.3.post2)
idna (2.7)
iniparse (0.4)
ipaddress (1.0.16)
IPy (0.75)
jsonschema (2.6.0)
kitchen (1.1.1)
perf (0.1)
pip (8.1.2)
policycoreutils-default-encoding (0.1)
pycurl (7.19.0)
pygobject (3.22.0)
pygpgme (0.3)
pyliblzma (0.5.3)
python-linux-procfs (0.4.9)
pyudev (0.15)
pyxattr (0.5.1)
PyYAML (3.13)
requests (2.21.0)
schedutils (0.4)
seobject (0.1)
sepolicy (1.1)
setuptools (0.9.8)
six (1.12.0)
slip (0.4.0)
slip.dbus (0.4.0)
texttable (0.9.1)
urlgrabber (3.10)
urllib3 (1.24.1)
websocket-client (0.54.0)
yum-metadata-parser (1.1.4)
```
## Expected Result

No warning.

## Actual Result

Warning.

## System Information

    $ python -m requests.help

```
# python -m requests.help
/usr/lib/python2.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.24.1) or chardet (2.2.1) doesn't match a supported version!
  RequestsDependencyWarning)
{
  "chardet": {
    "version": "2.2.1"
  }, 
  "cryptography": {
    "version": ""
  }, 
  "idna": {
    "version": "2.7"
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.5"
  }, 
  "platform": {
    "release": "3.10.0-957.el7.x86_64", 
    "system": "Linux"
  }, 
  "pyOpenSSL": {
    "openssl_version": "", 
    "version": null
  }, 
  "requests": {
    "version": "2.21.0"
  }, 
  "system_ssl": {
    "version": "100020bf"
  }, 
  "urllib3": {
    "version": "1.24.1"
  }, 
  "using_pyopenssl": false
}
```

OS: 

```
# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core) 
```

Should have read the source:
https://github.com/requests/requests/blob/master/requests/__init__.py#L57-L71

 - urllib3: < 1.23
 - charded: < 3.0.0

Any reason why this is not enforced via `requirements.txt`? (I am new to Python, so excuse my ignorance.) 

It is enforced via Python packaging methods but downstream redistributors do whatever they want. :man_shrugging: 

As an update to Till's list for future googlers, here's the current version spec:

    urllib3 >= 1.21.1, <= 1.24
    chardet >= 3.0.2, < 3.1.0

Notably, `chardet` is in the early 3.x range. You can copy/paste that into your requirements.txt, no changes needed.

Follow the source to see what has changed since I posted this: https://github.com/requests/requests/blob/master/requests/__init__.py#L57-L71
gevent 1.4.0 and urllib3 1.22 does not work with requests 2.20.1

## Expected Result

My python module is executed successfully with the following:

gevent 1.4.0, urllib3 1.22  work with requests 2.10.0 with no error

## Actual Result

but with 

gevent 1.4.0, urllib3 1.22  with requests 2.20.1 gives

  File "/home/abc/xyz/lib/python2.7/site-packages/urllib3/util/wait.py", line 33, in wait_for_read
    return _wait_for_io_events(socks, EVENT_READ, timeout)
  File "/home/abc/xyz/lib/python2.7/site-packages/urllib3/util/wait.py", line 22, in _wait_for_io_events
    with DefaultSelector() as selector:
  File "/home/abc/xyz/lib/python2.7/site-packages/urllib3/util/selectors.py", line 581, in DefaultSelector
    return _DEFAULT_SELECTOR()
  File "/home/abc/xyz/lib/python2.7/site-packages/urllib3/util/selectors.py", line 394, in __init__
    self._epoll = select.epoll()
AttributeError: 'module' object has no attribute 'epoll'

There is absolutely no changes in our code and gevent.monkey.patch_all() is the first statement before any imports.

## System Information

    $ python -m requests.help

{
  "chardet": {
    "version": "3.0.4"
  }, 
  "cryptography": {
    "version": ""
  }, 
  "idna": {
    "version": "2.6"
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.15"
  }, 
  "platform": {
    "release": "3.10.0-693.2.2.el7.x86_64", 
    "system": "Linux"
  }, 
  "pyOpenSSL": {
    "openssl_version": "", 
    "version": null
  }, 
  "requests": {
    "version": "2.21.0"
  }, 
  "system_ssl": {
    "version": "1000105f"
  }, 
  "urllib3": {
    "version": "1.22"
  }, 
  "using_pyopenssl": false
}



You should try upgrading to a newer version of urllib3. When we first added selectors we had many people coming to us with monkey-patching problems. Essentially due to incorrect import ordering urllib3 used to detect that `select.epoll` was available but then gevent patches it out and then when we go to use `select.epoll` it's no longer there and we get this error. This is fixed in newer versions of urllib3 (even though it's not *really* our fault).

2.20.1 isn't supported any longer

Works with the following:

"requests": {
    "version": "2.21.0"
  }, 
  "urllib3": {
    "version": "1.24.1"
  },
  "gevent": {
    "version": "1.4.0"
  } 

Thanks !
CURL command works from command line but fails when using requests

Please refer to our [Stack Overflow tag](https://stackoverflow.com/questions/tagged/python-requests) for guidance.


Using CURL POST works in UBUNTU command line works but does not work from a python script 
**Curl command:**
```
curl -X POST \
            https://wstest3.edensprings.com/API/mobile/GetData \
            -H 'cache-control: no-cache' \
            -H 'content-type: application/json' \
            -H 'postman-token: GGGGG-HHHHH' \
            -d '{
                    "ApplicationName": "AAAAA",
                    "ApplicationCode": "BBBBB",
                    "LanguageCode": "HE",
                    "Country": "IL",
                    "Market": "IL",
                    "CustomerNumber" : "CCCCC",
                    "PhoneNumber" : "DDDDD",
                    "Last4Digits" : "EEEEE",
                    "AddressID": "FFFFF"
            }'

```
**Python script**
```
url='https://wstest3.edensprings.com/API/mobile/GetData'
payload ={'ApplicationName':'AAAAA',
                    'ApplicationCode': 'BBBBB',
                    'LanguageCode': 'HE',
                    'Country': 'IL',
                    'Market': 'IL',
                    'CustomerNumber' : 'CCCCC',
                    'PhoneNumber' : 'DDDDD',
                    'Last4Digits' : 'EEEEE',
                    'AddressID': 'FFFFF'
                }
headers={'cache-control': 'no-cache',
                     'content-type': 'application/json',
                     'postman-token': 'GGGGG-HHHHH'}
r = requests.post(url, data=payload, headers=headers)

```
**The error that is returned by a remote server to the requests.post() is :**
{"Message":"An error has occurred.","ExceptionMessage":"Object reference not set to an instance of an object.","ExceptionType":"System.NullReferenceException","StackTrace":"   at EdenSprings.Gateway.WebApiControllerBase.Process[RequestType,ResponseType](String MethodName, RequestType Request, Action`2 Method, Boolean UseAuthentication) .....



The problem was solved. It was encoding utf8 reasons...
API reference doesn't include Response.raw

Look under http://docs.python-requests.org/en/master/api/#requests.Response - `raw` attribute is not listed, even though it's described in http://docs.python-requests.org/en/master/user/quickstart/#raw-response-content , etc.

## Expected Result

The API reference should be complete. 

## Actual Result

See above.

## Reproduction Steps

See above.

## System Information

The model.py has it like this (f37daf25006a4b475b1bea90fecdd035146dbc3b):

~~~
        #: File-like object representation of response (for advanced usage).
        #: Use of ``raw`` requires that ``stream=True`` be set on the request.
        # This requirement does not apply for use internally to Requests.
        self.raw = None
~~~

Note that last comment line doesn't start with `#:`. Maybe that causes the entire comment block to **not** be treated as doc-comment and the field is excluded from docs.


Ping?