Raibek/requests-github-issues huggingface dataset:
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

Hi dear community,
I am using windows 10 and python3.6. When i have multple threads(concurrent.futures ThreadPoolExecutor) making requests with the same parameter and the same proxy settings, i got this error.

>  response = requests.get(random.choice(get_urls()), proxies={"http": proxy, "https": proxy})
>   File "C:\Users\peja\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\api.py", line 72, in get
>     return request('get', url, params=params, **kwargs)
>   File "C:\Users\peja\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\api.py", line 58, in request
>     return session.request(method=method, url=url, **kwargs)
>   File "C:\Users\peja\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 508, in request
>     resp = self.send(prep, **send_kwargs)
>   File "C:\Users\peja\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 618, in send
>     r = adapter.send(request, **kwargs)
>   File "C:\Users\peja\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\adapters.py", line 502, in send
>     raise ProxyError(e, request=request)
> requests.exceptions.ProxyError: HTTPSConnectionPool(host='restcountries.eu', port=443): Max retries exceeded with url: /rest/v2/callingcode/372 (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x124BEB10>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',)))
> Unhandled exception in thread started by <bound method ProxyTools.call_url of <__main__.ProxyTools object at 0x006983F0>>
> Traceback (most recent call last):
>   File "C:\Users\peja\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3\connection.py", line 141, in _new_conn
>     (self.host, self.port), self.timeout, **extra_kw)
>   File "C:\Users\peja\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3\util\connection.py", line 83, in create_connection
>     raise err
>   File "C:\Users\peja\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
>     sock.connect(sa)
> ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
> 
Is there any solution for this?

Hi @ilkerceng, this looks like a request for assistance rather than a defect report for Requests which is the primary use of this issue tracker. Our standard policy is to have questions like these posted on [StackOverflow](https://www.stackoverflow.com). Thanks!

hi @nateprewitt, thanks for your response.
certificate verify failed

Caused by SSLError(SSLError(1, '_ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed'
In my code, i used verify=False,why

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
Reverting redirect examples back to intended URLs

A find/replace from a few weeks ago made the redirection sample not demonstrate redirection anymore

# [Codecov](https://codecov.io/gh/requests/requests/pull/4839?src=pr&el=h1) Report
> Merging [#4839](https://codecov.io/gh/requests/requests/pull/4839?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/d2962f1d55a211a4606d8387b6333de7a21e630d?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4839/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4839?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4839   +/-   ##
=======================================
  Coverage   66.79%   66.79%           
=======================================
  Files          15       15           
  Lines        1572     1572           
=======================================
  Hits         1050     1050           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4839?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4839?src=pr&el=footer). Last update [d2962f1...5fdc25b](https://codecov.io/gh/requests/requests/pull/4839?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Good spot, thanks @rabdill!

My pleasure @nateprewitt, thanks for helping maintain this project. Much appreciated.
Requests does not let me send an iterable object as a data with known size

I am sending huge amounts of data in a request via a POST (gigabytes) where I know the data length, and would like to send it to the other side in the http headers (so it can do some optimization on allocation).

Since this data it generated itself via a stream (in chunks of lots of megabytes each), I am implementing my own class that I send to the data parameter.

My options today are (all bad):
A. Don't implement a \_\_len__/len, and implement \_\_Iter__, this way I can control how much data is send per iteration (I can even send it all in one shot), but the data is sent as chunked and the other side doesn't know how big the request is.
B. Implement \_\_len__/len, but then I need to implement read, which defaults to 8196 size reads (because of httplib) which I have to fake and return more (since 8196 kills performance).
C. Use toolbelt's StreamingIterator which returns to the performance issues of B (it implements read).

A side issue as well, I don't understand why a class with \_\_len__/len and read, needs to modify \_\_len__/len to what is left after read if it's passed as a files (multipart) parameter, but not if it's sent as a data parameter (confusing).

## Expected Result

I should be able to send a object with a known length, and only \_\_iter__ implemented so it won't be limited to 8192 cases in B, and C above.


This is effectively a feature request if I'm understanding correctly. You may have noticed there was a template for that which explicitly tells you that this project is under a feature freeze. That means that our default response to all new feature requests is "No".

I think it's a bug if I send an object with both len and __iter__ (and no read) and it throws an error...

Except we explicity document the behaviour of having `__iter__` defined regardless of `len`. This is also your first mention of it throwing an error. Regardless, nothing claims to support the behaviour you're proposing which in turn makes it a feature request
Allow urllib3 1.24

A new version of has been released. Most notably it ships https://github.com/urllib3/urllib3/pull/1346

That, plus https://github.com/requests/requests/pull/4718 on our side will resolve https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-18074. We just need to release after merging this.

cc @sigmavirus24 @nateprewitt 

Hi @ofek, we’ve got comms available in #4830 and the patch for 2.20 is staged in #4835.
Allow urllib3 1.24

Changelog: https://github.com/urllib3/urllib3/blob/1.24/CHANGES.rst#124-2018-10-16

# [Codecov](https://codecov.io/gh/requests/requests/pull/4834?src=pr&el=h1) Report
> Merging [#4834](https://codecov.io/gh/requests/requests/pull/4834?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/2c6a8426aebd853966747f2c851f551c583cb21a?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4834/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4834?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4834   +/-   ##
=======================================
  Coverage   66.79%   66.79%           
=======================================
  Files          15       15           
  Lines        1572     1572           
=======================================
  Hits         1050     1050           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4834?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4834?src=pr&el=footer). Last update [2c6a842...4b5b118](https://codecov.io/gh/requests/requests/pull/4834?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hi @AymericDu, thanks for contributing to Requests. We're aware of the 1.24 release and have an announcement up in the issues detailing next steps. Once we've been able to vet the release, we'll be pushing a new version of Requests with the pin move. Thanks for your patience.
Running prepare_headers on PreparedRequest break it

After calling prepare() on a request, PreparedRequest object is returned. If its a PATCH request (GET is working as expected), and prepare_headers is called, it's not possible to send this request.

## Expected Result

Headers should be added to a prepared request and it should be possible to send it later.

## Actual Result

An exception is thrown:

```
Traceback (most recent call last):
  File "r.py", line 7, in <module>
    s.send(r)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 466, in send
    low_conn.send(hex(len(i))[2:].encode('utf-8'))
TypeError: object of type 'int' has no len()
```

## Reproduction Steps

```python
import requests

r = requests.Request('PATCH', 'http://domain.com/objects/32', json={'name': 'new name'})
r = r.prepare()

s = requests.Session()
r.prepare_headers({'Authorization': 'Token 123'})
s.send(r)
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.6"
  },
  "platform": {
    "release": "4.9.87-linuxkit-aufs",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.19.1"
  },
  "system_ssl": {
    "version": "1010006f"
  },
  "urllib3": {
    "version": "1.23"
  },
  "using_pyopenssl": false
}
```


The error occurs while `HTTPAdapter::send` method tries to read the length of `request.body`.

@mlubinmow what do you think should be happening here? I'm not convinced this is a bug in Requests but rather in using undocumented methods.

Specifically http://docs.python-requests.org/en/master/user/advanced/?highlight=PreparedRequest#prepared-requests tells you how you're expected to interact with PreparedRequests and makes no guarantees about the `prepare_*` methods on the object.

@sigmavirus24 I disagree, function is listed in Devs interface and is inconsistent, as it works for GET request. I would expect it to.. just work ;) As mentioned in an issue.

http://docs.python-requests.org/en/master/api/#lower-lower-level-classes

@mlubimow just because something is included in auto-generated documentation doesn't mean it just magically works as you expect it to.

To be clearer, the documentation I linked to explains how to actually work with the prepared request flow. If something happens to work in one case when you do something the system wasn't designed for, it doesn't mean it must work in a different case.
Follow suit of urllib3 and drop python2.6

- Already untested (?)
- Resolves https://github.com/urllib3/urllib3/issues/1456

I used [pyupgrade](https://github.com/asottile/pyupgrade) for the syntax changes.  Most of the motivation was to resolve the urllib3 issue.

Hi @asottile, thanks for the pull request. As we noted in the urllib3 issue, this is already addressed and will be in our release within the next week once the changes in urllib3 have had some time to air out.

I’ve created #4830 for visibility. Thanks again @asottile!

@nateprewitt disregarding the urllib3 issue -- is this PR not ok? does requests intend to support python2.6 (it doesn't test against it)

There’s nothing wrong with the PR, but we already did this work a year ago. We haven’t supported python2.6 in a while but hadn’t intended to forcibly break compat until 3.0.0. 

The reason we declined to merge this is it’s more work to rebase the release commits and would remove credit from the original author. It also introduces some breakages that will affect a number of our downstream dependencies which are unrelated to 2.6.
compat: py 2.7 has ordered dicts

Use it if available rather than relying on urllib3 to provide it.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4828?src=pr&el=h1) Report
> Merging [#4828](https://codecov.io/gh/requests/requests/pull/4828?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/2c6a8426aebd853966747f2c851f551c583cb21a?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4828/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4828?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4828   +/-   ##
=======================================
  Coverage   66.79%   66.79%           
=======================================
  Files          15       15           
  Lines        1572     1572           
=======================================
  Hits         1050     1050           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4828?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4828?src=pr&el=footer). Last update [2c6a842...864e82f](https://codecov.io/gh/requests/requests/pull/4828?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hi @cHYzZQo, thanks for contributing to Requests! Unfortunately, we're already addressing the urllib3 1.24 related changes in our upcoming release patch. If you upgrade to 1.24 before we move the versioning pin, we won't be able to guarantee things work. Our current plan is to have a release ready within the next week after any major bugs in urllib3 have had a chance to surface.

I see you got about 5 of these, so I'm fine with letting this drop. Unfortunately pip is a mess and depending on what order things happen in it doesn't always respect requests' pinning. Are your coming changes anywhere in public yet?

nvmd just saw the v2.20.0 PR., tx
CVE URLs update: www sub-subdomain no longer valid


Thanks @webmaven!

You're welcome @nateprewitt. Was wondering if adding the new CVE (http://cve.mitre.org/cgi-bin/cvename.cgi?name=2018-18074 eg. #4718) would also be an appropriate documentation change at this point, but would need to know next release's version number (eg. 2.19.2, or 2.20?) to include it.

@webmaven, thanks for the offer! We’re likely going to be doing a release sometime next week, and I think we’ll get all of that info bundled during the release process.  

@nateprewitt and @webmaven Do you have any insight as to whether or not this change could be applied to versions 2.6.0 and 2.7.0 and, if so, would doing so address the security concern outlined in the CVE? We're distributing those versions in a public repo and are considering just applying this change instead of forcing users through an upgrade quite far forward. Any thoughts would be much appreciated. :)

Hi @cachedout, I think you *could* apply the patch in #4718 (or a derivative) to the head of 2.6 or 2.7. We don’t have any intention to maintain that in Requests though since both of those release are approaching 4 years since release and are 13 versions behind.

If you choose to go down that path for Saltstack, we probably want to make it clear that it’s a forked version of Requests at that point. If you’re already vendoring copies though, that may not be a problem.

@nateprewitt Totally understood and thanks for the quick reply. That gives me what I need. Thanks!
Memory leaks suspected (gc objects increases)

I'm running Telegram bot and noticed the free memory degradation when running bot for a long time. Firstly, I suspect my code; then I suspect bot and finally I came to requests. :)
I used **len(gc.get_objects())** to identify that problem exists. I located the communication routines, then cleared all bot code and comes to the example that raises the count of gc objects on every iteration.

## Expected Result

**len(gc.get_objects())** should give the **same** result on every loop iteration

## Actual Result

The value of **len(gc.get_objects())** increases on every loop iteration.
```
Test N2
GetObjects len: 27959
Test N3
GetObjects len: 27960
Test N4
GetObjects len: 27961
Test N5
GetObjects len: 27962
Test N6
GetObjects len: 27963
Test N7
GetObjects len: 27964
```

## Reproduction Steps

```
token = "XXX:XXX"
chat_id = '111'
proxy = {'https':'socks5h://ZZZ'} #You may need proxy to run this in Russia

from time import sleep
import gc, requests

def garbage_info():
    res = ""
    res += "\nGetObjects len: " + str(len(gc.get_objects()))
    return res

def tester():
    count = 0
    while(True):
        sleep(1)
        count += 1
        msg = "\nTest N{0}".format(count) + garbage_info()
        print(msg)

        method_url = r'sendMessage'
        payload = {'chat_id': str(chat_id), 'text': msg}

        request_url = "https://api.telegram.org/bot{0}/{1}".format(token, method_url)
        method_name = 'get'

        session = requests.session()
        req = requests.Request(
            method=method_name.upper(),
            url=request_url,
            params=payload
        )
        prep = session.prepare_request(req)

        settings = session.merge_environment_settings(
            prep.url, None, None, None, None)
#            prep.url, proxy, None, None, None)  #Change the line to enable proxy
        send_kwargs = {
            'timeout': None,
            'allow_redirects': None,
        }
        send_kwargs.update(settings)
        resp = session.send(prep, **send_kwargs)

        # For more clean output
        gc.collect()

tester()
```

## System Information

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.3.1"
  },
  "idna": {
    "version": "2.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.6"
  },
  "platform": {
    "release": "4.15.0-36-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010009f",
    "version": "17.5.0"
  },
  "requests": {
    "version": "2.19.1"
  },
  "system_ssl": {
    "version": "1010007f"
  },
  "urllib3": {
    "version": "1.23"
  },
  "using_pyopenssl": true
}
```


In the future, please search **closed and** open issues before creating new ones that are duplicates.

Duplicate of #4601
Add auth.HTTPHeaderAuth

Since authentication using 'Authorization' header is quite popular situation, I added `HTTPHeaderAuth` class to make it easier.

With this, instead of doing this

```python
requests.get(url, header={'Authorization': 'token xxx'})
```

(this will be overwritten by netrc!)

or

```python
def auth_header(r):
    r.haeders['Authorization'] = 'token xxx'
    return r
requests.get(url, auth=auth_header)
```

you can do

```python
requests.get(url, auth=HTTPHeaderAuth('token xxx'))
```

This is easier, clearer and no overwrite by netrc file.

Thanks for sending this over. Unfortunately we're under a feature freeze and cannot accept this at this time.
Add read() to Response object.

The traditional 'read' interface is useful in some cases.
At very least it may be easier for code that expectes file-handle
like objects to just add requests support.

Due to gzip/deflate content-encoding it is not really possible for
the caller to use just use Response.raw.read().  If gzip was used
that would return compressed bytes.

Hi,
I realize the implementation probably needs some work, but I wanted to see what you think of the general idea before I polished it up.

I've been using this in a downstream project at https://git.launchpad.net/simplestreams/tree/simplestreams/contentsource.py#n356 for a while.


Hi there, Requests is currently under a feature-freeze and the `read()` method isn't really the only thing that's necessary for code or libraries that require a file-like object.

Further, it is possible to get the raw, compressed bytes from `Response.raw.read` so this functionality is an unnecessary duplication of the logic in urllib3 + extra work to provide functionality that already exists.

The problem with Response.raw.read() is that if content-encoding of gzip, it will return gzipped' content.
The user is not expecting gzipped content.

Here is attempt to use requests and 'raw.read' as you suggest.
It returns gzip compressed data even though the url opened has normal json content.  This fact is not something the user of requests should have to expect or handle.

    >>> import requests
    >>> url = "http://cloud-images.ubuntu.com/releases/streams/v1/com.brightbox:official.json"
    >>> r = requests.get(url, stream=True)
    >>> r.headers.get('content-encoding')
    'gzip'
    >>> r.raw.read(32)
    b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03\xcdWMs\x9b0\x10\xbd\xfbW0\\\x1b\x08\xb2)&\xfc\x89^z\xcf'

Here is the function that my change provides.  Now, read() returns
the uncompressed data.

    >>> r = requests.get(url, stream=True)
    >>> r.read(32)
    b'{\n   "updated": "Thu, 11 Oct 201'


With regard to feature-freeze, I don't need this to be landed right now.  I was really looking for feedback as to whether or not you were open to a 'read' interface on Response.

> With regard to feature-freeze, I don't need this to be landed right now. I was really looking for feedback as to whether or not you were open to a 'read' interface on Response.

The feature-freeze is relevant because it defaults the answer to No
Are connections reused when response is read through json() method?

From [Advanced Usage page](http://docs.python-requests.org/en/master/user/advanced/#keep-alive) (emphasis mine):

> ## Keep-Alive
>
> Excellent news — thanks to urllib3, keep-alive is 100% automatic within a session! Any requests that you make within a session will automatically reuse the appropriate connection!
> 
> Note that connections are only released back to the pool for reuse once all body data has been read; be sure to either set `stream` to `False` or **read the `content` property** of the `Response` object.

Could you please clarify if the connection would be released to the pool for reuse if the response is read though `.json()` method instead of the `content` property?

If you're not using `stream=True` the connection will always be released to the pool.

If you do use `stream=True` then anything that reads the body of the response (`.content`, `.text`, `.json()`) will release the connection upon successfully consuming the body.
Unit tests failing

Running the unit tests, many failures. Am I doing this wrong? Couldn't find docs to run the tests from source.

## Expected Result

All tests to pass

## Actual Result

Many failed

## Reproduction Steps

```
git clone https://github.com/requests/requests
cd requests
sudo make init
sudo pipenv run detox
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  }, 
  "cryptography": {
    "version": "2.2.2"
  }, 
  "idna": {
    "version": "2.6"
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.13"
  }, 
  "platform": {
    "release": "4.15.13-041513-generic", 
    "system": "Linux"
  }, 
  "pyOpenSSL": {
    "openssl_version": "1000207f", 
    "version": "0.15.1"
  }, 
  "requests": {
    "version": "2.19.1"
  }, 
  "system_ssl": {
    "version": "1010003f"
  }, 
  "urllib3": {
    "version": "1.22"
  }, 
  "using_pyopenssl": true
}

```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Is that right you use `sudo make init` and `sudo pipenv run detox`? It sounds a little bit strange to me that you need su rights for running tests.

@hook-s3c what errors do you get?

No response
Fix pep8 in docs code snippets

Fixed some pep8 issues in docs code snippets.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4816?src=pr&el=h1) Report
> Merging [#4816](https://codecov.io/gh/requests/requests/pull/4816?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/a6cd380c640087218695bc7c62311a4843777e43?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4816/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4816?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4816   +/-   ##
=======================================
  Coverage   66.79%   66.79%           
=======================================
  Files          15       15           
  Lines        1572     1572           
=======================================
  Hits         1050     1050           
  Misses        522      522
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4816?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4816?src=pr&el=footer). Last update [a6cd380...17ea176](https://codecov.io/gh/requests/requests/pull/4816?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This project doesn't observe pep8

ok, this is your problem.
import eats 600MB of RAM

Importing requests and doing nothing takes huge amount of RAM.

## Expected Result

Don't waste memory

## Actual Result

```
    MB
4.872^                                                                    #
     |                                                           @:@::::::#::
     |                                                    @::::@@@:@::::::#::
     |                                                  @@@::::@ @:@::::::#::
     |                                               ::@@@@::::@ @:@::::::#::
     |                                   @     :::@::::@@@@::::@ @:@::::::#::
     |                                   @ :::::::@::::@@@@::::@ @:@::::::#::
     |                                   @::: ::::@::::@@@@::::@ @:@::::::#::
     |                                  @@::: ::::@::::@@@@::::@ @:@::::::#::
     |                               @@:@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |                              :@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |                          :::::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |                     @:::@:: ::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |                 @@::@:::@:: ::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |            ::::@@@::@:::@:: ::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |            :: :@@@::@:::@:: ::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |        @@:::: :@@@::@:::@:: ::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |      ::@ : :: :@@@::@:::@:: ::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |    ::::@ : :: :@@@::@:::@:: ::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
     |  ::::::@ : :: :@@@::@:::@:: ::@ :@@::: ::::@::::@@@@::::@ @:@::::::#:::
   0 +----------------------------------------------------------------------->Mi
     0                                                                   653.3
```

## Reproduction Steps
```bash
valgrind --trace-children=yes --tool=massif python3.6
```

```python
import requests

```

```
ms_print massif.out.* | less
```

## System Information

    $ python -m requests.help

```
$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.3.1"
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.5"
  },
  "platform": {
    "release": "3.10.0-862.2.3.el7.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010009f",
    "version": "17.5.0"
  },
  "requests": {
    "version": "2.19.1"
  },
  "system_ssl": {
    "version": "100020ff"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": true
}
```


w/o import requests it's only ~80MB

my bad... Mi is not MB... Closing this issue. Also if anybody is interested - use massif-visualizer tool for a better understanding
How to add keystore password in requests.post

my code is 
`import requests`
`requests.post(URL,cert=(".jks/.pem file path","keystore password"))`

It is not working though


Please provide us with the output of

```
python -m requests.help
```

If that is unavailable on your version of Requests please provide some basic information about your system (Python version, operating system, etc).

"implementation":CPython,version:3.6.6
platform: release-10,system-windows
requests version:2.19.1
urlib3 version:1.23

`requests.post("https://{url}",cert=("z:/certificate.pem","z:/keystore.key","password"))`

The error is `openSSL error:[('x509 certificate routines', 'X509_check_private_key', 'key values mismatch')]`
netrc parsing is overly strict

This started out as a [bug report for httpie](https://github.com/jakubroztocil/httpie/issues/717#issuecomment-426125261) and they referred me here.

> it appears that auth via netrc is broken if `~/.netrc` includes entries that are not exactly login/password tuples. For example, I have the following entries for circle ci and heroku:
> 
> ```
> machine api.heroku.com
>   login <redacted>
>   password <redacted>
>   method interactive
> machine circleci.com
>   login <redacted>
> ```
>
> both of these entries prevent my entry for `github.com` from working with `httpie` (but curl works just fine).


## Expected Result

`https api.github.com/users` should return 200

## Actual Result

`https api.github.com/users` should returned 401

## Reproduction Steps

See summary section for `.netrc` configs that don't parse.

## System Information

    $ python -m requests.help

> I'm not sure if this is the output you wanted since I don't know enough about python to know if the requests found this way is the same one that httpie depends on.

```
{
  "chardet": {
    "version": "3.0.4"
  }, 
  "cryptography": {
    "version": ""
  }, 
  "idna": {
    "version": "2.6"
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.15"
  }, 
  "platform": {
    "release": "18.0.0", 
    "system": "Darwin"
  }, 
  "pyOpenSSL": {
    "openssl_version": "", 
    "version": null
  }, 
  "requests": {
    "version": "2.18.4"
  }, 
  "system_ssl": {
    "version": "1000210f"
  }, 
  "urllib3": {
    "version": "1.22"
  }, 
  "using_pyopenssl": false
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).


Hi there,

What happens if you do:

```py
import netrc
import os.path

netrc.netrc(os.path.expanduser('~/.netrc')).authenticators('api.github.com')
```

I ask, because [we rely on the standard library](https://github.com/requests/requests/blob/a6cd380c640087218695bc7c62311a4843777e43/requests/utils.py#L166..L210) to parse the file. That means if anything is actually overly strict then it's Python.

```
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    netrc.netrc(os.path.expanduser('~/.netrc')).authenticators('api.github.com')
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/netrc.py", line 35, in __init__
    self._parse(file, fp, default_netrc)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/netrc.py", line 88, in _parse
    file, lexer.lineno)
netrc.NetrcParseError: malformed machine entry circleci.com terminated by 'machine' (/Users/ian/.netrc, line 3)
```

In this case, I only had circleci.com and api.github.com in netrc

Your traceback shows that this is in fact not the fault of Requests and there's little for us to do to work around it.

Perhaps a more modern version of Python fixes the issue with the standard library.
Fix no_proxy being too greedy.

**Summary:** Setting `no_proxy` on `requests.com` used to enable it on `somerequests.com` also, which was unexpected and potentially dangerous. (see #4795)

Note that this delegates to urllib more often, as stated [here](https://github.com/requests/requests/issues/4795#issuecomment-425886286). I don't have the history behind `should_bypass_proxies` implementation, therefore my change is the tiniest possible (minimum thing to fix the greedy behavior). I don't know exactly why requests tries some things first and then delegates to urllib though.

Fixes #4795 

Apparently [CPython 3.4 `no_proxy` implementation was as naive as requests'](https://github.com/python/cpython/blob/cd1d5c554c40b5b348de2e77d65986aa154e4477/Lib/urllib/request.py#L2349), hence the failing build. Not sure giving the final word to `proxy_bypass` is a great idea. Can't we just use the same implementation as in [CPython 3.7](https://github.com/python/cpython/blob/655608a1112e592cd6a9155ebe774dd285f561f3/Lib/urllib/request.py#L2487)?
Decode as UTF-8 when charset is none and content-type header is "application/json"



## Actual Result

When content-type header is "application/json" and charset is none, `response.encoding` is empty.


```python:server.py
from flask import Flask, make_response, jsonify
import json
app = Flask(__name__)

@app.route('/test')
def test():
    resp = jsonify({"result":"test"})
    resp.headers['Content-Type'] = 'application/json;'
    return resp

```

```python:client.py
r = requests.get("http://localhost:5000/test")
print(r.encoding)
// empty
```


## Feature request

In RFC8259, character encoding of JSON must be UTF-8. Please ensure it is set correctly.

>JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 [RFC3629].

https://www.rfc-editor.org/rfc/rfc8259.txt

Hi @yuji38kwmt, it's important to note RFC8259 specifies that it:
> MUST be **encoded** using UTF-8

That's a directive for a client before sending JSON data outside of its ecosystem. Requests should do this [here](https://github.com/requests/requests/blob/a6cd380c640087218695bc7c62311a4843777e43/requests/models.py#L462-L468). As for receiving data, the very next line of the RFC states:
>  Previous specifications of JSON have not required the use of UTF-8
   when transmitting JSON text.

Many of these implementations still exist in the wild and are actively used. We can't make a decision to blacklist or break compatibility with those clients when a new RFC is released. Requests generally works with the philosophy of ensuring what we emit conforms to the current spec, but that we tolerate receiving many of the commonly observed deviations from it.

The encoding attribute is only used on the `text` method, as noted in the documentation so it's OK for it to be None. You should find though that calling the `json` method on your response returns a properly encoded string if it is in fact utf-8. Are you observing something different?

Thanks.

>You should find though that calling the json method on your response returns a properly encoded string if it is in fact utf-8. Are you observing something different?

I didn't know `json` method , so I used `json.loads(response.text)` insted of `json` method.
I close this issue.
Using requests with unpatched socket

Hi, 

In my python environment, I am using patched socket using eventlet. But sometimes the thread is getting stuck because of eventlet library limitation,
[https://github.com/eventlet/eventlet/issues/492](https://github.com/eventlet/eventlet/issues/492)

therefore According to event I need to use eventlet.patcher.original("requests"),  while using this I am gettting this error : 
``    requests = eventlet.patcher.original("requests")
  File "/usr/lib/python2.7/site-packages/eventlet/patcher.py", line 194, in original
    real_mod = __import__(modname, {}, {}, modname.split('.')[:-1])
  File "/usr/lib/python2.7/site-packages/requests/__init__.py", line 63, in <module>
    from . import utils
ImportError: cannot import name utils ``


When I debug it request.request shows me `func_code = <code object request at 0x7f4443cc7bb0, file "/usr/lib/python2.7/site-packages/requests/api.py", line 16>`

but socket.socket gives me "<class 'eventlet.greenio.base.GreenSocket'>".

So, is it possible pass unpatched socket (which comes from eventlet.patcher.original("socket"))  while using requests.request ?

Based on your traceback, this seems to be an issue with how `eventlet.patcher` works. This is not an issue in Requests. It would seem eventlet doesn't understand import semantics or how to use import machinery. By default Requests uses `socket.socket` (unpatched, not eventlet) so to answer your question, yes it is possible to use that. It would be even more possible if eventlet were using the import machinery appropriately.
response cookies missing

cookies not found in response

## Expected Result

`foo=bar` shown in cookies

## Actual Result

empty cookiejar

## Reproduction Steps

```python

In [14]: r = requests.get('http://httpbin.org/cookies/set?foo=bar')

In [15]: r.cookies
Out[15]: <RequestsCookieJar[]>

```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.2.2"
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.5"
  },
  "platform": {
    "release": "17.5.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "1010008f",
    "version": "17.5.0"
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "100020ff"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": true
}

```

I have figured out why, there is a redirection. The cookies were set in first response, which is a 302 redirect, and the second response set no cookies. So r.cookies is empty, but cookies are found in s.cookies. Is this a intended behavior?

It is entirely intended behaviour. I believe it is documented as well.
SSLError: [Errno 8] _ssl.c:508: EOF occurred in violation of protocol

Hi Experts,

I am getting the following error while running a python script on my Linux box.

Script: https://github.com/coreyhines/Arista/blob/master/bugalertUpdate.py

Traceback (most recent call last):
  File "./bugalertUpdate-2.py", line 84, in <module>
    result = requests.post(url, data=json.dumps(jsonpost))
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 88, in post
    return request('post', url, data=data, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 438, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 331, in send
    raise SSLError(e)
SSLError: [Errno 8] _ssl.c:508: EOF occurred in violation of protocol

Request your help on this issue.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.
How to set requests 'user-agent' header globally

I want to set requests 'user-agent' header globally,But I can't find any way to do it,Can anyone help me?


Read the documentation: http://docs.python-requests.org/en/master/user/advanced/

```
s = requests.Session()
s.headers.update({'x-test': 'true'})

# both 'x-test' and 'x-test2' are sent
s.get('http://httpbin.org/headers', headers={'x-test2': 'true'})
```






Hi @beruhan, @therumbler is correct. For future reference, we use the issue tracker for defects in Requests. Questions and requests for assistance should be directed to StackOverflow.com with the python-requests tag. Thanks!

Long time later, but is there a way to set the user agent even for implicit requests? Like if I use a function from a library that makes a request, then it won't fall under the Session if I go that way. What would be the way in that case?

@aadibajpai Quoting literally the last comment here:

> Questions and requests for assistance should be directed to StackOverflow.com with the python-requests tag.

> @aadibajpai Quoting literally the last comment here:
> 
> > Questions and requests for assistance should be directed to StackOverflow.com with the python-requests tag.

Been there, found something (https://stackoverflow.com/questions/52497739/how-to-set-requests-user-agent-header-globally), didn't help and since the question is same then a new question would be marked a duplicate :)

Thta doesn't magically make this the right place to ask questions since this is officially not the place for questions or requests for support to be asked

> Thta doesn't magically make this the right place to ask questions since this is officially not the place for questions or requests for support to be asked

Fair enough, do you have a recommendation then? 🙂
Remove dead link to unixstickers.com

The link does not link Requests stickers as the text suggests.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4804?src=pr&el=h1) Report
> Merging [#4804](https://codecov.io/gh/requests/requests/pull/4804?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/207e83f6b63250c2251f9654f31507ea5cc7edc6?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4804/graphs/tree.svg?width=650&token=mZGswIiXLa&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4804?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4804   +/-   ##
=======================================
  Coverage   66.77%   66.77%           
=======================================
  Files          15       15           
  Lines        1568     1568           
=======================================
  Hits         1047     1047           
  Misses        521      521
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4804?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4804?src=pr&el=footer). Last update [207e83f...e71396a](https://codecov.io/gh/requests/requests/pull/4804?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Issue Passing All Tests in Suite

Hello,

In my efforts to start contributing to the requests codebase, I've run into an issue with one of the test.

<img width="1306" alt="screen shot 2018-09-22 at 8 59 54 am" src="https://user-images.githubusercontent.com/1155601/45918043-6538a780-be46-11e8-9acf-44a6fc2042a4.png">

This issue seems to relate to https://github.com/requests/requests/issues/3732 in some way but I've had no luck in tracing down the exact culprit.

The two items that stand out to me are:

> pytest-httpbin server hit an exception serving request: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:661)
> attempting to ignore so the rest of the tests can run

and

> Failed: DID NOT RAISE <class 'requests.exceptions.ConnectionError'>

Steps to reproduce against my current configuration:

I'm executing these test by tying in "Pytest" in the requests directory.

Any help will be greatly appreciated.


Hi @josephlane, it sounds like you have something on your network that may be intercepting this request and routing it somewhere else. Typically, this won't resolve to a valid IP address since it's not an actual service. Since the request is not over https, we don't have a way to prevent interception from happening. I would verify you don't have some sort of network manager, proxy, or possibly a bad actor intercepting this request.
Please document how to make requests trust system SSL cert repository

The documentation explains how to use a specific file or directory of certificates to trust, but I cannot find how to make it simply trust the system set (for the current system). With older versions of requests this was the default, which made it rather easy to install some extra intranet root certs on all sorts of different machines (different Linux flavours, OSX, possibly even Windows) and have the intranet certs be trusted by everything from browsers to requests-based programs. It is unclear how to get this behaviour with the current version of requests and certifi. 


In the future, please search **closed and** open issues before creating new ones that are duplicates.