Raibek/requests-github-issues huggingface dataset:
chardet is licensed under LGPL

`chardet` is a dependency of `requests` and is licensed under the LGPL 2.1.

Aren't GPL and LGPL components incompatible with the Apache 2.0 license used by `requests`, as per https://www.apache.org/legal/resolved.html#category-x and https://www.apache.org/licenses/GPL-compatibility.html?


@asieira it really depends on how you interpret the language.

First, you're linking to a section titled: `Which licenses may NOT be included within Apache products?` I believe that means projects developed specifically by the ASF not under the Apache License.

LGPL might cause license issues if we were modifying chardet and including that in Requests. That would affect our licensing.

FWIW, in case anyone else finds this closed issue, https://github.com/psf/requests/issues/4848#issuecomment-544558645 lists an MIT licensed alternative to `chardet`.
Requests doesn't respect socket default timeout ( socket.setdefaulttimeout() )

urllib3 and Requests has different semantic when handling unspecified timeout (which means no keyword arg given)

## Actual Result

**urllib3** 
- when timeout is None -> no timeout
- when timeout is not given -> internal _Default objects is used which leads to `socket.getdefaulttimeout()`
see https://github.com/shazow/urllib3/blob/master/urllib3/connectionpool.py#L373

**requests**
both None and not specified timeout is handled in same way. urllib3 `Timeout` object is created
with None values -> no timeout at all while `socket.getdefaulttimeout()` is ignored.
see https://github.com/requests/requests/blob/master/requests/adapters.py#L431
timeout arg is always passed into underlaying urllib3

## Expected Result

I think it should be better handle it same as urllib3. When timeout is not specified on request level then don't replace it with infinite timeout but just pass unspecified to timeout to urllib3

## System Information

reported on requests version 2.18.4


Thanks, this is a known issue and should be resolved in 3.0.
PATCH request with form-data file has wrong content-length

I have a script that submits either a POST or PATCH request with a particular binary file. It fails when using PATCH but is successful using POST. Testing with Postman gives expected results.

## Expected Result

Post & Patch with same content-length

## Actual Result

Patch method has much smaller content-length (not sure if being calculated wrong or if the body is malformed)

## Reproduction Steps

```python
import requests

testFile = open('testfile.png', 'rb')

postResp = requests.post(
    "https://httpbin.org/post", 
    files = {'file': testFile}
)

patchResp = requests.patch(
    "https://httpbin.org/patch", 
    files = {'file': testFile}
)

print("Post content length:" + postResp.headers['content-length'])
print("Patch content length:" + patchResp.headers['content-length'])
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.3"
  },
  "platform": {
    "release": "7",
    "system": "Windows"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "100020bf"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": false
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

Requests consumes the file on the first request which means that the file object is at the end of the file and there's nothing for us to send in the second request. You either need to reopen it or `seek(0, 0)`. Cheers!
If it possible to let requests stop (but don't raise Exception) when it get to the max redirection limit?

When I use **requests**, I find that you can't let **requests** stop when redirected a fixed times.
But I think someone(like me) will need headers or something else among those redirections.

## Expected Result
If I set a flag to let requests.session stop when it comes to the max redirection limit, and then, It won't raise a **TooManyRedirects**, but will stop and let you could get the headers or anything in this request.

## Actual Result

It will raise a **TooManyRedirects** Exception.

## Reproduction Steps
It will work like that:
```python
import requests

session  = reqeusts.Session()
session.max_redirects = 3
session.stopflag = True    # flag that will let redirects stop when it comes to max redirect time.
r = session.get(will_redirect_many_times_url)    # it will stop when redirect 3 times
# And then I can get the infomation at this request.
h = r.headers
```
Actually I modified my requests a little bit, 

I add that code:

```python
if i == self.max_redirects:
    break
```
before:
```python
if i >= self.max_redirects:
    raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects)
```
At `requests/session.py` at 113 lines
and It will stop when comes to max redirect limit.

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  }, 
  "cryptography": {
    "version": "1.2.3"
  }, 
  "idna": {
    "version": "2.6"
  }, 
  "implementation": {
    "name": "CPython", 
    "version": "2.7.12"
  }, 
  "platform": {
    "release": "4.4.0-98-generic", 
    "system": "Linux"
  }, 
  "pyOpenSSL": {
    "openssl_version": "1000207f", 
    "version": "0.15.1"
  }, 
  "requests": {
    "version": "2.18.4"
  }, 
  "system_ssl": {
    "version": "1000207f"
  }, 
  "urllib3": {
    "version": "1.22"
  }, 
  "using_pyopenssl": true
}
```

If you need the information from the most recent response you should do

```py
try:
    r = session.get(...)
except requests.exceptions.TooManyRedirects as exc:
    r = exc.response
```

because we include the most recent response on that exception. 🎉 

We're in a feature-freeze and we're only adding features that are absolutely necessary. This feature can be achieved in the above manner as well as a few other ways, so this feature really isn't necessary or entirely desirable to the 95% of our users.
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)

 i am facing `requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)` issue.
i am using python 

> v2.7.13

 , requests 

> v2.7.0.

my service is started with https using bottle-paste. i have created certificates using openssl. for server-side i am using **.pem** and from client-side i am using **.pkcs12**. 
and my service using **OpenSSL** . i am using **sslcontext** to start service `SSL.Context(SSL.SSLv23_METHOD).`
and as you guys know requests module using ssl.
**same service can hit from java , postman** with using client certificates( .pkcs12 ). not sure why is it failing here ? can any one help me here ??

Python's ssl module does not support PKCS12 files. You have to extract the PKCS12 and dump the key and cert to disk. You may also need to dump the CA and intermediate chain.

Something like this should do the trick. ``cert.pem`` contains key, cert and maybe chain.

```
$ openssl pkcs12 -in file.p12 -out client.pem -chain -nodes
```

Thanks for that @tiran. I think this should help you understand why your having trouble @munaAchyuta. Cheers!

Thanks @tiran for your solution. Yes it solved my issue. Thanks for responding.
Empty file stream should be set to Content-Length:0

I have trouble in uploading empty files  to AWS S3 using requests, the point is that AWS S3 doesn't support Transfer-Encoding: chunked to upload files.

Dealing with the file pointer(fp) which points to an empty file, requests  set it to Transfer-Encoding: chunked. In fact, the length of empty file can be calculated by the function super_len() without raising exception, the length is zero, but requests treats it same with None(caused by exceptions)!
```python
try:
    length = super_len(data)
except (TypeError, AttributeError, UnsupportedOperation):
    length = None
```
I touched an empty file named empty.txt, and the result is 'Transfer-Encoding': 'chunked' instead of 'Content-Length': '0'. In this situation, i think the result should be 'Content-Length': '0', since the length of the file can be calculated!
```python
>>> import requests
>>> fp = open('empty.txt', 'rb')
>>> r = requests.put('http://httpbin.org/put', data=fp)
>>> r.request.headers
{'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'User-Agent': 'python-requests/2.18.4'}
>>> fp.close()
'''


The travis have failed, the trouble is shown in the snip code in super_len(), requests treate None(length can't be determined automatically) to zero. So, how can i get the header set to Content-Length:0, when the file is empty, but the total_length can be calculated out, the total_length is  absolutely determined 0!

Do you agree that return None to distinguish between None and zero!
```
if total_length is None:
    total_length = 0
    #return None
```

Sorry, this behaviour is intentional. Some file like objects return a length of zero but do, in fact, contain data. For example, Unix named pipes. That is why this behaviour was introduced. Any fix will need to continue to support that use-case.
requests.get() always pumps out errors in my Spyder3.2.4

Summary.

I was trying to get data from a website by using requests module in my spyder3.2.4.
I used requests.get() to realize the goal.
It worked well. But, recentely, when I used requests.get(), spyder would pump out errors.
Really confusing.


What you expected.

## import requests
r = requests.get(url)
r.status_code

200

What happened instead.

**AttributeError: module 'requests' has no attribute 'get'**



```python
import requests

```

## System Information

    $ python -m requests.help

```
<paste here>
```

This command is only available on Requests v2.16.4 and






 greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

I have a screenshot of the problem. But I cannot upload it..... When I tried, it always told me that something went really wrong, and it cannnot process that file! 
pity....
help me..... The problem is abusing me .....

@gfkdcadet, this is almost certainly an issue with you having a file name "requests.py" in your project that is causing namespace conflicts.

If not, please either open an issue on Stackoverflow or you can email me at the email in my profile. I'm going to close this out since it's not a defect with Requests.

Sorry for my ignorance. Yes, as u said, it is the problem of the inproper file name "requests.py", which caused namespace conflicts.@nateprewitt
Connection timeout doesn't work for chunk-encoded requests

Original issue was #1422, but I was able to reproduce in 2.18.4.

## Expected Result

Chunk-encoded requests timeout after the specified timeout if the server does not respond.

## Actual Result

Request holds connection open and never times out.

## Reproduction Steps

First, run a test server that accepts POST requests and waits 10 seconds before responding:
```python
from http.server import BaseHTTPRequestHandler, HTTPServer
import time

class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        time.sleep(10)
        self.send_response(200)
        self.end_headers()

if __name__ == '__main__':
    httpd = HTTPServer(('', 8000), Handler)
    print('Listening on :8000')
    httpd.serve_forever()
```

Then, in another REPL, verify timeout works for non-chunk-encoded requests:

```python
import requests
requests.post('http://localhost:8000', timeout=2.0, data='hello'.encode('utf-8'))
````
A timeout exception should be thrown.

Then, verify timeout does not work for chunk-encoded requests:
```python
def gen():
    yield 'hello'.encode('utf-8')
requests.post('http://localhost:8000', timeout=2.0, data=gen())
```
A `<Response [200]>` will be returned, indicating that the timeout did not work properly (a timeout exception _should_ have been thrown instead).

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.3"
  },
  "platform": {
    "release": "4.4.0-43-Microsoft",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "1000207f"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": false
}
```

## Workaround

Insert the following on [line 454 of requests/adapters.py](https://github.com/requests/requests/blob/master/requests/adapters.py#L454):
```python
low_conn.timeout = timeout.connect_timeout
```

This would be resolved by moving to use urllib3's chunked encoding logic as well. I concur with @sigmavirus24's notes on another issue: if we can get some socket-level tests of our chunked handling, we can move to using urllib3's logic to resolve this issue.

In the consequence of that bug, when using with sessions, the chunked request have a timeout from the http request previously executed on that connection (!) if any... I've spent some time to explain why sometimes timeout happened and sometimes not.

@ddzialak comment is important: it's a nasty issue to debug.

I assume this is fixed by #5128, merged but not released yet.
@nateprewitt when can we expect a new maintenance release?


Requests is now using urllib3's chunking API
Find cause of status code 0

Hi All,

I'm trying to debug why sometimes my requests fail (Doing some perf test through `locust.io`) but I can't find the cause. My nginx logs have no error attached and what I see is requests giving status code 0.

I thought it could be timeout but I timed the request and I see <1sec.

Is there a way for me to find out why I'm getting this behavior? 

I'm basically getting `status_code==0` but I can't find a reason to that.

I'm using python 2.7 and locust to run the perf test

I believe we pull the status code [here](https://github.com/requests/requests/blob/d1fb1a29ab949223d8d64797e0fcc9a7d2690483/requests/adapters.py#L267) which means fundamentally this is a urllib3 bug. Requests defaults the status code to `None` but has specific handling for [this case](https://github.com/requests/requests/blob/d1fb1a29ab949223d8d64797e0fcc9a7d2690483/requests/models.py#L820). That aside, is this a non-exceptional case? I've never seen a status code of 0 for a non-exceptional case.

I see. And do you know any way for me to debug the reason why urllib3 is failing to do something?

Looking at `response.raw` would be a good place to start. You didn't really answer my question though: Is the response object you're receiving coming from an exception that's been handled?

I believe so.

This is the line:
https://github.com/locustio/locust/blob/a4c377efc7914e8d90701d9226a5cd843859b790/locust/clients.py#L161

Did you notice https://github.com/locustio/locust/blob/a4c377efc7914e8d90701d9226a5cd843859b790/locust/clients.py#L164 ?

Closing as a downstream bug.
pandas.read_html(url)  

Summary.

## Expected Result

import pandas as pd
for i in range(1,10):
    url="https://myip.ms/browse/sites/{0}/ipID/23.227.38.0/ipIDii/23.227.38.255/sort/2/asc/1/".format(i)
    dfs=pd.read_html(url)
    data=dfs[13]
    select_index=data.index[data.index%2==0]
    jg=data.iloc[select_index,:7]
    jg.to_excel("d:\{}.xls".format(i))

## Actual Result

don't need one by one，get all！

What a wonder！

I hope you can try

I believe you've filed this against the wrong project. I don't see anything related to this project.
Could we set the request json's dump charset?

As you know, I use the requests  post some data to the server, my JSON data is
```
{'name':'张三'} 
```
## Expected Result
origin json data,like {'name':'张三'}

But I catch the  package, it translated into 
## Actual Result
```
{'name':'\u5f20\u4e09'}
```
But the server couldn't recognize it.
## Reproduction Steps

```python
import requests
request.post('http://example.com/xxx',json={'name':'张三'})
```

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.2"
  },
  "platform": {
    "release": "16.7.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "100020df"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": false
}

```
I trace the source code, I found requests call the json.dumps
in models.py at  about line 454,
```python 
        if not data and json is not None:
            # urllib3 requires a bytes-like body. Python 2's json.dumps
            # provides this natively, but Python 3 gives a Unicode string.
            content_type = 'application/json'
            body = complexjson.dumps(json)
            if not isinstance(body, bytes):
                body = body.encode('utf-8')

        is_stream = all([
            hasattr(data, '__iter__'),
            not isinstance(data, (basestring, list, tuple, collections.Mapping))
        ])
```
the dumps haven't any parameter, so how could I set the parameter ensure_ascii?


You can call `json.dumps` yourself to get that effect. We don't provide anything other than the 98% use case. This is an advanced use-case and we expect that you can do this yourself (including setting the `Content-Type`).

Cheers,
Ian

@sigmavirus24 Could you please reconsider that? I am sure non-ascii data usage is well beyond 2%.

I could write a patch if you guys agree: i see 2 possible solutions adding a parameter `json_unicode` (where json dumps would be called with `ensure_ascii=False`, or an extra `json_ensure_ascii`. 

@schettino72 quoting myself:

> This is an advanced use-case and we expect that you can do this yourself (including setting the Content-Type).

In other words:

```py
requests.request(method, url, data=json.dumps(...).encode(...), headers={'Content-Type': ...})
```

The API is in feature-freeze so we're not adding new keywords.
HTTPDigestAuth Loosing Credentials in Redirect

Loosing credentials over redirect
`https://e4ftl01.cr.usgs.gov/SRTM/SRTMGL3.003/2000.02.11/N01E041.SRTMGL3.hgt.zip`

Cannot send credentials to the website.

## Actual Result

401 - Unauthorized :angry: 

## Reproduction Steps

```python
import requests
from requests.auth import HTTPDigestAuth

cred = ('usr', 'pwd')
s = requests.Session()
s.auth = HTTPDigestAuth(*cred)

r = s.get('https://e4ftl01.cr.usgs.gov/SRTM/SRTMGL3.003/2000.02.11/N01E041.SRTMGL3.hgt.zip')
>>> <Response 401>
print(r.headers)
>>> {'Server': 'nginx/1.10.2', 'Date': 'Wed, 22 Nov 2017 13:38:23 GMT', 'Content-Type': 'text/html; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'X-Frame-Options': 'SAMEORIGIN', 'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'WWW-Authenticate': 'Basic realm="Please enter your Earthdata Login credentials. If you do not have a Earthdata Login, create one at https://urs.earthdata.nasa.gov//users/new"', 'Cache-Control': 'no-cache', 'X-Request-Id': 'a40e5c15-2072-4ac2-832d-293db65a87d2', 'X-Runtime': '0.040991', 'Strict-Transport-Security': 'max-age=31536000'}
```

## Solution

```python
import requests
from requests.auth import HTTPDigestAuth

cred = ('usr', 'pwd')
s = requests.Session()
s.auth = HTTPDigestAuth(*cred)

r = s.get('https://e4ftl01.cr.usgs.gov/SRTM/SRTMGL3.003/2000.02.11/N01E041.SRTMGL3.hgt.zip')
r = s.get(r.url, stream=True)

with open('/tmp/file', 'wb') as f:
    for d in r.iter_content():
        f.write(d)
```

## System Information

    $ python3 -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.6.3"
  },
  "platform": {
    "release": "4.12.0-1-amd64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "1010007f"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": false
}
```

I am able to reproduce this problem with the master branch. In my case, the HTTP status code of the redirect is 307.

@miili Could you also share the output for `r.history` after the 401?

Oh, actually, my server didn't support HTTP Digest authentication. Looking at the headers of the response from the URL which you are trying to hit, I only see:
```
WWW-Authenticate: Basic realm="Please enter your Earthdata Login credentials. If you do not have a Earthdata Login, create one at https://urs.earthdata.nasa.gov//users/new"
```
I maybe wrong, but it could be the case that the server only supports Basic Auth?

@nehaljwani you're correct. `WWW-Authenticate: <type> [optional parameters]` indicates the kind of authentication that the server expects. In this case `type` is `Basic` meaning that using `HTTPDigestAuth` is inappropriate. If it wanted Digest Authentication it would look like `WWW-Authenticate: Digest realm="..." [etc]`
Get error "requests.exceptions.ConnectionError: ('Connection aborted.', error(54, 'Connection reset by peer'))"

Summary.
With python 2.7.10, I got this error but it's ok with python 3.6. I tried to ask a co-worker to try my code with his computer, it's still work with python2.7.10. It's strange
## Expected Result

## Actual Result

What happened instead.

## Reproduction Steps

```python
import requests

```

## System Information

    $ python -m requests.help
```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": "2.1.3"
  },
  "idna": {
    "version": "2.6"
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.10"
  },
  "platform": {
    "release": "16.7.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "9081df",
    "version": "0.13.1"
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "9081df"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": true
}
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

We need to know more about what’s happening here: can we get some sample code? All this means is that the remote server is closing the connection when we aren’t expecting it.

This has gone 5 days without information necessary to investigate anything. As such, I'm closing this. If you have the chance to provide us with more information, we may be able to reopen this and help investigate.
Add documentation for available status codes

There was no way to determine what actual names were available outside
of looking at the source code. They were not listed in the documentation
or accessible through the interactive help.

In addition, doing `pydoc requests.status_codes` displayed some pretty
unhelpful information - the utf-8 encoding string was included in the
module name, there was no description, and internal variables used for
initialisation leaked into the module scope:

    DATA
        code = 511
        codes = <lookup 'status_codes'>
        title = 'network_authentication'
        titles = ('network_authentication_required', 'network_auth', ...

This change prevents the internal variables from leaking, adds a
docstring (which has the side-effect of correcting the module name), and
appends information on the allowed status code names to the docstring
when the module is initialised.

The improved module documentation is then used in the API documentation
to provide another easy reference to the complete list of status codes.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4394?src=pr&el=h1) Report
> Merging [#4394](https://codecov.io/gh/requests/requests/pull/4394?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/e68ef0ecad271f335377804a747afe676d17024d?src=pr&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4394/graphs/tree.svg?token=mZGswIiXLa&src=pr&height=150&width=650)](https://codecov.io/gh/requests/requests/pull/4394?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4394      +/-   ##
==========================================
+ Coverage   89.05%   89.08%   +0.03%     
==========================================
  Files          18       18              
  Lines        2055     2062       +7     
==========================================
+ Hits         1830     1837       +7     
  Misses        225      225
```


| [Impacted Files](https://codecov.io/gh/requests/requests/pull/4394?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/status\_codes.py](https://codecov.io/gh/requests/requests/pull/4394?src=pr&el=tree#diff-cmVxdWVzdHMvc3RhdHVzX2NvZGVzLnB5) | `100% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4394?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4394?src=pr&el=footer). Last update [e68ef0e...c250701](https://codecov.io/gh/requests/requests/pull/4394?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Quickstart guide is missing section on cookies.

The quickstart guide has a section for cookies in the table of contents, but there isn't actually anything there.

http://docs.python-requests.org/en/master/user/quickstart/#cookies

@eliotxchan, is this happening consistently on page refreshes and across different browsers? I can access that page fine on multiple devices.

If it still isn't showing up, there's either a network or browser issue somewhere on your end.

Additionally, you can find the section on cookies here: https://github.com/requests/requests/blob/master/docs/user/quickstart.rst#cookies

Was actually my ad blocker blocking that. Very odd.
requests.session ignores second `verify` argument?

Am doing some testing for HTTPS certificates, and am seeing some unexpected behaviour from the requests "session" object:

```
import requests
# www.ottg.eu has a known invalid https certificate:
requests.session().get('https://www.ottg.eu', verify=True)
# raises error as expected

session = requests.session()
session.get('https://www.ottg.eu', verify=False)
# as expected does not raise, prints InsecureRequestWarning, 

session.get('https://www.ottg.eu', verify=True)
# unexpectedly, this seems to ignore the verify parameter, and it does not raise
# it also still prints the InsecureRequestWarning
```

That doesn't seem right?

tested against requests 2.18.4 on ubuntu 17.10

(just repro'd in both python2 and python3, and on windows)

maybe this is expected behaviour?  because of connection pooling?

(with apologies for the stream-of-consciousness) -- it definitely is because of connection pooling, isn't it.  behaviour surprising but maybe not unexpected after all?  i suppose a fix would be to somehow keep track of connections and changes in the `verify` argument?    probably quite a heavyweight fix for an edge case...

We started working on a fix for this but it took us down a deep and very dark rabbit hole that led to very bad places if I remember correctly. I think the PR might still be open on urllib3 but I don't think it's advisable. A similar yet different solution might be to have two `https://` connection pools in an adapter one for verified connections and one for unverified connections to hack around this ugly behaviour. (Or have a session with 3 httpadapters such that one adapter only ever handles unverified TLS connections. I think that's less elegant though)

I think this is fixed in proposed/3.0.0 with https://github.com/requests/requests/pull/3109 and https://github.com/requests/requests/pull/4173

Yup, this is fixed in 3.0.0 as I recall.

Alright, I think we can close this out since it's fixed in the 3.0.0 branch and it doesn't seem like we have any action we can take to mitigate this in the 2.x branch.

Looks like v3 is still not out over 2 years later, and I have a fix for this for v2 in PR #5406
[docs; tiny] Clarify that Response.ok will *only* return True/False

This is a tiny documentation fix, just to be super-duper explicit about the return value of `response.ok`.

(I wanted to check I could use its value in a function that’s meant to return a boolean, but the docs weren’t clear about what a non-Truthy return value would be.)

# [Codecov](https://codecov.io/gh/requests/requests/pull/4390?src=pr&el=h1) Report
> Merging [#4390](https://codecov.io/gh/requests/requests/pull/4390?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/c3f2553114fdaf83b9c646a73987bbd2f13285e1?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4390/graphs/tree.svg?token=mZGswIiXLa&width=650&height=150&src=pr)](https://codecov.io/gh/requests/requests/pull/4390?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4390   +/-   ##
=======================================
  Coverage   88.36%   88.36%           
=======================================
  Files          18       18           
  Lines        2055     2055           
=======================================
  Hits         1816     1816           
  Misses        239      239
```


| [Impacted Files](https://codecov.io/gh/requests/requests/pull/4390?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/models.py](https://codecov.io/gh/requests/requests/pull/4390?src=pr&el=tree#diff-cmVxdWVzdHMvbW9kZWxzLnB5) | `93.45% <ø> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4390?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4390?src=pr&el=footer). Last update [c3f2553...775cde0](https://codecov.io/gh/requests/requests/pull/4390?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks @alexwlchan
FileNotFoundError: [Errno 2] No such file or directory: './README.rst'

Summary.

## Expected Result

pip install request
Installed OK

## Actual Result

pip install request
Collecting request
  Using cached request-0.0.13.tar.gz
Collecting get (from request)
  Using cached get-0.0.21.tar.gz
Collecting post (from request)
  Using cached post-0.0.13.tar.gz
Collecting setupfiles (from request)
  Using cached setupfiles-0.0.50.tar.gz
Collecting query_string (from get->request)
  Using cached query_string-0.0.12.tar.gz
Collecting public (from query_string->get->request)
  Using cached public-0.0.41.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-bex9djjl/public/setup.py", line 28, in <module>
        setup_cfg_data = read_configuration(path)
      File "/tmp/pip-build-bex9djjl/public/setup.py", line 23, in read_configuration
        val = open(val).read()
    FileNotFoundError: [Errno 2] No such file or directory: './README.rst'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-bex9djjl/public/

What happened instead.

Library installation failed

## Reproduction Steps

```python
pip install request
```

## System Information

    $ python -m requests.help

```
No module named requests.help
```

This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).
![selection_999 1555](https://user-images.githubusercontent.com/30954037/32990180-bca9465a-ccf2-11e7-8025-55fa1a7d985d.png)

![image](https://user-images.githubusercontent.com/30954037/32990178-b2fb6b88-ccf2-11e7-8882-71dcc56747e2.png)


@lvallejou85 it looks like you're missing the final "s" in the package name in your install command. It should be `pip install requests`.
pip install : Unable to determine SOCKS version

i have try all the method, it just not work !

@torfhub If you really have found a problem and expect to have it fixed, what you wrote is not enough.

Start by reading [CONTRIBUTING.md](https://github.com/requests/requests/blob/master/CONTRIBUTING.md)
--
At least post here the command and the output.


Hi @torfhub,

We will need a lot more information in order to help you. Without that information, I'm going to close this, for now. If you leave a comment with information, I'll be happy to reopen this and help you investigate.

---

@HelioGuilherme66 we appreciate your attempt to help gather some basic information with @torfhub. Please, in the future, try to phrase your comments a bit more gently. @torfhub seems rather new to open source software and gruff and unwelcoming responses are not typically helpful for either them or the maintainers.
not able to extract key:value from Json 

I am able to get whole json data but not able to get specific key:values from json data (url). Have someone seen this issue ?

## Expected Result
Specific Key:Value Pair i.e ( ID : 123456)

What you expected.
I want to extract key-value from json data

## Actual Result
Getting below error

print (data["id"])
Traceback (most recent call last):
File "", line 1, in 
KeyError: 'id'
What happened instead.

## Reproduction Steps <code>

import urllib, json
import requests
from requests.auth import HTTPBasicAuth
url = requests.get('http://ac-infinibox10.com/api/rest/pools', auth=HTTPBasicAuth('test', 'test'))
data = url.json()

print(data) ###### this works

print (data['id']) ###### this does not works
==================
System information:
Python 2.7
RHEL 7

Hi - can you print output of `print(data)` ?

{u'metadata': {u'ready': True, u'number_of_objects': 3, u'pages_total': 1, u'page': 1, u'page_size': 50}, u'result': [{u'volumes_count': 1, u'updated_at': 1487594704819, u'max_extend': -1, u'allocated_physical_space': 10000007168, u'free_virtual_space': 99990000369664, u'id': 4880, u'reserved_capacity': 107374313472, u'filesystems_count': 0, u'ssd_enabled': True, u'snapshots_count': 0, u'state': u'NORMAL', u'free_physical_space': 69990000099328, u'entities_count': 1, u'owners': [], u'capacity_savings': 0, u'name': u'AMDC-IBOX04_AIX_Storage_Pool', u'virtual_capacity': 100000000376832, u'created_at': 1487594704819, u'filesystem_snapshots_count': 0, u'compression_enabled': True, u'physical_capacity_warning': 90, u'physical_capacity': 70000000106496, u'physical_capacity_critical': 95}, {u'volumes_count': 28, u'updated_at': 1508346569350, u'max_extend': -1, u'allocated_physical_space': 111404476229632, u'free_virtual_space': 78739426377728, u'id': 15285, u'reserved_capacity': 107374313472, u'filesystems_count': 0, u'ssd_enabled': True, u'snapshots_count': 0, u'state': u'NORMAL', u'free_physical_space': 88595524130816, u'entities_count': 28, u'owners': [],

Try posting it in a code block - it has been clipped here. 

import urllib, json


import requests
from requests.auth import HTTPBasicAuth
url = requests.get('http://ac-infinibox10.com/api/rest/pools', auth=HTTPBasicAuth('test', 'test'))
data = url.json()
print(data)

print (data['id'])

I feel your dictionary structure might be different from what you're expecting.

You can try: `print(data.keys())` - perhaps you need to access the `id` key at a different level. 

Thanks for you quick reply on the issue. I am new to python and need to write script to fetch set of information from json data.

I tried below command and I m getting output. How can I extract a specific data like free_physical_space=free_physical_space from json data ?

>>> print(data.keys())
[u'metadata', u'result', u'error']


Maybe something like this:

```
for result in data['result']:
    print(result['free_physical_space'])
```

Please close this issue as it is not really a `requests` bug.

Thank you so much!  It worked.
Closing the issue.
Correct HTTP -> HTML in quickstart doc

The quickstart doc apparently wanted to mention that "HTML and XML have ways of specifying their encoding in their body".

# [Codecov](https://codecov.io/gh/requests/requests/pull/4384?src=pr&el=h1) Report
> Merging [#4384](https://codecov.io/gh/requests/requests/pull/4384?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/d55de2d391d1350766221ce0aa36e670dc2a0169?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4384/graphs/tree.svg?width=650&height=150&src=pr&token=mZGswIiXLa)](https://codecov.io/gh/requests/requests/pull/4384?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4384   +/-   ##
=======================================
  Coverage   88.36%   88.36%           
=======================================
  Files          18       18           
  Lines        2055     2055           
=======================================
  Hits         1816     1816           
  Misses        239      239
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4384?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4384?src=pr&el=footer). Last update [d55de2d...7f08ad3](https://codecov.io/gh/requests/requests/pull/4384?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
How can I get session storage variable in Python requests library?

I want to get session storage variable when I call any url.
Best Regards

here is my code
```

> self.session = requests.Session()
> self.session.mount('https://', HTTPAdapter(max_retries=4))
> resp = self.session.get("https://m.betclic.it/sport/sportList/")

```
this is session storage when I visit this site
![uta](https://user-images.githubusercontent.com/24641385/32818822-82fd67e6-ca00-11e7-890b-9bdf2cfc88e2.png)


This isn't a bug or feature request and is therefore better suited to a question and answer site like [StackOverflow](https://stackoverflow.com/questions/tagged/python-requests).
utils: winreg module may not exist like on windows universal platform.

UWP has no API to get access to windows registry, so python has no winreg module on UWP.
This fixes unexpected error on UWP.

# [Codecov](https://codecov.io/gh/requests/requests/pull/4380?src=pr&el=h1) Report
> Merging [#4380](https://codecov.io/gh/requests/requests/pull/4380?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/d1fb1a29ab949223d8d64797e0fcc9a7d2690483?src=pr&el=desc) will **decrease** coverage by `0.08%`.
> The diff coverage is `66.66%`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4380/graphs/tree.svg?src=pr&width=650&token=mZGswIiXLa&height=150)](https://codecov.io/gh/requests/requests/pull/4380?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master   #4380      +/-   ##
=========================================
- Coverage   89.08%     89%   -0.09%     
=========================================
  Files          18      18              
  Lines        2062    2065       +3     
=========================================
+ Hits         1837    1838       +1     
- Misses        225     227       +2
```


| [Impacted Files](https://codecov.io/gh/requests/requests/pull/4380?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/utils.py](https://codecov.io/gh/requests/requests/pull/4380?src=pr&el=tree#diff-cmVxdWVzdHMvdXRpbHMucHk=) | `86.35% <66.66%> (-0.38%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4380?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4380?src=pr&el=footer). Last update [d1fb1a2...351ec98](https://codecov.io/gh/requests/requests/pull/4380?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@Lukasa I've updated the PR

@Lukasa done

rebased on current master
#4373, fix possible winreg value type difference


# [Codecov](https://codecov.io/gh/requests/requests/pull/4377?src=pr&el=h1) Report
> Merging [#4377](https://codecov.io/gh/requests/requests/pull/4377?src=pr&el=desc) into [master](https://codecov.io/gh/requests/requests/commit/e4fc3539b43416f9e9ba6837d73b1b7392d4b242?src=pr&el=desc) will **increase** coverage by `0.68%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/requests/requests/pull/4377/graphs/tree.svg?height=150&width=650&token=mZGswIiXLa&src=pr)](https://codecov.io/gh/requests/requests/pull/4377?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4377      +/-   ##
==========================================
+ Coverage   88.36%   89.05%   +0.68%     
==========================================
  Files          18       18              
  Lines        2055     2055              
==========================================
+ Hits         1816     1830      +14     
+ Misses        239      225      -14
```


| [Impacted Files](https://codecov.io/gh/requests/requests/pull/4377?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [requests/utils.py](https://codecov.io/gh/requests/requests/pull/4377?src=pr&el=tree#diff-cmVxdWVzdHMvdXRpbHMucHk=) | `86.72% <100%> (+3.31%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/requests/requests/pull/4377?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/requests/requests/pull/4377?src=pr&el=footer). Last update [e4fc353...7e1fb02](https://codecov.io/gh/requests/requests/pull/4377?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


It seems that `test_should_bypass_proxies_win_registry` only monkeypatches WinReg but never tests anything (no assert~). Will adding another monkeypatch there be sufficient?

Since we already have multiple parameterized tests, I simply rotate the `ProxyEnable` Windows registry key type per test to simulate the case. Also I have added an assertion to test the proxy selection as it was supposed to be there.

Looks like the tests are failing: mind addressing the test failures?

Hi @mingyuan-xia the tests are passing but the branch isn't up-to-date with master. If you can fix that up, I think we can finally accept this work. 👍 

Ok I will pull from the master and please squash my commits into one to make things more concise.
Handle HTTPDigestAuth requests that return stale=true

I'm using the requests library to communicate with a Wildfly management API. In one specific environment, I am getting sporadic 401 responses that include in the 'WWW-Authenticate' the 'stale=true' flag. My understanding of the spec on this header is that the server sees that the authentication is valid but the nonce is old so it sends a new nonce with the stale flag to signal that it wants you to try again with the newly returned nonce. 

My question is how do I do that with requests? It seems that if I can find a way to build the Authorization header I could manage sending the response myself. However, ideally, I would like to set a limit on the number of times that request would retry this with the new nonce but I don't see that this is an option. In fact, it appears it is currently limited to the 2 attempts (the original 401 and the following response). 

How do I go about handling this?

So the Requests standard Digest Handler does not handle any of this, so you'll need to write a custom auth handler. This is very straightforward, and you can use the DigestAuth handler in the repository as a jumping off point: take that code and make the edits you need.
pyinstaller make a exe file which contain requests run failed on no requests' windows

Help me, I just being crazy....
My Pyinstaller version is 3.2.1, Python 2.7.13 and I make a exe which contain requests. But it can not run on other's windows whose without requests.
So, which version should I try?

I'm afraid this is an issue with PyInstaller, not Requests: we can't really help you with PyInstaller. You'll need to take this up with the developers of PyInstaller.
On Windows, getting wrong proxy settings from Windows registry

For some windows machines (not all), a confusing Windows registry key type (`HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings \ ProxyEnable`) fails the following code in `requests/requests/utils.py`.

```
            proxyEnable = winreg.QueryValueEx(internetSettings,
                                              'ProxyEnable')[0]
            proxyOverride = winreg.QueryValueEx(internetSettings,
                                                'ProxyOverride')[0]
```
The case is if proxyEnable is a string , everything is fine. But `ProxyEnable` could be `REG_SZ` on some Windows machines (no idea why, but it happens). After user deletes this key or set if to `REG_DWORD 0x00000000(0)`, things get right. But I believe better type checks in requests will help.

First found: https://stackoverflow.com/questions/46260169/python-locationvalueerror-no-host-specified
Aggregated discussion: https://github.com/conda/conda/issues/3962


## Expected Result

Type checks. no failure for  `REG_SZ` values.


I'd be happy to merge a patch that resolves this issue.

PR: https://github.com/requests/requests/pull/4377

@mingyuan-xia Could you please open a PR with that patch? I'd really love to see that get fixed in the next release of requests.

https://github.com/requests/requests/pull/4377