Repository: psf/requests

Issue #2949: Session's Authorization header isn't sent on redirect
Created: 2015-12-28T21:44:14Z
I'm using requests to hit developer-api.nest.com and setting an Authorization header with a bearer token. On some requests, that API responds with an 307 redirect. When that happens, I still need the Authorization header to be sent on the subsequent request. I've tried using requests.get() as well as a session.
I suppose I could work around this by not allowing redirects, detecting the 307 and then issuing the new request myself but I'm wondering if this is a bug. Should I expect that the Authorization header would be sent on all requests made within the context of a session?
In [41]: s = requests.Session()

In [42]: s.headers
Out[42]: {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Connection': 'keep-alive', 'User-Agent': 'python-requests/2.7.0 CPython/3.4.3 Darwin/15.2.0'}

In [43]: s.headers['Authorization'] = "Bearer <snip>"

In [45]: s.get("https://developer-api.nest.com/devices/thermostats/")
Out[45]: <Response [401]>

In [46]: s.get("https://developer-api.nest.com/devices/thermostats/")
Out[46]: <Response [200]>

In [49]: Out[45].history
Out[49]: [<Response [307]>]

In [50]: Out[46].history
Out[50]: []

In [51]: Out[45].request.headers
Out[51]: {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Connection': 'keep-alive', 'User-Agent': 'python-requests/2.7.0 CPython/3.4.3 Darwin/15.2.0'}

In [52]: Out[46].request.headers
Out[52]: {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Connection': 'keep-alive', 'User-Agent': 'python-requests/2.7.0 CPython/3.4.3 Darwin/15.2.0', 'Authorization': 'Bearer <snip>'}

Duplicate issue #4301: Sessions apparently corrupting header values on subsequent requests
Created: 2017-09-22T13:02:09Z
After reading http://docs.python-requests.org/en/master/user/advanced/#session-objects I tried setting request headers on a session to ensure that all requests made using that session object would include those headers with their values unchanged. However, I am running into a strange corruption issue that I would like to share with you.
Expected Result
That the X-Api-Key header I set on the session would be correctly sent to the server, unchanged, on all requests made using the session object.
Actual Result
For some reason, the value of the X-Api-Key header is being changed and causing a 401 error when authenticating to the API endpoint I am trying to contact.
Reproduction Steps
import requests

print requests.__version__

s = requests.Session()
s.headers.update({
    "X-Api-Key": '<redacted>TU=',
    "Accept-Encoding": "gzip, deflate",
    "User-Agent": "magnet-sdk-python",
    "Accept": "application/json"
})

response = s.request('GET', 'https://api.niddel.com/v2/me')
print response.status_code

response = s.request('GET', 'https://api.niddel.com/v2/me')
print response.status_code
The expected output for this is:
2.18.4
200
200

What I consistently get instead is:
2.18.4
200
401

More importantly, on the server-side logs I enabled logging of the first and last few characters of the API key for troubleshooting purposes. On the first case, they match the (valid) API key I wrote on the test code. In the failed request, the last few characters inexplicably changed from xTU= to +w==.
So for some reason the X-Api-Key header contents are being corrupted on the 2nd and later requests of a session. Curiously enough, the same thing does not happen if I perform completely separate requests:
import requests

headers = {"X-Api-Key": '<redacted>TU=',
           "Accept-Encoding": "gzip, deflate",
           "User-Agent": "magnet-sdk-python",
           "Accept": "application/json"}

response = requests.request('GET', 'https://api.niddel.com/v2/me', headers=headers)
print response.status_code

response = requests.request('GET', 'https://api.niddel.com/v2/me', headers=headers)
print response.status_code

Which gives me two 200 status codes, as expected.
I thought long and hard about whether this could be a server-side error related to multiple requests on persistent connections. But I have plenty of code that uses libcurl and also does requests like these using persistent connections at a large scale, and this error never manifested itself. So I am thinking this might be an issue on requests (or perhaps urllib3).
System Information
$ python -m requests.help

{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": ""
  },
  "implementation": {
    "name": "CPython",
    "version": "2.7.10"
  },
  "platform": {
    "release": "16.7.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.18.4"
  },
  "system_ssl": {
    "version": "9081df"
  },
  "urllib3": {
    "version": "1.22"
  },
  "using_pyopenssl": false
}

Closing comment (2017-10-06T10:19:27Z): I may be hitting this problem as well. Basically I do (pseudocode):
url = 'https://remote.com/v0/api'
headers = {'Authorization': 'Bearer <access_token>'}

while True:

    client = requests.Session()
    client.headers.update(headers)
    client.stream = True

    response = client.get(url, params=params, timeout=timeout)

    if response.status_code != 200:
        time.delay(10)
        continue
    
    for line in response.iter_lines():
        if not line:
            continue
        process_line(line)
This works all week just fine, but once in a week (on Monday morning), I start receiving this 401 response status codes non-stop ("Insufficient authorization to perform request."). Now, if I kill the process and restart it it will work just fine (i.e.: no code change, same headers, same API URL, same params...).
Note that I am actually creating a new Session after each failure, not reusing the old one.
Could it be related to this issue? Do not know if it could be related to #2949 instead (or as well).
@asieira Which workaround did you use?