Requests on pyth in 3 doesn't work the same as it does in python 2

I'm having a problem with making an `OPTIONS` request. If I run the following code in `python 2.7` with `requests 2.7.0` it works fine and I get a `204` back. If I run it with `python 3.8.5` I get a `403`:
```
import requests
from requests_toolbelt.utils import dump

def print_raw_http(response):
    data = dump.dump_all(response, request_prefix=b'', response_prefix=b'')
    print('\n' * 2 + data.decode('utf-8'))

headers = {
    'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_2 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Mobile/15B202 NETGEAR/v1 (iOS Vuezone)',
}
session = requests.Session()
r = session.options('https://ocapi-app.arlo.com/api/auth', headers=headers)
print_raw_http(r)
r.raise_for_status()
```

```
$ python -V
Python 2.7.16

$ python3 -V
Python 3.8.5

$ pip freeze | grep requests
requests==2.7.0

$ pip3 freeze | grep requests
requests==2.7.0
```

`python 2.7`:
```
$ python blah.py 


OPTIONS /api/auth HTTP/1.1
Host: ocapi-app.arlo.com
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_2 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Mobile/15B202 NETGEAR/v1 (iOS Vuezone)
Content-Length: 0


HTTP/1.1 204 No Content
```

`python 3.8.5`:
```
$ python3 blah.py 


OPTIONS /api/auth HTTP/1.1
Host: ocapi-app.arlo.com
User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_2 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Mobile/15B202 NETGEAR/v1 (iOS Vuezone)
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Length: 0


HTTP/1.1 403 Forbidden
```

Works fine with cURL too:
```
curl -vvvv -X OPTIONS "https://ocapi-app.arlo.com/api/auth" --output --http1.1 --no-alpn --no-npn -H "Host: ocapi-app.arlo.com" -H "Connection: keep-alive" -H "Accept-Encoding: gzip, deflate" -H "Accept: */*" -H "User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_2 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Mobile/15B202 NETGEAR/v1 (iOS Vuezone)" -H "Content-length: 0"
```
```
> OPTIONS /api/auth HTTP/1.1
> Host: ocapi-app.arlo.com
> Connection: keep-alive
> Accept-Encoding: gzip, deflate
> Accept: */*
> User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_2 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Mobile/15B202 NETGEAR/v1 (iOS Vuezone)
> Content-length: 0
> 
< HTTP/1.1 204 No Content
```


When pasting code blocks it is much easier to read if you use three `` tics around the code block. See [these docs](https://docs.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks) for more information.


Yes, I know how to use markdown. That was a copy/paste fail while doing this from my phone.

See here if you want a formatted version: https://github.com/psf/requests/issues/5801#issuecomment-828925212

Requests 2.7.0 is ancient and unsupported

@sigmavirus24 I'm a little confused about the versioning. According to pypi, requests is on [2.25.1](https://pypi.org/project/requests/)? What is the lastest stable version of requests?

@jeffreydwalter Your pip freeze in the original post indicates 2.7.0, not 2.25.1. Is this the case?

@sethmlarson, I was using 2.7.0, but I just installed requests==2.25.1, and the problem persists.

@sethmlarson @sigmavirus24  can you please reopen this issue. This issue is relevant and happens with 2.25.1 as well.

Thank you. Now that we're back in business, any thoughts about what the problem might be? The sample script I provided illustrates the problem.

It seems like that requires auth. Can you reproduce this with some other website or just Arlo?

It's an OPTION request. If you run my test script in python 2.7 (or curl)
you'll see you get back a 2XX response and a 403 in python 3.x.

On Mon, May 24, 2021, 5:43 PM Ian Stapleton Cordasco <
***@***.***> wrote:

> It seems like that requires auth. Can you reproduce this with some other
> website or just Arlo?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/psf/requests/issues/5817#issuecomment-847397642>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAEKOBZRXO2INN7JSKAW4STTPLI7TANCNFSM446AL4DQ>
> .
>


So, if you look at the response you're receiving @jeffreydwalter, you can see you're getting a CAPTCHA challenge from Cloudflare (not Arlo) because you've been identified as a bot.

I did a couple quick checks and the issue _appears_ to be related how Requests is interacting with Urllib3's cipher setting. Cloudflare is doing fingerprinting and anything using Requests on Python 3.8+ is getting auto blocked. I happened upon this finding by downgrading urllib3 to 1.24 which works. However, both 1.23 and 1.24.1 do not.

This is the [commit](https://github.com/urllib3/urllib3/pull/1463/) that was added back to 1.24.1 which fixed accidental removal of setting a default cipher in 1.24.0. I'd need to sit down for a minute to work out what we're passing into urllib3 for Python 3.8. This issue isn't reproducible using urllib3 alone, so it's specific to Requests.

There's also a question of whether we want to be in the business of trying to circumvent safety measures on services that have chosen to set them up. I'll do some more investigation if I find time but this sits pretty low on the priority list.

That's fair. I appreciate you illuminating the issue. I assume Arlo put
that in place to block people from using python to automate management of
their personal cameras. There are 3-4 python libraries out there that
copied my library and all of them have been thwarted by this. I'd venture
there are a couple thousand users with broken home automation.

I'm happy to take a look at the code if you can point me in the right
direction. If not and you don't think this will get fixed in a reasonable
amount of time, I can rewrite my library to use httpx or something else.

Thanks again!

On Mon, May 24, 2021, 6:49 PM Nate Prewitt ***@***.***> wrote:

> There's also a question of whether we want to be in the business of trying
> to circumvent safety measures on services that have chosen to set them up.
> I'll do some more investigation if I find time but this sits pretty low on
> the priority list.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/psf/requests/issues/5817#issuecomment-847426348>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAEKOBYP73EY5J7A3ECLFA3TPLQXXANCNFSM446AL4DQ>
> .
>


@nateprewitt I can confirm that downgrading my urllib3 to 1.24 makes the issue go away. I think this is worth digging into because I don't think it's just a case of CloudFlare blocking use of python or requests. There's probably a subtle protocol issue at play which is causing them to reject the requests.

Yeah this isn't something that Requests can do much about, you're being detected as "requiring additional auth" but the goalposts for what individual services like Cloudflare will decide can change any day so closing this as not actionable by our team. Thanks for the report!

@sethmlarson what are you saying? You guys made a change to `urllib3` and/or `requests` that caused this issue. Downgrading to `urllib3` to `1.24` fixes the issue. The thing you can do about it is, fix the regression...