Requests 2.32.0 Not supported URL scheme http+docker

<!-- Summary. -->
Newest version of requests 2.32.0 has an incompatibility with python lib `docker`

```

INTERNALERROR> Traceback (most recent call last):

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/requests/adapters.py", line 532, in send

INTERNALERROR>     conn = self._get_connection(request, verify, proxies=proxies, cert=cert)

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/requests/adapters.py", line 400, in _get_connection

INTERNALERROR>     conn = self.poolmanager.connection_from_host(

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/urllib3/poolmanager.py", line 304, in connection_from_host

INTERNALERROR>     return self.connection_from_context(request_context)

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/urllib3/poolmanager.py", line 326, in connection_from_context

INTERNALERROR>     raise URLSchemeUnknown(scheme)

INTERNALERROR> urllib3.exceptions.URLSchemeUnknown: Not supported URL scheme http+docker

INTERNALERROR> 

INTERNALERROR> During handling of the above exception, another exception occurred:

INTERNALERROR> 

INTERNALERROR> Traceback (most recent call last):

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/docker/api/client.py", line 213, in _retrieve_server_version

INTERNALERROR>     return self.version(api_version=False)["ApiVersion"]

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/docker/api/daemon.py", line 181, in version

INTERNALERROR>     return self._result(self._get(url), json=True)

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/docker/utils/decorators.py", line 44, in inner

INTERNALERROR>     return f(self, *args, **kwargs)

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/docker/api/client.py", line 236, in _get

INTERNALERROR>     return self.get(url, **self._set_request_timeout(kwargs))

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/requests/sessions.py", line 602, in get

INTERNALERROR>     return self.request("GET", url, **kwargs)

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/requests/sessions.py", line 589, in request

INTERNALERROR>     resp = self.send(prep, **send_kwargs)

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/requests/sessions.py", line 703, in send

INTERNALERROR>     r = adapter.send(request, **kwargs)

INTERNALERROR>   File "/var/lib/jenkins/workspace/Development_sm_master/gravity/.nox/lib/python3.10/site-packages/requests/adapters.py", line 534, in send

INTERNALERROR>     raise InvalidURL(e, request=request)

INTERNALERROR> requests.exceptions.InvalidURL: Not supported URL scheme http+docker
```

## Expected Result

Normal initalization of docker client 

## Actual Result

<!-- What happened instead. -->

Stack trace posted above

## Reproduction Steps

```sh
mkvirtualenv debug_issue
pip install docker
pip install 'requests>=2.32.0'
python
```
```python
import docker
docker.from_env()
```

## System Information

    $ python -m requests.help

```json
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "3.3.2"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.7"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.12"
  },
  "platform": {
    "release": "6.5.0-28-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.31.0"
  },
  "system_ssl": {
    "version": "30000020"
  },
  "urllib3": {
    "version": "2.2.1"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}
```

<!-- This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c). -->


I believe this was introduce with this line change which started enforcing the URL scheme:
https://github.com/psf/requests/commit/c0813a2d910ea6b4f8438b91d315b8d181302356#diff-a4598bf8444e0feca489e60704c809c0dbfc47b90c4364216c931b497b3312d7R512

The issue is also being tracked over in `docker-py` repo:
https://github.com/docker/docker-py/issues/3256

Hi @joshzcold, could you clarify more about the expected functionality with a `http+docker` scheme vs calling the URI with just `http`. Is docker-py using a custom Requests adapter or was this just being passed as generated to Requests?

This appears to be a byproduct of fixing CVE-2024-35195. urllib3 is expecting the scheme to be either `http` or `https` [here](https://github.com/urllib3/urllib3/blob/b8589ec9f8c4da91511e601b632ac06af7e7c10e/src/urllib3/poolmanager.py#L323-L325) in the context construction.

It looks like there's a proposed patch for docker [here](https://github.com/docker/docker-py/pull/3257). Their sub-adapter seems to reasonably have been relying on implementing `get_connection`. We'll use this issue to track the change in #6655 regarding `get_connection` and use the docker issue linked about the docker adapter.

@nateprewitt do you think the fix in https://github.com/docker/docker-py/pull/3257 is good enough for now, or do you have a suggestion for a better way to work around this problem, at least for now? I guess it would be great if Docker SDK for Python wouldn't need to depend on internal behavior of requests, but changing that in a proper way likely needs a larger rewrite / rearchitecturing of that part of Docker SDK for Python...

Since it looks like docker-py needs apply the fix on their end I believe we can close this PR.

We wrote the #6655 fix trying to avoid breaking users that were relying on the existing behavior of `get_connection` in their custom Adapters but missed the other side where they implemented a custom `get_connection` but were still using the default `send`.

I don't think what `docker-py` was doing is unreasonable, but unfortunately I don't see a clear way to mitigate this in Requests while maintaining the security fix. https://github.com/docker/docker-py/pull/3257 seems like the least invasive option at the moment.

I have a hunch `docker-py` may not be the only package that hits this issue so I'd like to track these _kinds_ of problems to evaluate if we need to take further action on our side. I'll leave this open for now to gather any other cases.

`requests-unixsocket`, which implements the `http+unix` scheme, is also broken by this change.

I would guess requests-file or whichever fork of that is still maintained would be affected if anyone uses it

> `requests-unixsocket`, which implements the `http+unix` scheme, is also broken by this change.

It implements it very similarly to Docker SDK for Python, i.e. it also overwrites `get_connection()`. Adding a `_get_connection()` similar to the one in https://github.com/docker/docker-py/pull/3257 should fix it as well.

> I would guess requests-file or whichever fork of that is still maintained would be affected if anyone uses it

I don't know requests-file, but the code in https://github.com/dashea/requests-file overwrites `send()` itself (and doesn't use `HTTPAdapter`, but `BaseAdapter`), so it should not be affected. Or at least not by the same change, maybe it's affected by something else ;-)


@felixfontein and everyone else following along, we've iterated through a few alternatives but each has similar impact on a different user group so a "fix" would only be shifting the problem.

We've decided to move `_get_connection` to a new public API on HTTPAdapter called `get_connection_with_tls_context`. Going forward this will be the expanded API signature that enables us and other libraries to properly handle TLS settings when fetching connections. The existing `get_connection` will be marked as deprecated and users will be pointed to the new API.

This will avoid the clunkiness of needing to override a private method and gives a clear path forward for the API. It's likely other custom adapters implementing `get_connection` may also suffer from something similar to CVE-2024-35195 and will need this behavior. While we didn't intend to suddenly force this shift, we appreciate everyone's patience while we get things stabilized.

I anticipate we'll have a patch staged at some point today and intend to push that out in 2.32.2 asap.

> @felixfontein and everyone else following along, we've iterated through a few alternatives but each has similar impact on a different user group so a "fix" would only be shifting the problem.
> 
> We've decided to move `_get_connection` to a new public API on HTTPAdapter called `get_connection_with_tls_context`. Going forward this will be the expanded API signature that enables us and other libraries to properly handle TLS settings when fetching connections. The existing `get_connection` will be marked as deprecated and users will be pointed to the new API.
> 
> This will avoid the clunkiness of needing to override a private method and gives a clear path forward for the API. It's likely other custom adapters implementing `get_connection` may also suffer from something similar to [CVE-2024-35195](https://github.com/advisories/GHSA-9wx4-h78v-vm56) and will need this behavior. While we didn't intend to suddenly force this shift, we appreciate everyone's patience while we get things stabilized.
> 
> I anticipate we'll have a patch staged at some point today and intend to push that out in 2.32.2 asap.

I am still facing the problem with version 2.32.2. Was the patch to fix the issue included in that release?

@sergiosmcr the patch needs to be done in docker-py which is already staged in https://github.com/docker/docker-py/pull/3257. Requests 2.23.2 was just to make the integration point for docker-py easier. As I noted at the top, we cannot fix this in Requests. Any changes will just move the current problem on to another package using Requests.

This should be resolved with docker-py 7.1.0, no longer requiring pinning Requests<2.32. I'll leave this open for a few more hours for comment and then plan to resolve this as completed by the end of today.

I am facing same issue with Docker-compose

Can someone help me what are the exact commands I should follow to fix the issue? 

Some of the answers from the net says "pinning requests version to 2.31.0" I am a newbie, wondering what is pinning? how to do? 

I am using WSL2 , Ubuntu 22 LTS.
docker-compose version 1.29.2, build unknown
docker                        7.1.0
docker-py                     1.10.6
docker-pycreds                0.4.0

Error Message I am receiving is below -->

  File "/home/rkk/.local/lib/python3.10/site-packages/requests/adapters.py", line 532, in send
    conn = self._get_connection(request, verify, proxies=proxies, cert=cert)
  File "/home/rkk/.local/lib/python3.10/site-packages/requests/adapters.py", line 400, in _get_connection
    conn = self.poolmanager.connection_from_host(
  File "/home/rkk/.local/lib/python3.10/site-packages/urllib3/poolmanager.py", line 304, in connection_from_host
    return self.connection_from_context(request_context)
  File "/home/rkk/.local/lib/python3.10/site-packages/urllib3/poolmanager.py", line 326, in connection_from_context
    raise URLSchemeUnknown(scheme)
urllib3.exceptions.URLSchemeUnknown: Not supported URL scheme http+docker


@rkkilari the old docker-compose Python package (which is End of Life for two years now!) does not work with Docker SDK for Python 7.1.0, which is the first version that works with Requests 2.32.x. You have to stick to `docker < 7.0.0` and `requests < 2.32.0`.

Also please note that you installed both `docker-py` and `docker`; these are incompatible and destroy each other. Uninstall both first, and then reinstall only one of them. If you install `docker`, make sure to install `docker < 7.0.0`, since Docker SDK for Python 7 is no longer compatible with docker-compose. `docker-py` shouldn't be used anymore anyway, except if you need Python 2.6 compatibility.

And to your actual question: pinning means that you make sure a fixed version is installed for a package. In this case, make sure that `requests` is installed with version 2.31.0 (or more generally < 2.32.0). How to do that depends on how you install the Python packages. If you are using a `requirements.txt` file, simply replace the line with `requests` by `requests < 2.32.0` (if there is no such line, simply add such a line).