Repository: streamlink/streamlink

Issue:
HTTP NO SSL Verify broken
### Checklist

- [X] This is a bug report and not [a different kind of issue](https://github.com/streamlink/streamlink/issues/new/choose)
- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
- [X] [I have checked the list of open and recently closed bug reports](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22bug%22)
- [X] [I have checked the commit log of the master branch](https://github.com/streamlink/streamlink/commits/master)

### Collaboration

- [X] [I will provide feedback should a pull request be opened with a fix for the problem](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#pull-request-feedback)

### Streamlink version

[cli][info] Your Streamlink version (6.10.0) is up to date!

### Description

Recently, I noticed that streams with SSL enabled do not play on streamlink. 

By default, I get such message : 
``
[cli][info] Found matching plugin hls for URL https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0
error: Unable to open URL: https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0 (HTTPSConnectionPool(host='s2.callofliberty.fr', port=443): Max retries exceeded with url: /coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)'))))
``

With SSL Verify disabled : 
``
streamlink --http-no-ssl-verify "https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0"
[cli][info] Found matching plugin hls for URL https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0
error: Unable to open URL: https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0 (Cannot set verify_mode to CERT_NONE when check_hostname is enabled.)
``

### Debug log

```text
SSL Verify Enabled

streamlink -l debug "https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0"
[session][debug] Loading plugin: hls
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.12.2
[cli][debug] OpenSSL:    OpenSSL 3.0.13 30 Jan 2024
[cli][debug] Streamlink: 6.10.0
[cli][debug] Dependencies:
[cli][debug]  certifi: 2024.2.2
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 5.1.0
[cli][debug]  pycountry: 23.12.11
[cli][debug]  pycryptodome: 3.20.0
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.31.0
[cli][debug]  trio: 0.25.0
[cli][debug]  trio-websocket: 0.11.1
[cli][debug]  typing-extensions: 4.10.0
[cli][debug]  urllib3: 2.2.1
[cli][debug]  websocket-client: 1.7.0
[cli][debug]  exceptiongroup: 1.2.0
[cli][debug] Arguments:
[cli][debug]  url=https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0
[cli][debug]  --loglevel=debug
[cli][debug]  --title=0
[cli][debug]  --hls-audio-select=['*']
[cli][debug]  --ffmpeg-ffmpeg=C:\Program Files\Python312\Lib\site-packages\ffmpeg\ffmpeg.exe
[cli][debug]  --http-header=[('User-Agent', 'Mozilla/5.0')]
[cli][debug]  --http-disable-dh=True
[cli][info] Found matching plugin hls for URL https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0
[plugins.hls][debug] URL=https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0; params={}
[utils.l10n][debug] Language code: en_US
error: Unable to open URL: https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0 (HTTPSConnectionPool(host='s2.callofliberty.fr', port=443): Max retries exceeded with url: /coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)'))))


SSL Verify Disabled

streamlink -l debug --http-no-ssl-verify "https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0"
[session][debug] Loading plugin: hls
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.12.2
[cli][debug] OpenSSL:    OpenSSL 3.0.13 30 Jan 2024
[cli][debug] Streamlink: 6.10.0
[cli][debug] Dependencies:
[cli][debug]  certifi: 2024.2.2
[cli][debug]  isodate: 0.6.1
[cli][debug]  lxml: 5.1.0
[cli][debug]  pycountry: 23.12.11
[cli][debug]  pycryptodome: 3.20.0
[cli][debug]  PySocks: 1.7.1
[cli][debug]  requests: 2.31.0
[cli][debug]  trio: 0.25.0
[cli][debug]  trio-websocket: 0.11.1
[cli][debug]  typing-extensions: 4.10.0
[cli][debug]  urllib3: 2.2.1
[cli][debug]  websocket-client: 1.7.0
[cli][debug]  exceptiongroup: 1.2.0
[cli][debug] Arguments:
[cli][debug]  url=https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0
[cli][debug]  --loglevel=debug
[cli][debug]  --title=0
[cli][debug]  --hls-audio-select=['*']
[cli][debug]  --ffmpeg-ffmpeg=C:\Program Files\Python312\Lib\site-packages\ffmpeg\ffmpeg.exe
[cli][debug]  --http-header=[('User-Agent', 'Mozilla/5.0')]
[cli][debug]  --http-no-ssl-verify=False
[cli][debug]  --http-disable-dh=True
[cli][info] Found matching plugin hls for URL https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0
[plugins.hls][debug] URL=https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0; params={}
[utils.l10n][debug] Language code: en_US
error: Unable to open URL: https://s2.callofliberty.fr/coldtv/RTS1/index.m3u8?callofliberty=c4eb578121a1f142fb270fabe28ba3b2568c9b0 (Cannot set verify_mode to CERT_NONE when check_hostname is enabled.)
```


Discussion:

Patch:
diff --git a/src/streamlink/session/http.py b/src/streamlink/session/http.py
index 52b936f3..1b120ac6 100644
--- a/src/streamlink/session/http.py
+++ b/src/streamlink/session/http.py
@@ -222,6 +222,16 @@ class SSLContextAdapter(HTTPAdapter):
         kwargs["ssl_context"] = self.poolmanager.connection_pool_kw["ssl_context"]
         return super().proxy_manager_for(*args, **kwargs)
 
+    def send(self, *args, verify=True, **kwargs):
+        # Always update the `check_hostname` and `verify_mode` attributes of our custom `SSLContext` before sending a request:
+        # If `verify` is `False`, then `requests` will set `cert_reqs=ssl.CERT_NONE` on the `HTTPSConnectionPool` object,
+        # which leads to `SSLContext` incompatibilities later on in `urllib3.connection._ssl_wrap_socket_and_match_hostname()`
+        # due to the default values of our `SSLContext`, namely `check_hostname=True` and `verify_mode=ssl.CERT_REQUIRED`.
+        if ssl_context := self.poolmanager.connection_pool_kw.get("ssl_context"):  # pragma: no branch
+            ssl_context.check_hostname = bool(verify)
+            ssl_context.verify_mode = ssl.CERT_NONE if not verify else ssl.CERT_REQUIRED
+        return super().send(*args, verify=verify, **kwargs)
+
 
 class TLSNoDHAdapter(SSLContextAdapter):
     def get_ssl_context(self) -> ssl.SSLContext:


Test patch:
diff --git a/tests/session/test_http.py b/tests/session/test_http.py
index 8a6b5c97..c66de9a3 100644
--- a/tests/session/test_http.py
+++ b/tests/session/test_http.py
@@ -1,10 +1,14 @@
+import ssl
 from ssl import SSLContext
 from typing import Optional
 from unittest.mock import Mock, PropertyMock, call
 
 import pytest
 import requests
+from requests.adapters import HTTPAdapter
+from urllib3.response import HTTPResponse
 
+from streamlink import Streamlink
 from streamlink.exceptions import PluginError, StreamlinkDeprecationWarning
 from streamlink.session.http import HTTPSession, SSLContextAdapter, TLSNoDHAdapter, TLSSecLevel1Adapter
 from streamlink.session.http_useragents import FIREFOX
@@ -132,3 +136,52 @@ class TestHTTPAdapters:
         ssl_context_poolmanager = adapter.poolmanager.connection_pool_kw.get("ssl_context")
         ssl_context_proxymanager = proxymanager.connection_pool_kw.get("ssl_context")
         assert ssl_context_poolmanager is ssl_context_proxymanager
+
+
+class TestHTTPSessionVerifyAndCustomSSLContext:
+    @pytest.fixture()
+    def adapter(self, session: Streamlink):
+        # The http-disable-dh session option mounts the TLSNoDHAdapter with a custom SSLContext
+        session.set_option("http-disable-dh", True)
+
+        adapter = session.http.adapters.get("https://")
+        assert isinstance(adapter, TLSNoDHAdapter)
+
+        return adapter
+
+    @pytest.fixture(autouse=True)
+    def _fake_request(self, monkeypatch: pytest.MonkeyPatch, session: Streamlink, adapter: HTTPAdapter):
+        class FakeHTTPResponse(HTTPResponse):
+            def stream(self, *_, **__):
+                yield b"mocked"
+
+        # Can't use requests_mock here, as it monkeypatches the adapter's send() method, which we want to test
+        req = requests.PreparedRequest()
+        resp = FakeHTTPResponse(status=200)
+        # noinspection PyTypeChecker
+        response = adapter.build_response(req, resp)
+        monkeypatch.setattr("requests.adapters.HTTPAdapter.send", Mock(return_value=response))
+
+        assert session.http.get("https://mocked/").text == "mocked"
+
+    @pytest.mark.parametrize(
+        ("session", "check_hostname", "verify_mode"),
+        [
+            pytest.param({"http-ssl-verify": True}, True, ssl.CERT_REQUIRED, id="verify"),
+            pytest.param({"http-ssl-verify": False}, False, ssl.CERT_NONE, id="no-verify"),
+        ],
+        indirect=["session"],
+    )
+    def test_ssl_context_attributes(
+        self,
+        session: Streamlink,
+        adapter: HTTPAdapter,
+        check_hostname: bool,
+        verify_mode: ssl.VerifyMode,
+    ):
+        assert session.http.verify is session.get_option("http-ssl-verify")
+
+        ssl_context = adapter.poolmanager.connection_pool_kw.get("ssl_context")
+        assert isinstance(ssl_context, SSLContext)
+        assert ssl_context.check_hostname is check_hostname
+        assert ssl_context.verify_mode is verify_mode