Repository: streamlink/streamlink

Issue:
How to avoid "Input is not padded or padding is corrupt" error?
<!--
Thanks for reporting a bug!
USE THE TEMPLATE. Otherwise your bug report may be rejected.

First, see the contribution guidelines:
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink

Also check the list of open and closed bug reports:
https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22bug%22

Please see the text preview to avoid unnecessary formatting errors.
-->


## Bug Report

<!-- Replace [ ] with [x] in order to check the box -->
- [x] This is a bug report and I have read the contribution guidelines.


### Description

<!-- Explain the bug as thoroughly as you can. Don't leave out information which is necessary for us to reproduce and debug this issue. -->

Sometimes, on streams that involve some kind of decryption, I get a `streamlink.exceptions.StreamError: Input is not padded or padding is corrupt, got padding size of xxx`

Whenever this happens, streamlink completely stalls until it reaches the timeout value, and then it just quits. This is very frustrating as this is not reproducible, and I have no idea how it is going to happen or when it is going to happen. In some cases, I can be recording the same exact same stream in two different commands running at the same time, and one will quit and the other will keep recording. In the case of downloading a replay of a stream, I can just restart the download and streamlink always gets past the point where it crashed previously with no issue.

What I would like to know is how I can avoid this error. For example if the error comes up, how can I force streamlink to try redownload/redecrypt the segment?

### Expected / Actual behavior

Streamlink downloads the stream with no issues.

### Reproduction steps / Explicit stream URLs to test

<!-- How can we reproduce this? Please note the exact steps below using the list format supplied. If you need more steps please add them. -->

NA. The stream needs to have some sort of encryption. I have received this error across Fox Sports Go streams, ESPN+ streams, and live streams of Turner websites (TNT/TBS).

### Log output

<!--
TEXT LOG OUTPUT IS REQUIRED for a bug report!
Use the `--loglevel debug` parameter and avoid using parameters which suppress log output.
https://streamlink.github.io/cli.html#cmdoption-l

Make sure to **remove usernames and passwords**
You can copy the output to https://gist.github.com/ or paste it below.
-->

```
streamlink 'hlsvariant://https://turnerlive.akamaized.net/hls/live/572100/true/aes_slate/de/master.m3u8?hdnts=exp=1553034983~acl=/hls/live/572100/true/*!/hls/live/572100-b/true/*!/hls/live/630682/true/*!/hls/live/630682-b/true/*~id=c5c9ce531e437434c07dc7ff65b0ab94~hmac=112e16086c033e16ffcb3b487ff24f3f4f6626ccc2c57f0be36a60f30d269aee' best -o trutv.ts
[cli][info] Found matching plugin hls for URL hlsvariant://https://turnerlive.akamaized.net/hls/live/572100/true/aes_slate/de/master.m3u8?hdnts=exp=1553034983~acl=/hls/live/572100/true/*!/hls/live/572100-b/true/*!/hls/live/630682/true/*!/hls/live/630682-b/true/*~id=c5c9ce531e437434c07dc7ff65b0ab94~hmac=112e16086c033e16ffcb3b487ff24f3f4f6626ccc2c57f0be36a60f30d269aee
[cli][info] Available streams: 360p_alt2 (worst), 360p_alt, 360p, 480p_alt, 480p, 540p_alt, 540p, 720p_alt, 720p (best)
[cli][info] Opening stream: 720p (hls)                                                                                                                                              [download][trutv.ts] Written 763.8 MB (28m49s @ 424.4 KB/s)                                                                                                                                               Exception in thread Thread-HLSStreamWriter:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/streamlink/stream/segmented.py", line 167, in run
    self.write(segment, result)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/streamlink/stream/hls.py", line 140, in write
    self.reader.buffer.write(pkcs7_decode(decrypted_chunk))
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/streamlink/stream/hls.py", line 31, in pkcs7_decode
    raise StreamError("Input is not padded or padding is corrupt, got padding size of {0}".format(val))
streamlink.exceptions.StreamError: Input is not padded or padding is corrupt, got padding size of 156

error: Error when reading from stream: Read timeout, exiting
[cli][info] Stream ended
[cli][info] Closing currently open stream...
```

Discussion:
bastimeyer: Regarding the fix for encrypted HLS streams, this causes the HTTPSession to re-download the faulty/truncated segment. If the server however continues to respond with an incorrect Content-Length header or truncated response body, then there's not much that can be done.
marmeladema: I have installed a patched version of streamlink and will report here in the next couple of days if I encounter the issue again.

Patch:
diff --git a/src/streamlink/plugin/api/http_session.py b/src/streamlink/plugin/api/http_session.py
index d56739be..6cd3764c 100644
--- a/src/streamlink/plugin/api/http_session.py
+++ b/src/streamlink/plugin/api/http_session.py
@@ -1,5 +1,7 @@
 import time
 
+import requests.adapters
+import urllib3
 from requests import Session
 
 from streamlink.exceptions import PluginError
@@ -7,16 +9,39 @@ from streamlink.packages.requests_file import FileAdapter
 from streamlink.plugin.api import useragents
 from streamlink.utils import parse_json, parse_xml
 
-try:
-    from requests.packages import urllib3
 
+try:
     # We tell urllib3 to disable warnings about unverified HTTPS requests,
     # because in some plugins we have to do unverified requests intentionally.
     urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
-except (ImportError, AttributeError):
+except AttributeError:
     pass
 
-__all__ = ["HTTPSession"]
+
+class _HTTPResponse(urllib3.response.HTTPResponse):
+    def __init__(self, *args, **kwargs):
+        # Always enforce content length validation!
+        # This fixes a bug in requests which doesn't raise errors on HTTP responses where
+        # the "Content-Length" header doesn't match the response's body length.
+        # https://github.com/psf/requests/issues/4956#issuecomment-573325001
+        #
+        # Summary:
+        # This bug is related to urllib3.response.HTTPResponse.stream() which calls urllib3.response.HTTPResponse.read() as
+        # a wrapper for http.client.HTTPResponse.read(amt=...), where no http.client.IncompleteRead exception gets raised
+        # due to "backwards compatiblity" of an old bug if a specific amount is attempted to be read on an incomplete response.
+        #
+        # urllib3.response.HTTPResponse.read() however has an additional check implemented via the enforce_content_length
+        # parameter, but it doesn't check by default and requests doesn't set the parameter for enabling it either.
+        #
+        # Fix this by overriding urllib3.response.HTTPResponse's constructor and always setting enforce_content_length to True,
+        # as there is no way to make requests set this parameter on its own.
+        kwargs.update({"enforce_content_length": True})
+        super().__init__(*args, **kwargs)
+
+
+# override all urllib3.response.HTTPResponse references in requests.adapters.HTTPAdapter.send
+urllib3.connectionpool.HTTPConnectionPool.ResponseCls = _HTTPResponse
+requests.adapters.HTTPResponse = _HTTPResponse
 
 
 def _parse_keyvalue_list(val):
@@ -29,12 +54,10 @@ def _parse_keyvalue_list(val):
 
 
 class HTTPSession(Session):
-    def __init__(self, *args, **kwargs):
-        Session.__init__(self, *args, **kwargs)
-
-        if self.headers['User-Agent'].startswith('python-requests'):
-            self.headers['User-Agent'] = useragents.FIREFOX
+    def __init__(self):
+        super().__init__()
 
+        self.headers['User-Agent'] = useragents.FIREFOX
         self.timeout = 20.0
 
         self.mount('file://', FileAdapter())
@@ -123,12 +146,16 @@ class HTTPSession(Session):
 
         while True:
             try:
-                res = Session.request(self, method, url,
-                                      headers=headers,
-                                      params=params,
-                                      timeout=timeout,
-                                      proxies=proxies,
-                                      *args, **kwargs)
+                res = super().request(
+                    method,
+                    url,
+                    headers=headers,
+                    params=params,
+                    timeout=timeout,
+                    proxies=proxies,
+                    *args,
+                    **kwargs
+                )
                 if raise_for_status and res.status_code not in acceptable_status:
                     res.raise_for_status()
                 break
@@ -136,8 +163,7 @@ class HTTPSession(Session):
                 raise
             except Exception as rerr:
                 if retries >= total_retries:
-                    err = exception("Unable to open URL: {url} ({err})".format(url=url,
-                                                                               err=rerr))
+                    err = exception(f"Unable to open URL: {url} ({rerr})")
                     err.err = rerr
                     raise err
                 retries += 1
@@ -150,3 +176,6 @@ class HTTPSession(Session):
             res = schema.validate(res.text, name="response text", exception=PluginError)
 
         return res
+
+
+__all__ = ["HTTPSession"]
diff --git a/src/streamlink/plugins/twitch.py b/src/streamlink/plugins/twitch.py
index 4ef7f1d1..47a8ebe5 100644
--- a/src/streamlink/plugins/twitch.py
+++ b/src/streamlink/plugins/twitch.py
@@ -513,7 +513,7 @@ class Twitch(Plugin):
             (?:
                 /video/(?P<video_id>\d+)
                 |
-                /clip/(?P<clip_name>[\w]+)
+                /clip/(?P<clip_name>[\w-]+)
             )?
         )
     """, re.VERBOSE)
diff --git a/src/streamlink/plugins/zattoo.py b/src/streamlink/plugins/zattoo.py
index b0fbf4ac..bf527f7b 100644
--- a/src/streamlink/plugins/zattoo.py
+++ b/src/streamlink/plugins/zattoo.py
@@ -17,7 +17,6 @@ log = logging.getLogger(__name__)
 class Zattoo(Plugin):
     API_CHANNELS = '{0}/zapi/v2/cached/channels/{1}?details=False'
     API_HELLO = '{0}/zapi/session/hello'
-    API_HELLO_V2 = '{0}/zapi/v2/session/hello'
     API_HELLO_V3 = '{0}/zapi/v3/session/hello'
     API_LOGIN = '{0}/zapi/v2/account/login'
     API_LOGIN_V3 = '{0}/zapi/v3/account/login'
@@ -157,19 +156,16 @@ class Zattoo(Plugin):
         # a new session is required for the app_token
         self.session.http.cookies = cookiejar_from_dict({})
         if self.base_url == 'https://zattoo.com':
-            app_token_url = 'https://zattoo.com/client/token-2fb69f883fea03d06c68c6e5f21ddaea.json'
+            app_token_url = 'https://zattoo.com/token-46a1dfccbd4c3bdaf6182fea8f8aea3f.json'
         elif self.base_url == 'https://www.quantum-tv.com':
             app_token_url = 'https://www.quantum-tv.com/token-4d0d61d4ce0bf8d9982171f349d19f34.json'
         else:
             app_token_url = self.base_url
 
         res = self.session.http.get(app_token_url)
-        if self.base_url == 'https://www.quantum-tv.com':
+        if self.base_url == 'https://www.quantum-tv.com' or self.base_url == 'https://zattoo.com':
             app_token = self.session.http.json(res)["session_token"]
             hello_url = self.API_HELLO_V3.format(self.base_url)
-        elif self.base_url == 'https://zattoo.com':
-            app_token = self.session.http.json(res)['app_tid']
-            hello_url = self.API_HELLO_V2.format(self.base_url)
         else:
             match = self._app_token_re.search(res.text)
             app_token = match.group(1)
@@ -182,19 +178,12 @@ class Zattoo(Plugin):
             self._session_attributes.set(
                 'uuid', __uuid, expires=self.TIME_SESSION)
 
-        if self.base_url == 'https://zattoo.com':
-            params = {
-                'uuid': __uuid,
-                'app_tid': app_token,
-                'app_version': '1.0.0'
-            }
-        else:
-            params = {
-                'client_app_token': app_token,
-                'uuid': __uuid,
-            }
+        params = {
+            'client_app_token': app_token,
+            'uuid': __uuid,
+        }
 
-        if self.base_url == 'https://www.quantum-tv.com':
+        if self.base_url == 'https://www.quantum-tv.com' or self.base_url == 'https://zattoo.com':
             params['app_version'] = '3.2028.3'
         else:
             params['lang'] = 'en'


Test patch:
diff --git a/tests/plugins/test_twitch.py b/tests/plugins/test_twitch.py
index 50abd993..7d26639b 100644
--- a/tests/plugins/test_twitch.py
+++ b/tests/plugins/test_twitch.py
@@ -18,6 +18,7 @@ class TestPluginCanHandleUrlTwitch(PluginCanHandleUrl):
         'https://www.twitch.tv/twitch',
         'https://www.twitch.tv/videos/150942279',
         'https://clips.twitch.tv/ObservantBenevolentCarabeefPhilosoraptor',
+        'https://www.twitch.tv/weplaydota/clip/FurryIntelligentDonutAMPEnergyCherry-akPRxv7Y3w58WmFq'
         'https://www.twitch.tv/twitch/video/292713971',
         'https://www.twitch.tv/twitch/v/292713971',
     ]
diff --git a/tests/test_api_http_session.py b/tests/test_api_http_session.py
index 694e435f..3b692a1a 100644
--- a/tests/test_api_http_session.py
+++ b/tests/test_api_http_session.py
@@ -5,23 +5,29 @@ import requests
 
 from streamlink.exceptions import PluginError
 from streamlink.plugin.api.http_session import HTTPSession
+from streamlink.plugin.api.useragents import FIREFOX
 
 
 class TestPluginAPIHTTPSession(unittest.TestCase):
+    def test_session_init(self):
+        session = HTTPSession()
+        self.assertEqual(session.headers.get("User-Agent"), FIREFOX)
+        self.assertEqual(session.timeout, 20.0)
+        self.assertIn("file://", session.adapters.keys())
+
     @patch("streamlink.plugin.api.http_session.time.sleep")
-    @patch("streamlink.plugin.api.http_session.Session.request")
+    @patch("streamlink.plugin.api.http_session.Session.request", side_effect=requests.Timeout)
     def test_read_timeout(self, mock_request, mock_sleep):
-        mock_request.side_effect = requests.Timeout
         session = HTTPSession()
 
         with self.assertRaises(PluginError) as cm:
             session.get("http://localhost/", timeout=123, retries=3, retry_backoff=2, retry_max_backoff=5)
         self.assertTrue(str(cm.exception).startswith("Unable to open URL: http://localhost/"))
         self.assertEqual(mock_request.mock_calls, [
-            call(session, "GET", "http://localhost/", headers={}, params={}, timeout=123, proxies={}, allow_redirects=True),
-            call(session, "GET", "http://localhost/", headers={}, params={}, timeout=123, proxies={}, allow_redirects=True),
-            call(session, "GET", "http://localhost/", headers={}, params={}, timeout=123, proxies={}, allow_redirects=True),
-            call(session, "GET", "http://localhost/", headers={}, params={}, timeout=123, proxies={}, allow_redirects=True),
+            call("GET", "http://localhost/", headers={}, params={}, timeout=123, proxies={}, allow_redirects=True),
+            call("GET", "http://localhost/", headers={}, params={}, timeout=123, proxies={}, allow_redirects=True),
+            call("GET", "http://localhost/", headers={}, params={}, timeout=123, proxies={}, allow_redirects=True),
+            call("GET", "http://localhost/", headers={}, params={}, timeout=123, proxies={}, allow_redirects=True),
         ])
         self.assertEqual(mock_sleep.mock_calls, [
             call(2),