Repository: streamlink/streamlink
Pull request #2802 (2020-02-16 23:26:28)

Fix tvplayer plugin and tests
Let's try again.

closes https://github.com/streamlink/streamlink/issues/2456

codecov[bot]: # [Codecov](https://codecov.io/gh/streamlink/streamlink/pull/2802?src=pr&el=h1) Report
> Merging [#2802](https://codecov.io/gh/streamlink/streamlink/pull/2802?src=pr&el=desc) into [master](https://codecov.io/gh/streamlink/streamlink/commit/7f4f8f0ea53959e55a7d6ce5908149c874aad6f8?src=pr&el=desc) will **decrease** coverage by `0.02%`.
> The diff coverage is `56.25%`.


```diff
@@            Coverage Diff             @@
##           master    #2802      +/-   ##
==========================================
- Coverage   52.87%   52.85%   -0.03%     
==========================================
  Files         248      248              
  Lines       15481    15476       -5     
==========================================
- Hits         8186     8180       -6     
- Misses       7295     7296       +1
```

bastimeyer: Thanks for the code fixes. This is much better now.
gravyboat: @bastimeyer Anything else? If not let's get this merged.
bastimeyer: If it's working according to #2456, then go ahead and merge. I can't say much here due to being geo-blocked. The rest is looking fine I guess.
mkbloke: Are you going to merge this into master?  I completely understand that you may wish to make improvements to the code, but surely that can be done in another PR?  In the meantime, master has no working tvplayer plugin...
gravyboat: @mkbloke We've got a huge PR for flake8 fixes here: https://github.com/streamlink/streamlink/pull/2804 that I just merged (had to get this in before anything else because it had so many changes). Sorry about the delay on getting this in. I was hesitant on the order because this now has some conflicts because of the flake8 support. Can you fix those up real quick and then we'll get this merged? Sorry about the inconvenience.
mkbloke: Thanks for the response, @gravyboat.

I had somewhat realised that the flake8 stuff may have been an issue.

As far as I can see (and having checked locally), the PR does not invalidate flake8 compliance for the plugin.  I did not get an answer on flake8 regarding: https://github.com/streamlink/streamlink/pull/2802#discussion_r380358698 for test_tvplayer.py, so that's still an unknown.
bastimeyer: > I did not get an answer on flake8 regarding: [#2802 (comment)](https://github.com/streamlink/streamlink/pull/2802#discussion_r380358698)

Just make it so that it doesn't produce any linting errors/warnings and that the test is legible. Everything else is up to you.

The merge conflicts are here because your PR now has a diff from an older commit, even though the resulting changes are probably the same. It should be relatively simple to resolve though.

If you need help with solving the merge conflicts and rebasing your PR branch, just ask.
mkbloke: @bastimeyer, thanks for the response.  Can you advise how to resolve ```W605 invalid escape sequence '\/'``` where the HTML fragment contains, for example:
```
"thumbnail":"https:\/\/d537y3nbkeq75.cloudfront.net\/tvp\/epg\/e3e45caf-79fe-3827-82b4-ecdfee52acfb.jpg?width=700&lang=en"
```
Thanks.
mkbloke: Shall I just delete that single long line?  It would solve the E501 and W605 issues flagged by flake8.
gravyboat: Looks good, thanks @mkbloke!

Patch:
diff --git a/src/streamlink/plugins/tvplayer.py b/src/streamlink/plugins/tvplayer.py
index aa90f399..e21803f7 100644
--- a/src/streamlink/plugins/tvplayer.py
+++ b/src/streamlink/plugins/tvplayer.py
@@ -1,3 +1,4 @@
+import logging
 import re
 
 from streamlink.plugin import Plugin, PluginArguments, PluginArgument
@@ -6,44 +7,38 @@ from streamlink.plugin.api import useragents
 from streamlink.stream import HLSStream
 
 
+log = logging.getLogger(__name__)
+
+
 class TVPlayer(Plugin):
-    context_url = "http://tvplayer.com/watch/context"
-    api_url = "http://api.tvplayer.com/api/v2/stream/live"
-    login_url = "https://tvplayer.com/account/login"
+    api_url = "https://v1-streams-elb.tvplayer-cdn.com/api/live/stream/"
+    stream_url = "https://live.tvplayer.com/stream-live.php"
+    login_url = "https://v1-auth.tvplayer-cdn.com/login?responseType=redirect&redirectUri=https://tvplayer.com/login&lang=en"
     update_url = "https://tvplayer.com/account/update-detail"
     dummy_postcode = "SE1 9LT"  # location of ITV HQ in London
 
     url_re = re.compile(r"https?://(?:www.)?tvplayer.com/(:?watch/?|watch/(.+)?)")
-    stream_attrs_re = re.compile(r'data-(resource|token|channel-id)\s*=\s*"(.*?)"', re.S)
-    data_id_re = re.compile(r'data-id\s*=\s*"(.*?)"', re.S)
-    login_token_re = re.compile(r'input.*?name="token".*?value="(\w+)"')
+    stream_attrs_re = re.compile(r'data-player-(expiry|key|token|uvid)\s*=\s*"(.*?)"', re.S)
+    login_token_re = re.compile(r'input.*?name="_token".*?value="(\w+)"')
     stream_schema = validate.Schema({
-        "tvplayer": validate.Schema({
-            "status": u'200 OK',
-            "response": validate.Schema({
-                "stream": validate.url(scheme=validate.any("http", "https")),
-                validate.optional("drmToken"): validate.any(None, validate.text)
-            })
+        "response": validate.Schema({
+            "stream": validate.any(None, validate.text),
+            "drm": validate.any(None, validate.text)
         })
-    },
-        validate.get("tvplayer"),
-        validate.get("response"))
-    context_schema = validate.Schema({
-        "validate": validate.text,
-        validate.optional("token"): validate.text,
-        "platform": {
-            "key": validate.text
-        }
     })
     arguments = PluginArguments(
-        PluginArgument("email",
-                       help="The email address used to register with tvplayer.com.",
-                       metavar="EMAIL",
-                       requires=["password"]),
-        PluginArgument("password",
-                       sensitive=True,
-                       help="The password for your tvplayer.com account.",
-                       metavar="PASSWORD")
+        PluginArgument(
+            "email",
+            help="The email address used to register with tvplayer.com.",
+            metavar="EMAIL",
+            requires=["password"]
+        ),
+        PluginArgument(
+            "password",
+            sensitive=True,
+            help="The password for your tvplayer.com account.",
+            metavar="PASSWORD"
+        )
     )
 
     @classmethod
@@ -61,69 +56,67 @@ class TVPlayer(Plugin):
         token = match and match.group(1)
         res2 = self.session.http.post(
             self.login_url,
-            data=dict(email=username, password=password, token=token),
+            data=dict(email=username, password=password, _token=token),
             allow_redirects=False
         )
         # there is a 302 redirect on a successful login
         return res2.status_code == 302
 
-    def _get_stream_data(self, resource, channel_id, token, service=1):
-        # Get the context info (validation token and platform)
-        self.logger.debug("Getting stream information for resource={0}".format(resource))
-        context_res = self.session.http.get(
-            self.context_url,
-            params={
-                "resource": resource,
-                "gen": token
+    def _get_stream_data(self, expiry, key, token, uvid):
+        res = self.session.http.get(
+            self.api_url + uvid,
+            params=dict(key=key, platform="chrome"),
+            headers={
+                "Token": token,
+                "Token-Expiry": expiry,
+                "Uvid": uvid
             }
         )
-        context_data = self.session.http.json(context_res, schema=self.context_schema)
-        self.logger.debug("Context data: {0}", str(context_data))
-
-        # get the stream urls
-        res = self.session.http.post(
-            self.api_url,
-            data=dict(
-                service=service,
-                id=channel_id,
-                validate=context_data["validate"],
-                token=context_data.get("token"),
-                platform=context_data["platform"]["key"]
-            ),
-            raise_for_status=False
-        )
 
-        return self.session.http.json(res, schema=self.stream_schema)
+        res_schema = self.session.http.json(res, schema=self.stream_schema)
+
+        if res_schema["response"]["stream"] is None:
+            res = self.session.http.get(
+                self.stream_url,
+                params=dict(key=key),
+                headers={
+                    "Token": token,
+                    "Token-Expiry": expiry,
+                    "Uvid": uvid
+                }
+            ).json()
+            res_schema["response"]["stream"] = res["Streams"]["Adaptive"]
+
+        return res_schema
 
     def _get_stream_attrs(self, page):
         stream_attrs = dict((k.replace("-", "_"), v.strip('"')) for k, v in self.stream_attrs_re.findall(page.text))
 
-        if not stream_attrs.get("channel_id"):
-            m = self.data_id_re.search(page.text)
-            stream_attrs["channel_id"] = m and m.group(1)
-
-        self.logger.debug("Got stream attributes: {0}", str(stream_attrs))
+        log.debug("Got stream attributes: {0}", str(stream_attrs))
         valid = True
-        for a in ("channel_id", "resource", "token"):
+        for a in ("expiry", "key", "token", "uvid"):
             if a not in stream_attrs:
-                self.logger.debug("Missing '{0}' from stream attributes", a)
+                log.debug("Missing '{0}' from stream attributes", a)
                 valid = False
 
         return stream_attrs if valid else {}
 
     def _get_streams(self):
         if self.get_option("email") and self.get_option("password"):
-            self.logger.debug("Logging in as {0}".format(self.get_option("email")))
+            log.debug("Logging in as {0}".format(self.get_option("email")))
             if not self.authenticate(self.get_option("email"), self.get_option("password")):
-                self.logger.warning("Failed to login as {0}".format(self.get_option("email")))
+                log.warning("Failed to login as {0}".format(self.get_option("email")))
 
         # find the list of channels from the html in the page
         self.url = self.url.replace("https", "http")  # https redirects to http
         res = self.session.http.get(self.url)
 
         if "enter your postcode" in res.text:
-            self.logger.info("Setting your postcode to: {0}. "
-                             "This can be changed in the settings on tvplayer.com", self.dummy_postcode)
+            log.info(
+                "Setting your postcode to: {0}. "
+                "This can be changed in the settings on tvplayer.com",
+                self.dummy_postcode
+            )
             res = self.session.http.post(
                 self.update_url,
                 data=dict(postcode=self.dummy_postcode),
@@ -135,14 +128,16 @@ class TVPlayer(Plugin):
             stream_data = self._get_stream_data(**stream_attrs)
 
             if stream_data:
-                if stream_data.get("drmToken"):
-                    self.logger.error("This stream is protected by DRM can cannot be played")
+                if stream_data["response"]["drm"] is not None:
+                    log.error("This stream is protected by DRM can cannot be played")
                     return
                 else:
-                    return HLSStream.parse_variant_playlist(self.session, stream_data["stream"])
+                    return HLSStream.parse_variant_playlist(
+                        self.session, stream_data["response"]["stream"]
+                    )
         else:
             if "need to login" in res.text:
-                self.logger.error(
+                log.error(
                     "You need to login using --tvplayer-email/--tvplayer-password to view this stream")
 
 


Test patch:
diff --git a/tests/plugins/test_tvplayer.py b/tests/plugins/test_tvplayer.py
index 1448fade..dcbb404c 100644
--- a/tests/plugins/test_tvplayer.py
+++ b/tests/plugins/test_tvplayer.py
@@ -33,24 +33,40 @@ class TestPluginTVPlayer(unittest.TestCase):
     @patch('streamlink.plugins.tvplayer.HLSStream')
     def test_get_streams(self, hlsstream, mock_get_stream_data):
         mock_get_stream_data.return_value = {
-            "stream": "http://test.se/stream1"
+            "response": {"stream": "http://test.se/stream1", "drm": None}
         }
 
         page_resp = Mock()
         page_resp.text = u"""
-                    <div class="video-js theoplayer-skin theo-seekbar-above-controls content-box vjs-fluid"
-                 data-resource= "bbcone"
-                 data-token = "1324567894561268987948596154656418448489159"
-                                    data-content-type="live"
-                    data-environment="live"
-                    data-subscription="free"
-                    data-channel-id="89">
-                <div id="channel-info" class="channel-info">
-                    <div class="row visible-xs visible-sm">
+            <div class="col-xs-12">
+                <div id="live-player-root"
+                    data-player-library="videojs"
+                    data-player-id="ILWxqLKV91Ql8kF"
+                    data-player-key="2Pw1Eg0Px3Gy9Jm3Ry8Ar5Bi5Vc5Nk"
+                    data-player-uvid="139"
+                    data-player-token="275c808a685a09d6e36d0253ab3765af"
+                    data-player-expiry="1531958189"
+                    data-player-poster=""
+                    data-requested-channel-id="139"
+                    data-play-button="/assets/tvplayer/images/dist/play-button-epg.svg"
+                    data-update-url="/now-ajax"
+                    data-timezone-name="Europe/London"
+                    data-thumb-link-classes=""
+                    data-theme-name="tvplayer"
+                    data-base-entitlements="%5B%22free%22%5D"
+                    data-has-identity="0"
+                    data-require-identity-to-watch-free="1"
+                    data-stream-type="live"
+                    data-live-route="/watch/%25live_channel_id%25"
+                >
+                </div>
+            </div>
         """
 
         self.session.http.get.return_value = page_resp
-        hlsstream.parse_variant_playlist.return_value = {"test": HLSStream(self.session, "http://test.se/stream1")}
+        hlsstream.parse_variant_playlist.return_value = {
+            "test": HLSStream(self.session, "http://test.se/stream1")
+        }
 
         TVPlayer.bind(self.session, "test.tvplayer")
         plugin = TVPlayer("http://tvplayer.com/watch/dave")
@@ -60,21 +76,27 @@ class TestPluginTVPlayer(unittest.TestCase):
         self.assertTrue("test" in streams)
 
         # test the url is used correctly
-        self.session.http.get.assert_called_with("http://tvplayer.com/watch/dave")
+        self.session.http.get.assert_called_with(
+            "http://tvplayer.com/watch/dave"
+        )
         # test that the correct API call is made
         mock_get_stream_data.assert_called_with(
-            resource="bbcone",
-            channel_id="89",
-            token="1324567894561268987948596154656418448489159"
+            expiry="1531958189",
+            key="2Pw1Eg0Px3Gy9Jm3Ry8Ar5Bi5Vc5Nk",
+            token="275c808a685a09d6e36d0253ab3765af",
+            uvid="139"
         )
         # test that the correct URL is used for the HLSStream
-        hlsstream.parse_variant_playlist.assert_called_with(ANY, "http://test.se/stream1")
+        hlsstream.parse_variant_playlist.assert_called_with(
+            ANY,
+            "http://test.se/stream1"
+        )
 
     def test_get_invalid_page(self):
         page_resp = Mock()
         page_resp.text = u"""
-        var validate = "foo";
-        var resourceId = "1234";
+            var validate = "foo";
+            var resourceId = "1234";
         """
         self.session.http.get.return_value = page_resp
 
@@ -87,7 +109,9 @@ class TestPluginTVPlayer(unittest.TestCase):
 
         # test the url is used correctly
 
-        self.session.http.get.assert_called_with("http://tvplayer.com/watch/dave")
+        self.session.http.get.assert_called_with(
+            "http://tvplayer.com/watch/dave"
+        )
 
     def test_arguments(self):
         from streamlink_cli.main import setup_plugin_args
@@ -102,7 +126,10 @@ class TestPluginTVPlayer(unittest.TestCase):
 
         setup_plugin_args(session, parser)
 
-        self.assertSequenceEqual(plugin_parser.add_argument.mock_calls,
-                                 [call('--tvplayer-email', metavar="EMAIL", help=ANY),
-                                  call('--tvplayer-password', metavar="PASSWORD", help=ANY)],
-                                 )
+        self.assertSequenceEqual(
+            plugin_parser.add_argument.mock_calls,
+            [
+                call('--tvplayer-email', metavar="EMAIL", help=ANY),
+                call('--tvplayer-password', metavar="PASSWORD", help=ANY)
+            ],
+        )