Repository: streamlink/streamlink

Issue:
Svtplay's Plugin is currently broken
<!--
Thanks for reporting a plugin issue!
USE THE TEMPLATE. Otherwise your plugin issue may be rejected.

First, see the contribution guidelines:
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink

Also check the list of open and closed plugin issues:
https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22plugin+issue%22

Please see the text preview to avoid unnecessary formatting errors.
-->


## Plugin Issue

<!-- Replace [ ] with [x] in order to check the box -->
- [x] This is a plugin issue and I have read the contribution guidelines.


### Description

<!-- Explain the plugin issue as thoroughly as you can. -->

The svtplay plugin seems to be broken. Every url I have tested yield the same error. Other Swedish geo-restricted sites work fine.

I'm on the latest streamlink 1.5.0 on Windows 10.

### Reproduction steps / Explicit stream URLs to test

<!-- How can we reproduce this? Please note the exact steps below using the list format supplied. If you need more steps please add them. -->

1. Go to Svtplay.se with a Swedish Ip, Vpn or Proxy.
2. Pick some show and it's URL
3. Try to stream it with streamlink


### Log output

<!--
TEXT LOG OUTPUT IS REQUIRED for a plugin issue!
Use the `--loglevel debug` parameter and avoid using parameters which suppress log output.
https://streamlink.github.io/cli.html#cmdoption-l

Make sure to **remove usernames and passwords**
You can copy the output to https://gist.github.com/ or paste it below.
-->

```
streamlink https://www.svtplay.se/video/27096341/sportnytt/sportnytt-28-aug-22-00-3
[cli][info] Found matching plugin svtplay for URL https://www.svtplay.se/video/27096341/sportnytt/sportnytt-28-aug-22-00-3
error: Unable to parse JSON: Expecting value: line 1 column 1 (char 0) ('<!DOCTYPE html>\n\n<head>\n    <me ...)
```


### Additional comments, screenshots, etc.

Thanks a lot for your support!

[Love Streamlink? Please consider supporting our collective. Thanks!](https://opencollective.com/streamlink/donate)



Discussion:
@kvn1351, I have updated the regex.  You can get the whole plugin for testing from:

https://github.com/mkbloke/streamlink/blob/svtplay/src/streamlink/plugins/svtplay.py

Just select raw, then file/save as in your browser to download it.

~~Alternatively, you can just edit your existing plugin to update the regex on line 21 to:~~

~~```_id_re = re.compile(r'DATA_LAKE = {"content":{"id":"(?P<id>[^"]+)"')```~~

If the plugin works for you with that change, I'll create a PR for it.  Cheers.
@kvn1351, just to add, if you can test the change against a number of different URLs, that would be very helpful, thanks.
I've just pushed another minor improvement on this plugin, so rather than changing the single line as I mentioned previously, it would be better to download the whole updated plugin from:

https://github.com/mkbloke/streamlink/blob/svtplay/src/streamlink/plugins/svtplay.py

The minor improvement allows the plugin to also pick up the correct video page when given a landing page like:

https://www.svtplay.se/dokument-inifran-att-radda-ett-barn

Cheers.

It still does not work for the direct channel pages like:
https://www.svtplay.se/kanaler/svt1

Worked fine there before so the change they have done also makes this page fail.

Thanks for the report, @DanOscarsson.

I think I'm going to somewhat start from the beginning with this plugin, gathering the possible URLs under svtplay.se and oppetarkiv.se to start with, with a view to removing the support for the 'old' format if I cannot find any URLs where keeping it would be necessary.

I'll likely also drop support for svtflow.se unless I can find any reason for keeping it, as the one URL to content on svtflow.se that I've found with a Google search just results in a redirect to the home page on svtplay.se.
Let me know if I can assist in any way.
Hi all.  I've been testing this updated plugin on various URLs for both svtplay.se and oppetarkiv.se sites.  It seems to be working, so if anybody would like to test and report back, that would be most welcome.

I've pushed the most recent changes to: https://github.com/mkbloke/streamlink/blob/svtplay/src/streamlink/plugins/svtplay.py

This plugin now supports the ```--title``` option, where data is available, so you can for example use:

```--title '{title} - {author} - {category}'```

You can also multiplex in subtitles for VODs where they are available using the ```--svtplay-mux-subtitles``` option.

Cheers.

Hello!

I can confirm that it works for the following URLs:
- https://www.svtplay.se/kanaler/svt1?start=auto
- https://www.svtplay.se/kanaler/svt2?start=auto
- https://www.svtplay.se/kanaler/kunskapskanalen?start=auto
- https://www.svtplay.se/kanaler/svtbarn?start=auto
- https://www.svtplay.se/kanaler/svt24?start=auto

I have not tested the options though.

Great work!
Thanks for the feedback.

Resolving patch:
diff --git a/src/streamlink/plugins/svtplay.py b/src/streamlink/plugins/svtplay.py
--- a/src/streamlink/plugins/svtplay.py
+++ b/src/streamlink/plugins/svtplay.py
@@ -1,92 +1,148 @@
+import logging
 import re
 
-from streamlink.plugin import Plugin
-from streamlink.plugin.api import StreamMapper, validate
-from streamlink.stream import HLSStream, HDSStream
+from streamlink.compat import urljoin
+from streamlink.plugin import Plugin, PluginArguments, PluginArgument
+from streamlink.plugin.api import validate
+from streamlink.stream import DASHStream, HTTPStream
+from streamlink.stream.ffmpegmux import MuxedStream
 
-API_URL = "http://www.svt.se/videoplayer-api/video/{0}"
-
-_url_re = re.compile(r"""
-    http(s)?://
-    (www\.)?
-    (?:
-        svtplay |
-        svtflow |
-        oppetarkiv
-    )
-    .se
-""", re.VERBOSE)
-
-# Regex to match video ID
-_id_re = re.compile(r"""data-video-id=['"](?P<id>[^'"]+)['"]""")
-_old_id_re = re.compile(r"/(?:video|klipp)/(?P<id>[0-9]+)/")
-
-# New video schema used with API call
-_video_schema = validate.Schema(
-    {
-        "videoReferences": validate.all(
-            [{
-                "url": validate.text,
-                "format": validate.text
-            }],
-        ),
-    },
-    validate.get("videoReferences")
-)
-
-# Old video schema
-_old_video_schema = validate.Schema(
-    {
-        "video": {
-            "videoReferences": validate.all(
-                [{
-                    "url": validate.text,
-                    "playerType": validate.text
-                }],
-            ),
-        }
-    },
-    validate.get("video"),
-    validate.get("videoReferences")
-)
+log = logging.getLogger(__name__)
 
 
 class SVTPlay(Plugin):
+    api_url = 'https://api.svt.se/videoplayer-api/video/{0}'
+
+    author = None
+    category = None
+    title = None
+
+    url_re = re.compile(r'''
+        https?://(?:www\.)?(?:svtplay|oppetarkiv)\.se
+        (/(kanaler/)?.*)
+    ''', re.VERBOSE)
+
+    latest_episode_url_re = re.compile(r'''
+        class="play_titlepage__latest-video"\s+href="(?P<url>[^"]+)"
+    ''', re.VERBOSE)
+
+    live_id_re = re.compile(r'.*/(?P<live_id>[^?]+)')
+
+    vod_id_re = re.compile(r'''
+        (?:DATA_LAKE\s+=\s+{"content":{"id":|"svtId":|data-video-id=)
+        "(?P<vod_id>[^"]+)"
+    ''', re.VERBOSE)
+
+    _video_schema = validate.Schema({
+        validate.optional('programTitle'): validate.text,
+        validate.optional('episodeTitle'): validate.text,
+        'videoReferences': [{
+            'url': validate.url(),
+            'format': validate.text,
+        }],
+        validate.optional('subtitleReferences'): [{
+            'url': validate.url(),
+            'format': validate.text,
+        }],
+    })
+
+    arguments = PluginArguments(
+        PluginArgument(
+            'mux-subtitles',
+            action='store_true',
+            help="Automatically mux available subtitles in to the output stream.",
+        ),
+    )
+
     @classmethod
-    def can_handle_url(self, url):
-        return _url_re.match(url)
+    def can_handle_url(cls, url):
+        return cls.url_re.match(url) is not None
 
-    def _create_streams(self, stream_type, parser, video):
-        try:
-            streams = parser(self.session, video["url"])
-            return streams.items()
-        except IOError as err:
-            self.logger.error("Failed to extract {0} streams: {1}",
-                              stream_type, err)
+    def get_author(self):
+        if self.author is not None:
+            return self.author
 
-    def _get_streams(self):
-        # Retrieve URL page and search for new type of video ID
-        res = self.session.http.get(self.url)
-        match = _id_re.search(res.text)
+    def get_category(self):
+        if self.category is not None:
+            return self.category
 
-        # Use API if match, otherwise resort to old method
-        if match:
-            vid = match.group("id")
-            res = self.session.http.get(API_URL.format(vid))
+    def get_title(self):
+        if self.title is not None:
+            return self.title
 
-            videos = self.session.http.json(res, schema=_video_schema)
-            mapper = StreamMapper(cmp=lambda format, video: video["format"] == format)
-            mapper.map("hls", self._create_streams, "HLS", HLSStream.parse_variant_playlist)
-            mapper.map("hds", self._create_streams, "HDS", HDSStream.parse_manifest)
-        else:
-            res = self.session.http.get(self.url, params=dict(output="json"))
-            videos = self.session.http.json(res, schema=_old_video_schema)
+    def _set_metadata(self, data, category):
+        if 'programTitle' in data:
+            self.author = data['programTitle']
+
+        self.category = category
 
-            mapper = StreamMapper(cmp=lambda type, video: video["playerType"] == type)
-            mapper.map("ios", self._create_streams, "HLS", HLSStream.parse_variant_playlist)
-            mapper.map("flash", self._create_streams, "HDS", HDSStream.parse_manifest)
+        if 'episodeTitle' in data:
+            self.title = data['episodeTitle']
 
-        return mapper(videos)
+    def _get_live(self, path):
+        match = self.live_id_re.search(path)
+        if match is None:
+            return
+
+        live_id = "ch-{0}".format(match.group('live_id'))
+        log.debug("Live ID={0}".format(live_id))
+
+        res = self.session.http.get(self.api_url.format(live_id))
+        api_data = self.session.http.json(res, schema=self._video_schema)
+
+        self._set_metadata(api_data, 'Live')
+
+        for playlist in api_data['videoReferences']:
+            if playlist['format'] == 'dashhbbtv':
+                for s in DASHStream.parse_manifest(self.session, playlist['url']).items():
+                    yield s
+
+    def _get_vod(self):
+        res = self.session.http.get(self.url)
+        match = self.latest_episode_url_re.search(res.text)
+        if match:
+            res = self.session.http.get(
+                urljoin(self.url, match.group('url')),
+            )
+
+        match = self.vod_id_re.search(res.text)
+        if match is None:
+            return
+
+        vod_id = match.group('vod_id')
+        log.debug("VOD ID={0}".format(vod_id))
+
+        res = self.session.http.get(self.api_url.format(vod_id))
+        api_data = self.session.http.json(res, schema=self._video_schema)
+
+        self._set_metadata(api_data, 'VOD')
+
+        substreams = {}
+        if 'subtitleReferences' in api_data:
+            for subtitle in api_data['subtitleReferences']:
+                if subtitle['format'] == 'webvtt':
+                    log.debug("Subtitle={0}".format(subtitle['url']))
+                    substreams[subtitle['format']] = HTTPStream(
+                        self.session,
+                        subtitle['url'],
+                    )
+
+        for manifest in api_data['videoReferences']:
+            if manifest['format'] == 'dashhbbtv':
+                for q, s in DASHStream.parse_manifest(self.session, manifest['url']).items():
+                    if self.get_option('mux_subtitles') and substreams:
+                        yield q, MuxedStream(self.session, s, subtitles=substreams)
+                    else:
+                        yield q, s
+
+    def _get_streams(self):
+        path, live = self.url_re.match(self.url).groups()
+        log.debug("Path={0}".format(path))
+
+        if live:
+            return self._get_live(path)
+        else:
+            return self._get_vod()
 
 
 __plugin__ = SVTPlay


Test patch:
diff --git a/tests/plugins/test_svtplay.py b/tests/plugins/test_svtplay.py
--- a/tests/plugins/test_svtplay.py
+++ b/tests/plugins/test_svtplay.py
@@ -6,13 +6,29 @@
 class TestPluginSVTPlay(unittest.TestCase):
     def test_can_handle_url(self):
         should_match = [
-            'https://www.svtplay.se/video/16567350?start=auto',
+            'https://www.svtplay.se/kanaler/svt1',
+            'https://www.svtplay.se/kanaler/svt2?start=auto',
+            'https://www.svtplay.se/kanaler/svtbarn',
+            'https://www.svtplay.se/kanaler/kunskapskanalen?start=auto',
+            'https://www.svtplay.se/kanaler/svt24?start=auto',
+            'https://www.svtplay.se/video/27659457/den-giftbla-floden?start=auto',
+            'https://www.svtplay.se/video/27794015/100-vaginor',
+            'https://www.svtplay.se/video/28065172/motet/motet-sasong-1-det-skamtar-du-inte-om',
+            'https://www.svtplay.se/dokument-inifran-att-radda-ett-barn',
+            'https://www.oppetarkiv.se/video/1399273/varfor-ar-det-sa-ont-om-q-avsnitt-1-av-5',
+            'https://www.oppetarkiv.se/video/2781201/karlekens-mirakel',
+            'https://www.oppetarkiv.se/video/10354325/studio-pop',
+            'https://www.oppetarkiv.se/video/5792180/studio-pop-david-bowie',
+            'https://www.oppetarkiv.se/video/2923832/hipp-hipp-sasong-2-avsnitt-3-av-7',
+            'https://www.oppetarkiv.se/video/3945501/cccp-hockey-cccp-hockey',
         ]
         for url in should_match:
             self.assertTrue(SVTPlay.can_handle_url(url))
 
     def test_can_handle_url_negative(self):
         should_not_match = [
+            'http://www.svtflow.se/video/2020285/avsnitt-6',
+            'https://www.svtflow.se/video/2020285/avsnitt-6',
             'https://example.com/index.html',
         ]
         for url in should_not_match: