Repository: streamlink/streamlink

Issue:
plugins.ard_mediathek: The ARD plugin is not loading anymore.
### Checklist

- [X] This is a plugin issue and not a different kind of issue
- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
- [X] [I have checked the list of open and recently closed plugin issues](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22plugin+issue%22)
- [X] [I have checked the commit log of the master branch](https://github.com/streamlink/streamlink/commits/master)

### Streamlink version

Latest stable release

### Description

It seems that the ARD (German public broadcaster) has made some changes to their site. On their site, their stream is working much better. 

However, when trying to use Streamlink, the ARD stream will not open. 

[cli][info] Found matching plugin ard_mediathek for URL https://www.ardmediathek.de/daserste/live/Y3JpZDovL2Rhc2Vyc3RlLmRlL0xpdmVzdHJlYW0tRGFzRXJzdGU/
error: No playable streams found on this URL: https://www.ardmediathek.de/daserste/live/Y3JpZDovL2Rhc2Vyc3RlLmRlL0xpdmVzdHJlYW0tRGFzRXJzdGU/

streamlink --version
streamlink 3.0.1+1.g33ccd2e




### Debug log

```text
[cli][debug] OS:         Linux-5.4.0-90-generic-x86_64-with-glibc2.29
[cli][debug] Python:     3.8.10
[cli][debug] Streamlink: 3.0.1+1.g33ccd2e
[cli][debug] Requests(2.26.0), Socks(1.7.1), Websocket(1.2.1)
[cli][debug] Arguments:
[cli][debug]  url=https://www.ardmediathek.de/daserste/live/Y3JpZDovL2Rhc2Vyc3RlLmRlL0xpdmVzdHJlYW0tRGFzRXJzdGU/
[cli][debug]  stream=['540p']
[cli][debug]  --loglevel=debug
[cli][info] Found matching plugin ard_mediathek for URL https://www.ardmediathek.de/daserste/live/Y3JpZDovL2Rhc2Vyc3RlLmRlL0xpdmVzdHJlYW0tRGFzRXJzdGU/
error: No playable streams found on this URL: https://www.ardmediathek.de/daserste/live/Y3JpZDovL2Rhc2Vyc3RlLmRlL0xpdmVzdHJlYW0tRGFzRXJzdGU/
```



Discussion:
Does the existing plugin work for VoD content?

I have a fix for the live channels within `https://www.ardmediathek.de/live/`, but it would be easier to get some info from you on what is still working (if anything), so I can look at those URLs and decide if the existing `media_id` code should be kept.

Thanks.

It's the exact same error for VoD:

[cli][info] Found matching plugin ard_mediathek for URL https://www.ardmediathek.de/video/wataha-einsatz-an-der-grenze-europas/folge-1-anschlag-s01-e01/mdr/Y3JpZDovL21kci5kZS9iZWl0cmFnL2Ntcy9hOWQxNTg1Yy1iMzQyLTQ3NGYtYTM2NS0wZWY2M2JjZjNhMDY/
error: No playable streams found on this URL: https://www.ardmediathek.de/video/wataha-einsatz-an-der-grenze-europas/folge-1-anschlag-s01-e01/mdr/Y3JpZDovL21kci5kZS9iZWl0cmFnL2Ntcy9hOWQxNTg1Yy1iMzQyLTQ3NGYtYTM2NS0wZWY2M2JjZjNhMDY/

[cli][info] Found matching plugin ard_mediathek for URL https://www.ardmediathek.de/video/white-sands-strand-der-geheimnisse/folge-1-warmer-empfang-s01-e01/ndr/Y3JpZDovL25kci5kZS80MmEwZmQwMC03YTYwLTQ4N2EtYWZhZC1lZDBmYjYzZjgzMDQ/
error: No playable streams found on this URL: https://www.ardmediathek.de/video/white-sands-strand-der-geheimnisse/folge-1-warmer-empfang-s01-e01/ndr/Y3JpZDovL25kci5kZS80MmEwZmQwMC03YTYwLTQ4N2EtYWZhZC1lZDBmYjYzZjgzMDQ/

Live:

[cli][info] Found matching plugin ard_mediathek for URL https://www.ardmediathek.de/live/Y3JpZDovL2Rhc2Vyc3RlLmRlL2xpdmUvY2xpcC9hYmNhMDdhMy0zNDc2LTQ4NTEtYjE2Mi1mZGU4ZjY0NmQ0YzQ/
error: No playable streams found on this URL: https://www.ardmediathek.de/live/Y3JpZDovL2Rhc2Vyc3RlLmRlL2xpdmUvY2xpcC9hYmNhMDdhMy0zNDc2LTQ4NTEtYjE2Mi1mZGU4ZjY0NmQ0YzQ/

It works fine in the browser. The ZDF plugin is working fine. 

Resolving patch:
diff --git a/src/streamlink/plugins/ard_mediathek.py b/src/streamlink/plugins/ard_mediathek.py
--- a/src/streamlink/plugins/ard_mediathek.py
+++ b/src/streamlink/plugins/ard_mediathek.py
@@ -4,29 +4,7 @@
 from streamlink.plugin import Plugin, pluginmatcher
 from streamlink.plugin.api import validate
 from streamlink.stream.hls import HLSStream
-from streamlink.stream.http import HTTPStream
-from streamlink.utils.url import update_scheme
 
-MEDIA_URL = "http://www.ardmediathek.de/play/media/{0}"
-QUALITY_MAP = {
-    "auto": "auto",
-    4: "1080p",
-    3: "720p",
-    2: "544p",
-    1: "360p",
-    0: "144p"
-}
-
-_media_id_re = re.compile(r"/play/(?:media|config|sola)/(\d+)")
-_media_schema = validate.Schema({
-    "_mediaArray": [{
-        "_mediaStreamArray": [{
-            validate.optional("_server"): validate.text,
-            "_stream": validate.any(validate.text, [validate.text]),
-            "_quality": validate.any(int, validate.text)
-        }]
-    }]
-})
 
 log = logging.getLogger(__name__)
 
@@ -35,54 +13,63 @@
     r"https?://(?:(\w+\.)?ardmediathek\.de/|mediathek\.daserste\.de/)"
 ))
 class ARDMediathek(Plugin):
-    def _get_http_streams(self, info):
-        name = QUALITY_MAP.get(info["_quality"], "vod")
-        urls = info["_stream"]
-        if not isinstance(info["_stream"], list):
-            urls = [urls]
+    def _get_streams(self):
+        data_json = self.session.http.get(self.url, schema=validate.Schema(
+            validate.parse_html(),
+            validate.xml_findtext(".//script[@id='fetchedContextValue'][@type='application/json']"),
+            validate.any(None, validate.all(
+                validate.parse_json(),
+                {str: dict},
+                validate.transform(lambda obj: list(obj.items())),
+                validate.filter(lambda item: item[0].startswith("https://api.ardmediathek.de/page-gateway/pages/")),
+                validate.any(validate.get((0, 1)), [])
+            ))
+        ))
+        if not data_json:
+            return
 
-        for url in urls:
-            stream = HTTPStream(self.session, update_scheme("https://", url))
-            yield name, stream
+        schema_data = validate.Schema({
+            "id": str,
+            "widgets": validate.all(
+                [dict],
+                validate.filter(lambda item: item.get("mediaCollection")),
+                validate.get(0),
+                {
+                    "geoblocked": bool,
+                    "publicationService": {
+                        "name": str,
+                    },
+                    "title": str,
+                    "mediaCollection": {
+                        "embedded": {
+                            "_mediaArray": [{
+                                "_mediaStreamArray": [{
+                                    "_quality": validate.any(str, int),
+                                    "_stream": validate.url()
+                                }]
+                            }]
+                        }
+                    }
+                }
+            )
+        })
+        data = schema_data.validate(data_json)
 
-    def _get_hls_streams(self, info):
-        return HLSStream.parse_variant_playlist(self.session, update_scheme("https://", info["_stream"])).items()
+        log.debug(f"Found media id: {data['id']}")
+        data_media = data["widgets"]
 
-    def _get_streams(self):
-        res = self.session.http.get(self.url)
-        match = _media_id_re.search(res.text)
-        if match:
-            media_id = match.group(1)
-        else:
+        if data_media["geoblocked"]:
+            log.info("The content is not available in your region")
             return
 
-        log.debug("Found media id: {0}".format(media_id))
-        res = self.session.http.get(MEDIA_URL.format(media_id))
-        media = self.session.http.json(res, schema=_media_schema)
-        log.trace("{0!r}".format(media))
+        self.author = data_media["publicationService"]["name"]
+        self.title = data_media["title"]
 
-        for media in media["_mediaArray"]:
+        for media in data_media["mediaCollection"]["embedded"]["_mediaArray"]:
             for stream in media["_mediaStreamArray"]:
-                stream_ = stream["_stream"]
-                if isinstance(stream_, list):
-                    if not stream_:
-                        continue
-                    stream_ = stream_[0]
-
-                stream_ = update_scheme("https://", stream_)
-                if ".m3u8" in stream_:
-                    parser = self._get_hls_streams
-                    parser_name = "HLS"
-                elif (".mp4" in stream_ and ".f4m" not in stream_):
-                    parser = self._get_http_streams
-                    parser_name = "HTTP"
-                else:
-                    log.error("Unexpected stream type: '{0}'".format(stream_))
-
-                try:
-                    yield from parser(stream)
-                except OSError as err:
-                    log.error("Failed to extract {0} streams: {1}".format(parser_name, err))
+                if stream["_quality"] != "auto" or ".m3u8" not in stream["_stream"]:
+                    continue
+                return HLSStream.parse_variant_playlist(self.session, stream["_stream"])
 
 
 __plugin__ = ARDMediathek


Test patch: