Repository: streamlink/streamlink

Issue:
ard_mediathek plugin fails with 403 error
### Description

Whenever I try to watch the ARD livestream I can use neither of the available plugins. However, the most promising solutions seems to be the `ard_mediathek` plugin. Unfortunately, I'm getting a `403 Client error` when trying to use it.

### Reproduction steps / Stream URLs to test

```
$ streamlink http://www.ardmediathek.de/tv/Das-Erste/live?kanal=208 best
[cli][info] Found matching plugin ard_mediathek for URL http://www.ardmediathek.de/tv/Das-Erste/live?kanal=208
[plugin.ard_mediathek][error] Failed to extract SMIL streams: Unable to open URL: http://live-lh.daserste.de/z//daserste_de@446299/manifest.f4m?hdcore?hdcore=3.3.0 (403 Client Error: Forbidden for url: http://live-lh.daserste.de/z//daserste_de@446299/manifest.f4m?hdcore?hdcore=3.3.0)
[cli][info] Available streams: auto (worst, best)
[cli][info] Opening stream: auto (http)
[cli][info] Starting player: mpv
[cli][info] Stream ended
[cli][info] Closing currently open stream...
$
```

Opening the URL in any modern browser (Firefox, Chrome et al) works out of the box.

### Environment details (operating system, python version, etc.)

- Ubuntu 16.10
- Streamlink 0.3.2

```
$ cat .config/streamlink/config 
twitch-oauth-token=secret
player=mpv
hls-live-edge=60
hds-live-edge=60
hls-segment-attempts=10
hds-segment-attempts=10
hls-segment-threads=4
hds-segment-threads=4
```

### Comments, logs, screenshots, etc.

Notice how I wouldn't be be able to use the `ard_live` plugin either since its URL schema (`live.daserste.de`) redirects to `http://www.daserste.de/live/index.html`, which the plugin doesn't recognize.

Additionally, the redirection URL (`http://live-lh.daserste.de`) is not covered by any plugin at all yet.


Discussion:

Resolving patch:
diff --git a/src/streamlink/plugins/ard_mediathek.py b/src/streamlink/plugins/ard_mediathek.py
--- a/src/streamlink/plugins/ard_mediathek.py
+++ b/src/streamlink/plugins/ard_mediathek.py
@@ -2,7 +2,7 @@
 
 from streamlink.plugin import Plugin
 from streamlink.plugin.api import http, validate
-from streamlink.stream import HTTPStream, HDSStream, RTMPStream
+from streamlink.stream import HTTPStream, HDSStream, HLSStream
 
 MEDIA_URL = "http://www.ardmediathek.de/play/media/{0}"
 SWF_URL = "http://www.ardmediathek.de/ard/static/player/base/flash/PluginFlash.swf"
@@ -16,7 +16,7 @@
 }
 
 _url_re = re.compile(r"http(s)?://(\w+\.)?ardmediathek.de/tv")
-_media_id_re = re.compile(r"/play/config/(\d+)")
+_media_id_re = re.compile(r"/play/(?:media|config)/(\d+)")
 _media_schema = validate.Schema({
     "_mediaArray": [{
         "_mediaStreamArray": [{
@@ -33,6 +33,10 @@
             validate.get("base"),
             validate.url(scheme="http")
         ),
+        "cdn": validate.all(
+            validate.xml_find("head/meta"),
+            validate.get("cdn")
+        ),
         "videos": validate.all(
             validate.xml_findall("body/seq/video"),
             [validate.get("src")]
@@ -61,17 +65,8 @@ def _get_hds_streams(self, info):
         url = info["_stream"] + HDCORE_PARAMETER
         return HDSStream.parse_manifest(self.session, url, pvswf=SWF_URL).items()
 
-    def _get_rtmp_streams(self, info):
-        name = QUALITY_MAP.get(info["_quality"], "live")
-        params = {
-            "rtmp": info["_server"].strip(),
-            "playpath": info["_stream"],
-            "pageUrl": self.url,
-            "swfVfy": SWF_URL,
-            "live": True
-        }
-        stream = RTMPStream(self.session, params)
-        yield name, stream
+    def _get_hls_streams(self, info):
+        return HLSStream.parse_variant_playlist(self.session, info["_stream"]).items()
 
     def _get_smil_streams(self, info):
         res = http.get(info["_stream"])
@@ -79,7 +74,7 @@ def _get_smil_streams(self, info):
 
         for video in smil["videos"]:
             url = "{0}/{1}{2}".format(smil["base"], video, HDCORE_PARAMETER)
-            streams = HDSStream.parse_manifest(self.session, url, pvswf=SWF_URL)
+            streams = HDSStream.parse_manifest(self.session, url, pvswf=SWF_URL, is_akamai=smil["cdn"] == "akamai")
 
             # TODO: Replace with "yield from" when dropping Python 2.
             for stream in streams.items():
@@ -93,6 +88,8 @@ def _get_streams(self):
         else:
             return
 
+        self.logger.debug("Found media id: {0}", media_id)
+
         res = http.get(MEDIA_URL.format(media_id))
         media = http.json(res, schema=_media_schema)
 
@@ -104,25 +101,23 @@ def _get_streams(self):
                     if not stream_:
                         continue
                     stream_ = stream_[0]
-                stream_ = stream_.strip()
 
-                if server.startswith("rtmp://"):
-                    parser = self._get_rtmp_streams
-                    parser_name = "RTMP"
-                elif stream_.endswith(".f4m"):
+                if stream_.endswith(".f4m"):
                     parser = self._get_hds_streams
                     parser_name = "HDS"
                 elif stream_.endswith(".smil"):
                     parser = self._get_smil_streams
                     parser_name = "SMIL"
+                elif stream_.endswith(".m3u8"):
+                    parser = self._get_hls_streams
+                    parser_name = "HLS"
                 elif stream_.startswith("http"):
                     parser = self._get_http_streams
                     parser_name = "HTTP"
 
                 try:
-                    # TODO: Replace with "yield from" when dropping Python 2.
-                    for stream in parser(stream):
-                        yield stream
+                    for s in parser(stream):
+                        yield s
                 except IOError as err:
                     self.logger.error("Failed to extract {0} streams: {1}",
                                       parser_name, err)


Test patch: