Repository: streamlink/streamlink

Issue:
Youtube plugin not working with some vods
<!--
Thanks for reporting a plugin issue!
USE THE TEMPLATE. Otherwise your plugin issue may be rejected.

First, see the contribution guidelines:
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink

Also check the list of open and closed plugin issues:
https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22plugin+issue%22

Please see the text preview to avoid unnecessary formatting errors.
-->


## Plugin Issue

<!-- Replace [ ] with [x] in order to check the box -->
- [x ] This is a plugin issue and I have read the contribution guidelines.


### Description

<!-- Explain the plugin issue as thoroughly as you can. -->
On some vods i get no playable streams found and cant seem to identify why.


### Reproduction steps / Explicit stream URLs to test

<!-- How can we reproduce this? Please note the exact steps below using the list format supplied. If you need more steps please add them. -->

1. Try to load streamlink https://www.youtube.com/watch?v=FmDUUGOTFCM


### Log output

<!--
TEXT LOG OUTPUT IS REQUIRED for a plugin issue!
Use the `--loglevel debug` parameter and avoid using parameters which suppress log output.
https://streamlink.github.io/cli.html#cmdoption-l

Make sure to **remove usernames and passwords**
You can copy the output to https://gist.github.com/ or paste it below.
-->

```
streamlink https://www.youtube.com/watch?v=jXg93FFZXWk -l debug
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.6.6
[cli][debug] Streamlink: 1.3.0
[cli][debug] Requests(2.22.0), Socks(1.7.1), Websocket(0.56.0)
[cli][info] Found matching plugin youtube for URL https://www.youtube.com/watch?v=jXg93FFZXWk
[plugin.youtube][debug] Video ID from URL
[plugin.youtube][debug] Using video ID: jXg93FFZXWk
[plugin.youtube][debug] get_video_info - 1: Found data
[plugin.youtube][debug] MuxedStream: v 137 a 251 = 1080p
Available streams: audio_mp4, audio_webm, 360p (worst), 720p, 1080p (best)

streamlink https://www.youtube.com/watch?v=FmDUUGOTFCM -l debug
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.6.6
[cli][debug] Streamlink: 1.3.0
[cli][debug] Requests(2.22.0), Socks(1.7.1), Websocket(0.56.0)
[cli][info] Found matching plugin youtube for URL https://www.youtube.com/watch?v=FmDUUGOTFCM
[plugin.youtube][debug] Video ID from URL
[plugin.youtube][debug] Using video ID: FmDUUGOTFCM
[plugin.youtube][debug] get_video_info - 1: Found data
error: No playable streams found on this URL: https://www.youtube.com/watch?v=FmDUUGOTFCM
```


### Additional comments, screenshots, etc.



[Love Streamlink? Please consider supporting our collective. Thanks!](https://opencollective.com/streamlink/donate)



Discussion:
I've faced with similar issue with following link: https://www.youtube.com/watch?v=UhkgggX_-QE&feature=youtu.be

```
streamlink "https://www.youtube.com/watch?v=UhkgggX_-QE&feature=youtu.be" -l debug
[cli][debug] OS:         macOS 10.15.2
[cli][debug] Python:     3.8.0
[cli][debug] Streamlink: 1.3.0
[cli][debug] Requests(2.22.0), Socks(1.7.1), Websocket(0.56.0)
[cli][info] Found matching plugin youtube for URL https://www.youtube.com/watch?v=UhkgggX_-QE&feature=youtu.be
[plugin.youtube][debug] Video ID from URL
[plugin.youtube][debug] Using video ID: UhkgggX_-QE
[plugin.youtube][debug] get_video_info - 1: Found data
error: No playable streams found on this URL: https://www.youtube.com/watch?v=UhkgggX_-QE&feature=youtu.be
```
+1, repro here as well using 1.3.0+19.gad276c4 and video link https://www.youtube.com/watch?v=fCZsk0E-Y5A

Had a chance to poke around at this today. Completely new to the code base and domain so forgive me if this is not a useful observation. I dumped `parse_qsl(res.text)` just following https://github.com/streamlink/streamlink/blob/master/src/streamlink/plugins/youtube.py#L326 and found that the video info response text for each problematic link provided so far has no `fmt_list` or `url_encoded_fmt_stream_map` entries, so the plugin's `_get_streams()` call returns an empty dict.

These same videos all download properly in youtube-dl 2020.01.01 FWIW.
Its getting worse.

E:\tmp2\London>streamlink -l debug https://www.youtube.com/watch?v=c6RpWivuEoY 480p
[cli][debug] OS:         Windows 7
[cli][debug] Python:     3.8.0
[cli][debug] Streamlink: 1.3.0
[cli][debug] Requests(2.22.0), Socks(1.7.1), Websocket(0.56.0)
[cli][info] Found matching plugin youtube for URL https://www.youtube.com/watch?v=c6RpWivuEoY
[plugin.youtube][debug] Video ID from URL
[plugin.youtube][debug] Using video ID: c6RpWivuEoY
[plugin.youtube][debug] get_video_info - 1: Found data
error: No playable streams found on this URL: https://www.youtube.com/watch?v=c6RpWivuEoY
Having the same issue on Manjaro(linux)

$ streamlink https://www.youtube.com/watch?v=RG4IXfTMGSM best -l debug
[cli][debug] OS:         Linux-5.4.6-2-MANJARO-x86_64-with-glibc2.2.5
[cli][debug] Python:     3.8.1
[cli][debug] Streamlink: 1.3.0
[cli][debug] Requests(2.22.0), Socks(1.7.0), Websocket(0.57.0)
[cli][info] Found matching plugin youtube for URL https://www.youtube.com/watch?v=RG4IXfTMGSM
[plugin.youtube][debug] Video ID from URL
[plugin.youtube][debug] Using video ID: RG4IXfTMGSM
[plugin.youtube][debug] get_video_info - 1: Found data
error: No playable streams found on this URL: https://www.youtube.com/watch?v=RG4IXfTMGSM
Same for Raspian :
```streamlink -l debug 'https://www.youtube.com/watch?v=oSmUI3m2kLk' best
[cli][debug] OS:         Linux-4.19.42-v7+-armv7l-with-debian-9.9
[cli][debug] Python:     2.7.13
[cli][debug] Streamlink: 1.3.0
[cli][debug] Requests(2.22.0), Socks(1.7.1), Websocket(0.57.0)
[cli][info] Found matching plugin youtube for URL https://www.youtube.com/watch?v=oSmUI3m2kLk
[plugin.youtube][debug] Video ID from URL
[plugin.youtube][debug] Using video ID: oSmUI3m2kLk
[plugin.youtube][debug] get_video_info - 1: Found data
error: No playable streams found on this URL: https://www.youtube.com/watch?v=oSmUI3m2kLk
```
Doesn't work for me either
Verifying, having similar "No playable streams" difficulties. Originally encountered with a 4K video and chalked it up to Google being weird and splitting video and audio at super-high (>=1440p) qualities, but I've also run into this in run-of-the-mill 720p max videos too that don't have strict Content ID protection that would usually spit out the "use youtube-dl instead" fallback message.
same problem on raspbian (I replaced ID with ABC for privacy reasons):

```
[cli][debug] OS:         Linux-4.19.93-v7l+-armv7l-with-debian-10.2
[cli][debug] Python:     3.7.3
[cli][debug] Streamlink: 1.3.0
[cli][debug] Requests(2.21.0), Socks(1.6.7), Websocket(0.57.0)
[cli][info] Found matching plugin youtube for URL https://www.youtube.com/watch?v=ABC
[plugin.youtube][debug] Video ID from URL
[plugin.youtube][debug] Using video ID: ABC
[plugin.youtube][debug] get_video_info - 1: Found data
error: No playable streams found on this URL: https://www.youtube.com/watch?v=ABC
```

Getting the same thing here - Raspbian.  Seems any VOD link I try to play fails with "no playable streams" as above.  

EDIT:  Does seem to be JUST VOD links... as the NASA TV live stream works just fine e.g.
https://www.youtube.com/watch?v=21X5lGlDOfg
Any word from the Streamlink team  (@back-to) on this issue?  Thanks!
I don't even know if we're going to consider this an issue as livestreams are fine. It's just VODs that don't work so I don't know if anyone will pick this up since we've been moving further and further from VOD support. I don't have time to look at it currently but if anyone wants to investigate feel free to do so.
@gravyboat - I had no idea about the VOD vs Live stream priority - thanks for clarifying that.  

Unfortunately although the live stream part of streamlink has historically been my primary use - I've begun using the VOD capability with Youtube quite extensively.  I can see that may have been a mistake in view of the priorities for this project.  

I'll do my best to dig through the problem and find a solution - but I don't have full faith in my programming capabilities to pull out the solution.  Wish me luck!   
@Maqken - do you have any luck with any VOD now?  The VOD from your original post that WAS working is not working now either.   

@phytoporg - do you have any (still) working examples of a VOD video link?   

I'm looking for a working VOD so I can compare some additional debug dumps of working vs non-working VOD.  Although I am thinking the issue now is something has changed with the youtube API that is now making this an issue will all VOD's with streamlink.  

Thanks!  
@liamkennedy it was too random so i´m not using it for youtube vods anymore, i´m trying out youtube-dl for the time being and it's working fine. It has the downside that i have to teach my homebrew ui to use it instead of streamlink but it's a nice exercise.
You might want to try older vods, my last experience with youtube vods was that newer vods didn't work and older did. but i suppose that if they are spreading the change backwards eventually none will.
@Maqken thanks for the response.  I have tried out some 10-year-old videos on my YT Channel - and the problem is happening with those too - so it does look like it's maybe on all VOD's now.  

In my case I have a quite large installed base of users worldwide running as a console application on Raspbian (mostly stretch - but some are still on Jessie) - so there are a few more challenges I have to work out before I could switch to youtube-dl for this.  I may very well have to go that route - although my highest priority would be to keep with streamlink and just work out what the fix is.  

Thanks again
@liamkennedy Sure no problem. It's mostly just a stance we've had to take against adding new VOD only sites since that kind of defeats the purpose of Streamlink and we were getting a lot of requests for sites that already exist in tools like youtube-dl so I have to make sure to say that it's the main focus so no one comes back to quote this issue going "well you fixed youtube!". All the maintainers have been pretty busy with real life stuff sadly so we've only been hitting more critical issues as of late. 
Streamlink just doesn't work for any youtube video anymore.
I just noticed, lot of stuff does not work that connected to youtube. Like playing youtube videos in VLC.
Unless... I did some digging and found this:
https://addons.videolan.org/p/1154080/
or rather this:
https://raw.githubusercontent.com/videolan/vlc/master/share/lua/playlist/youtube.lua

Its fixes the issue for VLC. Maybe there are something in it which helps to fix streamlink aswell. 

Resolving patch:
diff --git a/src/streamlink/plugins/youtube.py b/src/streamlink/plugins/youtube.py
--- a/src/streamlink/plugins/youtube.py
+++ b/src/streamlink/plugins/youtube.py
@@ -16,75 +16,42 @@
 log = logging.getLogger(__name__)
 
 
-def parse_stream_map(stream_map):
-    if not stream_map:
-        return []
-
-    return [parse_query(s) for s in stream_map.split(",")]
-
-
-def parse_fmt_list(formatsmap):
-    formats = {}
-    if not formatsmap:
-        return formats
-
-    for format in formatsmap.split(","):
-        s = format.split("/")
-        (w, h) = s[1].split("x")
-        formats[int(s[0])] = "{0}p".format(h)
-
-    return formats
-
-
 _config_schema = validate.Schema(
     {
-        validate.optional("fmt_list"): validate.all(
-            validate.text,
-            validate.transform(parse_fmt_list)
-        ),
-        validate.optional("url_encoded_fmt_stream_map"): validate.all(
-            validate.text,
-            validate.transform(parse_stream_map),
-            [{
-                "itag": validate.all(
-                    validate.text,
-                    validate.transform(int)
-                ),
-                "quality": validate.text,
-                "url": validate.url(scheme="http"),
-                validate.optional("s"): validate.text,
-                validate.optional("stereo3d"): validate.all(
-                    validate.text,
-                    validate.transform(int),
-                    validate.transform(bool)
-                ),
-            }]
-        ),
-        validate.optional("adaptive_fmts"): validate.all(
-            validate.text,
-            validate.transform(parse_stream_map),
-            [{
-                validate.optional("s"): validate.text,
-                "type": validate.all(
-                    validate.text,
-                    validate.transform(lambda t: t.split(";")[0].split("/")),
-                    [validate.text, validate.text]
-                ),
-                "url": validate.all(
-                    validate.url(scheme="http")
-                )
-            }]
-        ),
-        validate.optional("hlsvp"): validate.text,
         validate.optional("player_response"): validate.all(
             validate.text,
             validate.transform(parse_json),
             {
                 validate.optional("streamingData"): {
                     validate.optional("hlsManifestUrl"): validate.text,
+                    validate.optional("formats"): [{
+                        "itag": int,
+                        validate.optional("url"): validate.text,
+                        validate.optional("cipher"): validate.text,
+                        "qualityLabel": validate.text
+                    }],
+                    validate.optional("adaptiveFormats"): [{
+                        "itag": int,
+                        "mimeType": validate.all(
+                            validate.text,
+                            validate.transform(
+                                lambda t:
+                                    [t.split(';')[0].split('/')[0], t.split(';')[1].split('=')[1].strip('"')]
+                            ),
+                            [validate.text, validate.text],
+                        ),
+                        validate.optional("url"): validate.url(scheme="http"),
+                        validate.optional("cipher"): validate.text,
+                        validate.optional("qualityLabel"): validate.text,
+                        validate.optional("bitrate"): int
+                    }]
                 },
                 validate.optional("videoDetails"): {
                     validate.optional("isLive"): validate.transform(bool),
+                    validate.optional("isLiveContent"): validate.transform(bool),
+                    validate.optional("author"): validate.text,
+                    validate.optional("title"): validate.all(validate.text,
+                                                             validate.transform(maybe_decode))
                 },
                 validate.optional("playabilityStatus"): {
                     validate.optional("status"): validate.text,
@@ -93,14 +60,6 @@ def parse_fmt_list(formatsmap):
                 },
             },
         ),
-        validate.optional("live_playback"): validate.transform(bool),
-        validate.optional("reason"): validate.all(validate.text, validate.transform(maybe_decode)),
-        validate.optional("livestream"): validate.text,
-        validate.optional("live_playback"): validate.text,
-        validate.optional("author"): validate.all(validate.text,
-                                                  validate.transform(maybe_decode)),
-        validate.optional("title"): validate.all(validate.text,
-                                                 validate.transform(maybe_decode)),
         "status": validate.text
     }
 )
@@ -146,6 +105,7 @@ class YouTube(Plugin):
         }
     )
 
+    # There are missing itags
     adp_video = {
         137: "1080p",
         299: "1080p60",  # HFR
@@ -155,6 +115,9 @@ class YouTube(Plugin):
         315: "2160p60",  # HFR
         138: "2160p",
         302: "720p60",  # HFR
+        135: "480p",
+        133: "240p",
+        160: "144p",
     }
     adp_audio = {
         140: 128,
@@ -231,16 +194,15 @@ def get_oembed(self):
         self.author = data["author_name"]
         self.title = data["title"]
 
-    def _create_adaptive_streams(self, info, streams, protected):
+    def _create_adaptive_streams(self, info, streams):
         adaptive_streams = {}
         best_audio_itag = None
 
-        # Extract audio streams from the DASH format list
-        for stream_info in info.get("adaptive_fmts", []):
-            if stream_info.get("s"):
-                protected = True
+        # Extract audio streams from the adaptive format list
+        streaming_data = info.get("player_response", {}).get("streamingData", {})
+        for stream_info in streaming_data.get("adaptiveFormats", []):
+            if "url" not in stream_info:
                 continue
-
             stream_params = dict(parse_qsl(stream_info["url"]))
             if "itag" not in stream_params:
                 continue
@@ -248,7 +210,8 @@ def _create_adaptive_streams(self, info, streams, protected):
             # extract any high quality streams only available in adaptive formats
             adaptive_streams[itag] = stream_info["url"]
 
-            stream_type, stream_format = stream_info["type"]
+            stream_type, stream_format = stream_info["mimeType"]
+
             if stream_type == "audio":
                 stream = HTTPStream(self.session, stream_info["url"])
                 name = "audio_{0}".format(stream_format)
@@ -272,7 +235,7 @@ def _create_adaptive_streams(self, info, streams, protected):
                                                 HTTPStream(self.session, vurl),
                                                 HTTPStream(self.session, aurl))
 
-        return streams, protected
+        return streams
 
     def _find_video_id(self, url):
 
@@ -325,16 +288,16 @@ def _get_stream_info(self, video_id):
 
             res = self.session.http.get(self._video_info_url, params=params)
             info_parsed = parse_query(res.content if is_py2 else res.text, name="config", schema=_config_schema)
-            if (info_parsed.get("player_response", {}).get("playabilityStatus", {}).get("status") != "OK"
-                    or info_parsed.get("status") == "fail"):
-                reason = (info_parsed.get("player_response", {}).get("playabilityStatus", {}).get("reason")
-                          or info_parsed.get("reason"))
+            player_response = info_parsed.get("player_response", {})
+            playability_status = player_response.get("playabilityStatus", {})
+            if (playability_status.get("status") != "OK"):
+                reason = playability_status.get("reason")
                 log.debug("get_video_info - {0}: {1}".format(
                     count, reason)
                 )
                 continue
-            self.author = info_parsed.get("author")
-            self.title = info_parsed.get("title")
+            self.author = player_response.get("videoDetails", {}).get("author")
+            self.title = player_response.get("videoDetails", {}).get("title")
             log.debug("get_video_info - {0}: Found data".format(count))
             break
 
@@ -354,35 +317,34 @@ def _get_streams(self):
             log.error("Could not get video info")
             return
 
-        if info.get("livestream") == '1' or info.get("live_playback") == '1' \
-                or info.get("player_response", {}).get("videoDetails", {}).get("isLive"):
+        if (info.get("player_response", {}).get("videoDetails", {}).get("isLiveContent")
+           or info.get("player_response", {}).get("videoDetails", {}).get("isLive")):
             log.debug("This video is live.")
             is_live = True
 
-        formats = info.get("fmt_list")
         streams = {}
         protected = False
-        for stream_info in info.get("url_encoded_fmt_stream_map", []):
-            if stream_info.get("s"):
-                protected = True
-                continue
+        if (info.get("player_response", {}).get("streamingData", {}).get("adaptiveFormats", [{}])[0].get("cipher")
+           or info.get("player_response", {}).get("streamingData", {}).get("formats", [{}])[0].get("cipher")):
+            protected = True
+            log.debug("This video may be protected.")
 
+        for stream_info in info.get("player_response", {}).get("streamingData", {}).get("formats", []):
+            if "url" not in stream_info:
+                continue
             stream = HTTPStream(self.session, stream_info["url"])
-            name = formats.get(stream_info["itag"]) or stream_info["quality"]
-
-            if stream_info.get("stereo3d"):
-                name += "_3d"
+            name = stream_info["qualityLabel"]
 
             streams[name] = stream
 
         if not is_live:
-            streams, protected = self._create_adaptive_streams(info, streams, protected)
+            streams = self._create_adaptive_streams(info, streams)
 
-        hls_playlist = info.get("hlsvp") or info.get("player_response", {}).get("streamingData", {}).get("hlsManifestUrl")
-        if hls_playlist:
+        hls_manifest = info.get("player_response", {}).get("streamingData", {}).get("hlsManifestUrl")
+        if hls_manifest:
             try:
                 hls_streams = HLSStream.parse_variant_playlist(
-                    self.session, hls_playlist, namekey="pixels"
+                    self.session, hls_manifest, namekey="pixels"
                 )
                 streams.update(hls_streams)
             except IOError as err:


Test patch: