Repository: streamlink/streamlink

Issue:
picarto.tv plugin broken
*Thanks for reporting an issue!*
*Please read the contribution guidelines first! (see the link above)*
*Also check the list of known issues before reporting an issue!*

*Feel free to use the following template. Be as detailed as possible.*
*Please see the text preview to avoid unnecessary formatting errors.*
*Don't forget to remove this text before submitting.*

----

### Checklist

- [x] This is a bug report.
- [ ] This is a feature request.
- [x] This is a plugin (improvement) request.
- [x] I have read the contribution guidelines.

### Description

Picarto.tv plugin erroneously reports channel offline

### Expected / Actual behavior

Command: streamlink https://picarto.tv/RedAxis best
causes player to open, stream to start in it.

Actually reports [plugin.picarto][error] The channel RedAxis is currently offline

When channel physically online and can be viewed in browser.

### Reproduction steps / Explicit stream URLs to test

1. Try open any picarto.tv stream
2. Fail
3. Check that stream works in browser

### Environment details

Operating system and version: Windows 7, OpenSUSE Tumbleweed
Streamlink and Python version: streamlink-script.py 0.8.1 for windows, streamlink 0.9.0 for Linux, Python 3.6

### Comments, logs, screenshots, etc.



[Love Streamlink? Please consider supporting our collective. Thanks!](https://opencollective.com/streamlink/donate)



Discussion:
OK, I've been digging into this a bit. It seems like the changes are mostly to the html of the Picarto stream page, and the plugin uses some semi-fragile regexes to scrape that page. Should be fairly simple to fix.

The only backend change I've noticed is the load balancer API page. It seems to now return json instead of just a url:

`$ curl https://picarto.tv/process/channel -d loadbalancinginfo=twokinds`

`{"edges":[{"label":"US East","id":"us-east1","ep":"edge5-us-east.picarto.tv"},{"label":"US West","id":"us-west1","ep":"edge5-us-east.picarto.tv"}],"preferedEdge":"us-east1","techs":[{"type":"application\/x-mpegurl","label":"HLS"},{"type":"video\/mp4","label":"MP4"},{"type":"rtmp\/mp4","label":"RTMP Flash"}]}`

In watching the behavior of the web client, it seems like the token field is just "public", so the token extraction code can probably be removed. Did this token serve any purpose previously?

Resolving patch:
diff --git a/src/streamlink/plugins/picarto.py b/src/streamlink/plugins/picarto.py
--- a/src/streamlink/plugins/picarto.py
+++ b/src/streamlink/plugins/picarto.py
@@ -1,6 +1,7 @@
 from __future__ import print_function
 
 import re
+import json
 
 from streamlink.plugin import Plugin
 from streamlink.plugin.api import http
@@ -9,50 +10,30 @@
 
 
 class Picarto(Plugin):
-    API_CHANNEL_INFO = "https://picarto.tv/process/channel"
+    CHANNEL_API_URL = "https://api.picarto.tv/v1/channel/name/{channel}"
+    VIDEO_API_URL = "https://picarto.tv/process/channel"
     RTMP_URL = "rtmp://{server}:1935/play/"
     RTMP_PLAYPATH = "golive+{channel}?token={token}"
     HLS_URL = "https://{server}/hls/{channel}/index.m3u8?token={token}"
 
+    # Regex for all usable URLs
     _url_re = re.compile(r"""
         https?://(?:\w+\.)?picarto\.tv/(?:videopopout/)?([^&?/]+)
     """, re.VERBOSE)
 
-    # divs with tech_switch class
-    _tech_switch_re = re.compile(r"""
-    <div\s+class=".*?tech_switch.*?"(.*?)>
-    """, re.VERBOSE)
-    # placeStream(channel, playerID, product, offlineImage, online, token, tech)
-    _place_stream_re = re.compile(r"""
-        <script>\s*placeStream\s*\((.*?)\);?\s*</script>
-    """, re.VERBOSE)
-    # <source ...>
-    _source_re = re.compile(r'''source src="(http[^"]+)"''')
+    # Regex for VOD extraction
+    _vod_re = re.compile(r'''vod: "(https?://[\S]+?/index.m3u8)",''')
 
     @classmethod
     def can_handle_url(cls, url):
         return cls._url_re.match(url) is not None
 
-    @classmethod
-    def _stream_online(cls, page):
-        match = cls._place_stream_re.search(page.text)
-        if match:
-            return match.group(1).split(",")[4].strip() == "1"
-        return False
-
-    @classmethod
-    def _get_steam_list(cls, page):
-        for match in cls._tech_switch_re.findall(page.text):
-            args = {}
-            for attr in match.strip().split(" "):
-                key, value = attr.split("=")
-                _, key = key.split("-")
-                args[key] = value.strip('"')
-            yield args
-
-    def _create_hls_stream(self, server, args):
+    def _create_hls_stream(self, server, channel, token):
         streams = HLSStream.parse_variant_playlist(self.session,
-                                                   self.HLS_URL.format(server=server, **args),
+                                                   self.HLS_URL.format(
+                                                       server=server,
+                                                       channel=channel,
+                                                       token=token),
                                                    verify=False)
         if len(streams) > 1:
             self.logger.debug("Multiple HLS streams found")
@@ -64,67 +45,68 @@ def _create_hls_stream(self, server, args):
             # one HLS streams, rename it to live
             return {"live": list(streams.values())[0]}
 
-    def _create_flash_stream(self, server, args):
+    def _create_flash_stream(self, server, channel, token):
         params = {
             "rtmp": self.RTMP_URL.format(server=server),
-            "playpath": self.RTMP_PLAYPATH.format(**args)
+            "playpath": self.RTMP_PLAYPATH.format(token=token, channel=channel)
         }
         return RTMPStream(self.session, params=params)
 
     def _get_vod_stream(self, page):
-        m = self._source_re.search(page.text)
+        m = self._vod_re.search(page.text)
         if m:
             return HLSStream.parse_variant_playlist(self.session, m.group(1))
 
     def _get_streams(self):
-        page = http.get(self.url)
+        url_channel_name = self._url_re.match(self.url).group(1)
 
-        page_channel = self._url_re.match(self.url).group(1)
-        if page_channel.endswith(".flv"):
+        # Handle VODs first, since their "channel name" is different
+        if url_channel_name.endswith(".flv"):
             self.logger.debug("Possible VOD stream...")
+            page = http.get(self.url)
             vod_streams = self._get_vod_stream(page)
             if vod_streams:
                 for s in vod_streams.items():
                     yield s
                 return
+            else:
+                self.logger.warning("Probably a VOD stream but no VOD found?")
+
+        ci = http.get(self.CHANNEL_API_URL.format(channel=url_channel_name), raise_for_status=False)
 
-        if "This channel does not exist" in page.text:
-            self.logger.error("The channel {0} does not exist".format(page_channel))
+        if ci.status_code == 404:
+            self.logger.error("The channel {0} does not exist".format(url_channel_name))
             return
 
-        if not self._stream_online(page):
-            self.logger.error("The channel {0} is currently offline".format(page_channel))
+        channel_api_json = json.loads(ci.text)
+
+        if channel_api_json["online"] != True:
+            self.logger.error("The channel {0} is currently offline".format(url_channel_name))
             return
 
         server = None
-        streams = list(self._get_steam_list(page))
-        multi = False
-
-        for args in streams:
-            channel, tech, token = args["channel"], args["tech"], args["token"]
-            if channel.lower() != page_channel.lower():
-                if not multi:
-                    self.logger.info("Skipping multi-channel stream for: {0}".format(channel))
-                    multi = True
-                continue
-
-            self.logger.debug("Found stream for {channel}; tech=\"{tech}\", token=\"{token}\"", **args)
-
-            # cache the load balancing info
-            if not server:
-                channel_server_res = http.post(self.API_CHANNEL_INFO, data={"loadbalancinginfo": channel})
-                server = channel_server_res.text
-                self.logger.debug("Using load balancing server {0} for channel {1}",
-                                  server,
-                                  channel)
-
-            # generate all the streams, for multi-channel streams also append the channel name
-            if tech == "hls":
-                for s in self._create_hls_stream(server, args).items():
+        token = "public"
+        channel = channel_api_json["name"]
+
+        # Extract preferred edge server and available techs from the undocumented channel API
+        channel_server_res = http.post(self.VIDEO_API_URL, data={"loadbalancinginfo": channel})
+        info_json = json.loads(channel_server_res.text)
+        pref = info_json["preferedEdge"]
+        for i in info_json["edges"]:
+            if i["id"] == pref:
+                server = i["ep"]
+                break
+        self.logger.debug("Using load balancing server {0} : {1} for channel {2}",
+                          pref,
+                          server,
+                          channel)
+
+        for i in info_json["techs"]:
+            if i["label"] == "HLS":
+                for s in self._create_hls_stream(server, channel, token).items():
                     yield s
-
-            elif tech == "flash":
-                stream = self._create_flash_stream(server, args)
+            elif i["label"] == "RTMP Flash":
+                stream = self._create_flash_stream(server, channel, token)
                 yield "live", stream
 
 


Test patch: