Repository: streamlink/streamlink
Pull request #3243 (2020-10-10 10:38:59)

plugins.livestream: remove AkamaiHDStream, use only secure HLSStream
closes https://github.com/streamlink/streamlink/issues/3240


Patch:
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
new file mode 100644
index 00000000..42531e08
--- /dev/null
+++ b/.github/pull_request_template.md
@@ -0,0 +1,13 @@
+<!--
+Thanks for opening a pull request!
+
+Before you continue, please make sure that you have read and understood the contribution guidelines, otherwise your changes may be rejected:
+https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink
+
+If possible, run the tests, perform code linting and build the documentation locally on your system first to avoid unnecessary build failures:
+https://streamlink.github.io/latest/developing.html#validating-changes
+
+Also don't forget to add a meaningful description of your changes, so that the reviewing process is as simple as possible for the maintainers.
+
+Thank you very much!
+-->
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 53edf5c2..9ba3e6ed 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -43,11 +43,17 @@ Plugins which fall under the following categories will not be implemented or con
 
 5. NSFW sites of a pornographic nature (cam sites, porn sites, etc.)
 
-6. Sites which are mostly hosting VODs (stolen or legitimate), or require large changes to existing code to watch VODs
+6. Sites which don't provide any real live streaming content, eg. only VODs or VODs being rebroadcasted
 
-7. Sites where the maintainer has requested we not add their site to Streamlink
+7. Sites which don't provide any video streaming content, eg. radio stations
 
-8. Sites which are unmaintained.
+8. Sites where the maintainer has requested we not add their site to Streamlink
+
+9. Sites which are unmaintained, are in beta or are undergoing heavy amounts of development and may change rapidly
+
+10. Sites which have no way to determine viewership numbers
+
+11. Sites which are static cameras of a physical location
 
 ## Pull requests
 
diff --git a/docs/developing.rst b/docs/developing.rst
new file mode 100644
index 00000000..3c253805
--- /dev/null
+++ b/docs/developing.rst
@@ -0,0 +1,116 @@
+Developing
+==========
+
+Setup
+-----
+
+Setting up the repository
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In order to start working on Streamlink, you must first install the latest stable version of ``git``, optionally fork the
+repository on Github onto your account if you want to submit changes in a pull request, and then locally clone the repository.
+
+.. code-block:: bash
+
+    mkdir streamlink
+    cd streamlink
+    git clone --origin=upstream 'https://github.com/streamlink/streamlink.git' .
+    git remote add fork 'git@github.com:<YOUR-USERNAME>/streamlink.git'
+    git remote -v
+    git fetch --all
+
+When submitting a pull request, commit and push your changes onto a different branch.
+
+.. code-block:: bash
+
+    git checkout master
+    git pull upstream master
+    git checkout -b new/feature/or/bugfix/branch
+    git add ./foo
+    git commit
+    git push fork new/feature/or/bugfix/branch
+
+
+Setting up a new environment
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+While working on any kind of python-based project, it is usually best to do this in a virtual environment which is isolated from
+the Python environment of the host system. This ensures that development can be done in a clean space which is free of version
+conflicts and other unrelated packages.
+
+First, make sure that you have the latest stable versions of Python and pip installed.
+
+.. code-block:: bash
+
+    python --version
+    pip --version
+
+As the second preparation step, install the latest version of ``virtualenv``, either via your system's package manager or pip,
+and create a new virtual environment. These environments can be created separately for different Python versions. Please refer
+to the `virtualenv documentation`_ for all available parameters. There are also several wrappers around virtualenv available.
+
+.. code-block:: bash
+
+    pip install --upgrade --user virtualenv
+    virtualenv --version
+
+    # replace ~/venvs/streamlink with your path of choice and give it a proper name
+    virtualenv --download --verbose ~/venvs/streamlink
+
+Now activate the virtual environment by sourcing the activation shell script.
+
+.. code-block:: bash
+
+    source ~/venvs/streamlink/bin/activate
+
+    # non-POSIX shells have their own activation script, eg. FISH
+    source ~/venvs/streamlink/bin/activate.fish
+
+    # on Windows, activation scripts are located in the Scripts/ subdirectory instead of bin/
+    source ~/venvs/streamlink/Scripts/activate
+
+.. _virtualenv documentation: https://virtualenv.pypa.io/en/latest/
+
+
+Installing Streamlink
+^^^^^^^^^^^^^^^^^^^^^
+
+After activating the new virtual environment, Streamlink's build dependencies and Streamlink itself need to be installed.
+Regular development dependencies and documentation related dependencies are listed in the text files shown below and need to
+be installed separately.
+
+.. code-block:: bash
+
+    # install additional dependencies
+    pip install -r dev-requirements.txt
+    pip install -r docs-requirements.txt
+
+    # install Streamlink from source
+    # check setup.py for optional dependencies and install those manually if you need to
+    pip install -e .
+
+    # validate that Streamlink is working
+    which streamlink
+    streamlink --version
+
+
+Validating changes
+------------------
+
+Before submitting a pull request, run tests, perform code linting and build the documentation on your system first, to see if
+your changes contain any mistakes or errors. This will be done automatically for each pull request on each change, but
+performing these checks locally avoids unnecessary build failures.
+
+.. code-block:: bash
+
+    # run automated tests
+    python -m pytest -ra
+    # or just run a subset of all tests
+    python -m pytest -ra path/to/test-file.py::TestClassName::test_method_name ...
+
+    # check code for linting errors
+    flake8
+
+    # build the documentation
+    make --directory=docs clean html
+    $BROWSER ./docs/_build/html/index.html
diff --git a/docs/index.rst b/docs/index.rst
index 2d2fa53a..1c31123a 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -64,6 +64,7 @@ See their respective sections for more information on how to use them.
     plugin_matrix
     players
     issues
+    developing
     api_guide
     api
     changelog
diff --git a/docs/plugin_matrix.rst b/docs/plugin_matrix.rst
index d383592f..bd8dd9bd 100644
--- a/docs/plugin_matrix.rst
+++ b/docs/plugin_matrix.rst
@@ -38,6 +38,7 @@ brightcove              players.brig... [6]_ Yes   Yes
 btsports                sport.bt.com         Yes   Yes   Requires subscription account
 btv                     btvplus.bg           Yes   No    Streams are geo-restricted to Bulgaria.
 canalplus               mycanal.fr           No    Yes   Streams may be geo-restricted to France.
+cbsnews                 cbsnews.com          Yes   No
 cdnbg                   - armymedia.bg       Yes   No    Streams may be geo-restricted to Bulgaria
                         - bgonair.bg
                         - bloombergtv.bg
@@ -111,7 +112,7 @@ live_russia_tv          live.russia.tv       Yes   --
 liveedu                 - liveedu.tv         Yes   --    Some streams require a login.
                         - livecoding.tv
 liveme                  liveme.com           Yes   --
-livestream              new.livestream.com   Yes   --
+livestream              livestream.com       Yes   --
 lrt                     lrt.lt               Yes   No
 ltv_lsm_lv              ltv.lsm.lv           Yes   No    Streams may be geo-restricted to Latvia.
 mediaklikk              mediaklikk.hu        Yes   No    Streams may be geo-restricted to Hungary.
@@ -219,7 +220,7 @@ turkuvaz                - atv.com.tr         Yes   No    Streams may be geo-rest
                         - minikacocuk.com.tr
                         - minikago.com.tr
                         - sabah.com.tr
-tv3cat                  tv3.cat              Yes   Yes   Streams may be geo-restricted to Spain.
+tv3cat                  ccma.cat             Yes   Yes   Streams may be geo-restricted to Spain.
 tv4play                 - tv4play.se         Yes   Yes   Streams may be geo-restricted to Sweden.
                                                          Only non-premium streams currently supported.
                         - fotbollskanalen.se
diff --git a/script/deploy-docs.sh b/script/deploy-docs.sh
index e64a81ca..cd83b487 100755
--- a/script/deploy-docs.sh
+++ b/script/deploy-docs.sh
@@ -4,7 +4,7 @@ set -eo pipefail
 
 [[ -n "${GITHUB_ACTIONS}" ]] || exit 1
 
-ROOT=$(realpath "$(dirname "$(readlink -f "${0}")")/..")
+ROOT=$(git rev-parse --show-toplevel 2>/dev/null || realpath "$(dirname "$(readlink -f "${0}")")/..")
 
 DOCS_REPO=${DOCS_REPO:-streamlink/streamlink.github.io}
 DOCS_BRANCH=${DOCS_BRANCH:-master}
@@ -69,7 +69,7 @@ cp --archive "${SOURCE}/." "${DEST}/"
 echo Building a new file list in \'${DEST}\'
 ( cd "${SOURCE}"; find . -type f | sort | sed "s/^\\.\//${DEST}\//" > "${TEMP}/${DEST}/${FILELIST}" )
 
-if git diff-index --quiet HEAD --; then
+if [[ -z "$(git status --porcelain)" ]]; then
     echo No changes to be committed. Exiting...
     exit 0
 fi
diff --git a/src/streamlink/plugins/cbsnews.py b/src/streamlink/plugins/cbsnews.py
new file mode 100644
index 00000000..4e2bb8c8
--- /dev/null
+++ b/src/streamlink/plugins/cbsnews.py
@@ -0,0 +1,38 @@
+import re
+
+from streamlink.plugin import Plugin
+from streamlink.plugin.api import validate
+from streamlink.stream import HLSStream
+from streamlink.utils import parse_json
+
+
+class CBSNews(Plugin):
+    url_re = re.compile(r"https://www\.cbsnews\.com/live/")
+    _re_default_payload = re.compile(r"CBSNEWS.defaultPayload = (\{.*)")
+
+    _schema_items = validate.Schema(
+        validate.transform(_re_default_payload.search),
+        validate.any(None, validate.all(
+            validate.get(1),
+            validate.transform(parse_json),
+            {"items": [validate.all({
+                "video": validate.url(),
+                "format": "application/x-mpegURL"
+            }, validate.get("video"))]},
+            validate.get("items")
+        ))
+    )
+
+    @classmethod
+    def can_handle_url(cls, url):
+        return cls.url_re.match(url) is not None
+
+    def _get_streams(self):
+        items = self.session.http.get(self.url, schema=self._schema_items)
+        if items:
+            for hls_url in items:
+                for s in HLSStream.parse_variant_playlist(self.session, hls_url).items():
+                    yield s
+
+
+__plugin__ = CBSNews
diff --git a/src/streamlink/plugins/dlive.py b/src/streamlink/plugins/dlive.py
index dac758e6..b457f3da 100644
--- a/src/streamlink/plugins/dlive.py
+++ b/src/streamlink/plugins/dlive.py
@@ -1,8 +1,14 @@
+import json
+import logging
 import re
 
+from streamlink.compat import is_py3, unquote_plus
 from streamlink.plugin import Plugin, PluginError
+from streamlink.plugin.api import validate
 from streamlink.stream import HLSStream
-from streamlink.compat import unquote_plus, is_py3
+
+
+log = logging.getLogger(__name__)
 
 
 QUALITY_WEIGHTS = {
@@ -11,18 +17,43 @@ QUALITY_WEIGHTS = {
 
 
 class DLive(Plugin):
-    """
-    Plugin for dlive.tv
-    """
-
-    _url_re = re.compile(r"https?://(?:www\.)?dlive\.tv/")
-    _playback_re = re.compile(r"""(?<=playbackUrl":")(.+?)(?=")""")
-    _livestream_re = re.compile(r""""livestream":null""")
-    _username_re = re.compile(r"(?<=user:)(\w|-)+")
+    _re_url = re.compile(r"""
+        https?://(?:www\.)?dlive\.tv/
+        (?:
+            (?:p/(?P<video>[^/]+))
+            |
+            (?P<channel>[^/]+)
+        )
+    """, re.VERBOSE)
+    _re_videoPlaybackUrl = re.compile(r'"playbackUrl"\s*:\s*"([^"]+\.m3u8)"')
+
+    _schema_userByDisplayName = validate.Schema({
+        "data": {
+            "userByDisplayName": {
+                "livestream": validate.any(None, {
+                    "title": validate.text
+                }),
+                "username": validate.text
+            }
+        }},
+        validate.get("data"),
+        validate.get("userByDisplayName")
+    )
+    _schema_videoPlaybackUrl = validate.Schema(
+        validate.transform(_re_videoPlaybackUrl.search),
+        validate.any(None, validate.all(
+            validate.get(1),
+            validate.transform(unquote_plus),
+            validate.transform(lambda url: (
+                bytes(url, "utf-8").decode("unicode_escape") if is_py3 else url.decode("unicode_escape")
+            )),
+            validate.url()
+        ))
+    )
 
     @classmethod
     def can_handle_url(cls, url):
-        return cls._url_re.match(url)
+        return cls._re_url.match(url)
 
     @classmethod
     def stream_weight(cls, key):
@@ -32,31 +63,62 @@ class DLive(Plugin):
 
         return Plugin.stream_weight(key)
 
-    def _get_streams(self):
-        res = self.session.http.get(self.url)
+    def __init__(self, *args, **kwargs):
+        super(DLive, self).__init__(*args, **kwargs)
+        self.author = None
+        self.title = None
 
-        m = self._playback_re.search(res.text)
+        match = self._re_url.match(self.url)
+        self.video = match.group("video")
+        self.channel = match.group("channel")
 
-        if m:
-            hls_url = m.group(0)
+    def get_author(self):
+        return self.author
 
-            if is_py3:
-                hls_url = bytes(unquote_plus(hls_url), "utf-8").decode(
-                    "unicode_escape")
-            else:
-                hls_url = unquote_plus(hls_url).decode("unicode_escape")
-        else:
-            if self._livestream_re.search(res.text) is not None:
-                raise PluginError('Stream is offline')
+    def get_title(self):
+        return self.title
 
-            m = self._username_re.search(res.text)
-            if m:
-                hls_url = "https://live.prd.dlive.tv/hls/live/{}.m3u8".format(
-                    m.group(0))
-            else:
-                raise PluginError('Could not find username')
+    def _get_streams_video(self):
+        log.debug("Getting video HLS streams for {0}".format(self.video))
+        try:
+            hls_url = self.session.http.get(self.url, schema=self._schema_videoPlaybackUrl)
+            if hls_url is None:
+                return
+        except PluginError:
+            return
 
         return HLSStream.parse_variant_playlist(self.session, hls_url)
 
+    def _get_streams_live(self):
+        log.debug("Getting live HLS streams for {0}".format(self.channel))
+        try:
+            data = json.dumps({"query": """query {{
+                userByDisplayName(displayname:"{displayname}") {{
+                    livestream {{
+                        title
+                    }}
+                    username
+                }}
+            }}""".format(displayname=self.channel)})
+            res = self.session.http.post("https://graphigo.prd.dlive.tv/", data=data)
+            res = self.session.http.json(res, schema=self._schema_userByDisplayName)
+            if res["livestream"] is None:
+                return
+        except PluginError:
+            return
+
+        self.author = self.channel
+        self.title = res["livestream"]["title"]
+
+        hls_url = "https://live.prd.dlive.tv/hls/live/{0}.m3u8".format(res["username"])
+
+        return HLSStream.parse_variant_playlist(self.session, hls_url)
+
+    def _get_streams(self):
+        if self.video:
+            return self._get_streams_video()
+        elif self.channel:
+            return self._get_streams_live()
+
 
 __plugin__ = DLive
diff --git a/src/streamlink/plugins/livestream.py b/src/streamlink/plugins/livestream.py
index f490c95b..6ca88738 100644
--- a/src/streamlink/plugins/livestream.py
+++ b/src/streamlink/plugins/livestream.py
@@ -1,133 +1,49 @@
+import logging
 import re
 
-from streamlink.compat import urljoin
 from streamlink.plugin import Plugin
 from streamlink.plugin.api import validate
 from streamlink.plugin.api.utils import parse_json
-from streamlink.stream import AkamaiHDStream, HLSStream
+from streamlink.stream import HLSStream
 
-_url_re = re.compile(r"http(s)?://(www\.)?livestream.com/")
-_stream_config_schema = validate.Schema({
-    "event": {
-        "stream_info": validate.any({
-            "is_live": bool,
-            "qualities": [{
-                "bitrate": int,
-                "height": int
-            }],
-            validate.optional("play_url"): validate.url(scheme="http"),
-            validate.optional("m3u8_url"): validate.url(
-                scheme="http",
-                path=validate.endswith(".m3u8")
-            ),
-        }, None)
-    },
-    validate.optional("playerUri"): validate.text,
-    validate.optional("viewerPlusSwfUrl"): validate.url(scheme="http"),
-    validate.optional("lsPlayerSwfUrl"): validate.text,
-    validate.optional("hdPlayerSwfUrl"): validate.text
-})
-_smil_schema = validate.Schema(validate.union({
-    "http_base": validate.all(
-        validate.xml_find("{http://www.w3.org/2001/SMIL20/Language}head/"
-                          "{http://www.w3.org/2001/SMIL20/Language}meta"
-                          "[@name='httpBase']"),
-        validate.xml_element(attrib={
-            "content": validate.text
-        }),
-        validate.get("content")
-    ),
-    "videos": validate.all(
-        validate.xml_findall("{http://www.w3.org/2001/SMIL20/Language}body/"
-                             "{http://www.w3.org/2001/SMIL20/Language}switch/"
-                             "{http://www.w3.org/2001/SMIL20/Language}video"),
-        [
-            validate.all(
-                validate.xml_element(attrib={
-                    "src": validate.text,
-                    "system-bitrate": validate.all(
-                        validate.text,
-                        validate.transform(int)
-                    )
-                }),
-                validate.transform(
-                    lambda e: (e.attrib["src"], e.attrib["system-bitrate"])
-                )
-            )
-        ],
-    )
-}))
+log = logging.getLogger(__name__)
 
 
 class Livestream(Plugin):
-    @classmethod
-    def default_stream_types(cls, streams):
-        return ["akamaihd", "hls"]
+    _url_re = re.compile(r"https?://(?:www\.)?livestream\.com/")
+    _config_re = re.compile(r"window.config = ({.+})")
+    _stream_config_schema = validate.Schema(validate.any({
+        "event": {
+            "stream_info": validate.any({
+                "is_live": bool,
+                "secure_m3u8_url": validate.url(scheme="https"),
+            }, None),
+        }
+    }, {}), validate.get("event", {}), validate.get("stream_info", {}))
 
     @classmethod
-    def can_handle_url(self, url):
-        return _url_re.match(url)
-
-    def _get_stream_info(self):
-        res = self.session.http.get(self.url)
-        match = re.search("window.config = ({.+})", res.text)
-        if match:
-            config = match.group(1)
-            return parse_json(config, "config JSON",
-                              schema=_stream_config_schema)
-
-    def _parse_smil(self, url, swf_url):
-        res = self.session.http.get(url)
-        smil = self.session.http.xml(res, "SMIL config", schema=_smil_schema)
-
-        for src, bitrate in smil["videos"]:
-            url = urljoin(smil["http_base"], src)
-            yield bitrate, AkamaiHDStream(self.session, url, swf=swf_url)
+    def can_handle_url(cls, url):
+        return cls._url_re.match(url) is not None
 
     def _get_streams(self):
-        info = self._get_stream_info()
-        if not info:
+        res = self.session.http.get(self.url)
+        m = self._config_re.search(res.text)
+        if not m:
+            log.debug("Unable to find _config_re")
             return
 
-        stream_info = info["event"]["stream_info"]
+        stream_info = parse_json(m.group(1), "config JSON",
+                                 schema=self._stream_config_schema)
+
+        log.trace("stream_info: {0!r}".format(stream_info))
         if not (stream_info and stream_info["is_live"]):
-            # Stream is not live
+            log.debug("Stream might be Off Air")
             return
 
-        play_url = stream_info.get("play_url")
-        if play_url:
-            swf_url = (
-                info.get("playerUri")
-                or info.get("hdPlayerSwfUrl")
-                or info.get("lsPlayerSwfUrl")
-                or info.get("viewerPlusSwfUrl")
-            )
-            if swf_url:
-                if not swf_url.startswith("http"):
-                    if swf_url.startswith("//"):
-                        swf_url = "http:" + swf_url
-                    else:
-                        swf_url = "http://" + swf_url
-
-                # Work around broken SSL.
-                swf_url = swf_url.replace("https://", "http://")
-
-            qualities = stream_info["qualities"]
-            for bitrate, stream in self._parse_smil(play_url, swf_url):
-                name = "{0:d}k".format(int(bitrate / 1000))
-                for quality in qualities:
-                    if quality["bitrate"] == bitrate:
-                        name = "{0}p".format(quality["height"])
-
-                yield name, stream
-
-        m3u8_url = stream_info.get("m3u8_url")
+        m3u8_url = stream_info.get("secure_m3u8_url")
         if m3u8_url:
-            streams = HLSStream.parse_variant_playlist(self.session, m3u8_url,
-                                                       namekey="pixels")
-            # TODO: Replace with "yield from" when dropping Python 2.
-            for stream in streams.items():
-                yield stream
+            for s in HLSStream.parse_variant_playlist(self.session, m3u8_url).items():
+                yield s
 
 
 __plugin__ = Livestream
diff --git a/src/streamlink/plugins/tv3cat.py b/src/streamlink/plugins/tv3cat.py
index 531fad8d..50bd3ba7 100644
--- a/src/streamlink/plugins/tv3cat.py
+++ b/src/streamlink/plugins/tv3cat.py
@@ -9,7 +9,7 @@ log = logging.getLogger(__name__)
 
 
 class TV3Cat(Plugin):
-    _url_re = re.compile(r"http://(?:www.)?ccma.cat/tv3/directe/(.+?)/")
+    _url_re = re.compile(r"https?://(?:www\.)?ccma\.cat/tv3/directe/(.+?)/")
     _stream_info_url = "http://dinamics.ccma.cat/pvideo/media.jsp" \
                        "?media=video&version=0s&idint={ident}&profile=pc&desplacament=0"
     _media_schema = validate.Schema({
@@ -39,7 +39,6 @@ class TV3Cat(Plugin):
                     return HLSStream.parse_variant_playlist(self.session, stream['url'], name_fmt="{pixels}_{bitrate}")
                 except PluginError:
                     log.debug("Failed to get streams for: {0}".format(stream['geo']))
-                    pass
 
 
 __plugin__ = TV3Cat
diff --git a/src/streamlink/plugins/twitch.py b/src/streamlink/plugins/twitch.py
index 616dbd11..fd5ca904 100644
--- a/src/streamlink/plugins/twitch.py
+++ b/src/streamlink/plugins/twitch.py
@@ -3,123 +3,25 @@ import argparse
 import json
 import logging
 import re
-import warnings
 from collections import namedtuple
 from random import random
 
 import requests
 
 from streamlink.compat import urlparse
-from streamlink.exceptions import NoStreamsError, PluginError, StreamError
+from streamlink.exceptions import NoStreamsError, PluginError
 from streamlink.plugin import Plugin, PluginArguments, PluginArgument
 from streamlink.plugin.api import validate
 from streamlink.plugin.api.utils import parse_json, parse_query
-from streamlink.stream import (
-    HTTPStream, HLSStream, FLVPlaylist, extract_flv_header_tags
-)
+from streamlink.stream import HTTPStream, HLSStream
 from streamlink.stream.hls import HLSStreamWorker
 from streamlink.stream.hls_filtered import FilteredHLSStreamWriter, FilteredHLSStreamReader
 from streamlink.stream.hls_playlist import M3U8, M3U8Parser, load as load_hls_playlist
 from streamlink.utils.times import hours_minutes_seconds
 
-try:
-    from itertools import izip as zip
-except ImportError:
-    pass
-
 
 log = logging.getLogger(__name__)
 
-QUALITY_WEIGHTS = {
-    "source": 1080,
-    "1080": 1080,
-    "high": 720,
-    "720": 720,
-    "medium": 480,
-    "480": 480,
-    "360": 360,
-    "low": 240,
-    "mobile": 120,
-}
-
-# Streamlink's client-id used for public API calls (don't steal this and register your own application on Twitch)
-TWITCH_CLIENT_ID = "pwkzresl8kj2rdj6g7bvxl9ys1wly3j"
-# Twitch's client-id used for private API calls (see issue #2680 for why we are doing this)
-TWITCH_CLIENT_ID_PRIVATE = "kimne78kx3ncx6brgo4mv6wki5h1ko"
-
-_url_re = re.compile(r"""
-    http(s)?://
-    (?:
-        (?P<subdomain>[\w\-]+)
-        \.
-    )?
-    twitch.tv/
-    (?:
-        videos/(?P<videos_id>\d+)|
-        (?P<channel>[^/]+)
-    )
-    (?:
-        /
-        (?P<video_type>[bcv])(?:ideo)?
-        /
-        (?P<video_id>\d+)
-    )?
-    (?:
-        /(?:clip/)?
-        (?P<clip_name>[\w]+)
-    )?
-""", re.VERBOSE)
-
-_access_token_schema = validate.Schema(
-    {
-        "token": validate.text,
-        "sig": validate.text
-    },
-    validate.union((
-        validate.get("sig"),
-        validate.get("token")
-    ))
-)
-_token_schema = validate.Schema(
-    {
-        "chansub": {
-            "restricted_bitrates": validate.all(
-                [validate.text],
-                validate.filter(
-                    lambda n: not re.match(r"(.+_)?archives|live|chunked", n)
-                )
-            )
-        }
-    },
-    validate.get("chansub")
-)
-_stream_schema = validate.Schema(
-    {
-        "stream": validate.any(None, {
-            "stream_type": validate.text,
-            "broadcast_platform": validate.text,
-            "channel": validate.any(None, {
-                "broadcaster_software": validate.text
-            })
-        })
-    },
-    validate.get("stream")
-)
-_video_schema = validate.Schema(
-    {
-        "chunks": {
-            validate.text: [{
-                "length": int,
-                "url": validate.any(None, validate.url(scheme="http")),
-                "upkeep": validate.any("pass", "fail", None)
-            }]
-        },
-        "restrictions": {validate.text: validate.text},
-        "start_offset": int,
-        "end_offset": int,
-    }
-)
-
 Segment = namedtuple("Segment", "uri duration title key discontinuity ad byterange date map prefetch")
 
 LOW_LATENCY_MAX_LIVE_EDGE = 2
@@ -262,11 +164,7 @@ class UsherService(object):
         params.update(extra_params)
 
         req = requests.Request("GET", url, params=params)
-        # prepare_request is only available in requests 2.0+
-        if hasattr(self.session.http, "prepare_request"):
-            req = self.session.http.prepare_request(req)
-        else:
-            req = req.prepare()
+        req = self.session.http.prepare_request(req)
 
         return req.url
 
@@ -279,69 +177,212 @@ class UsherService(object):
 
 
 class TwitchAPI(object):
-    def __init__(self, session, beta=False, version=3):
+    # Streamlink's client-id used for public API calls (don't steal this and register your own application on Twitch)
+    TWITCH_CLIENT_ID = "pwkzresl8kj2rdj6g7bvxl9ys1wly3j"
+    # Twitch's client-id used for private API calls (see issue #2680 for why we are doing this)
+    TWITCH_CLIENT_ID_PRIVATE = "kimne78kx3ncx6brgo4mv6wki5h1ko"
+
+    def __init__(self, session):
         self.session = session
-        self.subdomain = beta and "betaapi" or "api"
-        self.version = version
 
-    def call(self, path, format="json", schema=None, private=False, **extra_params):
-        params = dict(as3="t", **extra_params)
+    def _call(self, method="GET", subdomain="api", path="/", headers=None, private=False, data=None, **params):
+        url = "https://{0}.twitch.tv{1}".format(subdomain, path)
+        headers = headers or dict()
+        headers.update({
+            "Client-ID": self.TWITCH_CLIENT_ID if not private else self.TWITCH_CLIENT_ID_PRIVATE
+        })
 
-        if len(format) > 0:
-            url = "https://{0}.twitch.tv{1}.{2}".format(self.subdomain, path, format)
-        else:
-            url = "https://{0}.twitch.tv{1}".format(self.subdomain, path)
+        return self.session.http.request(method, url, data=data, params=params, headers=headers)
 
-        headers = {'Accept': 'application/vnd.twitchtv.v{0}+json'.format(self.version),
-                   'Client-ID': TWITCH_CLIENT_ID if not private else TWITCH_CLIENT_ID_PRIVATE}
+    def call(self, path, schema=None, **params):
+        headers = {"Accept": "application/vnd.twitchtv.v5+json"}
+        res = self._call(path=path, headers=headers, **params)
 
-        res = self.session.http.get(url, params=params, headers=headers)
+        return self.session.http.json(res, schema=schema)
 
-        if format == "json":
-            return self.session.http.json(res, schema=schema)
-        else:
-            return res
+    def call_gql(self, data, schema=None, **params):
+        res = self._call(method="POST", subdomain="gql", path="/gql", data=json.dumps(data), private=True, **params)
 
-    def call_subdomain(self, subdomain, path, format="json", schema=None, **extra_params):
-        subdomain_buffer = self.subdomain
-        self.subdomain = subdomain
-        try:
-            return self.call(path, format=format, schema=schema, **extra_params)
-        finally:
-            self.subdomain = subdomain_buffer
+        return self.session.http.json(res, schema=schema)
 
     # Public API calls
 
-    def users(self, **params):
-        return self.call("/kraken/users", **params)
-
-    def videos(self, video_id, **params):
-        return self.call("/kraken/videos/{0}".format(video_id), **params)
-
-    def channel_info(self, channel, **params):
-        return self.call("/kraken/channels/{0}".format(channel), **params)
-
-    def streams(self, channel_id, **params):
-        return self.call("/kraken/streams/{0}".format(channel_id), **params)
+    def channel_from_video_id(self, video_id):
+        return self.call("/kraken/videos/{0}".format(video_id), schema=validate.Schema(
+            {
+                "channel": {
+                    "_id": validate.any(int, validate.text),
+                    "name": validate.text
+                }
+            },
+            validate.get("channel"),
+            validate.union((
+                validate.all(validate.get("_id"), validate.transform(int)),
+                validate.all(validate.get("name"), validate.transform(lambda n: n.lower()))
+            ))
+        ))
+
+    def channel_from_login(self, channel):
+        return self.call("/kraken/users", login=channel, schema=validate.Schema(
+            {
+                "users": [{
+                    "_id": validate.any(int, validate.text)
+                }]
+            },
+            validate.get("users"),
+            validate.get(0),
+            validate.get("_id"),
+            validate.transform(int)
+        ))
+
+    def stream_rerun(self, channel_id):
+        return self.call("/kraken/streams/{0}".format(channel_id), schema=validate.Schema(
+            {
+                "stream": validate.any(None, validate.all({
+                    "stream_type": validate.text,
+                    "broadcast_platform": validate.text,
+                    "channel": validate.all({
+                        "broadcaster_software": validate.text
+                    }, validate.get("broadcaster_software"))
+                }, validate.union((
+                    validate.get("stream_type"),
+                    validate.get("broadcast_platform"),
+                    validate.get("channel")
+                ))))
+            },
+            validate.get("stream")
+        ))
+
+    def metadata_video(self, video_id):
+        return self.call("/kraken/videos/{0}".format(video_id), schema=validate.Schema(validate.any(
+            validate.all(
+                {
+                    "title": validate.text,
+                    "game": validate.text,
+                    "channel": validate.all(
+                        {"display_name": validate.text},
+                        validate.get("display_name")
+                    )
+                },
+                validate.transform(lambda data: (data["channel"], data["title"], data["game"]))
+            ),
+            validate.all({}, validate.transform(lambda _: (None,) * 3))
+        )))
+
+    def metadata_channel(self, channel_id):
+        return self.call("/kraken/streams/{0}".format(channel_id), schema=validate.Schema(
+            {
+                "stream": validate.any(
+                    validate.all(
+                        {"channel": {
+                            "display_name": validate.text,
+                            "game": validate.text,
+                            "status": validate.text
+                        }},
+                        validate.get("channel"),
+                        validate.transform(lambda ch: (ch["display_name"], ch["status"], ch["game"]))
+                    ),
+                    validate.all(None, validate.transform(lambda _: (None,) * 3))
+                )
+            },
+            validate.get("stream")
+        ))
 
     # Private API calls
 
-    def access_token(self, endpoint, asset, **params):
-        return self.call("/api/{0}/{1}/access_token".format(endpoint, asset), private=True, **params)
-
-    def hosted_channel(self, **params):
-        return self.call_subdomain("tmi", "/hosts", format="", **params)
-
-    # Unsupported/Removed private API calls
-
-    def channel_viewer_info(self, channel, **params):
-        warnings.warn("The channel_viewer_info API call is unsupported and may stop working at any time")
-        return self.call("/api/channels/{0}/viewer".format(channel), private=True, **params)
+    def access_token(self, endpoint, asset):
+        return self.call("/api/{0}/{1}/access_token".format(endpoint, asset), private=True, schema=validate.Schema(
+            {
+                "token": validate.text,
+                "sig": validate.text
+            },
+            validate.union((
+                validate.get("sig"),
+                validate.get("token")
+            ))
+        ))
+
+    def token(self, tokenstr):
+        return parse_json(tokenstr, schema=validate.Schema(
+            {
+                "chansub": {
+                    "restricted_bitrates": validate.all(
+                        [validate.text],
+                        validate.filter(
+                            lambda n: not re.match(r"(.+_)?archives|live|chunked", n)
+                        )
+                    )
+                }
+            },
+            validate.get("chansub"),
+            validate.get("restricted_bitrates")
+        ))
 
-    def channel_subscription(self, channel, **params):
-        warnings.warn("The channel_subscription API call has been removed and no longer works",
-                      category=DeprecationWarning)
-        return self.call("/api/channels/{0}/subscription".format(channel), private=True, **params)
+    def hosted_channel(self, channel_id):
+        return self.call("/hosts", subdomain="tmi", include_logins=1, host=channel_id, schema=validate.Schema(
+            {
+                "hosts": [{
+                    "host_id": int,
+                    "target_id": int,
+                    "target_login": validate.text,
+                    "target_display_name": validate.text
+                }]
+            },
+            validate.get("hosts"),
+            validate.get(0),
+            validate.union((
+                validate.get("host_id"),
+                validate.get("target_id"),
+                validate.get("target_login"),
+                validate.get("target_display_name")
+            ))
+        ))
+
+    # GraphQL API calls
+
+    def clips(self, clipname):
+        query = """{{
+            clip(slug: "{clipname}") {{
+                broadcaster {{
+                    displayName
+                }}
+                title
+                game {{
+                    name
+                }}
+                videoQualities {{
+                    quality
+                    sourceURL
+                }}
+            }}
+        }}""".format(clipname=clipname)
+
+        return self.call_gql({"query": query}, schema=validate.Schema(
+            {"data": {
+                "clip": validate.any(None, validate.all({
+                    "broadcaster": validate.all({"displayName": validate.text}, validate.get("displayName")),
+                    "title": validate.text,
+                    "game": validate.all({"name": validate.text}, validate.get("name")),
+                    "videoQualities": [validate.all({
+                        "quality": validate.all(
+                            validate.text,
+                            validate.transform(lambda q: "{0}p".format(q))
+                        ),
+                        "sourceURL": validate.url()
+                    }, validate.union((
+                        validate.get("quality"),
+                        validate.get("sourceURL")
+                    )))]
+                }, validate.union((
+                    validate.get("broadcaster"),
+                    validate.get("title"),
+                    validate.get("game"),
+                    validate.get("videoQualities")
+                ))))
+            }},
+            validate.get("data"),
+            validate.get("clip")
+        ))
 
 
 class Twitch(Plugin):
@@ -396,83 +437,31 @@ class Twitch(Plugin):
         )
     )
 
-    _schema_metadata_empty = validate.transform(lambda _: (None,) * 3)
-    _schema_metadata_channel = validate.Schema(
-        {
-            "stream": validate.any(
-                validate.all(
-                    {"channel": {
-                        "display_name": validate.text,
-                        "game": validate.text,
-                        "status": validate.text
-                    }},
-                    validate.get("channel"),
-                    validate.transform(lambda ch: (ch["display_name"], ch["status"], ch["game"]))
-                ),
-                validate.all(None, _schema_metadata_empty)
-            )
-        },
-        validate.get("stream")
-    )
-    _schema_metadata_video = validate.Schema(validate.any(
-        validate.all(
-            {
-                "title": validate.text,
-                "game": validate.text,
-                "channel": validate.all(
-                    {"display_name": validate.text},
-                    validate.get("display_name")
-                )
-            },
-            validate.transform(lambda data: (data["channel"], data["title"], data["game"]))
-        ),
-        validate.all({}, _schema_metadata_empty)
-    ))
-
-    @classmethod
-    def stream_weight(cls, key):
-        weight = QUALITY_WEIGHTS.get(key)
-        if weight:
-            return weight, "twitch"
-
-        return Plugin.stream_weight(key)
+    _re_url = re.compile(r"""
+        https?://(?:(?P<subdomain>[\w\-]+)\.)?twitch\.tv/
+        (?:
+            videos/(?P<videos_id>\d+)
+            |
+            (?P<channel>[^/]+)
+            (?:
+                /video/(?P<video_id>\d+)
+                |
+                /clip/(?P<clip_name>[\w]+)
+            )?
+        )
+    """, re.VERBOSE)
 
     @classmethod
     def can_handle_url(cls, url):
-        return _url_re.match(url)
-
-    def _get_metadata(self):
-        if self.video_id:
-            (self.author, self.title, self.category) = self.api.videos(self.video_id, schema=self._schema_metadata_video)
-        elif self.clip_name:
-            self._get_clips()
-        elif self._channel:
-            (self.author, self.title, self.category) = self.api.streams(self.channel_id, schema=self._schema_metadata_channel)
-
-    def get_title(self):
-        if self.title is None:
-            self._get_metadata()
-        return self.title
-
-    def get_author(self):
-        if self.author is None:
-            self._get_metadata()
-        return self.author
-
-    def get_category(self):
-        if self.category is None:
-            self._get_metadata()
-        return self.category
+        return cls._re_url.match(url)
 
     def __init__(self, url):
-        Plugin.__init__(self, url)
-        self._hosted_chain = []
-        match = _url_re.match(url).groupdict()
+        super(Twitch, self).__init__(url)
+        match = self._re_url.match(url).groupdict()
         parsed = urlparse(url)
         self.params = parse_query(parsed.query)
         self.subdomain = match.get("subdomain")
         self.video_id = None
-        self.video_type = None
         self._channel_id = None
         self._channel = None
         self.clip_name = None
@@ -483,270 +472,147 @@ class Twitch(Plugin):
         if self.subdomain == "player":
             # pop-out player
             if self.params.get("video"):
-                try:
-                    self.video_type = self.params["video"][0]
-                    self.video_id = self.params["video"][1:]
-                except IndexError:
-                    log.debug("Invalid video param: {0}".format(self.params["video"]))
+                self.video_id = self.params["video"]
             self._channel = self.params.get("channel")
         elif self.subdomain == "clips":
             # clip share URL
             self.clip_name = match.get("channel")
         else:
             self._channel = match.get("channel") and match.get("channel").lower()
-            self.video_type = match.get("video_type")
-            if match.get("videos_id"):
-                self.video_type = "v"
             self.video_id = match.get("video_id") or match.get("videos_id")
             self.clip_name = match.get("clip_name")
 
-        self.api = TwitchAPI(beta=self.subdomain == "beta",
-                             session=self.session,
-                             version=5)
+        self.api = TwitchAPI(session=self.session)
         self.usher = UsherService(session=self.session)
 
+    def get_title(self):
+        if self.title is None:
+            self._get_metadata()
+        return self.title
+
+    def get_author(self):
+        if self.author is None:
+            self._get_metadata()
+        return self.author
+
+    def get_category(self):
+        if self.category is None:
+            self._get_metadata()
+        return self.category
+
+    def _get_metadata(self):
+        if self.video_id:
+            (self.author, self.title, self.category) = self.api.metadata_video(self.video_id)
+        elif self.clip_name:
+            self._get_clips()
+        elif self._channel:
+            (self.author, self.title, self.category) = self.api.metadata_channel(self.channel_id)
+
     @property
     def channel(self):
         if not self._channel:
             if self.video_id:
-                cdata = self._channel_from_video_id(self.video_id)
-                self._channel = cdata["name"].lower()
-                self._channel_id = cdata["_id"]
+                self._channel_from_video_id(self.video_id)
         return self._channel
 
-    @channel.setter
-    def channel(self, channel):
-        self._channel = channel
-        # channel id becomes unknown
-        self._channel_id = None
-
     @property
     def channel_id(self):
         if not self._channel_id:
-            # If the channel name is set, use that to look up the ID
             if self._channel:
-                cdata = self._channel_from_login(self._channel)
-                self._channel_id = cdata["_id"]
-
-            # If the channel name is not set but the video ID is,
-            # use that to look up both ID and name
+                self._channel_from_login(self._channel)
             elif self.video_id:
-                cdata = self._channel_from_video_id(self.video_id)
-                self._channel = cdata["name"].lower()
-                self._channel_id = cdata["_id"]
+                self._channel_from_video_id(self.video_id)
         return self._channel_id
 
     def _channel_from_video_id(self, video_id):
-        vdata = self.api.videos(video_id)
-        if "channel" not in vdata:
+        try:
+            self._channel_id, self._channel = self.api.channel_from_video_id(video_id)
+        except PluginError:
             raise PluginError("Unable to find video: {0}".format(video_id))
-        return vdata["channel"]
 
     def _channel_from_login(self, channel):
-        cdata = self.api.users(login=channel)
-        if len(cdata["users"]):
-            return cdata["users"][0]
-        else:
+        try:
+            self._channel_id = self.api.channel_from_login(channel)
+        except PluginError:
             raise PluginError("Unable to find channel: {0}".format(channel))
 
-    def _create_playlist_streams(self, videos):
-        start_offset = int(videos.get("start_offset", 0))
-        stop_offset = int(videos.get("end_offset", 0))
-        streams = {}
-
-        for quality, chunks in videos.get("chunks").items():
-            if not chunks:
-                if videos.get("restrictions", {}).get(quality) == "chansub":
-                    log.warning("The quality '{0}' is not available since it requires a subscription.".format(quality))
-                continue
-
-            # Rename 'live' to 'source'
-            if quality == "live":
-                quality = "source"
-
-            chunks_filtered = list(filter(lambda c: c["url"], chunks))
-            if len(chunks) != len(chunks_filtered):
-                log.warning("The video '{0}' contains invalid chunks. There will be missing data.".format(quality))
-                chunks = chunks_filtered
-
-            chunks_duration = sum(c.get("length") for c in chunks)
-
-            # If it's a full broadcast we just use all the chunks
-            if start_offset == 0 and chunks_duration == stop_offset:
-                # No need to use the FLV concat if it's just one chunk
-                if len(chunks) == 1:
-                    url = chunks[0].get("url")
-                    stream = HTTPStream(self.session, url)
-                else:
-                    chunks = [HTTPStream(self.session, c.get("url")) for c in chunks]
-                    stream = FLVPlaylist(self.session, chunks,
-                                         duration=chunks_duration)
-            else:
-                try:
-                    stream = self._create_video_clip(chunks,
-                                                     start_offset,
-                                                     stop_offset)
-                except StreamError as err:
-                    log.error("Error while creating video '{0}': {1}".format(quality, err))
-                    continue
-
-            streams[quality] = stream
-
-        return streams
-
-    def _create_video_clip(self, chunks, start_offset, stop_offset):
-        playlist_duration = stop_offset - start_offset
-        playlist_offset = 0
-        playlist_streams = []
-        playlist_tags = []
-
-        for chunk in chunks:
-            chunk_url = chunk["url"]
-            chunk_length = chunk["length"]
-            chunk_start = playlist_offset
-            chunk_stop = chunk_start + chunk_length
-            chunk_stream = HTTPStream(self.session, chunk_url)
-
-            if chunk_start <= start_offset <= chunk_stop:
-                try:
-                    headers = extract_flv_header_tags(chunk_stream)
-                except IOError as err:
-                    raise StreamError("Error while parsing FLV: {0}", err)
-
-                if not headers.metadata:
-                    raise StreamError("Missing metadata tag in the first chunk")
-
-                metadata = headers.metadata.data.value
-                keyframes = metadata.get("keyframes")
-
-                if not keyframes:
-                    if chunk["upkeep"] == "fail":
-                        raise StreamError("Unable to seek into muted chunk, try another timestamp")
-                    else:
-                        raise StreamError("Missing keyframes info in the first chunk")
-
-                keyframe_offset = None
-                keyframe_offsets = keyframes.get("filepositions")
-                keyframe_times = [playlist_offset + t for t in keyframes.get("times")]
-                for time, offset in zip(keyframe_times, keyframe_offsets):
-                    if time > start_offset:
-                        break
-
-                    keyframe_offset = offset
-
-                if keyframe_offset is None:
-                    raise StreamError("Unable to find a keyframe to seek to "
-                                      "in the first chunk")
-
-                chunk_headers = dict(Range="bytes={0}-".format(int(keyframe_offset)))
-                chunk_stream = HTTPStream(self.session, chunk_url,
-                                          headers=chunk_headers)
-                playlist_streams.append(chunk_stream)
-                for tag in headers:
-                    playlist_tags.append(tag)
-            elif start_offset <= chunk_start < stop_offset:
-                playlist_streams.append(chunk_stream)
-
-            playlist_offset += chunk_length
-
-        return FLVPlaylist(self.session, playlist_streams,
-                           tags=playlist_tags, duration=playlist_duration)
-
-    def _get_video_streams(self):
-        log.debug("Getting video steams for {0} (type={1})".format(self.video_id, self.video_type))
-
-        if self.video_type == "b":
-            self.video_type = "a"
-
+    def _access_token(self, endpoint, asset):
         try:
-            videos = self.api.videos(self.video_type + self.video_id,
-                                     schema=_video_schema)
+            sig, token = self.api.access_token(endpoint, asset)
         except PluginError as err:
-            if "HTTP/1.1 0 ERROR" in str(err):
+            if "404 Client Error" in str(err):
                 raise NoStreamsError(self.url)
             else:
                 raise
 
-        # Parse the "t" query parameter on broadcasts and adjust
-        # start offset if needed.
-        time_offset = self.params.get("t")
-        if time_offset:
-            try:
-                time_offset = hours_minutes_seconds(time_offset)
-            except ValueError:
-                time_offset = 0
+        try:
+            restricted_bitrates = self.api.token(token)
+        except PluginError:
+            restricted_bitrates = []
 
-            videos["start_offset"] += time_offset
+        return sig, token, restricted_bitrates
 
-        return self._create_playlist_streams(videos)
+    def _switch_to_hosted_channel(self):
+        disabled = self.options.get("disable_hosting")
+        hosted_chain = [self.channel]
+        while True:
+            try:
+                host_id, target_id, login, display_name = self.api.hosted_channel(self.channel_id)
+            except PluginError:
+                return False
 
-    def _access_token(self, type="live"):
-        try:
-            if type == "live":
-                endpoint = "channels"
-                value = self.channel
-            elif type == "video":
-                endpoint = "vods"
-                value = self.video_id
-
-            sig, token = self.api.access_token(endpoint, value,
-                                               schema=_access_token_schema)
-        except PluginError as err:
-            if "404 Client Error" in str(err):
-                raise NoStreamsError(self.url)
-            else:
-                raise
+            log.info("{0} is hosting {1}".format(self.channel, login))
+            if disabled:
+                log.info("hosting was disabled by command line option")
+                return True
 
-        return sig, token
+            if login in hosted_chain:
+                loop = " -> ".join(hosted_chain + [login])
+                log.error("A loop of hosted channels has been detected, cannot find a playable stream. ({0})".format(loop))
+                return True
 
-    def _check_for_host(self):
-        host_info = self.api.hosted_channel(include_logins=1, host=self.channel_id).json()["hosts"][0]
-        if "target_login" in host_info and host_info["target_login"].lower() != self.channel.lower():
-            log.info("{0} is hosting {1}".format(self.channel, host_info["target_login"]))
-            return host_info["target_login"]
+            hosted_chain.append(login)
+            log.info("switching to {0}".format(login))
+            self._channel_id = target_id
+            self._channel = login
+            self.author = display_name
 
     def _check_for_rerun(self):
-        stream = self.api.streams(self.channel_id, schema=_stream_schema)
+        if not self.options.get("disable_reruns"):
+            return False
 
-        return stream and (
-            stream["stream_type"] != "live"
-            or stream["broadcast_platform"] == "rerun"
-            or stream["channel"] and stream["channel"]["broadcaster_software"] == "watch_party_rerun"
-        )
+        try:
+            stream_type, broadcast_platform, broadcaster_software = self.api.stream_rerun(self.channel_id)
+            if stream_type != "live" or broadcast_platform == "rerun" or broadcaster_software == "watch_party_rerun":
+                log.info("Reruns were disabled by command line option")
+                return True
+        except (PluginError, TypeError):
+            pass
 
-    def _get_hls_streams(self, stream_type="live"):
-        log.debug("Getting {0} HLS streams for {1}".format(stream_type, self.channel))
-        self._hosted_chain.append(self.channel)
+        return False
 
-        if stream_type == "live":
-            if self.options.get("disable_reruns") and self._check_for_rerun():
-                log.info("Reruns were disabled by command line option")
-                return {}
+    def _get_hls_streams_live(self):
+        if self._switch_to_hosted_channel():
+            return
+        if self._check_for_rerun():
+            return
 
-            hosted_channel = self._check_for_host()
-            if hosted_channel and self.options.get("disable_hosting"):
-                log.info("hosting was disabled by command line option")
-            elif hosted_channel:
-                log.info("switching to {0}".format(hosted_channel))
-                if hosted_channel in self._hosted_chain:
-                    log.error(
-                        u"A loop of hosted channels has been detected, "
-                        "cannot find a playable stream. ({0})".format(
-                            u" -> ".join(self._hosted_chain + [hosted_channel])))
-                    return {}
-                self.channel = hosted_channel
-                return self._get_hls_streams(stream_type)
-
-            # only get the token once the channel has been resolved
-            sig, token = self._access_token(stream_type)
-            url = self.usher.channel(self.channel, sig=sig, token=token, fast_bread=True)
-        elif stream_type == "video":
-            sig, token = self._access_token(stream_type)
-            url = self.usher.video(self.video_id, nauthsig=sig, nauth=token)
-        else:
-            log.debug("Unknown HLS stream type: {0}".format(stream_type))
-            return {}
+        # only get the token once the channel has been resolved
+        log.debug("Getting live HLS streams for {0}".format(self.channel))
+        sig, token, restricted_bitrates = self._access_token("channels", self.channel)
+        url = self.usher.channel(self.channel, sig=sig, token=token, fast_bread=True)
 
+        return self._get_hls_streams(url, restricted_bitrates)
+
+    def _get_hls_streams_video(self):
+        log.debug("Getting video HLS streams for {0}".format(self.channel))
+        sig, token, restricted_bitrates = self._access_token("vods", self.video_id)
+        url = self.usher.video(self.video_id, nauthsig=sig, nauth=token)
+
+        # If the stream is a VOD that is still being recorded, the stream should start at the beginning of the recording
+        return self._get_hls_streams(url, restricted_bitrates, force_restart=True)
+
+    def _get_hls_streams(self, url, restricted_bitrates, **extra_params):
         time_offset = self.params.get("t", 0)
         if time_offset:
             try:
@@ -755,14 +621,7 @@ class Twitch(Plugin):
                 time_offset = 0
 
         try:
-            # If the stream is a VOD that is still being recorded the stream should start at the
-            # beginning of the recording
-            streams = TwitchHLSStream.parse_variant_playlist(
-                self.session,
-                url,
-                start_offset=time_offset,
-                force_restart=not stream_type == "live"
-            )
+            streams = TwitchHLSStream.parse_variant_playlist(self.session, url, start_offset=time_offset, **extra_params)
         except IOError as err:
             err = str(err)
             if "404 Client Error" in err or "Failed to parse playlist" in err:
@@ -770,55 +629,28 @@ class Twitch(Plugin):
             else:
                 raise PluginError(err)
 
-        try:
-            token = parse_json(token, schema=_token_schema)
-            for name in token["restricted_bitrates"]:
-                if name not in streams:
-                    log.warning("The quality '{0}' is not available since it requires a subscription.".format(name))
-        except PluginError:
-            pass
+        for name in restricted_bitrates:
+            if name not in streams:
+                log.warning("The quality '{0}' is not available since it requires a subscription.".format(name))
 
         return streams
 
     def _get_clips(self):
-        data = json.dumps({'query': '''{{
-            clip(slug: "{0}") {{
-                broadcaster {{
-                    displayName
-                }}
-                title
-                videoQualities {{
-                    quality
-                    sourceURL
-                }}
-            }}
-        }}'''.format(self.clip_name)})
-        clip_data = self.session.http.post('https://gql.twitch.tv/gql',
-                                           data=data,
-                                           headers={'Client-ID': TWITCH_CLIENT_ID_PRIVATE},
-                                           ).json()['data']['clip']
-        log.trace('{0!r}'.format(clip_data))
-        if not clip_data:
+        try:
+            (self.author, self.title, self.category, streams) = self.api.clips(self.clip_name)
+        except (PluginError, TypeError):
             return
 
-        self.author = clip_data['broadcaster']['displayName']
-        self.title = clip_data['title']
-
-        streams = {}
-        for quality_option in clip_data['videoQualities']:
-            streams['{0}p'.format(quality_option['quality'])] = HTTPStream(self.session, quality_option['sourceURL'])
-        return streams
+        for quality, stream in streams:
+            yield quality, HTTPStream(self.session, stream)
 
     def _get_streams(self):
         if self.video_id:
-            if self.video_type == "v":
-                return self._get_hls_streams("video")
-            else:
-                return self._get_video_streams()
+            return self._get_hls_streams_video()
         elif self.clip_name:
             return self._get_clips()
         elif self._channel:
-            return self._get_hls_streams("live")
+            return self._get_hls_streams_live()
 
 
 __plugin__ = Twitch


Test patch:
diff --git a/tests/plugins/test_cbsnews.py b/tests/plugins/test_cbsnews.py
new file mode 100644
index 00000000..addc4ac7
--- /dev/null
+++ b/tests/plugins/test_cbsnews.py
@@ -0,0 +1,24 @@
+import unittest  # noqa: F401
+import pytest
+from streamlink.plugins.cbsnews import CBSNews
+
+
+class TestCBSNews:
+    valid_urls = [
+        ("https://www.cbsnews.com/live/cbs-sports-hq/",),
+        ("https://www.cbsnews.com/live/cbsn-local-bay-area/",),
+        ("https://www.cbsnews.com/live/",),
+    ]
+    invalid_urls = [
+        ("https://www.cbsnews.com/feature/election-2020/",),
+        ("https://www.cbsnews.com/48-hours/",),
+        ("https://twitch.tv/",)
+    ]
+
+    @pytest.mark.parametrize(["url"], valid_urls)
+    def test_can_handle_url(self, url):
+        assert CBSNews.can_handle_url(url), "url should be handled"
+
+    @pytest.mark.parametrize(["url"], invalid_urls)
+    def test_can_handle_url_negative(self, url):
+        assert not CBSNews.can_handle_url(url), "url should not be handled"
diff --git a/tests/plugins/test_dlive.py b/tests/plugins/test_dlive.py
index 5e81eb27..b3233a51 100644
--- a/tests/plugins/test_dlive.py
+++ b/tests/plugins/test_dlive.py
@@ -7,6 +7,7 @@ class TestPluginDLive(unittest.TestCase):
     def test_can_handle_url(self):
         # should match
         self.assertTrue(DLive.can_handle_url("https://dlive.tv/pewdiepie"))
+        self.assertTrue(DLive.can_handle_url("https://dlive.tv/p/pdp+K6DqqtYWR"))
 
         # shouldn't match
-        self.assertFalse(DLive.can_handle_url("https://twitch.tv/lirik"))
+        self.assertFalse(DLive.can_handle_url("https://dlive.tv/"))
diff --git a/tests/plugins/test_livestream.py b/tests/plugins/test_livestream.py
index 77301a52..4c61cb8c 100644
--- a/tests/plugins/test_livestream.py
+++ b/tests/plugins/test_livestream.py
@@ -8,13 +8,14 @@ class TestPluginLivestream(unittest.TestCase):
         should_match = [
             'https://livestream.com/',
             'https://www.livestream.com/',
+            'https://livestream.com/accounts/22300508/events/6675945',
         ]
         for url in should_match:
-            self.assertTrue(Livestream.can_handle_url(url))
+            self.assertTrue(Livestream.can_handle_url(url), url)
 
     def test_can_handle_url_negative(self):
         should_not_match = [
             'https://example.com/index.html',
         ]
         for url in should_not_match:
-            self.assertFalse(Livestream.can_handle_url(url))
+            self.assertFalse(Livestream.can_handle_url(url), url)
diff --git a/tests/plugins/test_tv3cat.py b/tests/plugins/test_tv3cat.py
index 6caa4150..db8857de 100644
--- a/tests/plugins/test_tv3cat.py
+++ b/tests/plugins/test_tv3cat.py
@@ -6,8 +6,14 @@ from streamlink.plugins.tv3cat import TV3Cat
 class TestPluginTV3Cat(unittest.TestCase):
     def test_can_handle_url(self):
         should_match = [
+            'http://ccma.cat/tv3/directe/tv3/',
+            'http://ccma.cat/tv3/directe/324/',
+            'https://ccma.cat/tv3/directe/tv3/',
+            'https://ccma.cat/tv3/directe/324/',
             'http://www.ccma.cat/tv3/directe/tv3/',
             'http://www.ccma.cat/tv3/directe/324/',
+            'https://www.ccma.cat/tv3/directe/tv3/',
+            'https://www.ccma.cat/tv3/directe/324/',
         ]
         for url in should_match:
             self.assertTrue(TV3Cat.can_handle_url(url))
diff --git a/tests/plugins/test_twitch.py b/tests/plugins/test_twitch.py
index bb90349d..d66833a9 100644
--- a/tests/plugins/test_twitch.py
+++ b/tests/plugins/test_twitch.py
@@ -291,11 +291,11 @@ class TestTwitchMetadata(unittest.TestCase):
 
     def subject_channel(self, data=True, failure=False):
         self.mock.get(
-            "https://api.twitch.tv/kraken/users.json?login=foo",
+            "https://api.twitch.tv/kraken/users?login=foo",
             json={"users": [{"_id": 1234}]}
         )
         self.mock.get(
-            "https://api.twitch.tv/kraken/streams/1234.json",
+            "https://api.twitch.tv/kraken/streams/1234",
             status_code=200 if not failure else 404,
             json={"stream": None} if not data else {"stream": {
                 "channel": {
@@ -309,7 +309,7 @@ class TestTwitchMetadata(unittest.TestCase):
 
     def subject_video(self, data=True, failure=False):
         self.mock.get(
-            "https://api.twitch.tv/kraken/videos/1337.json",
+            "https://api.twitch.tv/kraken/videos/1337",
             status_code=200 if not failure else 404,
             json={} if not data else {
                 "title": "video title",
@@ -350,23 +350,134 @@ class TestTwitchMetadata(unittest.TestCase):
             self.subject_video(failure=True)
 
 
+@patch("streamlink.plugins.twitch.log")
+class TestTwitchHosting(unittest.TestCase):
+    def subject(self, channel, hosts=None, disable=False):
+        with requests_mock.Mocker() as mock:
+            mock.get(
+                "https://api.twitch.tv/kraken/users?login=foo",
+                json={"users": [{"_id": 1}]}
+            )
+            if hosts is None:
+                mock.get("https://tmi.twitch.tv/hosts", json={})
+            else:
+                mock.get(
+                    "https://tmi.twitch.tv/hosts",
+                    [{"json": {
+                        "hosts": [dict(
+                            host_id=host_id,
+                            target_id=target_id,
+                            target_login=target_login,
+                            target_display_name=target_display_name
+                        )]}
+                      } for host_id, target_id, target_login, target_display_name in hosts]
+                )
+
+            session = Streamlink()
+            Twitch.bind(session, "tests.plugins.test_twitch")
+            plugin = Twitch("https://twitch.tv/{0}".format(channel))
+            plugin.options.set("disable-hosting", disable)
+
+            res = plugin._switch_to_hosted_channel()
+            return res, plugin.channel, plugin._channel_id, plugin.author
+
+    def test_hosting_invalid_host_data(self, mock_log):
+        res, channel, channel_id, author = self.subject("foo")
+        self.assertFalse(res, "Doesn't stop HLS resolve procedure")
+        self.assertEqual(channel, "foo", "Doesn't switch channel")
+        self.assertEqual(channel_id, 1, "Doesn't switch channel id")
+        self.assertEqual(author, None, "Doesn't override author metadata")
+        self.assertEqual(mock_log.info.mock_calls, [], "Doesn't log anything to info")
+        self.assertEqual(mock_log.error.mock_calls, [], "Doesn't log anything to error")
+
+    def test_hosting_no_host_data(self, mock_log):
+        res, channel, channel_id, author = self.subject("foo", [(1, None, None, None)])
+        self.assertFalse(res, "Doesn't stop HLS resolve procedure")
+        self.assertEqual(channel, "foo", "Doesn't switch channel")
+        self.assertEqual(channel_id, 1, "Doesn't switch channel id")
+        self.assertEqual(author, None, "Doesn't override author metadata")
+        self.assertEqual(mock_log.info.mock_calls, [], "Doesn't log anything to info")
+        self.assertEqual(mock_log.error.mock_calls, [], "Doesn't log anything to error")
+
+    def test_hosting_host_single(self, mock_log):
+        res, channel, channel_id, author = self.subject("foo", [(1, 2, "bar", "Bar"), (2, None, None, None)])
+        self.assertFalse(res, "Doesn't stop HLS resolve procedure")
+        self.assertEqual(channel, "bar", "Switches channel")
+        self.assertEqual(channel_id, 2, "Switches channel id")
+        self.assertEqual(author, "Bar", "Overrides author metadata")
+        self.assertEqual(mock_log.info.mock_calls, [
+            call("foo is hosting bar"),
+            call("switching to bar")
+        ])
+        self.assertEqual(mock_log.error.mock_calls, [], "Doesn't log anything to error")
+
+    def test_hosting_host_single_disable(self, mock_log):
+        res, channel, channel_id, author = self.subject("foo", [(1, 2, "bar", "Bar")], disable=True)
+        self.assertTrue(res, "Stops HLS resolve procedure")
+        self.assertEqual(channel, "foo", "Doesn't switch channel")
+        self.assertEqual(channel_id, 1, "Doesn't switch channel id")
+        self.assertEqual(author, None, "Doesn't override author metadata")
+        self.assertEqual(mock_log.info.mock_calls, [
+            call("foo is hosting bar"),
+            call("hosting was disabled by command line option")
+        ])
+        self.assertEqual(mock_log.error.mock_calls, [], "Doesn't log anything to error")
+
+    def test_hosting_host_multiple(self, mock_log):
+        res, channel, channel_id, author = self.subject("foo", [
+            (1, 2, "bar", "Bar"),
+            (2, 3, "baz", "Baz"),
+            (3, 4, "qux", "Qux"),
+            (4, None, None, None)
+        ])
+        self.assertFalse(res, "Doesn't stop HLS resolve procedure")
+        self.assertEqual(channel, "qux", "Switches channel")
+        self.assertEqual(channel_id, 4, "Switches channel id")
+        self.assertEqual(author, "Qux", "Overrides author metadata")
+        self.assertEqual(mock_log.info.mock_calls, [
+            call("foo is hosting bar"),
+            call("switching to bar"),
+            call("bar is hosting baz"),
+            call("switching to baz"),
+            call("baz is hosting qux"),
+            call("switching to qux")
+        ])
+        self.assertEqual(mock_log.error.mock_calls, [], "Doesn't log anything to error")
+
+    def test_hosting_host_multiple_loop(self, mock_log):
+        res, channel, channel_id, author = self.subject("foo", [
+            (1, 2, "bar", "Bar"),
+            (2, 3, "baz", "Baz"),
+            (3, 1, "foo", "Foo")
+        ])
+        self.assertTrue(res, "Stops HLS resolve procedure")
+        self.assertEqual(channel, "baz", "Has switched channel")
+        self.assertEqual(channel_id, 3, "Has switched channel id")
+        self.assertEqual(author, "Baz", "Has overridden author metadata")
+        self.assertEqual(mock_log.info.mock_calls, [
+            call("foo is hosting bar"),
+            call("switching to bar"),
+            call("bar is hosting baz"),
+            call("switching to baz"),
+            call("baz is hosting foo")
+        ])
+        self.assertEqual(mock_log.error.mock_calls, [
+            call("A loop of hosted channels has been detected, cannot find a playable stream. (foo -> bar -> baz -> foo)")
+        ])
+
+
 @patch("streamlink.plugins.twitch.log")
 class TestTwitchReruns(unittest.TestCase):
     log_call = call("Reruns were disabled by command line option")
 
-    class StopError(Exception):
-        """Stop when trying to get an access token in _get_hls_streams..."""
-
-    @patch("streamlink.plugins.twitch.Twitch._check_for_host", return_value=None)
-    @patch("streamlink.plugins.twitch.Twitch._access_token", side_effect=StopError())
-    def start(self, *mocked, **params):
+    def subject(self, **params):
         with requests_mock.Mocker() as mock:
-            mocked_users = mock.get(
-                "https://api.twitch.tv/kraken/users.json?login=foo",
+            mock.get(
+                "https://api.twitch.tv/kraken/users?login=foo",
                 json={"users": [{"_id": 1234}]}
             )
-            mocked_stream = mock.get(
-                "https://api.twitch.tv/kraken/streams/1234.json",
+            mock.get(
+                "https://api.twitch.tv/kraken/streams/1234",
                 json={"stream": None} if params.pop("offline", False) else {"stream": {
                     "stream_type": params.pop("stream_type", "live"),
                     "broadcast_platform": params.pop("broadcast_platform", "live"),
@@ -380,66 +491,33 @@ class TestTwitchReruns(unittest.TestCase):
             Twitch.bind(session, "tests.plugins.test_twitch")
             plugin = Twitch("https://www.twitch.tv/foo")
             plugin.options.set("disable-reruns", params.pop("disable", True))
-            try:
-                streams = plugin.streams()
-            except TestTwitchReruns.StopError:
-                streams = True
-                pass
 
-            return streams, mocked_users, mocked_stream, mocked[0]
+            return plugin._check_for_rerun()
 
     def test_disable_reruns_live(self, mock_log):
-        streams, api_users, api_stream, access_token = self.start()
-        self.assertTrue(api_users.called_once)
-        self.assertTrue(api_stream.called_once)
-        self.assertTrue(access_token.called_once)
-        self.assertTrue(streams)
+        self.assertFalse(self.subject())
         self.assertNotIn(self.log_call, mock_log.info.call_args_list)
 
     def test_disable_reruns_not_live(self, mock_log):
-        streams, api_users, api_stream, access_token = self.start(stream_type="rerun")
-        self.assertTrue(api_users.called_once)
-        self.assertTrue(api_stream.called_once)
-        self.assertFalse(access_token.called)
-        self.assertDictEqual(streams, {})
+        self.assertTrue(self.subject(stream_type="rerun"))
         self.assertIn(self.log_call, mock_log.info.call_args_list)
 
     def test_disable_reruns_mixed(self, mock_log):
-        streams, api_users, api_stream, access_token = self.start(stream_type="rerun", broadcast_platform="live")
-        self.assertTrue(api_users.called_once)
-        self.assertTrue(api_stream.called_once)
-        self.assertFalse(access_token.called)
-        self.assertDictEqual(streams, {})
+        self.assertTrue(self.subject(stream_type="rerun", broadcast_platform="live"))
         self.assertIn(self.log_call, mock_log.info.call_args_list)
 
     def test_disable_reruns_mixed2(self, mock_log):
-        streams, api_users, api_stream, access_token = self.start(stream_type="live", broadcast_platform="rerun")
-        self.assertTrue(api_users.called_once)
-        self.assertTrue(api_stream.called_once)
-        self.assertFalse(access_token.called)
-        self.assertDictEqual(streams, {})
+        self.assertTrue(self.subject(stream_type="live", broadcast_platform="rerun"))
         self.assertIn(self.log_call, mock_log.info.call_args_list)
 
     def test_disable_reruns_broadcaster_software(self, mock_log):
-        streams, api_users, api_stream, access_token = self.start(broadcaster_software="watch_party_rerun")
-        self.assertTrue(api_users.called_once)
-        self.assertTrue(api_stream.called_once)
-        self.assertTrue(access_token.called_once)
-        self.assertDictEqual(streams, {})
+        self.assertTrue(self.subject(broadcaster_software="watch_party_rerun"))
         self.assertIn(self.log_call, mock_log.info.call_args_list)
 
     def test_disable_reruns_offline(self, mock_log):
-        streams, api_users, api_stream, access_token = self.start(offline=True)
-        self.assertTrue(api_users.called_once)
-        self.assertTrue(api_stream.called_once)
-        self.assertTrue(access_token.called_once)
-        self.assertTrue(streams)
+        self.assertFalse(self.subject(offline=True))
         self.assertNotIn(self.log_call, mock_log.info.call_args_list)
 
     def test_enable_reruns(self, mock_log):
-        streams, api_users, api_stream, access_token = self.start(disable=False)
-        self.assertFalse(api_users.called)
-        self.assertFalse(api_stream.called)
-        self.assertTrue(access_token.called_once)
-        self.assertTrue(streams)
+        self.assertFalse(self.subject(stream_type="rerun", disable=False))
         self.assertNotIn(self.log_call, mock_log.info.call_args_list)