Repository: streamlink/streamlink

Issue:
plugins.twitch: Returns "Channel is offline" for short period after a stream has started
### Checklist

- [X] This is a [plugin issue](https://streamlink.github.io/plugins.html) and not [a different kind of issue](https://github.com/streamlink/streamlink/issues/new/choose)
- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
- [X] [I have checked the list of open and recently closed plugin issues](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22plugin+issue%22)
- [X] [I have checked the commit log of the master branch](https://github.com/streamlink/streamlink/commits/master)

### Streamlink version

6.4.2+6.ge58fe243

### Description

After https://github.com/streamlink/streamlink/pull/5704, streams will now return "Channel is offline" for a short period after the stream has started, even when the stream is viewable in a web browser. The period seems to vary but I've seen it as high as around 30 seconds.

You should be able to reproduce this using a personal Twitch channel, open the channel in a browser window, prepare a sample streamlink command (`streamlink -o /dev/null https://twitch.tv/channel best`), then start streaming to the channel. The stream should start autoplaying in the browser a short time later, once this happens attempt to run the streamlink command. It should fail with "Channel is offline", it will keep returning this for another 15~30 seconds in my experience before working properly.

Before the linked commit, streamlink would be able to play a stream immediately after it started, the same time it was viewable in the browser.

### Debug log

```text
[13:22:47.708671][cli][debug] OS:         Linux-6.1.65-1-lts-x86_64-with-glibc2.38
[13:22:47.708918][cli][debug] Python:     3.11.6
[13:22:47.709048][cli][debug] OpenSSL:    OpenSSL 3.1.4 24 Oct 2023
[13:22:47.709126][cli][debug] Streamlink: 6.4.2+6.ge58fe243
[13:22:47.709739][cli][debug] Dependencies:
[13:22:47.711347][cli][debug]  certifi: 2023.11.17
[13:22:47.711896][cli][debug]  isodate: 0.6.1
[13:22:47.712349][cli][debug]  lxml: 4.9.3
[13:22:47.713059][cli][debug]  pycountry: 22.3.5
[13:22:47.713634][cli][debug]  pycryptodome: 3.19.0
[13:22:47.714240][cli][debug]  PySocks: 1.7.1
[13:22:47.714741][cli][debug]  requests: 2.31.0
[13:22:47.715192][cli][debug]  trio: 0.23.1
[13:22:47.715626][cli][debug]  trio-websocket: 0.11.1
[13:22:47.716044][cli][debug]  typing-extensions: 4.8.0
[13:22:47.716471][cli][debug]  urllib3: 2.1.0
[13:22:47.717358][cli][debug]  websocket-client: 1.7.0
[13:22:47.717598][cli][debug] Arguments:
[13:22:47.717679][cli][debug]  url=https://www.twitch.tv/<channel>
[13:22:47.717743][cli][debug]  stream=['best']
[13:22:47.717811][cli][debug]  --loglevel=all
[13:22:47.717869][cli][debug]  --logfile=2023_12_05_21_22_47_(UTC).log
[13:22:47.717935][cli][debug]  --output=2023_12_05_21_22_47_(UTC).ts
[13:22:47.718014][cli][debug]  --force-progress=True
[13:22:47.718085][cli][debug]  --retry-streams=5.0
[13:22:47.718142][cli][debug]  --retry-max=12
[13:22:47.718195][cli][debug]  --retry-open=10
[13:22:47.718252][cli][debug]  --stream-segment-attempts=20
[13:22:47.718305][cli][debug]  --stream-segment-threads=10
[13:22:47.718359][cli][debug]  --stream-segment-timeout=60.0
[13:22:47.718413][cli][debug]  --stream-timeout=240.0
[13:22:47.718469][cli][debug]  --hls-playlist-reload-attempts=10
[13:22:47.718554][cli][debug]  --hls-playlist-reload-time=segment
[13:22:47.718615][cli][debug]  --hls-live-restart=True
[13:22:47.718684][cli][debug]  --webbrowser=False
[13:22:47.718755][cli][debug]  --twitch-disable-hosting=True
[13:22:47.719307][cli][info] Found matching plugin twitch for URL https://www.twitch.tv/<channel>
[13:22:47.860846][plugins.twitch][error] Channel is offline
[13:22:47.861003][cli][info] Waiting for streams, retrying every 5.0 second(s)
[13:22:52.934465][plugins.twitch][error] Channel is offline
[13:22:58.009304][plugins.twitch][debug] Getting live HLS streams for <channel>
[13:22:58.106980][plugins.twitch][debug] {'adblock': False, 'geoblock_reason': '', 'hide_ads': False, 'server_ads': True, 'show_ads': True}
[13:22:58.153457][utils.l10n][debug] Language code: en_US
[13:22:58.306960][cli][info] Available streams: audio_only, 720p60 (worst, best)
[13:22:58.307176][cli][info] Opening stream: 720p60 (hls)
```



Discussion:
This is because their GQL API has a slight delay, same issue with the stream metadata retrieval when polling the stream previously using `--retry-streams` (#4411 and all the linked duplicate threads).

How would you suggest fixing this otherwise? The reason why the API gets checked for the live status now is that just getting the access token results in streams being returned incorrectly for offline streams, hence the change. Then there's the issue with geo-blocked content on the `primevideo` channel, which was the reason why the error handling got changed recently. Simply checking for a 404 HTTP status code when trying to access the multivariant or media playlists doesn't help. The only other idea I have is checking if other API endpoints don't have this delay.

I don't think this slight delay is an issue. It even fixes said metadata retrieval issue.
I think this issue would cause a lot more inconvenience than the extremely rare issue it was meant to fix, I've been using streamlink on Twitch channels for years and this is the first time I've ever seen a channel with a weird broken playlist like that. Even the Twitch site doesn't handle it correctly, it just infinitely loads the video and never says the channel is offline.

It seems like the individual variant playlists properly returns a `404` code, even when the multivariant returns a `200`. Maybe a `404` on the first variant playlist fetch could signal an offline stream? I don't know if this would lead to any false positives though, would need testing probably.
If the is-live API check gets removed, then the following things need to be checked:

- **online channels**

  Nothing interesting here...

- **invalid/unknown channels**

  This is covered by the access token API response, so HLS stream error handling is not involved here.

- **`4xx` multivariant responses**

  `4xx` HTTP responses when accessing the multivariant playlist **sometimes** include JSON data with error messages.

  Previously, this JSON data has been ignored, but I've updated the error handling recently in #5591 because of geo-blocked streams like `primevideo`, so a better error message could be logged.

  However, this also negatively affected regular offline streams in some cases, and it has already caused confusion (#5656).

  - **regular offline channels**

    Currently logs one of these garbage error messages:

    - `twirp error not_found: transcode does not exist` (from the JSON data)
    - `Could not access HLS playlist` (no JSON data)

  - **geo-blocked streams**

    Useful error logging:

    - `Content Restricted In Region`

- **`200` multivariant playlist responses with `4xx` media playlist responses**

  This is currently the case on the `paymoneywubby` channel, which was the motivation for the is-live API check. I don't think this is an edge case. This is probably because of some offline-hosting settings on this channel.

  Setting `check_streams=True` when parsing the multivariant playlist allows detecting this case, but this comes at the cost of delaying all stream listings and individual stream starts, because each media playlist gets checked **consecutively**, meaning 6 or more HTTP requests will be made in order and each response gets awaited until we know that any channel is actually live or not.

Resolving patch:
diff --git a/src/streamlink/plugins/twitch.py b/src/streamlink/plugins/twitch.py
--- a/src/streamlink/plugins/twitch.py
+++ b/src/streamlink/plugins/twitch.py
@@ -487,26 +487,6 @@ def clips(self, clipname):
             ),
         ))
 
-    def stream_metadata(self, channel):
-        query = self._gql_persisted_query(
-            "StreamMetadata",
-            "1c719a40e481453e5c48d9bb585d971b8b372f8ebb105b17076722264dfa5b3e",
-            channelLogin=channel,
-        )
-
-        return self.call(query, schema=validate.Schema(
-            {
-                "data": {
-                    "user": validate.none_or_all({
-                        "stream": validate.none_or_all({
-                            "id": str,
-                        }),
-                    }),
-                },
-            },
-            validate.get(("data", "user")),
-        ))
-
 
 class TwitchClientIntegrity:
     URL_P_SCRIPT = "https://k.twitchcdn.net/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/p.js"
@@ -816,21 +796,7 @@ def _access_token(self, is_live, channel_or_vod):
 
         return sig, token, restricted_bitrates
 
-    def _check_is_live(self) -> bool:
-        data = self.api.stream_metadata(self.channel)
-        if data is None:
-            log.error("Unknown channel")
-            return False
-        if data["stream"] is None:
-            log.error("Channel is offline")
-            return False
-
-        return True
-
     def _get_hls_streams_live(self):
-        if not self._check_is_live():
-            return
-
         # only get the token once the channel has been resolved
         log.debug(f"Getting live HLS streams for {self.channel}")
         self.session.http.headers.update({
@@ -863,6 +829,7 @@ def _get_hls_streams(self, url, restricted_bitrates, **extra_params):
                 self.session,
                 url,
                 start_offset=time_offset,
+                check_streams=True,
                 disable_ads=self.get_option("disable-ads"),
                 low_latency=self.get_option("low-latency"),
                 **extra_params,
@@ -871,7 +838,6 @@ def _get_hls_streams(self, url, restricted_bitrates, **extra_params):
             # TODO: fix the "err" attribute set by HTTPSession.request()
             orig = getattr(err, "err", None)
             if isinstance(orig, HTTPError) and orig.response.status_code >= 400:
-                error = None
                 with suppress(PluginError):
                     error = validate.Schema(
                         validate.parse_json(),
@@ -881,7 +847,7 @@ def _get_hls_streams(self, url, restricted_bitrates, **extra_params):
                         }],
                         validate.get((0, "error")),
                     ).validate(orig.response.text)
-                log.error(error or "Could not access HLS playlist")
+                    log.error(error or "Could not access HLS playlist")
                 return
             raise PluginError(err) from err
 


Test patch:
diff --git a/tests/plugins/test_twitch.py b/tests/plugins/test_twitch.py
--- a/tests/plugins/test_twitch.py
+++ b/tests/plugins/test_twitch.py
@@ -396,53 +396,6 @@ def test_hls_low_latency_no_ads_reload_time(self):
         assert self.thread.reader.worker.playlist_reload_time == pytest.approx(23 / 3)
 
 
-class TestTwitchIsLive:
-    @pytest.fixture()
-    def plugin(self, request: pytest.FixtureRequest):
-        return Twitch(Streamlink(), "https://twitch.tv/channelname")
-
-    @pytest.fixture()
-    def mock(self, request: pytest.FixtureRequest, requests_mock: rm.Mocker):
-        mock = requests_mock.post("https://gql.twitch.tv/gql", **getattr(request, "param", {"json": {}}))
-        yield mock
-        assert mock.call_count > 0
-        payload = mock.last_request.json()  # type: ignore[union-attr]
-        assert tuple(sorted(payload.keys())) == ("extensions", "operationName", "variables")
-        assert payload.get("operationName") == "StreamMetadata"
-        assert payload.get("extensions") == {
-            "persistedQuery": {
-                "sha256Hash": "1c719a40e481453e5c48d9bb585d971b8b372f8ebb105b17076722264dfa5b3e",
-                "version": 1,
-            },
-        }
-        assert payload.get("variables") == {"channelLogin": "channelname"}
-
-    @pytest.mark.parametrize(("mock", "expected", "log"), [
-        pytest.param(
-            {"json": {"data": {"user": None}}},
-            False,
-            [("streamlink.plugins.twitch", "error", "Unknown channel")],
-            id="no-user",
-        ),
-        pytest.param(
-            {"json": {"data": {"user": {"stream": None}}}},
-            False,
-            [("streamlink.plugins.twitch", "error", "Channel is offline")],
-            id="no-stream",
-        ),
-        pytest.param(
-            {"json": {"data": {"user": {"stream": {"id": "1234567890"}}}}},
-            True,
-            [],
-            id="is-live",
-        ),
-    ], indirect=["mock"])
-    def test_is_live(self, plugin: Twitch, caplog: pytest.LogCaptureFixture, mock: rm.Mocker, expected: bool, log: list):
-        caplog.set_level(1, "streamlink")
-        assert plugin._check_is_live() is expected
-        assert [(record.name, record.levelname, record.message) for record in caplog.records] == log
-
-
 class TestTwitchAPIAccessToken:
     @pytest.fixture(autouse=True)
     def _client_integrity_token(self, monkeypatch: pytest.MonkeyPatch):
@@ -676,7 +629,7 @@ def plugin(self, request: pytest.FixtureRequest, requests_mock: rm.Mocker, sessi
             },
             nullcontext(),
             None,
-            [("streamlink.plugins.twitch", "error", "Could not access HLS playlist")],
+            [],
             id="non-json error response",
         ),
     ], indirect=["plugin"])