Repository: streamlink/streamlink

Issue:
plugins.twitch: incorrect discontinuity tags lead to warnings being spammed
### Checklist

- [X] This is a [plugin issue](https://streamlink.github.io/plugins.html) and not [a different kind of issue](https://github.com/streamlink/streamlink/issues/new/choose)
- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
- [X] [I have checked the list of open and recently closed plugin issues](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22plugin+issue%22)
- [X] [I have checked the commit log of the master branch](https://github.com/streamlink/streamlink/commits/master)

### Streamlink version

streamlink 6.7.4

### Description

Recently, I've noticed constant warnings about stream discontinuities despite not seeing any actually happen.

According to the verbose log the playlists seem to keep having the X-DISCONTINUITY tag in them even though I'm not getting served some sort of mid-roll or pre-roll due to having a subscription to the channel in question. I'm still receiving the live broadcast.

There are also instances of repetitively getting this warning even after getting past the initial pre-roll in the event I don't have a subscription to a particular streamer.

I'm not sure what's going on in the Twitch side of things, is this some attempt at trying to break streamlink usage again?

The verbose log is too large for the issue report so I'm attaching it as a file:
[streamlink.txt.gz](https://github.com/user-attachments/files/15524324/streamlink.txt.gz)



### Debug log

```text
https://github.com/user-attachments/files/15524324/streamlink.txt.gz
```



Discussion:
This has nothing to do with Twitch. It's a bug in the HLS implementation which warns when there's a stream discontinuity, which is not supported by Streamlink, as two different bitstreams are concatenated. This happens during ad transitions. The warning however is not supposed to be written to the log output every time. This looks like a bug in regards to the ad segment filtering.
After having another closer look at the logs, the warning behavior seems to be working correctly. What's wrong is the discontinuity tag in between two "live" segments where there's no actual discontinuity.

No idea why Twitch is inserting this here, but it's clearly wrong. You said it yourself, you get the regular live content without ads.

I guess this could be covered by the Twitch plugin's HLS subclasses. The discontinuity status of a segment would need to be cleared when it and the segment before it are both considered "live" segments. 

Prefetch segments however might cause a few problems:
https://github.com/streamlink/streamlink/blob/6.7.4/src/streamlink/plugins/twitch.py#L87
I took another log after a streamer ran a several minute long midroll ad break:
[streamlink.txt.gz](https://github.com/user-attachments/files/15527113/streamlink.txt.gz)

I might try seeing what happens if I don't use the "Low Latency" option, but this is really strange behavior coming from the playlists.
There is no point doing this unless you're using the branch of the linked pull request (#6016).
https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#pull-request-feedback
Apologies, I noticed the pull request a bit late; I grabbed the `twitch.py` from it and ran this test.

The fix seems to work from my testing; I only get a warning on the actual adbreak segments.
[streamlink.txt.gz](https://github.com/user-attachments/files/15527285/streamlink.txt.gz)

Something else bizarre happened towards the end, after the adbreak the video froze and I just keep seeing streamlink reloading the playlist, but mpv receives nothing; just stuck on the last frame of the "Commercial Break In Progress" video. That's a separate issue I'm trying to understand the underlying cause of that's not related to this issue though.
You are not filtering out ads using `--twitch-disable-ads` ([1](https://streamlink.github.io/cli.html#cmdoption-twitch-disable-ads), [2](https://streamlink.github.io/cli/plugins/twitch.html#embedded-ads)), hence why you're seeing the "commercial break in progress" video loop placeholder. This is a loop of about 30 seconds which overrides the actual live segments.

> after the adbreak the video froze and I just keep seeing streamlink reloading the playlist, but mpv receives nothing; just stuck on the last frame of the "Commercial Break In Progress" video

The reason for this are incorrect timestamps of the "commercial break in progress" video container, which messes with your player's demuxer during the stream discontinuity, or rather discontinuities, because when repeating the "commercial break in progress" video, there's also a discontinuity. There is unfortunately not much that can be done here. You should always be filtering out "ads" / the placeholder in order to prevent this.

----

The changes of #6016 are working fine. I've checked multiple streams with and without ad filtering, as well as with low latency streaming and without. I was worried about the changed logic of the line I linked in my previous comment, which filtered out prefetch segments if the last actual segment contained a discontinuity. This is not necessary anymore, as the discontinuities are properly annotated in the prefetch segments now, which was the reason for the log warning spam. I'm going to merge #6016 now.
I was about to file a new issue against streamlink 6.8.1

It worked fine yesterday.  Seems like Twitch decided to break something today (Tuesday 2024-06-18).

I'm observing this on at least the two streamers I just happened to try that I'm still subscribed to.

Lots of lines like:
[stream.hls][warning] Encountered a stream discontinuity. This is unsupported and will result in incoherent output data.

https://gist.github.com/mjevans/616101c98666baa31f6048218b94c21a

I have re-tested with PR #6022 included, the behavior and output appear identical.  This PR might be a building block towards the fix, but it alone is insufficient.
I posted this in the linked PR, and the only thing that helped for me was turning off `twitch-low-latency`. Without that I get discontinuities (on some streams), and actual discontinuities in the video feed. Based on turning off low latency fixing it, I think it might be an issue with low latency as of today?
Removing --twitch-low-latency AND using the PR does seem to avoid the issue.
Though it would be nice if low latency worked too, that's what I usually used in scripts to try to be almost as live as the web browser and mobile clients for interactive streams.
This seems to be a regression, it does not happen in versions 6.3.1 nor 5.5.1.
> This seems to be a regression, it does not happen in versions 6.3.1 nor 5.5.1.

This is not a regression in Streamlink, because Twitch has made changes. They've recently started to include discontinuity tags between two "live" segments in some cases. Unless there's an actual discontinuity, which is unlikely considering it's two "live" segments and considering it's included repeatedly, this is incorrect metadata.
https://datatracker.ietf.org/doc/html/rfc8216#section-4.3.2.3

If you read Streamlink's changelog or check the diff, you'll see that there were no relevant changes between those versions.

```sh
git diff 6.3.1...6.8.1 -- src/streamlink/plugins/twitch.py src/streamlink/stream/hls/
```

So if you want this issue to be resolved, instead of making wild claims, consider providing useful log outputs, as I've explained in the linked pull request.
https://github.com/streamlink/streamlink/pull/6022#issuecomment-2178541757
FWIW the pull request had a green checkmark so I thought there wasn't anything else to discuss there.

**Patch Author Requested Data Collection**

`streamlink --twitch-disable-ads --twitch-low-latency --loglevel=all https://twitch.tv/${STREAMER} ${QUALITY} 2>&1 | tee -a all.log`

Note I am also using a streamlink config ~/.config/streamlink/config.twitch which inclueds

```
# follow https://streamlink.github.io/cli/plugins/twitch.html#authentication
twitch-api-header=Authorization=OAuth REDACTED
twitch-disable-ads
twitch-disable-hosting
player=mpv
player-args=--really-quiet
```

Unfortunately I didn't know these things _last_ night when I was trying it out with two streamers (from the general Chicago ingress area) which both had that issue.  Today only one of the streamers I'm subscribed to has streamed thus far.  That stream has not presented the issue.  I'll continue testing with other streamers and see if a reproduction occurs.



Patch:
diff --git a/src/streamlink/plugins/twitch.py b/src/streamlink/plugins/twitch.py
index 7ae078e9ad5..06278b094d4 100644
--- a/src/streamlink/plugins/twitch.py
+++ b/src/streamlink/plugins/twitch.py
@@ -71,26 +71,45 @@ class TwitchM3U8Parser(M3U8Parser[TwitchM3U8, TwitchHLSSegment, HLSPlaylist]):
     __m3u8__: ClassVar[Type[TwitchM3U8]] = TwitchM3U8
     __segment__: ClassVar[Type[TwitchHLSSegment]] = TwitchHLSSegment
 
+    @parse_tag("EXT-X-TWITCH-LIVE-SEQUENCE")
+    def parse_ext_x_twitch_live_sequence(self, value):
+        # Unset discontinuity state if the previous segment was not an ad,
+        # as the following segment won't be an ad
+        if self.m3u8.segments and not self.m3u8.segments[-1].ad:
+            self._discontinuity = False
+
     @parse_tag("EXT-X-TWITCH-PREFETCH")
     def parse_tag_ext_x_twitch_prefetch(self, value):
         segments = self.m3u8.segments
         if not segments:  # pragma: no cover
             return
         last = segments[-1]
+
         # Use the average duration of all regular segments for the duration of prefetch segments.
         # This is better than using the duration of the last segment when regular segment durations vary a lot.
         # In low latency mode, the playlist reload time is the duration of the last segment.
         duration = last.duration if last.prefetch else sum(segment.duration for segment in segments) / float(len(segments))
+
         # Use the last duration for extrapolating the start time of the prefetch segment, which is needed for checking
         # whether it is an ad segment and matches the parsed date ranges or not
         date = last.date + timedelta(seconds=last.duration)
+
         # Always treat prefetch segments after a discontinuity as ad segments
+        # (discontinuity tag inserted after last regular segment)
+        # Don't reset discontinuity state: the date extrapolation might be inaccurate,
+        # so all following prefetch segments should be considered an ad after a discontinuity
         ad = self._discontinuity or self._is_segment_ad(date)
+
+        # Since we don't reset the discontinuity state in prefetch segments for the purpose of ad detection,
+        # set the prefetch segment's discontinuity attribute based on ad transitions
+        discontinuity = ad != last.ad
+
         segment = dataclass_replace(
             last,
             uri=self.uri(value),
             duration=duration,
             title=None,
+            discontinuity=discontinuity,
             date=date,
             ad=ad,
             prefetch=True,
@@ -108,6 +127,15 @@ def get_segment(self, uri: str, **data) -> TwitchHLSSegment:
         ad = self._is_segment_ad(self._date, self._extinf.title if self._extinf else None)
         segment: TwitchHLSSegment = super().get_segment(uri, ad=ad, prefetch=False)  # type: ignore[assignment]
 
+        # Special case where Twitch incorrectly inserts discontinuity tags between segments of the live content
+        if (
+            segment.discontinuity
+            and not segment.ad
+            and self.m3u8.segments
+            and not self.m3u8.segments[-1].ad
+        ):
+            segment.discontinuity = False
+
         return segment
 
     def _is_segment_ad(self, date: Optional[datetime], title: Optional[str] = None) -> bool:


Test patch:
diff --git a/tests/plugins/test_twitch.py b/tests/plugins/test_twitch.py
index e8997dbbc22..00bdd0237cf 100644
--- a/tests/plugins/test_twitch.py
+++ b/tests/plugins/test_twitch.py
@@ -363,6 +363,7 @@ def test_hls_low_latency_has_prefetch_disable_ads_no_preroll_with_prefetch_ads(s
             Tag("EXT-X-DISCONTINUITY"),
             TagDateRangeAd(start=DATETIME_BASE + timedelta(seconds=3), duration=4),
         ]
+        tls = Tag("EXT-X-TWITCH-LIVE-SEQUENCE", 7)
         # noinspection PyTypeChecker
         segments = self.subject([
             # regular stream data with prefetch segments
@@ -374,9 +375,9 @@ def test_hls_low_latency_has_prefetch_disable_ads_no_preroll_with_prefetch_ads(s
             # still no prefetch segments while ads are playing
             Playlist(3, [*ads, Seg(3), Seg(4), Seg(5), Seg(6)]),
             # new prefetch segments on the first regular segment occurrence
-            Playlist(4, [*ads, Seg(4), Seg(5), Seg(6), Seg(7), Pre(8), Pre(9)]),
-            Playlist(5, [*ads, Seg(5), Seg(6), Seg(7), Seg(8), Pre(9), Pre(10)]),
-            Playlist(6, [*ads, Seg(6), Seg(7), Seg(8), Seg(9), Pre(10), Pre(11)]),
+            Playlist(4, [*ads, Seg(4), Seg(5), Seg(6), tls, Seg(7), Pre(8), Pre(9)]),
+            Playlist(5, [*ads, Seg(5), Seg(6), tls, Seg(7), Seg(8), Pre(9), Pre(10)]),
+            Playlist(6, [*ads, Seg(6), tls, Seg(7), Seg(8), Seg(9), Pre(10), Pre(11)]),
             Playlist(7, [Seg(7), Seg(8), Seg(9), Seg(10), Pre(11), Pre(12)], end=True),
         ], streamoptions={"disable_ads": True, "low_latency": True})
 
@@ -415,6 +416,36 @@ def test_hls_low_latency_no_ads_reload_time(self):
         self.await_read(read_all=True)
         assert self.thread.reader.worker.playlist_reload_time == pytest.approx(23 / 3)
 
+    @patch("streamlink.stream.hls.hls.log")
+    def test_hls_prefetch_after_discontinuity(self, mock_log):
+        segments = self.subject([
+            Playlist(0, [Segment(0), Segment(1)]),
+            Playlist(2, [Segment(2), Segment(3), Tag("EXT-X-DISCONTINUITY"), SegmentPrefetch(4), SegmentPrefetch(5)]),
+            Playlist(6, [Segment(6), Segment(7)], end=True),
+        ], streamoptions={"disable_ads": True, "low_latency": True})
+
+        self.await_write(8)
+        assert self.await_read(read_all=True) == self.content(segments, cond=lambda seg: seg.num not in (4, 5))
+        assert mock_log.warning.mock_calls == [
+            call("Encountered a stream discontinuity. This is unsupported and will result in incoherent output data."),
+        ]
+
+    @patch("streamlink.stream.hls.hls.log")
+    def test_hls_ignored_discontinuity(self, mock_log):
+        Seg, Pre = Segment, SegmentPrefetch
+        discontinuity = Tag("EXT-X-DISCONTINUITY")
+        tls = Tag("EXT-X-TWITCH-LIVE-SEQUENCE", 1234)  # value is irrelevant
+        segments = self.subject([
+            Playlist(0, [Seg(0), discontinuity, Seg(1)]),
+            Playlist(2, [Seg(2), Seg(3), discontinuity, Seg(4), Seg(5)]),
+            Playlist(6, [Seg(6), Seg(7), discontinuity, tls, Pre(8), Pre(9)]),
+            Playlist(10, [Seg(10), Seg(11), discontinuity, tls, Pre(12), discontinuity, tls, Pre(13)], end=True),
+        ], streamoptions={"disable_ads": True, "low_latency": True})
+
+        self.await_write(14)
+        assert self.await_read(read_all=True) == self.content(segments)
+        assert mock_log.warning.mock_calls == []
+
 
 class TestTwitchAPIAccessToken:
     @pytest.fixture(autouse=True)