Repository: streamlink/streamlink
Pull request #3213 (2020-09-30 05:02:33)

plugins.twitch: filter ads by EXT-X-DATERANGE tag
As mentioned in https://github.com/streamlink/streamlink/issues/3210#issuecomment-700571743, simply filtering ads by comparing each segment's title isn't good enough anymore and checking for the title not being equal to "live" can break the implementation if Twitch decides to change the title of live stream segments or if they decide to add titles to VOD segments.

So here's a better filtering approach, based on the [`EXT-X-DATERANGE`](https://tools.ietf.org/html/draft-pantos-http-live-streaming-23#section-4.3.2.7) and [`EXT-X-PROGRAM-DATE-TIME`](https://tools.ietf.org/html/draft-pantos-http-live-streaming-23#section-4.3.2.6) tags.

Example playlists can be found here:
- https://gist.github.com/drmthrowaway/99ddc2308ba431e76945e456da837b75
- https://gist.github.com/tp0/f4e6798301fb8361c192e17734a0ca16

----

Basic description of each commit:

1. Adds the `EXT-X-DATERANGE` tag to the HLS playlist parser and makes the `EXT-X-PROGRAM-DATE-TIME` tag return a datetime object instead of a string. Since I couldn't find any usage of the `EXT-X-PROGRAM-DATE-TIME` tag in the existing code, changing its return value should be fine. Also adds support for returning a custom M3U8 class by the parser and adds a `is_date_in_daterange` method to the existing `M3U8` class. This isn't an ideal solution, but I couldn't find a better place for this method so that it can be called when parsing is done apart from turning the `Segment` namedtuple into a regular class. Also worth to note is that I haven't implemented all possible attribute values of the `EXT-X-DATERANGE tag`, because I didn't need them (scte35 attributes).

2. Makes the Twitch plugin filter the parsed dateranges by attributes. It tries to find "ad dateranges" by the values of the ID and CLASS attributes and key names of custom attributes by Twitch. This is the crucial part of the ad detection. If a segment is then part of this daterange, it is marked as an ad and filtered out by the existing methods.

----

I've added tests for all "ad daterange" values I know of, **but since I still don't get any ads by Twitch, it's hard to know if my implementation is working correctly**. So please go ahead and test this on live data. Thank you!

gravyboat: I'm still seeing the following error from a random "big" stream pulled off twitch:
```
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.8.4
[cli][debug] Requests(2.24.0), Socks(1.7.1), Websocket(0.57.0)
[cli][info] Found matching plugin twitch for URL twitch.tv/ludwig
[cli][debug] Plugin specific arguments:
[cli][debug]  --twitch-disable-ads=True (disable_ads)
[cli][debug]  --twitch-low-latency=True (low_latency)
[plugin.twitch][debug] Getting live HLS streams for ludwig
[utils.l10n][debug] Language code: en_US
[cli][info] Available streams: audio_only, 160p (worst), 360p, 480p, 720p, 720p60, 1080p60 (best)
[cli][info] Opening stream: 1080p60 (hls)
[plugin.twitch][info] Will skip ad segments
[plugin.twitch][info] Low latency streaming (HLS live edge: 2)
[stream.hls][debug] Reloading playlist
[cli][error] Try 1/1: Could not open stream <HLSStream('https://video-weaver.sea01.hls.ttvnw.net/v1/playlist/CvYD5-nuyHKhhuV5zmXiVfVH9PjoXmeyFFvmStbY5iRPPTWO6ArtWycw3wwb9tPgFbDLQP3D10SnixFmyfoS-uMDvD7CLxTxGv8daUybYYNo1pKnPuxOcwlyIe5ns0t0Ysw5jp-Td30eXpa_uVO2cZhEjlDgu-n6osjGZH9dzrzdJoefH3yWqTHMwlAWlSjmJElqRkG_RkbGoSw0-6Jb9ccLuU6-1wpt3dVI5n9l-af_T36wq__5UZ3G4V_nNbAIE6V_Qvi7rXdywN4S1N-XU1cpKlGrlTyU3q2rB-H_6zyI-Idhq8o5B_Myo7BeFOJBCzNUz9DgKtoXgIRot7NGdW-LvorDiFOknnmCO41NwDTdTBPXD8nwo8r7YiCOiAu0-7dAvbRqLVwJdAFXq_1d7q1L0uaV6EpNthepMWYDFjY68lYMi_Jp5HqqivDKEgJsG_7VM4LP0BAm1r25818W7SSch6xL9Yx8BeWQQ5qDpNbnK5Ojp0rJQYVf_nos_DErya9e5gbGk-xXCVZTGAq0CsQjDxFnLYgZKdpJrr-2Hn3UZlDG0Wfc_HaKsjRCxxJZLUSGbFAGIUZfdrfTdIn7yv_RBNsTwV_DKDRcetFtzZ6EHXLw_7BHw8OD-IXpx9Ish3jOUL9Sub9gHdC-hYxn0aj4YVieZd1AQxIQLmf5mJRpJTbwzOTajgzEERoMmTIkBI1gQnXYCk4F.m3u8')> (Could not open stream: invalid literal for int() with base 10: '15.183')
error: Could not open stream <HLSStream('https://video-weaver.sea01.hls.ttvnw.net/v1/playlist/CvYD5-nuyHKhhuV5zmXiVfVH9PjoXmeyFFvmStbY5iRPPTWO6ArtWycw3wwb9tPgFbDLQP3D10SnixFmyfoS-uMDvD7CLxTxGv8daUybYYNo1pKnPuxOcwlyIe5ns0t0Ysw5jp-Td30eXpa_uVO2cZhEjlDgu-n6osjGZH9dzrzdJoefH3yWqTHMwlAWlSjmJElqRkG_RkbGoSw0-6Jb9ccLuU6-1wpt3dVI5n9l-af_T36wq__5UZ3G4V_nNbAIE6V_Qvi7rXdywN4S1N-XU1cpKlGrlTyU3q2rB-H_6zyI-Idhq8o5B_Myo7BeFOJBCzNUz9DgKtoXgIRot7NGdW-LvorDiFOknnmCO41NwDTdTBPXD8nwo8r7YiCOiAu0-7dAvbRqLVwJdAFXq_1d7q1L0uaV6EpNthepMWYDFjY68lYMi_Jp5HqqivDKEgJsG_7VM4LP0BAm1r25818W7SSch6xL9Yx8BeWQQ5qDpNbnK5Ojp0rJQYVf_nos_DErya9e5gbGk-xXCVZTGAq0CsQjDxFnLYgZKdpJrr-2Hn3UZlDG0Wfc_HaKsjRCxxJZLUSGbFAGIUZfdrfTdIn7yv_RBNsTwV_DKDRcetFtzZ6EHXLw_7BHw8OD-IXpx9Ish3jOUL9Sub9gHdC-hYxn0aj4YVieZd1AQxIQLmf5mJRpJTbwzOTajgzEERoMmTIkBI1gQnXYCk4F.m3u8')>, tried 1 times, exiting
```
Billy2011: > Every answer to any question about this I've found is "use python-dateutil", but I can't just add a dependency here, so a custom parser had to be written

Why don’t you use the isodate module for parsing, which is also used for the dash manifest parser?
azizLIGHT: I think it's working sometimes (line 58)  but I might be doing this wrong but here's my output. Let me know if I should adjust something: https://gist.github.com/azizLIGHT/e60250ecb8091daa5619a4902a449efe


bastimeyer: @Billy2011 Thanks... Don't ask how I've missed that. :disappointed: 
@azizLIGHT Regarding the error, you could try it again with the new changes. I will check this later, since I have to go now.
azizLIGHT: > @azizLIGHT Regarding the error, you could try it again with the new changes. I will check this later, since I have to go now.

I got the latest version and tried again but got same errors about [`Could not open stream <HLSStream`]( https://gist.github.com/azizLIGHT/5863d9bf4382df59f69e98d8634bdc8a).
I will try again when there's more changes added


Patch:
diff --git a/docs/plugin_matrix.rst b/docs/plugin_matrix.rst
index 76d73de5..606522da 100644
--- a/docs/plugin_matrix.rst
+++ b/docs/plugin_matrix.rst
@@ -20,6 +20,7 @@ albavision              - tvc.com.ec         Yes   No    Some streams are geo-re
                         - rts.com.ec
                         - elnueve.com.ar
                         - atv.pe
+aljazeeraen             aljazeera.com        Yes   Yes   English version of the site.
 animelab                animelab.com         --    Yes   Requires a login. Streams may be geo-restricted to Australia and New Zealand.
 app17                   17app.co             Yes   --
 ard_live                daserste.de          Yes   Yes   Streams may be geo-restricted to Germany.
diff --git a/src/streamlink/plugin/api/useragents.py b/src/streamlink/plugin/api/useragents.py
index ddfee8b5..c3d1b365 100644
--- a/src/streamlink/plugin/api/useragents.py
+++ b/src/streamlink/plugin/api/useragents.py
@@ -1,11 +1,11 @@
 ANDROID = ("Mozilla/5.0 (Linux; Android 7.1.1; SM-J510FN Build/NMF26X) "
            "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Mobile Safari/537.36")
-CHROME = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36"
+CHROME = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36"
 CHROME_OS = ("Mozilla/5.0 (X11; CrOS armv7l 10718.34.0) "
              "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.40 Safari/537.36")
 EDGE = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36 Edg/80.0.361.111"
-FIREFOX = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:81.0) Gecko/20100101 Firefox/81.0"
-FIREFOX_MAC = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:81.0) Gecko/20100101 Firefox/81.0"
+FIREFOX = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0"
+FIREFOX_MAC = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:78.0) Gecko/20100101 Firefox/78.0"
 IE_6 = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; WOW64; Trident/4.0; SLCC1)"
 IE_7 = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; WOW64; Trident/4.0; SLCC1)"
 IE_8 = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; SLCC1)"
diff --git a/src/streamlink/plugins/.removed b/src/streamlink/plugins/.removed
index feac9162..24623241 100644
--- a/src/streamlink/plugins/.removed
+++ b/src/streamlink/plugins/.removed
@@ -5,7 +5,6 @@ afreecatv
 aftonbladet
 aliez
 alieztv
-aljazeeraen
 antenna
 apac
 arconai
diff --git a/src/streamlink/plugins/aljazeeraen.py b/src/streamlink/plugins/aljazeeraen.py
new file mode 100644
index 00000000..9d8431f6
--- /dev/null
+++ b/src/streamlink/plugins/aljazeeraen.py
@@ -0,0 +1,39 @@
+import re
+
+from streamlink.plugin import Plugin
+from streamlink.plugins.brightcove import BrightcovePlayer
+from streamlink.stream import RTMPStream
+
+
+class AlJazeeraEnglish(Plugin):
+    url_re = re.compile(r"https?://(?:\w+\.)?aljazeera\.com")
+    account_id = 665003303001
+    render_re = re.compile(r'''RenderPagesVideo\((?P<q>['"])(?P<id>\d+)(?P=q)''')  # VOD
+    video_id_re = re.compile(r'''videoId=(?P<id>\d+)["']''')  # Live
+
+    @classmethod
+    def can_handle_url(cls, url):
+        return cls.url_re.match(url) is not None
+
+    def _get_streams(self):
+        res = self.session.http.get(self.url)
+
+        # check two different styles to include the video id in the page
+        video_id_m = self.render_re.search(res.text) or self.video_id_re.search(res.text)
+        video_id = video_id_m and video_id_m.group("id")
+
+        if not video_id:
+            self.logger.error("Could not find a video ID on this page")
+            return
+
+        # Use BrightcovePlayer class to get the streams
+        self.logger.debug("Found video ID: {0}", video_id)
+        bp = BrightcovePlayer(self.session, self.account_id)
+
+        for q, s in bp.get_streams(video_id):
+            # RTMP Streams are listed, but they do not appear to work
+            if not isinstance(s, RTMPStream):
+                yield q, s
+
+
+__plugin__ = AlJazeeraEnglish
diff --git a/src/streamlink/plugins/twitch.py b/src/streamlink/plugins/twitch.py
index 6bdfd452..202d6949 100644
--- a/src/streamlink/plugins/twitch.py
+++ b/src/streamlink/plugins/twitch.py
@@ -19,7 +19,7 @@ from streamlink.stream import (
 )
 from streamlink.stream.hls import HLSStreamWorker
 from streamlink.stream.hls_filtered import FilteredHLSStreamWriter, FilteredHLSStreamReader
-from streamlink.stream.hls_playlist import M3U8Parser, load as load_hls_playlist
+from streamlink.stream.hls_playlist import M3U8, M3U8Parser, load as load_hls_playlist
 from streamlink.utils.times import hours_minutes_seconds
 
 try:
@@ -125,19 +125,29 @@ Segment = namedtuple("Segment", "uri duration title key discontinuity ad byteran
 LOW_LATENCY_MAX_LIVE_EDGE = 2
 
 
-class TwitchM3U8Parser(M3U8Parser):
-    def parse_extinf(self, value):
-        duration, title = super(TwitchM3U8Parser, self).parse_extinf(value)
-        if title and str(title).startswith("Amazon"):
-            self.state["ad"] = True
+class TwitchM3U8(M3U8):
+    def __init__(self):
+        super(TwitchM3U8, self).__init__()
+        self.dateranges_ads = []
 
-        return duration, title
 
+class TwitchM3U8Parser(M3U8Parser):
     def parse_tag_ext_x_twitch_prefetch(self, value):
         segments = self.m3u8.segments
         if segments:
             segments.append(segments[-1]._replace(uri=self.uri(value), prefetch=True))
 
+    def parse_tag_ext_x_daterange(self, value):
+        super(TwitchM3U8Parser, self).parse_tag_ext_x_daterange(value)
+        daterange = self.m3u8.dateranges[-1]
+        is_ad = (
+            daterange.classname == "twitch-stitched-ad"
+            or str(daterange.id or "").startswith("stitched-ad-")
+            or any(attr_key.startswith("X-TV-TWITCH-AD-") for attr_key in daterange.x.keys())
+        )
+        if is_ad:
+            self.m3u8.dateranges_ads.append(daterange)
+
     def get_segment(self, uri):
         byterange = self.state.pop("byterange", None)
         extinf = self.state.pop("extinf", (0, None))
@@ -145,7 +155,7 @@ class TwitchM3U8Parser(M3U8Parser):
         map_ = self.state.get("map")
         key = self.state.get("key")
         discontinuity = self.state.pop("discontinuity", False)
-        ad = self.state.pop("ad", False)
+        ad = any(self.m3u8.is_date_in_daterange(date, daterange) for daterange in self.m3u8.dateranges_ads)
 
         return Segment(
             uri,
@@ -167,7 +177,7 @@ class TwitchHLSStreamWorker(HLSStreamWorker):
         super(TwitchHLSStreamWorker, self).__init__(reader, *args, **kwargs)
 
     def _reload_playlist(self, *args):
-        return load_hls_playlist(*args, parser=TwitchM3U8Parser)
+        return load_hls_playlist(*args, parser=TwitchM3U8Parser, m3u8=TwitchM3U8)
 
     def _playlist_reload_time(self, playlist, sequences):
         if self.stream.low_latency and sequences:
diff --git a/src/streamlink/stream/hls_playlist.py b/src/streamlink/stream/hls_playlist.py
index c3ea8851..130061c3 100644
--- a/src/streamlink/stream/hls_playlist.py
+++ b/src/streamlink/stream/hls_playlist.py
@@ -3,10 +3,14 @@ import logging
 
 from binascii import unhexlify
 from collections import namedtuple
+from datetime import timedelta
 from itertools import starmap
 
+from isodate import parse_datetime
+
 from streamlink.compat import urljoin, urlparse
 
+
 log = logging.getLogger(__name__)
 
 __all__ = ["load", "M3U8Parser"]
@@ -15,6 +19,9 @@ __all__ = ["load", "M3U8Parser"]
 # EXT-X-BYTERANGE
 ByteRange = namedtuple("ByteRange", "range offset")
 
+# EXT-X-DATERANGE
+DateRange = namedtuple("DateRange", "id classname start_date end_date duration planned_duration end_on_next x")
+
 # EXT-X-KEY
 Key = namedtuple("Key", "method uri iv key_format key_format_versions")
 
@@ -54,8 +61,24 @@ class M3U8(object):
 
         self.media = []
         self.playlists = []
+        self.dateranges = []
         self.segments = []
 
+    @classmethod
+    def is_date_in_daterange(cls, date, daterange):
+        if date is None or daterange.start_date is None:
+            return None
+
+        if daterange.end_date is not None:
+            return daterange.start_date <= date < daterange.end_date
+
+        duration = daterange.duration or daterange.planned_duration
+        if duration is not None:
+            end = daterange.start_date + duration
+            return daterange.start_date <= date < end
+
+        return daterange.start_date <= date
+
 
 class M3U8Parser(object):
     _extinf_re = re.compile(r"(?P<duration>\d+(\.\d+)?)(,(?P<title>.+))?")
@@ -64,8 +87,10 @@ class M3U8Parser(object):
     _tag_re = re.compile(r"#(?P<tag>[\w-]+)(:(?P<value>.+))?")
     _res_re = re.compile(r"(\d+)x(\d+)")
 
-    def __init__(self, base_uri=None, **kwargs):
+    def __init__(self, base_uri=None, m3u8=M3U8, **kwargs):
         self.base_uri = base_uri
+        self.m3u8 = m3u8()
+        self.state = {}
 
     def create_stream_info(self, streaminf, cls=None):
         program_id = streaminf.get("PROGRAM-ID")
@@ -131,6 +156,17 @@ class M3U8Parser(object):
 
         return unhexlify(value)
 
+    def parse_iso8601(self, value):
+        if value is None:
+            return None
+        try:
+            return parse_datetime(value)
+        except ValueError:
+            return None
+
+    def parse_timedelta(self, value):
+        return timedelta(seconds=float(value)) if value is not None else None
+
     def parse_resolution(self, value):
         match = self._res_re.match(value)
 
@@ -166,7 +202,21 @@ class M3U8Parser(object):
                                 attr.get("KEYFORMATVERSIONS"))
 
     def parse_tag_ext_x_program_date_time(self, value):
-        self.state["date"] = value
+        self.state["date"] = self.parse_iso8601(value)
+
+    def parse_tag_ext_x_daterange(self, value):
+        attr = self.parse_attributes(value)
+        daterange = DateRange(
+            attr.pop("ID", None),
+            attr.pop("CLASS", None),
+            self.parse_iso8601(attr.pop("START-DATE", None)),
+            self.parse_iso8601(attr.pop("END-DATE", None)),
+            self.parse_timedelta(attr.pop("DURATION", None)),
+            self.parse_timedelta(attr.pop("PLANNED-DURATION", None)),
+            self.parse_bool(attr.pop("END-ON-NEXT", None)),
+            attr
+        )
+        self.m3u8.dateranges.append(daterange)
 
     def parse_tag_ext_x_allow_cache(self, value):
         self.m3u8.allow_cache = self.parse_bool(value)
@@ -244,9 +294,6 @@ class M3U8Parser(object):
             self.m3u8.playlists.append(playlist)
 
     def parse(self, data):
-        self.state = {}
-        self.m3u8 = M3U8()
-
         lines = iter(filter(bool, data.splitlines()))
         try:
             line = next(lines)


Test patch:
diff --git a/tests/plugins/test_aljazeeraen.py b/tests/plugins/test_aljazeeraen.py
new file mode 100644
index 00000000..68f3bed8
--- /dev/null
+++ b/tests/plugins/test_aljazeeraen.py
@@ -0,0 +1,20 @@
+import unittest
+
+from streamlink.plugins.aljazeeraen import AlJazeeraEnglish
+
+
+class TestPluginAlJazeeraEnglish(unittest.TestCase):
+    def test_can_handle_url(self):
+        should_match = [
+            'https://www.aljazeera.com/live/',
+            'https://www.aljazeera.com/programmes/techknow/2017/04/science-sugar-170429141233635.html',
+        ]
+        for url in should_match:
+            self.assertTrue(AlJazeeraEnglish.can_handle_url(url))
+
+    def test_can_handle_url_negative(self):
+        should_not_match = [
+            'https://example.com/index.html',
+        ]
+        for url in should_not_match:
+            self.assertFalse(AlJazeeraEnglish.can_handle_url(url))
diff --git a/tests/plugins/test_twitch.py b/tests/plugins/test_twitch.py
index 3056c4cc..bfc5d99e 100644
--- a/tests/plugins/test_twitch.py
+++ b/tests/plugins/test_twitch.py
@@ -1,17 +1,42 @@
-import requests_mock
+from datetime import datetime, timedelta
 import unittest
 
-from tests.mixins.stream_hls import Playlist, Segment as Segment, TestMixinStreamHLS
+import requests_mock
+
+from tests.mixins.stream_hls import Playlist, Tag, Segment as _Segment, TestMixinStreamHLS
 from tests.mock import MagicMock, call, patch
 
 from streamlink import Streamlink
 from streamlink.plugins.twitch import Twitch, TwitchHLSStream
 
 
-class SegmentAd(Segment):
-    def __init__(self, *args, **kwargs):
-        super(SegmentAd, self).__init__(*args, **kwargs)
-        self.title = "Amazon|123456789"
+DATETIME_BASE = datetime(2000, 1, 1, 0, 0, 0, 0)
+DATETIME_FORMAT = "%Y-%m-%dT%H:%M:%S.%fZ"
+
+
+class TagDateRangeAd(Tag):
+    def __init__(self, start=DATETIME_BASE, duration=1, id="stitched-ad-1234", classname="twitch-stitched-ad", custom=None):
+        attrs = {
+            "ID": self.val_quoted_string(id),
+            "CLASS": self.val_quoted_string(classname),
+            "START-DATE": self.val_quoted_string(start.strftime(DATETIME_FORMAT)),
+            "DURATION": duration
+        }
+        if custom is not None:
+            attrs.update(**{key: self.val_quoted_string(value) for (key, value) in custom.items()})
+        super(TagDateRangeAd, self).__init__("EXT-X-DATERANGE", attrs)
+
+
+class Segment(_Segment):
+    def __init__(self, num, title="live", *args, **kwargs):
+        super(Segment, self).__init__(num, title, *args, **kwargs)
+        self.date = DATETIME_BASE + timedelta(seconds=num)
+
+    def build(self, namespace):
+        return "#EXT-X-PROGRAM-DATE-TIME:{0}\n{1}".format(
+            self.date.strftime(DATETIME_FORMAT),
+            super(Segment, self).build(namespace)
+        )
 
 
 class SegmentPrefetch(Segment):
@@ -51,11 +76,48 @@ class TestTwitchHLSStream(TestMixinStreamHLS, unittest.TestCase):
 
         return session
 
+    def test_hls_disable_ads_daterange_unknown(self):
+        daterange = TagDateRangeAd(start=DATETIME_BASE, duration=1, id="foo", classname="bar", custom=None)
+        thread, segments = self.subject([
+            Playlist(0, [daterange, Segment(0), Segment(1)], end=True)
+        ], disable_ads=True, low_latency=False)
+
+        self.assertEqual(self.await_read(read_all=True), self.content(segments), "Doesn't filter out segments")
+        self.assertTrue(all(self.called(s) for s in segments.values()), "Downloads all segments")
+
+    def test_hls_disable_ads_daterange_by_class(self):
+        daterange = TagDateRangeAd(start=DATETIME_BASE, duration=1, id="foo", classname="twitch-stitched-ad", custom=None)
+        thread, segments = self.subject([
+            Playlist(0, [daterange, Segment(0), Segment(1)], end=True)
+        ], disable_ads=True, low_latency=False)
+
+        self.assertEqual(self.await_read(read_all=True), segments[1].content, "Filters out ad segments")
+        self.assertTrue(all(self.called(s) for s in segments.values()), "Downloads all segments")
+
+    def test_hls_disable_ads_daterange_by_id(self):
+        daterange = TagDateRangeAd(start=DATETIME_BASE, duration=1, id="stitched-ad-1234", classname="/", custom=None)
+        thread, segments = self.subject([
+            Playlist(0, [daterange, Segment(0), Segment(1)], end=True)
+        ], disable_ads=True, low_latency=False)
+
+        self.assertEqual(self.await_read(read_all=True), segments[1].content, "Filters out ad segments")
+        self.assertTrue(all(self.called(s) for s in segments.values()), "Downloads all segments")
+
+    def test_hls_disable_ads_daterange_by_attr(self):
+        daterange = TagDateRangeAd(start=DATETIME_BASE, duration=1, id="foo", classname="/", custom={"X-TV-TWITCH-AD-URL": "/"})
+        thread, segments = self.subject([
+            Playlist(0, [daterange, Segment(0), Segment(1)], end=True)
+        ], disable_ads=True, low_latency=False)
+
+        self.assertEqual(self.await_read(read_all=True), segments[1].content, "Filters out ad segments")
+        self.assertTrue(all(self.called(s) for s in segments.values()), "Downloads all segments")
+
     @patch("streamlink.plugins.twitch.log")
     def test_hls_disable_ads_has_preroll(self, mock_log):
+        daterange = TagDateRangeAd(duration=4)
         thread, segments = self.subject([
-            Playlist(0, [SegmentAd(0), SegmentAd(1)]),
-            Playlist(2, [SegmentAd(2), SegmentAd(3)]),
+            Playlist(0, [daterange, Segment(0), Segment(1)]),
+            Playlist(2, [daterange, Segment(2), Segment(3)]),
             Playlist(4, [Segment(4), Segment(5)], end=True)
         ], disable_ads=True, low_latency=False)
 
@@ -72,9 +134,10 @@ class TestTwitchHLSStream(TestMixinStreamHLS, unittest.TestCase):
 
     @patch("streamlink.plugins.twitch.log")
     def test_hls_disable_ads_has_midstream(self, mock_log):
+        daterange = TagDateRangeAd(start=DATETIME_BASE + timedelta(seconds=2), duration=2)
         thread, segments = self.subject([
             Playlist(0, [Segment(0), Segment(1)]),
-            Playlist(2, [SegmentAd(2), SegmentAd(3)]),
+            Playlist(2, [daterange, Segment(2), Segment(3)]),
             Playlist(4, [Segment(4), Segment(5)], end=True)
         ], disable_ads=True, low_latency=False)
 
@@ -90,8 +153,9 @@ class TestTwitchHLSStream(TestMixinStreamHLS, unittest.TestCase):
 
     @patch("streamlink.plugins.twitch.log")
     def test_hls_no_disable_ads_has_preroll(self, mock_log):
+        daterange = TagDateRangeAd(duration=2)
         thread, segments = self.subject([
-            Playlist(0, [SegmentAd(0), SegmentAd(1)]),
+            Playlist(0, [daterange, Segment(0), Segment(1)]),
             Playlist(2, [Segment(2), Segment(3)], end=True)
         ], disable_ads=False, low_latency=False)
 
@@ -161,8 +225,9 @@ class TestTwitchHLSStream(TestMixinStreamHLS, unittest.TestCase):
 
     @patch("streamlink.plugins.twitch.log")
     def test_hls_low_latency_has_prefetch_has_preroll(self, mock_log):
+        daterange = TagDateRangeAd(duration=4)
         thread, segments = self.subject([
-            Playlist(0, [SegmentAd(0), SegmentAd(1), SegmentAd(2), SegmentAd(3)]),
+            Playlist(0, [daterange, Segment(0), Segment(1), Segment(2), Segment(3)]),
             Playlist(4, [Segment(4), Segment(5), Segment(6), Segment(7), SegmentPrefetch(8), SegmentPrefetch(9)], end=True)
         ], disable_ads=False, low_latency=True)
 
@@ -179,8 +244,9 @@ class TestTwitchHLSStream(TestMixinStreamHLS, unittest.TestCase):
 
     @patch("streamlink.plugins.twitch.log")
     def test_hls_low_latency_has_prefetch_disable_ads_has_preroll(self, mock_log):
+        daterange = TagDateRangeAd(duration=4)
         self.subject([
-            Playlist(0, [SegmentAd(0), SegmentAd(1), SegmentAd(2), SegmentAd(3)]),
+            Playlist(0, [daterange, Segment(0), Segment(1), Segment(2), Segment(3)]),
             Playlist(4, [Segment(4), Segment(5), Segment(6), Segment(7), SegmentPrefetch(8), SegmentPrefetch(9)], end=True)
         ], disable_ads=True, low_latency=True)
 
@@ -193,8 +259,9 @@ class TestTwitchHLSStream(TestMixinStreamHLS, unittest.TestCase):
 
     @patch("streamlink.plugins.twitch.log")
     def test_hls_low_latency_no_prefetch_disable_ads_has_preroll(self, mock_log):
+        daterange = TagDateRangeAd(duration=4)
         self.subject([
-            Playlist(0, [SegmentAd(0), SegmentAd(1), SegmentAd(2), SegmentAd(3)]),
+            Playlist(0, [daterange, Segment(0), Segment(1), Segment(2), Segment(3)]),
             Playlist(4, [Segment(4), Segment(5), Segment(6), Segment(7)], end=True)
         ], disable_ads=True, low_latency=True)
 
diff --git a/tests/resources/hls/test_date.m3u8 b/tests/resources/hls/test_date.m3u8
new file mode 100644
index 00000000..e3657e50
--- /dev/null
+++ b/tests/resources/hls/test_date.m3u8
@@ -0,0 +1,33 @@
+#EXTM3U
+
+#EXT-X-TARGETDURATION:120
+
+#EXT-X-DATERANGE:ID="start-invalid",START-DATE="invalid"
+#EXT-X-DATERANGE:ID="start-no-frac",START-DATE="2000-01-01T00:00:00Z"
+#EXT-X-DATERANGE:ID="start-with-frac",START-DATE="2000-01-01T00:00:00.000Z"
+
+#EXT-X-DATERANGE:ID="with-class",CLASS="bar",START-DATE="2000-01-01T00:00:00.000Z"
+
+#EXT-X-DATERANGE:ID="duration",START-DATE="2000-01-01T00:00:00.000Z",DURATION=30.5
+#EXT-X-DATERANGE:ID="planned-duration",START-DATE="2000-01-01T00:00:00.000Z",PLANNED-DURATION=15
+#EXT-X-DATERANGE:ID="duration-precedence",START-DATE="2000-01-01T00:00:00.000Z",DURATION=30.5,PLANNED-DURATION=15
+#EXT-X-DATERANGE:ID="end",START-DATE="2000-01-01T00:00:00.000Z",END-DATE="2000-01-01T00:01:00.000Z"
+#EXT-X-DATERANGE:ID="end-precedence",START-DATE="2000-01-01T00:00:00.000Z",END-DATE="2000-01-01T00:01:00.000Z",DURATION=30.5
+
+#EXT-X-DATERANGE:X-CUSTOM="value"
+
+#EXT-X-PROGRAM-DATE-TIME:2000-01-01T00:00:00.000Z
+#EXTINF:15.000,live
+segment0-15.ts
+
+#EXT-X-PROGRAM-DATE-TIME:2000-01-01T00:00:15.000Z
+#EXTINF:15.500,live
+segment15-30.5.ts
+
+#EXT-X-PROGRAM-DATE-TIME:2000-01-01T00:00:30.500Z
+#EXTINF:29.500,live
+segment30.5-60.ts
+
+#EXT-X-PROGRAM-DATE-TIME:2000-01-01T00:01:00.000Z
+#EXTINF:60.000,live
+segment60-.ts
diff --git a/tests/streams/test_hls_playlist.py b/tests/streams/test_hls_playlist.py
index 1b1357d6..0f39479a 100644
--- a/tests/streams/test_hls_playlist.py
+++ b/tests/streams/test_hls_playlist.py
@@ -1,10 +1,15 @@
 # -*- coding: utf-8 -*-
 from __future__ import unicode_literals
 
-from streamlink.stream.hls_playlist import load, StreamInfo, Resolution, Media
+from datetime import datetime, timedelta
 import unittest
+
+from isodate import tzinfo
+
 from tests.resources import text
 
+from streamlink.stream.hls_playlist import load, StreamInfo, Resolution, Media, DateRange, Segment
+
 
 class TestHLSPlaylist(unittest.TestCase):
     def test_load(self):
@@ -66,3 +71,81 @@ class TestHLSPlaylist(unittest.TestCase):
                            subtitles='subs')
             ]
         )
+
+    def test_parse_date(self):
+        with text("hls/test_date.m3u8") as m3u8_fh:
+            playlist = load(m3u8_fh.read(), "http://test.se/")
+
+        start_date = datetime(year=2000, month=1, day=1, hour=0, minute=0, second=0, microsecond=0, tzinfo=tzinfo.UTC)
+        end_date = datetime(year=2000, month=1, day=1, hour=0, minute=1, second=0, microsecond=0, tzinfo=tzinfo.UTC)
+        delta_15 = timedelta(seconds=15)
+        delta_30 = timedelta(seconds=30, milliseconds=500)
+        delta_60 = timedelta(seconds=60)
+
+        self.assertEqual(playlist.target_duration, 120)
+
+        self.assertEqual(
+            [daterange for daterange in playlist.dateranges],
+            [
+                DateRange(id="start-invalid",
+                          start_date=None,
+                          classname=None, end_date=None, duration=None, planned_duration=None, end_on_next=False, x={}),
+                DateRange(id="start-no-frac",
+                          start_date=start_date,
+                          classname=None, end_date=None, duration=None, planned_duration=None, end_on_next=False, x={}),
+                DateRange(id="start-with-frac",
+                          start_date=start_date,
+                          classname=None, end_date=None, duration=None, planned_duration=None, end_on_next=False, x={}),
+                DateRange(id="with-class",
+                          start_date=start_date, classname="bar",
+                          end_date=None, duration=None, planned_duration=None, end_on_next=False, x={}),
+                DateRange(id="duration",
+                          start_date=start_date, duration=delta_30,
+                          classname=None, end_date=None, planned_duration=None, end_on_next=False, x={}),
+                DateRange(id="planned-duration",
+                          start_date=start_date, planned_duration=delta_15,
+                          classname=None, end_date=None, duration=None, end_on_next=False, x={}),
+                DateRange(id="duration-precedence",
+                          start_date=start_date, duration=delta_30, planned_duration=delta_15,
+                          classname=None, end_date=None, end_on_next=False, x={}),
+                DateRange(id="end",
+                          start_date=start_date, end_date=end_date,
+                          classname=None, duration=None, planned_duration=None, end_on_next=False, x={}),
+                DateRange(id="end-precedence",
+                          start_date=start_date, end_date=end_date, duration=delta_30,
+                          classname=None, planned_duration=None, end_on_next=False, x={}),
+                DateRange(x={"X-CUSTOM": "value"},
+                          id=None, start_date=None, end_date=None, duration=None,
+                          classname=None, planned_duration=None, end_on_next=False)
+            ]
+        )
+        self.assertEqual(
+            [segment for segment in playlist.segments],
+            [
+                Segment(uri="http://test.se/segment0-15.ts", duration=15.0, title="live", date=start_date,
+                        key=None, discontinuity=False, byterange=None, map=None),
+                Segment(uri="http://test.se/segment15-30.5.ts", duration=15.5, title="live", date=start_date + delta_15,
+                        key=None, discontinuity=False, byterange=None, map=None),
+                Segment(uri="http://test.se/segment30.5-60.ts", duration=29.5, title="live", date=start_date + delta_30,
+                        key=None, discontinuity=False, byterange=None, map=None),
+                Segment(uri="http://test.se/segment60-.ts", duration=60.0, title="live", date=start_date + delta_60,
+                        key=None, discontinuity=False, byterange=None, map=None)
+            ]
+        )
+
+        self.assertEqual(
+            [playlist.is_date_in_daterange(playlist.segments[0].date, daterange) for daterange in playlist.dateranges],
+            [None, True, True, True, True, True, True, True, True, None]
+        )
+        self.assertEqual(
+            [playlist.is_date_in_daterange(playlist.segments[1].date, daterange) for daterange in playlist.dateranges],
+            [None, True, True, True, True, False, True, True, True, None]
+        )
+        self.assertEqual(
+            [playlist.is_date_in_daterange(playlist.segments[2].date, daterange) for daterange in playlist.dateranges],
+            [None, True, True, True, False, False, False, True, True, None]
+        )
+        self.assertEqual(
+            [playlist.is_date_in_daterange(playlist.segments[3].date, daterange) for daterange in playlist.dateranges],
+            [None, True, True, True, False, False, False, False, False, None]
+        )