Repository: yt-dlp/yt-dlp
Pull request #2714 (2022-02-10 11:35:09)

fallback to lockf()
### Before submitting a *pull request* make sure you have:
- [ ] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [x] Bug fix
- [ ] Improvement
- [ ] New extractor
- [ ] New feature

---

#1066 introduced regression #2670, this _may_ fix, please test.


RedirectCleaner: this seems to fix only a part of the problem, it now downloads the file but still throws an error

[debug] Command-line config: ['-v', 'https://www.youtube.com/watch?v=9s_XAmM_vb4']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2022.02.04 [c1653e9ef] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: a3125791c
[debug] Python version 3.10.2 (CPython 64bit) - Linux-4.14.180-lineageos-g9695abe891c8-aarch64-with-libc
[debug] exe versions: ffmpeg 5.0 (setts), ffprobe 5.0, rtmpdump 2.4
[debug] Optional libraries: sqlite
[debug] Proxy map: {}
[debug] [youtube] Extracting URL: https://www.youtube.com/watch?v=9s_XAmM_vb4
[youtube] 9s_XAmM_vb4: Downloading webpage
[youtube] 9s_XAmM_vb4: Downloading android player API JSON
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 9s_XAmM_vb4: Downloading 1 format(s): 248+251
[debug] Invoking downloader on "https://rr3---sn-h0jeenl6.googlevideo.com/videoplayback?expire=1644518137&ei=mQYFYrzyCsuq-gaborPgCQ&ip=62.178.198.204&id=o-AA0Dh52TzcY_x3B2YSQD4WAR3XmO6QUfmkAElLf6pUQ7&itag=248&source=youtube&requiressl=yes&mh=W8&mm=31%2C26&mn=sn-h0jeenl6%2Csn-4g5ednz7&ms=au%2Conr&mv=m&mvi=3&pl=18&initcwndbps=1913750&vprv=1&mime=video%2Fwebm&gir=yes&clen=37572106&dur=160.193&lmt=1640262185659293&mt=1644496181&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=5437434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRAIgGakCCD9DA6IXV5yYTrc3uw-dKgW6LvjYCD_hjjnYqhECIFt5_SNJG3Uo5ASpXnLm-AhGtptFwEmWq4rqi89o4prG&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAKqNULmc4NNcC-9vr3uSI8Bd_JyXh_NZ0Nk2vhDDnhaPAiEA3qKsQqyeAXYQA6wMLGXlkPnug9rmVqc8t9TIEez8L5w%3D"
[download] Resuming download at byte 37572106
[download] dicht & ergreifend – Nix gibt´s [9s_XAmM_vb4].f248.webm has already been downloaded
[download] 100% of 35.83MiB
[debug] Invoking downloader on "https://rr3---sn-h0jeenl6.googlevideo.com/videoplayback?expire=1644518137&ei=mQYFYrzyCsuq-gaborPgCQ&ip=62.178.198.204&id=o-AA0Dh52TzcY_x3B2YSQD4WAR3XmO6QUfmkAElLf6pUQ7&itag=251&source=youtube&requiressl=yes&mh=W8&mm=31%2C26&mn=sn-h0jeenl6%2Csn-4g5ednz7&ms=au%2Conr&mv=m&mvi=3&pl=18&initcwndbps=1913750&vprv=1&mime=audio%2Fwebm&gir=yes&clen=2605622&dur=160.221&lmt=1640260597790651&mt=1644496181&fvip=3&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=5432434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAPsGpH-sEB1m2cocAVFS7wLhl3LS0dvFLjAQfNtJeJpEAiAoKUTRkjhe6Agj9VLxJWevV4DZm3l9PP350VkYcAXnAQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAKqNULmc4NNcC-9vr3uSI8Bd_JyXh_NZ0Nk2vhDDnhaPAiEA3qKsQqyeAXYQA6wMLGXlkPnug9rmVqc8t9TIEez8L5w%3D"
[download] Destination: dicht & ergreifend – Nix gibt´s [9s_XAmM_vb4].f251.webm
[download] 100% of 2.48MiB at  3.60MiB/s ETA 00:00ERROR: unable to download video data: [Errno 38] Function not implemented
Traceback (most recent call last):
  File "/data/data/com.termux/files/home/git/yt-dlp/yt_dlp/YoutubeDL.py", line 3068, in process_info
    partial_success, real_download = self.dl(fname, new_info)
  File "/data/data/com.termux/files/home/git/yt-dlp/yt_dlp/YoutubeDL.py", line 2801, in dl
    return fd.download(name, new_info, subtitle)
  File "/data/data/com.termux/files/home/git/yt-dlp/yt_dlp/downloader/common.py", line 440, in download
    ret = self.real_download(filename, info_dict)
  File "/data/data/com.termux/files/home/git/yt-dlp/yt_dlp/downloader/http.py", line 372, in real_download
    return download()
  File "/data/data/com.termux/files/home/git/yt-dlp/yt_dlp/downloader/http.py", line 344, in download
    ctx.stream.close()
  File "/data/data/com.termux/files/home/git/yt-dlp/yt_dlp/utils.py", line 2207, in close
    self.__exit__(self, *args, value=False, traceback=False)
  File "/data/data/com.termux/files/home/git/yt-dlp/yt_dlp/utils.py", line 2187, in __exit__
    _unlock_file(self.f)
  File "/data/data/com.termux/files/home/git/yt-dlp/yt_dlp/utils.py", line 2157, in _unlock_file
    fcntl.flock(f, fcntl.LOCK_UN)
OSError: [Errno 38] Function not implemented

RedirectCleaner: as far as i understand the patch, android implements lockf, so why not use it, when it's available
pukkandan: I totally forgot https://github.com/yt-dlp/yt-dlp/issues/2080
jakeogh: @RedirectCleaner excellent, that is the expected result before https://github.com/yt-dlp/yt-dlp/pull/2656 is merged.

Looks good to merge.

Patch:
diff --git a/README.md b/README.md
index 7446cc2c2..1aab0ba94 100644
--- a/README.md
+++ b/README.md
@@ -951,7 +951,7 @@ ## Post-Processing Options:
                                      (currently supported: srt|vtt|ass|lrc)
                                      (Alias: --convert-subtitles)
     --convert-thumbnails FORMAT      Convert the thumbnails to another format
-                                     (currently supported: jpg|png)
+                                     (currently supported: jpg|png|webp)
     --split-chapters                 Split video into multiple files based on
                                      internal chapters. The "chapter:" prefix
                                      can be used with "--paths" and "--output"
@@ -982,15 +982,17 @@ ## Post-Processing Options:
                                      semicolon ";" delimited list of NAME=VALUE.
                                      The "when" argument determines when the
                                      postprocessor is invoked. It can be one of
-                                     "pre_process" (after extraction),
-                                     "before_dl" (before video download),
-                                     "post_process" (after video download;
-                                     default), "after_move" (after moving file
-                                     to their final locations), "after_video"
-                                     (after downloading and processing all
-                                     formats of a video), or "playlist" (end of
-                                     playlist). This option can be used multiple
-                                     times to add different postprocessors
+                                     "pre_process" (after video extraction),
+                                     "after_filter" (after video passes filter),
+                                     "before_dl" (before each video download),
+                                     "post_process" (after each video download;
+                                     default), "after_move" (after moving video
+                                     file to it's final locations),
+                                     "after_video" (after downloading and
+                                     processing all formats of a video), or
+                                     "playlist" (at end of playlist). This
+                                     option can be used multiple times to add
+                                     different postprocessors
 
 ## SponsorBlock Options:
 Make chapter entries for, or remove various segments (sponsor,
@@ -1663,6 +1665,7 @@ #### youtube
 
 #### youtubetab (YouTube playlists, channels, feeds, etc.)
 * `skip`: One or more of `webpage` (skip initial webpage download), `authcheck` (allow the download of playlists requiring authentication when no initial webpage is downloaded. This may cause unwanted behavior, see [#1122](https://github.com/yt-dlp/yt-dlp/pull/1122) for more details)
+* `approximate_date`: Extract approximate `upload_date` in flat-playlist. This may cause date-based filters to be slightly off
 
 #### funimation
 * `language`: Languages to extract. Eg: `funimation:language=english,japanese`
@@ -1695,6 +1698,10 @@ #### tiktok
 * `app_version`: App version to call mobile APIs with - should be set along with `manifest_app_version`. (e.g. `20.2.1`)
 * `manifest_app_version`: Numeric app version to call mobile APIs with. (e.g. `221`)
 
+#### rokfinchannel
+* `tab`: Which tab to download. One of `new`, `top`, `videos`, `podcasts`, `streams`, `stacks`. (E.g. `rokfinchannel:tab=streams`)
+
+
 NOTE: These options may be changed/removed in the future without concern for backward compatibility
 
 <!-- MANPAGE: MOVE "INSTALLATION" SECTION HERE -->
diff --git a/devscripts/prepare_manpage.py b/devscripts/prepare_manpage.py
index 069d99eeb..6612723f7 100644
--- a/devscripts/prepare_manpage.py
+++ b/devscripts/prepare_manpage.py
@@ -75,21 +75,17 @@ def filter_options(readme):
     section = re.search(r'(?sm)^# USAGE AND OPTIONS\n.+?(?=^# )', readme).group(0)
     options = '# OPTIONS\n'
     for line in section.split('\n')[1:]:
-        if line.lstrip().startswith('-'):
-            split = re.split(r'\s{2,}', line.lstrip())
-            # Description string may start with `-` as well. If there is
-            # only one piece then it's a description bit not an option.
-            if len(split) > 1:
-                option, description = split
-                split_option = option.split(' ')
-
-                if not split_option[-1].startswith('-'):  # metavar
-                    option = ' '.join(split_option[:-1] + [f'*{split_option[-1]}*'])
-
-                # Pandoc's definition_lists. See http://pandoc.org/README.html
-                options += f'\n{option}\n:   {description}\n'
-                continue
-        options += line.lstrip() + '\n'
+        mobj = re.fullmatch(r'\s{4}(?P<opt>-(?:,\s|[^\s])+)(?:\s(?P<meta>([^\s]|\s(?!\s))+))?(\s{2,}(?P<desc>.+))?', line)
+        if not mobj:
+            options += f'{line.lstrip()}\n'
+            continue
+        option, metavar, description = mobj.group('opt', 'meta', 'desc')
+
+        # Pandoc's definition_lists. See http://pandoc.org/README.html
+        option = f'{option} *{metavar}*' if metavar else option
+        description = f'{description}\n' if description else ''
+        options += f'\n{option}\n:   {description}'
+        continue
 
     return readme.replace(section, options, 1)
 
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
index 36b2b37c0..10eebecf2 100644
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -1037,8 +1037,7 @@ def validate_outtmpl(cls, outtmpl):
     @staticmethod
     def _copy_infodict(info_dict):
         info_dict = dict(info_dict)
-        for key in ('__original_infodict', '__postprocessors'):
-            info_dict.pop(key, None)
+        info_dict.pop('__postprocessors', None)
         return info_dict
 
     def prepare_outtmpl(self, outtmpl, info_dict, sanitize=False):
@@ -1585,6 +1584,7 @@ def process_ie_result(self, ie_result, download=True, extra_info=None):
 
             self._playlist_level += 1
             self._playlist_urls.add(webpage_url)
+            self._fill_common_fields(ie_result, False)
             self._sanitize_thumbnails(ie_result)
             try:
                 return self.__process_playlist(ie_result, download)
@@ -2309,6 +2309,58 @@ def check_thumbnails(thumbnails):
         else:
             info_dict['thumbnails'] = thumbnails
 
+    def _fill_common_fields(self, info_dict, is_video=True):
+        # TODO: move sanitization here
+        if is_video:
+            # playlists are allowed to lack "title"
+            info_dict['fulltitle'] = info_dict.get('title')
+            if 'title' not in info_dict:
+                raise ExtractorError('Missing "title" field in extractor result',
+                                     video_id=info_dict['id'], ie=info_dict['extractor'])
+            elif not info_dict.get('title'):
+                self.report_warning('Extractor failed to obtain "title". Creating a generic title instead')
+                info_dict['title'] = f'{info_dict["extractor"]} video #{info_dict["id"]}'
+
+        if info_dict.get('duration') is not None:
+            info_dict['duration_string'] = formatSeconds(info_dict['duration'])
+
+        for ts_key, date_key in (
+                ('timestamp', 'upload_date'),
+                ('release_timestamp', 'release_date'),
+                ('modified_timestamp', 'modified_date'),
+        ):
+            if info_dict.get(date_key) is None and info_dict.get(ts_key) is not None:
+                # Working around out-of-range timestamp values (e.g. negative ones on Windows,
+                # see http://bugs.python.org/issue1646728)
+                try:
+                    upload_date = datetime.datetime.utcfromtimestamp(info_dict[ts_key])
+                    info_dict[date_key] = upload_date.strftime('%Y%m%d')
+                except (ValueError, OverflowError, OSError):
+                    pass
+
+        live_keys = ('is_live', 'was_live')
+        live_status = info_dict.get('live_status')
+        if live_status is None:
+            for key in live_keys:
+                if info_dict.get(key) is False:
+                    continue
+                if info_dict.get(key):
+                    live_status = key
+                break
+            if all(info_dict.get(key) is False for key in live_keys):
+                live_status = 'not_live'
+        if live_status:
+            info_dict['live_status'] = live_status
+            for key in live_keys:
+                if info_dict.get(key) is None:
+                    info_dict[key] = (live_status == key)
+
+        # Auto generate title fields corresponding to the *_number fields when missing
+        # in order to always have clean titles. This is very common for TV series.
+        for field in ('chapter', 'season', 'episode'):
+            if info_dict.get('%s_number' % field) is not None and not info_dict.get(field):
+                info_dict[field] = '%s %d' % (field.capitalize(), info_dict['%s_number' % field])
+
     def process_video_result(self, info_dict, download=True):
         assert info_dict.get('_type', 'video') == 'video'
         self._num_videos += 1
@@ -2318,14 +2370,6 @@ def process_video_result(self, info_dict, download=True):
         elif not info_dict.get('id'):
             raise ExtractorError('Extractor failed to obtain "id"', ie=info_dict['extractor'])
 
-        info_dict['fulltitle'] = info_dict.get('title')
-        if 'title' not in info_dict:
-            raise ExtractorError('Missing "title" field in extractor result',
-                                 video_id=info_dict['id'], ie=info_dict['extractor'])
-        elif not info_dict.get('title'):
-            self.report_warning('Extractor failed to obtain "title". Creating a generic title instead')
-            info_dict['title'] = f'{info_dict["extractor"]} video #{info_dict["id"]}'
-
         def report_force_conversion(field, field_not, conversion):
             self.report_warning(
                 '"%s" field is not %s - forcing %s conversion, there is an error in extractor'
@@ -2366,45 +2410,7 @@ def sanitize_numeric_fields(info):
         if info_dict.get('display_id') is None and 'id' in info_dict:
             info_dict['display_id'] = info_dict['id']
 
-        if info_dict.get('duration') is not None:
-            info_dict['duration_string'] = formatSeconds(info_dict['duration'])
-
-        for ts_key, date_key in (
-                ('timestamp', 'upload_date'),
-                ('release_timestamp', 'release_date'),
-                ('modified_timestamp', 'modified_date'),
-        ):
-            if info_dict.get(date_key) is None and info_dict.get(ts_key) is not None:
-                # Working around out-of-range timestamp values (e.g. negative ones on Windows,
-                # see http://bugs.python.org/issue1646728)
-                try:
-                    upload_date = datetime.datetime.utcfromtimestamp(info_dict[ts_key])
-                    info_dict[date_key] = upload_date.strftime('%Y%m%d')
-                except (ValueError, OverflowError, OSError):
-                    pass
-
-        live_keys = ('is_live', 'was_live')
-        live_status = info_dict.get('live_status')
-        if live_status is None:
-            for key in live_keys:
-                if info_dict.get(key) is False:
-                    continue
-                if info_dict.get(key):
-                    live_status = key
-                break
-            if all(info_dict.get(key) is False for key in live_keys):
-                live_status = 'not_live'
-        if live_status:
-            info_dict['live_status'] = live_status
-            for key in live_keys:
-                if info_dict.get(key) is None:
-                    info_dict[key] = (live_status == key)
-
-        # Auto generate title fields corresponding to the *_number fields when missing
-        # in order to always have clean titles. This is very common for TV series.
-        for field in ('chapter', 'season', 'episode'):
-            if info_dict.get('%s_number' % field) is not None and not info_dict.get(field):
-                info_dict[field] = '%s %d' % (field.capitalize(), info_dict['%s_number' % field])
+        self._fill_common_fields(info_dict)
 
         for cc_kind in ('subtitles', 'automatic_captions'):
             cc = info_dict.get(cc_kind)
@@ -2512,8 +2518,6 @@ def is_wellformed(f):
         if '__x_forwarded_for_ip' in info_dict:
             del info_dict['__x_forwarded_for_ip']
 
-        # TODO Central sorting goes here
-
         if self.params.get('check_formats') is True:
             formats = LazyList(self._check_formats(formats[::-1]), reverse=True)
 
@@ -2526,6 +2530,12 @@ def is_wellformed(f):
 
         info_dict, _ = self.pre_process(info_dict)
 
+        if self._match_entry(info_dict) is not None:
+            return info_dict
+
+        self.post_extract(info_dict)
+        info_dict, _ = self.pre_process(info_dict, 'after_filter')
+
         # The pre-processors may have modified the formats
         formats = info_dict.get('formats', [info_dict])
 
@@ -2610,15 +2620,12 @@ def is_wellformed(f):
                     + ', '.join([f['format_id'] for f in formats_to_download]))
             max_downloads_reached = False
             for i, fmt in enumerate(formats_to_download):
-                formats_to_download[i] = new_info = dict(info_dict)
-                # Save a reference to the original info_dict so that it can be modified in process_info if needed
+                formats_to_download[i] = new_info = self._copy_infodict(info_dict)
                 new_info.update(fmt)
-                new_info['__original_infodict'] = info_dict
                 try:
                     self.process_info(new_info)
                 except MaxDownloadsReached:
                     max_downloads_reached = True
-                new_info.pop('__original_infodict')
                 # Remove copied info
                 for key, val in tuple(new_info.items()):
                     if info_dict.get(key) == val:
@@ -2826,7 +2833,7 @@ def existing_file(self, filepaths, *, default_overwrite=True):
         return None
 
     def process_info(self, info_dict):
-        """Process a single resolved IE result. (Modified it in-place)"""
+        """Process a single resolved IE result. (Modifies it in-place)"""
 
         assert info_dict.get('_type', 'video') == 'video'
         original_infodict = info_dict
@@ -2834,18 +2841,22 @@ def process_info(self, info_dict):
         if 'format' not in info_dict and 'ext' in info_dict:
             info_dict['format'] = info_dict['ext']
 
+        # This is mostly just for backward compatibility of process_info
+        # As a side-effect, this allows for format-specific filters
         if self._match_entry(info_dict) is not None:
             info_dict['__write_download_archive'] = 'ignore'
             return
 
+        # Does nothing under normal operation - for backward compatibility of process_info
         self.post_extract(info_dict)
-        self._num_downloads += 1
 
         # info_dict['_filename'] needs to be set for backward compatibility
         info_dict['_filename'] = full_filename = self.prepare_filename(info_dict, warn=True)
         temp_filename = self.prepare_filename(info_dict, 'temp')
         files_to_move = {}
 
+        self._num_downloads += 1
+
         # Forced printings
         self.__forced_printings(info_dict, full_filename, incomplete=('format' not in info_dict))
 
@@ -3259,17 +3270,14 @@ def sanitize_info(info_dict, remove_private_keys=False):
             return info_dict
         info_dict.setdefault('epoch', int(time.time()))
         info_dict.setdefault('_type', 'video')
-        remove_keys = {'__original_infodict'}  # Always remove this since this may contain a copy of the entire dict
-        keep_keys = ['_type']  # Always keep this to facilitate load-info-json
+
         if remove_private_keys:
-            remove_keys |= {
+            reject = lambda k, v: v is None or (k.startswith('_') and k != '_type') or k in {
                 'requested_downloads', 'requested_formats', 'requested_subtitles', 'requested_entries',
                 'entries', 'filepath', 'infojson_filename', 'original_url', 'playlist_autonumber',
             }
-            reject = lambda k, v: k not in keep_keys and (
-                k.startswith('_') or k in remove_keys or v is None)
         else:
-            reject = lambda k, v: k in remove_keys
+            reject = lambda k, v: False
 
         def filter_fn(obj):
             if isinstance(obj, dict):
@@ -3296,14 +3304,8 @@ def actual_post_extract(info_dict):
                     actual_post_extract(video_dict or {})
                 return
 
-            post_extractor = info_dict.get('__post_extractor') or (lambda: {})
-            extra = post_extractor().items()
-            info_dict.update(extra)
-            info_dict.pop('__post_extractor', None)
-
-            original_infodict = info_dict.get('__original_infodict') or {}
-            original_infodict.update(extra)
-            original_infodict.pop('__post_extractor', None)
+            post_extractor = info_dict.pop('__post_extractor', None) or (lambda: {})
+            info_dict.update(post_extractor())
 
         actual_post_extract(info_dict or {})
 
diff --git a/yt_dlp/__init__.py b/yt_dlp/__init__.py
index b93f47ecc..c87c5b6df 100644
--- a/yt_dlp/__init__.py
+++ b/yt_dlp/__init__.py
@@ -474,8 +474,8 @@ def report_unplayable_conflict(opt_name, arg, default=False, allowed=None):
             'key': 'SponsorBlock',
             'categories': sponsorblock_query,
             'api': opts.sponsorblock_api,
-            # Run this immediately after extraction is complete
-            'when': 'pre_process'
+            # Run this after filtering videos
+            'when': 'after_filter'
         })
     if opts.parse_metadata:
         postprocessors.append({
diff --git a/yt_dlp/downloader/__init__.py b/yt_dlp/downloader/__init__.py
index 76841993b..96d484dee 100644
--- a/yt_dlp/downloader/__init__.py
+++ b/yt_dlp/downloader/__init__.py
@@ -30,6 +30,7 @@ def get_suitable_downloader(info_dict, params={}, default=NO_DEFAULT, protocol=N
 from .common import FileDownloader
 from .dash import DashSegmentsFD
 from .f4m import F4mFD
+from .fc2 import FC2LiveFD
 from .hls import HlsFD
 from .http import HttpFD
 from .rtmp import RtmpFD
@@ -58,6 +59,7 @@ def get_suitable_downloader(info_dict, params={}, default=NO_DEFAULT, protocol=N
     'ism': IsmFD,
     'mhtml': MhtmlFD,
     'niconico_dmc': NiconicoDmcFD,
+    'fc2_live': FC2LiveFD,
     'websocket_frag': WebSocketFragmentFD,
     'youtube_live_chat': YoutubeLiveChatFD,
     'youtube_live_chat_replay': YoutubeLiveChatFD,
diff --git a/yt_dlp/downloader/fc2.py b/yt_dlp/downloader/fc2.py
new file mode 100644
index 000000000..157bcf23e
--- /dev/null
+++ b/yt_dlp/downloader/fc2.py
@@ -0,0 +1,41 @@
+from __future__ import division, unicode_literals
+
+import threading
+
+from .common import FileDownloader
+from .external import FFmpegFD
+
+
+class FC2LiveFD(FileDownloader):
+    """
+    Downloads FC2 live without being stopped. <br>
+    Note, this is not a part of public API, and will be removed without notice.
+    DO NOT USE
+    """
+
+    def real_download(self, filename, info_dict):
+        ws = info_dict['ws']
+
+        heartbeat_lock = threading.Lock()
+        heartbeat_state = [None, 1]
+
+        def heartbeat():
+            try:
+                heartbeat_state[1] += 1
+                ws.send('{"name":"heartbeat","arguments":{},"id":%d}' % heartbeat_state[1])
+            except Exception:
+                self.to_screen('[fc2:live] Heartbeat failed')
+
+            with heartbeat_lock:
+                heartbeat_state[0] = threading.Timer(30, heartbeat)
+                heartbeat_state[0]._daemonic = True
+                heartbeat_state[0].start()
+
+        heartbeat()
+
+        new_info_dict = info_dict.copy()
+        new_info_dict.update({
+            'ws': None,
+            'protocol': 'live_ffmpeg',
+        })
+        return FFmpegFD(self.ydl, self.params or {}).download(filename, new_info_dict)
diff --git a/yt_dlp/downloader/fragment.py b/yt_dlp/downloader/fragment.py
index 19c0990d3..83a9f81b6 100644
--- a/yt_dlp/downloader/fragment.py
+++ b/yt_dlp/downloader/fragment.py
@@ -25,6 +25,7 @@
     error_to_compat_str,
     encodeFilename,
     sanitized_Request,
+    traverse_obj,
 )
 
 
@@ -136,7 +137,12 @@ def _download_fragment(self, ctx, frag_url, info_dict, headers=None, request_dat
         if fragment_info_dict.get('filetime'):
             ctx['fragment_filetime'] = fragment_info_dict.get('filetime')
         ctx['fragment_filename_sanitized'] = fragment_filename
-        return True, self._read_fragment(ctx)
+        try:
+            return True, self._read_fragment(ctx)
+        except FileNotFoundError:
+            if not info_dict.get('is_live'):
+                raise
+            return False, None
 
     def _read_fragment(self, ctx):
         down, frag_sanitized = self.sanitize_open(ctx['fragment_filename_sanitized'], 'rb')
@@ -172,7 +178,7 @@ def _prepare_frag_download(self, ctx):
         dl = HttpQuietDownloader(
             self.ydl,
             {
-                'continuedl': True,
+                'continuedl': self.params.get('continuedl', True),
                 'quiet': self.params.get('quiet'),
                 'noprogress': True,
                 'ratelimit': self.params.get('ratelimit'),
@@ -382,6 +388,7 @@ def download_and_append_fragments_multiple(self, *args, pack_func=None, finish_f
         max_workers = self.params.get('concurrent_fragment_downloads', 1)
         if max_progress > 1:
             self._prepare_multiline_status(max_progress)
+        is_live = any(traverse_obj(args, (..., 2, 'is_live'), default=[]))
 
         def thread_func(idx, ctx, fragments, info_dict, tpe):
             ctx['max_progress'] = max_progress
@@ -395,25 +402,43 @@ class FTPE(concurrent.futures.ThreadPoolExecutor):
             def __exit__(self, exc_type, exc_val, exc_tb):
                 pass
 
-        spins = []
         if compat_os_name == 'nt':
-            self.report_warning('Ctrl+C does not work on Windows when used with parallel threads. '
-                                'This is a known issue and patches are welcome')
+            def bindoj_result(future):
+                while True:
+                    try:
+                        return future.result(0.1)
+                    except KeyboardInterrupt:
+                        raise
+                    except concurrent.futures.TimeoutError:
+                        continue
+        else:
+            def bindoj_result(future):
+                return future.result()
+
+        def interrupt_trigger_iter(fg):
+            for f in fg:
+                if not interrupt_trigger[0]:
+                    break
+                yield f
+
+        spins = []
         for idx, (ctx, fragments, info_dict) in enumerate(args):
             tpe = FTPE(math.ceil(max_workers / max_progress))
-            job = tpe.submit(thread_func, idx, ctx, fragments, info_dict, tpe)
+            job = tpe.submit(thread_func, idx, ctx, interrupt_trigger_iter(fragments), info_dict, tpe)
             spins.append((tpe, job))
 
         result = True
         for tpe, job in spins:
             try:
-                result = result and job.result()
+                result = result and bindoj_result(job)
             except KeyboardInterrupt:
                 interrupt_trigger[0] = False
             finally:
                 tpe.shutdown(wait=True)
-        if not interrupt_trigger[0]:
+        if not interrupt_trigger[0] and not is_live:
             raise KeyboardInterrupt()
+        # we expect the user wants to stop and DO WANT the preceding postprocessors to run;
+        # so returning a intermediate result here instead of KeyboardInterrupt on live
         return result
 
     def download_and_append_fragments(
@@ -431,10 +456,11 @@ def download_and_append_fragments(
             pack_func = lambda frag_content, _: frag_content
 
         def download_fragment(fragment, ctx):
+            if not interrupt_trigger[0]:
+                return False, fragment['frag_index']
+
             frag_index = ctx['fragment_index'] = fragment['frag_index']
             ctx['last_error'] = None
-            if not interrupt_trigger[0]:
-                return False, frag_index
             headers = info_dict.get('http_headers', {}).copy()
             byte_range = fragment.get('byte_range')
             if byte_range:
@@ -500,8 +526,6 @@ def _download_fragment(fragment):
             self.report_warning('The download speed shown is only of one thread. This is a known issue and patches are welcome')
             with tpe or concurrent.futures.ThreadPoolExecutor(max_workers) as pool:
                 for fragment, frag_content, frag_index, frag_filename in pool.map(_download_fragment, fragments):
-                    if not interrupt_trigger[0]:
-                        break
                     ctx['fragment_filename_sanitized'] = frag_filename
                     ctx['fragment_index'] = frag_index
                     result = append_fragment(decrypt_fragment(fragment, frag_content), frag_index, ctx)
diff --git a/yt_dlp/downloader/http.py b/yt_dlp/downloader/http.py
index 34a1eb59b..10ba61024 100644
--- a/yt_dlp/downloader/http.py
+++ b/yt_dlp/downloader/http.py
@@ -5,7 +5,6 @@
 import socket
 import time
 import random
-import re
 
 from .common import FileDownloader
 from ..compat import (
@@ -16,6 +15,7 @@
     ContentTooShortError,
     encodeFilename,
     int_or_none,
+    parse_http_range,
     sanitized_Request,
     ThrottledDownload,
     write_xattr,
@@ -59,6 +59,9 @@ class DownloadContext(dict):
         ctx.chunk_size = None
         throttle_start = None
 
+        # parse given Range
+        req_start, req_end, _ = parse_http_range(headers.get('Range'))
+
         if self.params.get('continuedl', True):
             # Establish possible resume length
             if os.path.isfile(encodeFilename(ctx.tmpfilename)):
@@ -91,6 +94,9 @@ def establish_connection():
                               if not is_test and chunk_size else chunk_size)
             if ctx.resume_len > 0:
                 range_start = ctx.resume_len
+                if req_start is not None:
+                    # offset the beginning of Range to be within request
+                    range_start += req_start
                 if ctx.is_resume:
                     self.report_resuming_byte(ctx.resume_len)
                 ctx.open_mode = 'ab'
@@ -99,7 +105,17 @@ def establish_connection():
             else:
                 range_start = None
             ctx.is_resume = False
-            range_end = range_start + ctx.chunk_size - 1 if ctx.chunk_size else None
+
+            if ctx.chunk_size:
+                chunk_aware_end = range_start + ctx.chunk_size - 1
+                # we're not allowed to download outside Range
+                range_end = chunk_aware_end if req_end is None else min(chunk_aware_end, req_end)
+            elif req_end is not None:
+                # there's no need for chunked downloads, so download until the end of Range
+                range_end = req_end
+            else:
+                range_end = None
+
             if range_end and ctx.data_len is not None and range_end >= ctx.data_len:
                 range_end = ctx.data_len - 1
             has_range = range_start is not None
@@ -124,23 +140,19 @@ def establish_connection():
                 # https://github.com/ytdl-org/youtube-dl/issues/6057#issuecomment-126129799)
                 if has_range:
                     content_range = ctx.data.headers.get('Content-Range')
-                    if content_range:
-                        content_range_m = re.search(r'bytes (\d+)-(\d+)?(?:/(\d+))?', content_range)
+                    content_range_start, content_range_end, content_len = parse_http_range(content_range)
+                    if content_range_start is not None and range_start == content_range_start:
                         # Content-Range is present and matches requested Range, resume is possible
-                        if content_range_m:
-                            if range_start == int(content_range_m.group(1)):
-                                content_range_end = int_or_none(content_range_m.group(2))
-                                content_len = int_or_none(content_range_m.group(3))
-                                accept_content_len = (
-                                    # Non-chunked download
-                                    not ctx.chunk_size
-                                    # Chunked download and requested piece or
-                                    # its part is promised to be served
-                                    or content_range_end == range_end
-                                    or content_len < range_end)
-                                if accept_content_len:
-                                    ctx.data_len = content_len
-                                    return
+                        accept_content_len = (
+                            # Non-chunked download
+                            not ctx.chunk_size
+                            # Chunked download and requested piece or
+                            # its part is promised to be served
+                            or content_range_end == range_end
+                            or content_len < range_end)
+                        if accept_content_len:
+                            ctx.data_len = content_len
+                            return
                     # Content-Range is either not present or invalid. Assuming remote webserver is
                     # trying to send the whole file, resume is not possible, so wiping the local file
                     # and performing entire redownload
diff --git a/yt_dlp/extractor/abematv.py b/yt_dlp/extractor/abematv.py
new file mode 100644
index 000000000..66b12c72f
--- /dev/null
+++ b/yt_dlp/extractor/abematv.py
@@ -0,0 +1,488 @@
+import io
+import json
+import time
+import hashlib
+import hmac
+import re
+import struct
+from base64 import urlsafe_b64encode
+from binascii import unhexlify
+
+import typing
+if typing.TYPE_CHECKING:
+    from ..YoutubeDL import YoutubeDL
+
+from .common import InfoExtractor
+from ..aes import aes_ecb_decrypt
+from ..compat import (
+    compat_urllib_response,
+    compat_urllib_parse_urlparse,
+    compat_urllib_request,
+)
+from ..utils import (
+    ExtractorError,
+    decode_base,
+    int_or_none,
+    random_uuidv4,
+    request_to_url,
+    time_seconds,
+    update_url_query,
+    traverse_obj,
+    intlist_to_bytes,
+    bytes_to_intlist,
+    urljoin,
+)
+
+
+# NOTE: network handler related code is temporary thing until network stack overhaul PRs are merged (#2861/#2862)
+
+def add_opener(self: 'YoutubeDL', handler):
+    ''' Add a handler for opening URLs, like _download_webpage '''
+    # https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L426
+    # https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L605
+    assert isinstance(self._opener, compat_urllib_request.OpenerDirector)
+    self._opener.add_handler(handler)
+
+
+def remove_opener(self: 'YoutubeDL', handler):
+    '''
+    Remove handler(s) for opening URLs
+    @param handler Either handler object itself or handler type.
+    Specifying handler type will remove all handler which isinstance returns True.
+    '''
+    # https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L426
+    # https://github.com/python/cpython/blob/main/Lib/urllib/request.py#L605
+    opener = self._opener
+    assert isinstance(self._opener, compat_urllib_request.OpenerDirector)
+    if isinstance(handler, (type, tuple)):
+        find_cp = lambda x: isinstance(x, handler)
+    else:
+        find_cp = lambda x: x is handler
+
+    removed = []
+    for meth in dir(handler):
+        if meth in ["redirect_request", "do_open", "proxy_open"]:
+            # oops, coincidental match
+            continue
+
+        i = meth.find("_")
+        protocol = meth[:i]
+        condition = meth[i + 1:]
+
+        if condition.startswith("error"):
+            j = condition.find("_") + i + 1
+            kind = meth[j + 1:]
+            try:
+                kind = int(kind)
+            except ValueError:
+                pass
+            lookup = opener.handle_error.get(protocol, {})
+            opener.handle_error[protocol] = lookup
+        elif condition == "open":
+            kind = protocol
+            lookup = opener.handle_open
+        elif condition == "response":
+            kind = protocol
+            lookup = opener.process_response
+        elif condition == "request":
+            kind = protocol
+            lookup = opener.process_request
+        else:
+            continue
+
+        handlers = lookup.setdefault(kind, [])
+        if handlers:
+            handlers[:] = [x for x in handlers if not find_cp(x)]
+
+        removed.append(x for x in handlers if find_cp(x))
+
+    if removed:
+        for x in opener.handlers:
+            if find_cp(x):
+                x.add_parent(None)
+        opener.handlers[:] = [x for x in opener.handlers if not find_cp(x)]
+
+
+class AbemaLicenseHandler(compat_urllib_request.BaseHandler):
+    handler_order = 499
+    STRTABLE = '123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'
+    HKEY = b'3AF0298C219469522A313570E8583005A642E73EDD58E3EA2FB7339D3DF1597E'
+
+    def __init__(self, ie: 'AbemaTVIE'):
+        # the protcol that this should really handle is 'abematv-license://'
+        # abematv_license_open is just a placeholder for development purposes
+        # ref. https://github.com/python/cpython/blob/f4c03484da59049eb62a9bf7777b963e2267d187/Lib/urllib/request.py#L510
+        setattr(self, 'abematv-license_open', getattr(self, 'abematv_license_open'))
+        self.ie = ie
+
+    def _get_videokey_from_ticket(self, ticket):
+        to_show = self.ie._downloader.params.get('verbose', False)
+        media_token = self.ie._get_media_token(to_show=to_show)
+
+        license_response = self.ie._download_json(
+            'https://license.abema.io/abematv-hls', None, note='Requesting playback license' if to_show else False,
+            query={'t': media_token},
+            data=json.dumps({
+                'kv': 'a',
+                'lt': ticket
+            }).encode('utf-8'),
+            headers={
+                'Content-Type': 'application/json',
+            })
+
+        res = decode_base(license_response['k'], self.STRTABLE)
+        encvideokey = bytes_to_intlist(struct.pack('>QQ', res >> 64, res & 0xffffffffffffffff))
+
+        h = hmac.new(
+            unhexlify(self.HKEY),
+            (license_response['cid'] + self.ie._DEVICE_ID).encode('utf-8'),
+            digestmod=hashlib.sha256)
+        enckey = bytes_to_intlist(h.digest())
+
+        return intlist_to_bytes(aes_ecb_decrypt(encvideokey, enckey))
+
+    def abematv_license_open(self, url):
+        url = request_to_url(url)
+        ticket = compat_urllib_parse_urlparse(url).netloc
+        response_data = self._get_videokey_from_ticket(ticket)
+        return compat_urllib_response.addinfourl(io.BytesIO(response_data), headers={
+            'Content-Length': len(response_data),
+        }, url=url, code=200)
+
+
+class AbemaTVBaseIE(InfoExtractor):
+    def _extract_breadcrumb_list(self, webpage, video_id):
+        for jld in re.finditer(
+                r'(?is)</span></li></ul><script[^>]+type=(["\']?)application/ld\+json\1[^>]*>(?P<json_ld>.+?)</script>',
+                webpage):
+            jsonld = self._parse_json(jld.group('json_ld'), video_id, fatal=False)
+            if jsonld:
+                if jsonld.get('@type') != 'BreadcrumbList':
+                    continue
+                trav = traverse_obj(jsonld, ('itemListElement', ..., 'name'))
+                if trav:
+                    return trav
+        return []
+
+
+class AbemaTVIE(AbemaTVBaseIE):
+    _VALID_URL = r'https?://abema\.tv/(?P<type>now-on-air|video/episode|channels/.+?/slots)/(?P<id>[^?/]+)'
+    _NETRC_MACHINE = 'abematv'
+    _TESTS = [{
+        'url': 'https://abema.tv/video/episode/194-25_s2_p1',
+        'info_dict': {
+            'id': '194-25_s2_p1',
+            'title': '第1話 「チーズケーキ」　「モーニング再び」',
+            'series': '異世界食堂２',
+            'series_number': 2,
+            'episode': '第1話 「チーズケーキ」　「モーニング再び」',
+            'episode_number': 1,
+        },
+        'skip': 'expired',
+    }, {
+        'url': 'https://abema.tv/channels/anime-live2/slots/E8tvAnMJ7a9a5d',
+        'info_dict': {
+            'id': 'E8tvAnMJ7a9a5d',
+            'title': 'ゆるキャン△ SEASON２ 全話一挙【無料ビデオ72時間】',
+            'series': 'ゆるキャン△ SEASON２',
+            'episode': 'ゆるキャン△ SEASON２ 全話一挙【無料ビデオ72時間】',
+            'series_number': 2,
+            'episode_number': 1,
+            'description': 'md5:9c5a3172ae763278f9303922f0ea5b17',
+        },
+        'skip': 'expired',
+    }, {
+        'url': 'https://abema.tv/video/episode/87-877_s1282_p31047',
+        'info_dict': {
+            'id': 'E8tvAnMJ7a9a5d',
+            'title': '第5話『光射す』',
+            'description': 'md5:56d4fc1b4f7769ded5f923c55bb4695d',
+            'thumbnail': r're:https://hayabusa\.io/.+',
+            'series': '相棒',
+            'episode': '第5話『光射す』',
+        },
+        'skip': 'expired',
+    }, {
+        'url': 'https://abema.tv/now-on-air/abema-anime',
+        'info_dict': {
+            'id': 'abema-anime',
+            # this varies
+            # 'title': '女子高生の無駄づかい 全話一挙【無料ビデオ72時間】',
+            'description': 'md5:55f2e61f46a17e9230802d7bcc913d5f',
+            'is_live': True,
+        },
+        'skip': 'Not supported until yt-dlp implements native live downloader OR AbemaTV can start a local HTTP server',
+    }]
+    _USERTOKEN = None
+    _DEVICE_ID = None
+    _TIMETABLE = None
+    _MEDIATOKEN = None
+
+    _SECRETKEY = b'v+Gjs=25Aw5erR!J8ZuvRrCx*rGswhB&qdHd_SYerEWdU&a?3DzN9BRbp5KwY4hEmcj5#fykMjJ=AuWz5GSMY-d@H7DMEh3M@9n2G552Us$$k9cD=3TxwWe86!x#Zyhe'
+
+    def _generate_aks(self, deviceid):
+        deviceid = deviceid.encode('utf-8')
+        # add 1 hour and then drop minute and secs
+        ts_1hour = int((time_seconds(hours=9) // 3600 + 1) * 3600)
+        time_struct = time.gmtime(ts_1hour)
+        ts_1hour_str = str(ts_1hour).encode('utf-8')
+
+        tmp = None
+
+        def mix_once(nonce):
+            nonlocal tmp
+            h = hmac.new(self._SECRETKEY, digestmod=hashlib.sha256)
+            h.update(nonce)
+            tmp = h.digest()
+
+        def mix_tmp(count):
+            nonlocal tmp
+            for i in range(count):
+                mix_once(tmp)
+
+        def mix_twist(nonce):
+            nonlocal tmp
+            mix_once(urlsafe_b64encode(tmp).rstrip(b'=') + nonce)
+
+        mix_once(self._SECRETKEY)
+        mix_tmp(time_struct.tm_mon)
+        mix_twist(deviceid)
+        mix_tmp(time_struct.tm_mday % 5)
+        mix_twist(ts_1hour_str)
+        mix_tmp(time_struct.tm_hour % 5)
+
+        return urlsafe_b64encode(tmp).rstrip(b'=').decode('utf-8')
+
+    def _get_device_token(self):
+        if self._USERTOKEN:
+            return self._USERTOKEN
+
+        self._DEVICE_ID = random_uuidv4()
+        aks = self._generate_aks(self._DEVICE_ID)
+        user_data = self._download_json(
+            'https://api.abema.io/v1/users', None, note='Authorizing',
+            data=json.dumps({
+                'deviceId': self._DEVICE_ID,
+                'applicationKeySecret': aks,
+            }).encode('utf-8'),
+            headers={
+                'Content-Type': 'application/json',
+            })
+        self._USERTOKEN = user_data['token']
+
+        # don't allow adding it 2 times or more, though it's guarded
+        remove_opener(self._downloader, AbemaLicenseHandler)
+        add_opener(self._downloader, AbemaLicenseHandler(self))
+
+        return self._USERTOKEN
+
+    def _get_media_token(self, invalidate=False, to_show=True):
+        if not invalidate and self._MEDIATOKEN:
+            return self._MEDIATOKEN
+
+        self._MEDIATOKEN = self._download_json(
+            'https://api.abema.io/v1/media/token', None, note='Fetching media token' if to_show else False,
+            query={
+                'osName': 'android',
+                'osVersion': '6.0.1',
+                'osLang': 'ja_JP',
+                'osTimezone': 'Asia/Tokyo',
+                'appId': 'tv.abema',
+                'appVersion': '3.27.1'
+            }, headers={
+                'Authorization': 'bearer ' + self._get_device_token()
+            })['token']
+
+        return self._MEDIATOKEN
+
+    def _real_initialize(self):
+        self._login()
+
+    def _login(self):
+        username, password = self._get_login_info()
+        # No authentication to be performed
+        if not username:
+            return True
+
+        if '@' in username:  # don't strictly check if it's email address or not
+            ep, method = 'user/email', 'email'
+        else:
+            ep, method = 'oneTimePassword', 'userId'
+
+        login_response = self._download_json(
+            f'https://api.abema.io/v1/auth/{ep}', None, note='Logging in',
+            data=json.dumps({
+                method: username,
+                'password': password
+            }).encode('utf-8'), headers={
+                'Authorization': 'bearer ' + self._get_device_token(),
+                'Origin': 'https://abema.tv',
+                'Referer': 'https://abema.tv/',
+                'Content-Type': 'application/json',
+            })
+
+        self._USERTOKEN = login_response['token']
+        self._get_media_token(True)
+
+    def _real_extract(self, url):
+        # starting download using infojson from this extractor is undefined behavior,
+        # and never be fixed in the future; you must trigger downloads by directly specifing URL.
+        # (unless there's a way to hook before downloading by extractor)
+        video_id, video_type = self._match_valid_url(url).group('id', 'type')
+        headers = {
+            'Authorization': 'Bearer ' + self._get_device_token(),
+        }
+        video_type = video_type.split('/')[-1]
+
+        webpage = self._download_webpage(url, video_id)
+        canonical_url = self._search_regex(
+            r'<link\s+rel="canonical"\s*href="(.+?)"', webpage, 'canonical URL',
+            default=url)
+        info = self._search_json_ld(webpage, video_id, default={})
+
+        title = self._search_regex(
+            r'<span\s*class=".+?EpisodeTitleBlock__title">(.+?)</span>', webpage, 'title', default=None)
+        if not title:
+            jsonld = None
+            for jld in re.finditer(
+                    r'(?is)<span\s*class="com-m-Thumbnail__image">(?:</span>)?<script[^>]+type=(["\']?)application/ld\+json\1[^>]*>(?P<json_ld>.+?)</script>',
+                    webpage):
+                jsonld = self._parse_json(jld.group('json_ld'), video_id, fatal=False)
+                if jsonld:
+                    break
+            if jsonld:
+                title = jsonld.get('caption')
+        if not title and video_type == 'now-on-air':
+            if not self._TIMETABLE:
+                # cache the timetable because it goes to 5MiB in size (!!)
+                self._TIMETABLE = self._download_json(
+                    'https://api.abema.io/v1/timetable/dataSet?debug=false', video_id,
+                    headers=headers)
+            now = time_seconds(hours=9)
+            for slot in self._TIMETABLE.get('slots', []):
+                if slot.get('channelId') != video_id:
+                    continue
+                if slot['startAt'] <= now and now < slot['endAt']:
+                    title = slot['title']
+                    break
+
+        # read breadcrumb on top of page
+        breadcrumb = self._extract_breadcrumb_list(webpage, video_id)
+        if breadcrumb:
+            # breadcrumb list translates to: (example is 1st test for this IE)
+            # Home > Anime (genre) > Isekai Shokudo 2 (series name) > Episode 1 "Cheese cakes" "Morning again" (episode title)
+            # hence this works
+            info['series'] = breadcrumb[-2]
+            info['episode'] = breadcrumb[-1]
+            if not title:
+                title = info['episode']
+
+        description = self._html_search_regex(
+            (r'<p\s+class="com-video-EpisodeDetailsBlock__content"><span\s+class=".+?">(.+?)</span></p><div',
+             r'<span\s+class=".+?SlotSummary.+?">(.+?)</span></div><div',),
+            webpage, 'description', default=None, group=1)
+        if not description:
+            og_desc = self._html_search_meta(
+                ('description', 'og:description', 'twitter:description'), webpage)
+            if og_desc:
+                description = re.sub(r'''(?sx)
+                    ^(.+?)(?:
+                        アニメの動画を無料で見るならABEMA！| # anime
+                        等、.+ # applies for most of categories
+                    )?
+                ''', r'\1', og_desc)
+
+        # canonical URL may contain series and episode number
+        mobj = re.search(r's(\d+)_p(\d+)$', canonical_url)
+        if mobj:
+            seri = int_or_none(mobj.group(1), default=float('inf'))
+            epis = int_or_none(mobj.group(2), default=float('inf'))
+            info['series_number'] = seri if seri < 100 else None
+            # some anime like Detective Conan (though not available in AbemaTV)
+            # has more than 1000 episodes (1026 as of 2021/11/15)
+            info['episode_number'] = epis if epis < 2000 else None
+
+        is_live, m3u8_url = False, None
+        if video_type == 'now-on-air':
+            is_live = True
+            channel_url = 'https://api.abema.io/v1/channels'
+            if video_id == 'news-global':
+                channel_url = update_url_query(channel_url, {'division': '1'})
+            onair_channels = self._download_json(channel_url, video_id)
+            for ch in onair_channels['channels']:
+                if video_id == ch['id']:
+                    m3u8_url = ch['playback']['hls']
+                    break
+            else:
+                raise ExtractorError(f'Cannot find on-air {video_id} channel.', expected=True)
+        elif video_type == 'episode':
+            api_response = self._download_json(
+                f'https://api.abema.io/v1/video/programs/{video_id}', video_id,
+                note='Checking playability',
+                headers=headers)
+            ondemand_types = traverse_obj(api_response, ('terms', ..., 'onDemandType'), default=[])
+            if 3 not in ondemand_types:
+                # cannot acquire decryption key for these streams
+                self.report_warning('This is a premium-only stream')
+
+            m3u8_url = f'https://vod-abematv.akamaized.net/program/{video_id}/playlist.m3u8'
+        elif video_type == 'slots':
+            api_response = self._download_json(
+                f'https://api.abema.io/v1/media/slots/{video_id}', video_id,
+                note='Checking playability',
+                headers=headers)
+            if not traverse_obj(api_response, ('slot', 'flags', 'timeshiftFree'), default=False):
+                self.report_warning('This is a premium-only stream')
+
+            m3u8_url = f'https://vod-abematv.akamaized.net/slot/{video_id}/playlist.m3u8'
+        else:
+            raise ExtractorError('Unreachable')
+
+        if is_live:
+            self.report_warning("This is a livestream; yt-dlp doesn't support downloading natively, but FFmpeg cannot handle m3u8 manifests from AbemaTV")
+            self.report_warning('Please consider using Streamlink to download these streams (https://github.com/streamlink/streamlink)')
+        formats = self._extract_m3u8_formats(
+            m3u8_url, video_id, ext='mp4', live=is_live)
+
+        info.update({
+            'id': video_id,
+            'title': title,
+            'description': description,
+            'formats': formats,
+            'is_live': is_live,
+        })
+        return info
+
+
+class AbemaTVTitleIE(AbemaTVBaseIE):
+    _VALID_URL = r'https?://abema\.tv/video/title/(?P<id>[^?/]+)'
+
+    _TESTS = [{
+        'url': 'https://abema.tv/video/title/90-1597',
+        'info_dict': {
+            'id': '90-1597',
+            'title': 'シャッフルアイランド',
+        },
+        'playlist_mincount': 2,
+    }, {
+        'url': 'https://abema.tv/video/title/193-132',
+        'info_dict': {
+            'id': '193-132',
+            'title': '真心が届く~僕とスターのオフィス・ラブ!?~',
+        },
+        'playlist_mincount': 16,
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+
+        playlist_title, breadcrumb = None, self._extract_breadcrumb_list(webpage, video_id)
+        if breadcrumb:
+            playlist_title = breadcrumb[-1]
+
+        playlist = [
+            self.url_result(urljoin('https://abema.tv/', mobj.group(1)))
+            for mobj in re.finditer(r'<li\s*class=".+?EpisodeList.+?"><a\s*href="(/[^"]+?)"', webpage)]
+
+        return self.playlist_result(playlist, playlist_title=playlist_title, playlist_id=video_id)
diff --git a/yt_dlp/extractor/arcpublishing.py b/yt_dlp/extractor/arcpublishing.py
index 1943fd5f8..8880e5c95 100644
--- a/yt_dlp/extractor/arcpublishing.py
+++ b/yt_dlp/extractor/arcpublishing.py
@@ -124,8 +124,7 @@ def _real_extract(self, url):
                 formats.extend(smil_formats)
             elif stream_type in ('ts', 'hls'):
                 m3u8_formats = self._extract_m3u8_formats(
-                    s_url, uuid, 'mp4', 'm3u8' if is_live else 'm3u8_native',
-                    m3u8_id='hls', fatal=False)
+                    s_url, uuid, 'mp4', live=is_live, m3u8_id='hls', fatal=False)
                 if all([f.get('acodec') == 'none' for f in m3u8_formats]):
                     continue
                 for f in m3u8_formats:
diff --git a/yt_dlp/extractor/bilibili.py b/yt_dlp/extractor/bilibili.py
index a775aa97f..1bbf7ca1c 100644
--- a/yt_dlp/extractor/bilibili.py
+++ b/yt_dlp/extractor/bilibili.py
@@ -225,10 +225,6 @@ def _real_extract(self, url):
                         'quality': -2 if 'hd.mp4' in backup_url else -3,
                     })
 
-                for a_format in formats:
-                    a_format.setdefault('http_headers', {}).update({
-                        'Referer': url,
-                    })
             for audio in audios:
                 formats.append({
                     'url': audio.get('baseUrl') or audio.get('base_url') or audio.get('url'),
@@ -252,6 +248,9 @@ def _real_extract(self, url):
                 'id': video_id,
                 'duration': float_or_none(durl.get('length'), 1000),
                 'formats': formats,
+                'http_headers': {
+                    'Referer': url,
+                },
             })
             break
 
diff --git a/yt_dlp/extractor/biqle.py b/yt_dlp/extractor/biqle.py
index 17ebbb257..2b57bade3 100644
--- a/yt_dlp/extractor/biqle.py
+++ b/yt_dlp/extractor/biqle.py
@@ -3,27 +3,28 @@
 
 from .common import InfoExtractor
 from .vk import VKIE
-from ..compat import (
-    compat_b64decode,
-    compat_urllib_parse_unquote,
+from ..compat import compat_b64decode
+from ..utils import (
+    int_or_none,
+    js_to_json,
+    traverse_obj,
+    unified_timestamp,
 )
-from ..utils import int_or_none
 
 
 class BIQLEIE(InfoExtractor):
     _VALID_URL = r'https?://(?:www\.)?biqle\.(?:com|org|ru)/watch/(?P<id>-?\d+_\d+)'
     _TESTS = [{
-        # Youtube embed
-        'url': 'https://biqle.ru/watch/-115995369_456239081',
-        'md5': '97af5a06ee4c29bbf9c001bdb1cf5c06',
+        'url': 'https://biqle.ru/watch/-2000421746_85421746',
+        'md5': 'ae6ef4f04d19ac84e4658046d02c151c',
         'info_dict': {
-            'id': '8v4f-avW-VI',
+            'id': '-2000421746_85421746',
             'ext': 'mp4',
-            'title': "PASSE-PARTOUT - L'ete c'est fait pour jouer",
-            'description': 'Passe-Partout',
-            'uploader_id': 'mrsimpsonstef3',
-            'uploader': 'Phanolito',
-            'upload_date': '20120822',
+            'title': 'Forsaken By Hope Studio Clip',
+            'description': 'Forsaken By Hope Studio Clip — Смотреть онлайн',
+            'upload_date': '19700101',
+            'thumbnail': r're:https://[^/]+/impf/7vN3ACwSTgChP96OdOfzFjUCzFR6ZglDQgWsIw/KPaACiVJJxM\.jpg\?size=800x450&quality=96&keep_aspect_ratio=1&background=000000&sign=b48ea459c4d33dbcba5e26d63574b1cb&type=video_thumb',
+            'timestamp': 0,
         },
     }, {
         'url': 'http://biqle.org/watch/-44781847_168547604',
@@ -32,53 +33,62 @@ class BIQLEIE(InfoExtractor):
             'id': '-44781847_168547604',
             'ext': 'mp4',
             'title': 'Ребенок в шоке от автоматической мойки',
+            'description': 'Ребенок в шоке от автоматической мойки — Смотреть онлайн',
             'timestamp': 1396633454,
-            'uploader': 'Dmitry Kotov',
             'upload_date': '20140404',
-            'uploader_id': '47850140',
+            'thumbnail': r're:https://[^/]+/c535507/u190034692/video/l_b84df002\.jpg',
         },
     }]
 
     def _real_extract(self, url):
         video_id = self._match_id(url)
         webpage = self._download_webpage(url, video_id)
-        embed_url = self._proto_relative_url(self._search_regex(
-            r'<iframe.+?src="((?:https?:)?//(?:daxab\.com|dxb\.to|[^/]+/player)/[^"]+)".*?></iframe>',
-            webpage, 'embed url'))
+
+        title = self._html_search_meta('name', webpage, 'Title', fatal=False)
+        timestamp = unified_timestamp(self._html_search_meta('uploadDate', webpage, 'Upload Date', default=None))
+        description = self._html_search_meta('description', webpage, 'Description', default=None)
+
+        global_embed_url = self._search_regex(
+            r'<script[^<]+?window.globEmbedUrl\s*=\s*\'((?:https?:)?//(?:daxab\.com|dxb\.to|[^/]+/player)/[^\']+)\'',
+            webpage, 'global Embed url')
+        hash = self._search_regex(
+            r'<script id="data-embed-video[^<]+?hash: "([^"]+)"[^<]*</script>', webpage, 'Hash')
+
+        embed_url = global_embed_url + hash
+
         if VKIE.suitable(embed_url):
             return self.url_result(embed_url, VKIE.ie_key(), video_id)
 
         embed_page = self._download_webpage(
-            embed_url, video_id, headers={'Referer': url})
-        video_ext = self._get_cookies(embed_url).get('video_ext')
-        if video_ext:
-            video_ext = compat_urllib_parse_unquote(video_ext.value)
-        if not video_ext:
-            video_ext = compat_b64decode(self._search_regex(
-                r'video_ext\s*:\s*[\'"]([A-Za-z0-9+/=]+)',
-                embed_page, 'video_ext')).decode()
-        video_id, sig, _, access_token = video_ext.split(':')
+            embed_url, video_id, 'Downloading embed webpage', headers={'Referer': url})
+
+        glob_params = self._parse_json(self._search_regex(
+            r'<script id="globParams">[^<]*window.globParams = ([^;]+);[^<]+</script>',
+            embed_page, 'Global Parameters'), video_id, transform_source=js_to_json)
+        host_name = compat_b64decode(glob_params['server'][::-1]).decode()
+
         item = self._download_json(
-            'https://api.vk.com/method/video.get', video_id,
-            headers={'User-Agent': 'okhttp/3.4.1'}, query={
-                'access_token': access_token,
-                'sig': sig,
-                'v': 5.44,
+            f'https://{host_name}/method/video.get/{video_id}', video_id,
+            headers={'Referer': url}, query={
+                'token': glob_params['video']['access_token'],
                 'videos': video_id,
+                'ckey': glob_params['c_key'],
+                'credentials': glob_params['video']['credentials'],
             })['response']['items'][0]
-        title = item['title']
 
         formats = []
         for f_id, f_url in item.get('files', {}).items():
             if f_id == 'external':
                 return self.url_result(f_url)
             ext, height = f_id.split('_')
-            formats.append({
-                'format_id': height + 'p',
-                'url': f_url,
-                'height': int_or_none(height),
-                'ext': ext,
-            })
+            height_extra_key = traverse_obj(glob_params, ('video', 'partial', 'quality', height))
+            if height_extra_key:
+                formats.append({
+                    'format_id': f'{height}p',
+                    'url': f'https://{host_name}/{f_url[8:]}&videos={video_id}&extra_key={height_extra_key}',
+                    'height': int_or_none(height),
+                    'ext': ext,
+                })
         self._sort_formats(formats)
 
         thumbnails = []
@@ -96,10 +106,9 @@ def _real_extract(self, url):
             'title': title,
             'formats': formats,
             'comment_count': int_or_none(item.get('comments')),
-            'description': item.get('description'),
+            'description': description,
             'duration': int_or_none(item.get('duration')),
             'thumbnails': thumbnails,
-            'timestamp': int_or_none(item.get('date')),
-            'uploader': item.get('owner_id'),
+            'timestamp': timestamp,
             'view_count': int_or_none(item.get('views')),
         }
diff --git a/yt_dlp/extractor/common.py b/yt_dlp/extractor/common.py
index 04d4c0733..d8bb21137 100644
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -1307,6 +1307,10 @@ def _og_search_video_url(self, html, name='video url', secure=True, **kargs):
     def _og_search_url(self, html, **kargs):
         return self._og_search_property('url', html, **kargs)
 
+    def _html_extract_title(self, html, name, **kwargs):
+        return self._html_search_regex(
+            r'(?s)<title>(.*?)</title>', html, name, **kwargs)
+
     def _html_search_meta(self, name, html, display_name=None, fatal=False, **kwargs):
         name = variadic(name)
         if display_name is None:
diff --git a/yt_dlp/extractor/dailymotion.py b/yt_dlp/extractor/dailymotion.py
index e71462061..95589d53a 100644
--- a/yt_dlp/extractor/dailymotion.py
+++ b/yt_dlp/extractor/dailymotion.py
@@ -259,9 +259,7 @@ def _real_extract(self, url):
                     continue
                 if media_type == 'application/x-mpegURL':
                     formats.extend(self._extract_m3u8_formats(
-                        media_url, video_id, 'mp4',
-                        'm3u8' if is_live else 'm3u8_native',
-                        m3u8_id='hls', fatal=False))
+                        media_url, video_id, 'mp4', live=is_live, m3u8_id='hls', fatal=False))
                 else:
                     f = {
                         'url': media_url,
diff --git a/yt_dlp/extractor/extractors.py b/yt_dlp/extractor/extractors.py
index 15bc74915..0f26dc24f 100644
--- a/yt_dlp/extractor/extractors.py
+++ b/yt_dlp/extractor/extractors.py
@@ -14,6 +14,10 @@
     ABCOTVSIE,
     ABCOTVSClipsIE,
 )
+from .abematv import (
+    AbemaTVIE,
+    AbemaTVTitleIE,
+)
 from .academicearth import AcademicEarthCourseIE
 from .acast import (
     ACastIE,
@@ -474,6 +478,7 @@
 from .fc2 import (
     FC2IE,
     FC2EmbedIE,
+    FC2LiveIE,
 )
 from .fczenit import FczenitIE
 from .filmmodu import FilmmoduIE
@@ -994,6 +999,9 @@
 from .nhk import (
     NhkVodIE,
     NhkVodProgramIE,
+    NhkForSchoolBangumiIE,
+    NhkForSchoolSubjectIE,
+    NhkForSchoolProgramListIE,
 )
 from .nhl import NHLIE
 from .nick import (
@@ -1003,11 +1011,12 @@
     NickNightIE,
     NickRuIE,
 )
-
 from .niconico import (
     NiconicoIE,
     NiconicoPlaylistIE,
     NiconicoUserIE,
+    NiconicoSeriesIE,
+    NiconicoHistoryIE,
     NicovideoSearchDateIE,
     NicovideoSearchIE,
     NicovideoSearchURLIE,
@@ -1325,6 +1334,11 @@
 from .rice import RICEIE
 from .rmcdecouverte import RMCDecouverteIE
 from .rockstargames import RockstarGamesIE
+from .rokfin import (
+    RokfinIE,
+    RokfinStackIE,
+    RokfinChannelIE,
+)
 from .roosterteeth import RoosterTeethIE, RoosterTeethSeriesIE
 from .rottentomatoes import RottenTomatoesIE
 from .rozhlas import RozhlasIE
diff --git a/yt_dlp/extractor/fc2.py b/yt_dlp/extractor/fc2.py
index 2c19a0c6e..7fc6b0e3d 100644
--- a/yt_dlp/extractor/fc2.py
+++ b/yt_dlp/extractor/fc2.py
@@ -1,14 +1,21 @@
 # coding: utf-8
 from __future__ import unicode_literals
 
+import re
+
 from .common import InfoExtractor
 from ..compat import (
     compat_parse_qs,
 )
 from ..utils import (
     ExtractorError,
+    WebSocketsWrapper,
+    has_websockets,
+    js_to_json,
     sanitized_Request,
+    std_headers,
     traverse_obj,
+    update_url_query,
     urlencode_postdata,
     urljoin,
 )
@@ -147,3 +154,146 @@ def _real_extract(self, url):
             'title': title,
             'thumbnail': thumbnail,
         }
+
+
+class FC2LiveIE(InfoExtractor):
+    _VALID_URL = r'https?://live\.fc2\.com/(?P<id>\d+)'
+    IE_NAME = 'fc2:live'
+
+    _TESTS = [{
+        'url': 'https://live.fc2.com/57892267/',
+        'info_dict': {
+            'id': '57892267',
+            'title': 'どこまで・・・',
+            'uploader': 'あつあげ',
+            'uploader_id': '57892267',
+            'thumbnail': r're:https?://.+fc2.+',
+        },
+        'skip': 'livestream',
+    }]
+
+    def _real_extract(self, url):
+        if not has_websockets:
+            raise ExtractorError('websockets library is not available. Please install it.', expected=True)
+        video_id = self._match_id(url)
+        webpage = self._download_webpage('https://live.fc2.com/%s/' % video_id, video_id)
+
+        self._set_cookie('live.fc2.com', 'js-player_size', '1')
+
+        member_api = self._download_json(
+            'https://live.fc2.com/api/memberApi.php', video_id, data=urlencode_postdata({
+                'channel': '1',
+                'profile': '1',
+                'user': '1',
+                'streamid': video_id
+            }), note='Requesting member info')
+
+        control_server = self._download_json(
+            'https://live.fc2.com/api/getControlServer.php', video_id, note='Downloading ControlServer data',
+            data=urlencode_postdata({
+                'channel_id': video_id,
+                'mode': 'play',
+                'orz': '',
+                'channel_version': member_api['data']['channel_data']['version'],
+                'client_version': '2.1.0\n [1]',
+                'client_type': 'pc',
+                'client_app': 'browser_hls',
+                'ipv6': '',
+            }), headers={'X-Requested-With': 'XMLHttpRequest'})
+        self._set_cookie('live.fc2.com', 'l_ortkn', control_server['orz_raw'])
+
+        ws_url = update_url_query(control_server['url'], {'control_token': control_server['control_token']})
+        playlist_data = None
+
+        self.to_screen('%s: Fetching HLS playlist info via WebSocket' % video_id)
+        ws = WebSocketsWrapper(ws_url, {
+            'Cookie': str(self._get_cookies('https://live.fc2.com/'))[12:],
+            'Origin': 'https://live.fc2.com',
+            'Accept': '*/*',
+            'User-Agent': std_headers['User-Agent'],
+        })
+        ws.__enter__()
+
+        self.write_debug('[debug] Sending HLS server request')
+
+        while True:
+            recv = ws.recv()
+            if not recv:
+                continue
+            data = self._parse_json(recv, video_id, fatal=False)
+            if not data or not isinstance(data, dict):
+                continue
+
+            if data.get('name') == 'connect_complete':
+                break
+        ws.send(r'{"name":"get_hls_information","arguments":{},"id":1}')
+
+        while True:
+            recv = ws.recv()
+            if not recv:
+                continue
+            data = self._parse_json(recv, video_id, fatal=False)
+            if not data or not isinstance(data, dict):
+                continue
+            if data.get('name') == '_response_' and data.get('id') == 1:
+                self.write_debug('[debug] Goodbye.')
+                playlist_data = data
+                break
+            elif self._downloader.params.get('verbose', False):
+                if len(recv) > 100:
+                    recv = recv[:100] + '...'
+                self.to_screen('[debug] Server said: %s' % recv)
+
+        if not playlist_data:
+            raise ExtractorError('Unable to fetch HLS playlist info via WebSocket')
+
+        formats = []
+        for name, playlists in playlist_data['arguments'].items():
+            if not isinstance(playlists, list):
+                continue
+            for pl in playlists:
+                if pl.get('status') == 0 and 'master_playlist' in pl.get('url'):
+                    formats.extend(self._extract_m3u8_formats(
+                        pl['url'], video_id, ext='mp4', m3u8_id=name, live=True,
+                        headers={
+                            'Origin': 'https://live.fc2.com',
+                            'Referer': url,
+                        }))
+
+        self._sort_formats(formats)
+        for fmt in formats:
+            fmt.update({
+                'protocol': 'fc2_live',
+                'ws': ws,
+            })
+
+        title = self._html_search_meta(('og:title', 'twitter:title'), webpage, 'live title', fatal=False)
+        if not title:
+            title = self._html_extract_title(webpage, 'html title', fatal=False)
+            if title:
+                # remove service name in <title>
+                title = re.sub(r'\s+-\s+.+$', '', title)
+        uploader = None
+        if title:
+            match = self._search_regex(r'^(.+?)\s*\[(.+?)\]$', title, 'title and uploader', default=None, group=(1, 2))
+            if match and all(match):
+                title, uploader = match
+
+        live_info_view = self._search_regex(r'(?s)liveInfoView\s*:\s*({.+?}),\s*premiumStateView', webpage, 'user info', fatal=False) or None
+        if live_info_view:
+            # remove jQuery code from object literal
+            live_info_view = re.sub(r'\$\(.+?\)[^,]+,', '"",', live_info_view)
+            live_info_view = self._parse_json(js_to_json(live_info_view), video_id)
+
+        return {
+            'id': video_id,
+            'title': title or traverse_obj(live_info_view, 'title'),
+            'description': self._html_search_meta(
+                ('og:description', 'twitter:description'),
+                webpage, 'live description', fatal=False) or traverse_obj(live_info_view, 'info'),
+            'formats': formats,
+            'uploader': uploader or traverse_obj(live_info_view, 'name'),
+            'uploader_id': video_id,
+            'thumbnail': traverse_obj(live_info_view, 'thumb'),
+            'is_live': True,
+        }
diff --git a/yt_dlp/extractor/generic.py b/yt_dlp/extractor/generic.py
index baedd7948..d975e4bdb 100644
--- a/yt_dlp/extractor/generic.py
+++ b/yt_dlp/extractor/generic.py
@@ -3991,12 +3991,16 @@ def filter_video(urls):
 
             # here's a fun little line of code for you:
             video_id = os.path.splitext(video_id)[0]
+            headers = {
+                'referer': full_response.geturl()
+            }
 
             entry_info_dict = {
                 'id': video_id,
                 'uploader': video_uploader,
                 'title': video_title,
                 'age_limit': age_limit,
+                'http_headers': headers,
             }
 
             if RtmpIE.suitable(video_url):
@@ -4014,11 +4018,11 @@ def filter_video(urls):
             elif ext == 'xspf':
                 return self.playlist_result(self._extract_xspf_playlist(video_url, video_id), video_id)
             elif ext == 'm3u8':
-                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_m3u8_formats_and_subtitles(video_url, video_id, ext='mp4')
+                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_m3u8_formats_and_subtitles(video_url, video_id, ext='mp4', headers=headers)
             elif ext == 'mpd':
-                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_mpd_formats_and_subtitles(video_url, video_id)
+                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_mpd_formats_and_subtitles(video_url, video_id, headers=headers)
             elif ext == 'f4m':
-                entry_info_dict['formats'] = self._extract_f4m_formats(video_url, video_id)
+                entry_info_dict['formats'] = self._extract_f4m_formats(video_url, video_id, headers=headers)
             elif re.search(r'(?i)\.(?:ism|smil)/manifest', video_url) and video_url != url:
                 # Just matching .ism/manifest is not enough to be reliably sure
                 # whether it's actually an ISM manifest or some other streaming
diff --git a/yt_dlp/extractor/gettr.py b/yt_dlp/extractor/gettr.py
index 179bd7c47..9842edd81 100644
--- a/yt_dlp/extractor/gettr.py
+++ b/yt_dlp/extractor/gettr.py
@@ -39,6 +39,7 @@ class GettrIE(GettrBaseIE):
             'thumbnail': r're:^https?://.+/out\.jpg',
             'timestamp': 1632782451058,
             'duration': 58.5585,
+            'tags': ['hornofafrica', 'explorations'],
         }
     }, {
         'url': 'https://gettr.com/post/p4iahp',
@@ -52,6 +53,7 @@ class GettrIE(GettrBaseIE):
             'thumbnail': r're:^https?://.+/out\.jpg',
             'timestamp': 1626594455017,
             'duration': 23,
+            'tags': 'count:12',
         }
     }]
 
@@ -84,7 +86,7 @@ def _real_extract(self, url):
 
         formats = self._extract_m3u8_formats(
             urljoin(self._MEDIA_BASE_URL, vid), post_id, 'mp4',
-            entry_protocol='m3u8_native', m3u8_id='hls') if vid else []
+            entry_protocol='m3u8_native', m3u8_id='hls', fatal=False) if vid else []
 
         if ovid:
             formats.append({
@@ -93,8 +95,6 @@ def _real_extract(self, url):
                 'ext': 'mp4',
                 'width': int_or_none(post_data.get('vid_wid')),
                 'height': int_or_none(post_data.get('vid_hgt')),
-                'source_preference': 1,
-                'quality': 1,
             })
 
         self._sort_formats(formats)
diff --git a/yt_dlp/extractor/imggaming.py b/yt_dlp/extractor/imggaming.py
index bae74b290..230dc86d3 100644
--- a/yt_dlp/extractor/imggaming.py
+++ b/yt_dlp/extractor/imggaming.py
@@ -96,7 +96,7 @@ def _real_extract(self, url):
                 continue
             if proto == 'hls':
                 m3u8_formats = self._extract_m3u8_formats(
-                    media_url, media_id, 'mp4', 'm3u8' if is_live else 'm3u8_native',
+                    media_url, media_id, 'mp4', live=is_live,
                     m3u8_id='hls', fatal=False, headers=self._MANIFEST_HEADERS)
                 for f in m3u8_formats:
                     f.setdefault('http_headers', {}).update(self._MANIFEST_HEADERS)
diff --git a/yt_dlp/extractor/manyvids.py b/yt_dlp/extractor/manyvids.py
index e8d7163e4..bd24f8853 100644
--- a/yt_dlp/extractor/manyvids.py
+++ b/yt_dlp/extractor/manyvids.py
@@ -89,4 +89,5 @@ def _real_extract(self, url):
             'view_count': view_count,
             'like_count': like_count,
             'formats': formats,
+            'uploader': self._html_search_regex(r'<meta[^>]+name="author"[^>]*>([^<]+)', webpage, 'uploader'),
         }
diff --git a/yt_dlp/extractor/mirrativ.py b/yt_dlp/extractor/mirrativ.py
index 81aea54f6..2111de615 100644
--- a/yt_dlp/extractor/mirrativ.py
+++ b/yt_dlp/extractor/mirrativ.py
@@ -19,9 +19,25 @@ def assert_error(self, response):
 class MirrativIE(MirrativBaseIE):
     IE_NAME = 'mirrativ'
     _VALID_URL = r'https?://(?:www\.)?mirrativ\.com/live/(?P<id>[^/?#&]+)'
-    LIVE_API_URL = 'https://www.mirrativ.com/api/live/live?live_id=%s'
 
     TESTS = [{
+        'url': 'https://mirrativ.com/live/UQomuS7EMgHoxRHjEhNiHw',
+        'info_dict': {
+            'id': 'UQomuS7EMgHoxRHjEhNiHw',
+            'title': 'ねむいぃ、。『参加型』🔰jcが初めてやるCOD✨初見さん大歓迎💗',
+            'is_live': True,
+            'description': 'md5:bfcd8f77f2fab24c3c672e5620f3f16e',
+            'thumbnail': r're:https?://.+',
+            'uploader': '# あ ち ゅ 。💡',
+            'uploader_id': '118572165',
+            'duration': None,
+            'view_count': 1241,
+            'release_timestamp': 1646229192,
+            'timestamp': 1646229167,
+            'was_live': False,
+        },
+        'skip': 'livestream',
+    }, {
         'url': 'https://mirrativ.com/live/POxyuG1KmW2982lqlDTuPw',
         'only_matching': True,
     }]
@@ -29,12 +45,11 @@ class MirrativIE(MirrativBaseIE):
     def _real_extract(self, url):
         video_id = self._match_id(url)
         webpage = self._download_webpage('https://www.mirrativ.com/live/%s' % video_id, video_id)
-        live_response = self._download_json(self.LIVE_API_URL % video_id, video_id)
+        live_response = self._download_json(f'https://www.mirrativ.com/api/live/live?live_id={video_id}', video_id)
         self.assert_error(live_response)
 
         hls_url = dict_get(live_response, ('archive_url_hls', 'streaming_url_hls'))
         is_live = bool(live_response.get('is_live'))
-        was_live = bool(live_response.get('is_archive'))
         if not hls_url:
             raise ExtractorError('Neither archive nor live is available.', expected=True)
 
@@ -42,55 +57,29 @@ def _real_extract(self, url):
             hls_url, video_id,
             ext='mp4', entry_protocol='m3u8_native',
             m3u8_id='hls', live=is_live)
-        rtmp_url = live_response.get('streaming_url_edge')
-        if rtmp_url:
-            keys_to_copy = ('width', 'height', 'vcodec', 'acodec', 'tbr')
-            fmt = {
-                'format_id': 'rtmp',
-                'url': rtmp_url,
-                'protocol': 'rtmp',
-                'ext': 'mp4',
-            }
-            fmt.update({k: traverse_obj(formats, (0, k)) for k in keys_to_copy})
-            formats.append(fmt)
         self._sort_formats(formats)
 
-        title = self._og_search_title(webpage, default=None) or self._search_regex(
-            r'<title>\s*(.+?) - Mirrativ\s*</title>', webpage) or live_response.get('title')
-        description = live_response.get('description')
-        thumbnail = live_response.get('image_url')
-
-        duration = try_get(live_response, lambda x: x['ended_at'] - x['started_at'])
-        view_count = live_response.get('total_viewer_num')
-        release_timestamp = live_response.get('started_at')
-        timestamp = live_response.get('created_at')
-
-        owner = live_response.get('owner', {})
-        uploader = owner.get('name')
-        uploader_id = owner.get('user_id')
-
         return {
             'id': video_id,
-            'title': title,
+            'title': self._og_search_title(webpage, default=None) or self._search_regex(
+                r'<title>\s*(.+?) - Mirrativ\s*</title>', webpage) or live_response.get('title'),
             'is_live': is_live,
-            'description': description,
+            'description': live_response.get('description'),
             'formats': formats,
-            'thumbnail': thumbnail,
-            'uploader': uploader,
-            'uploader_id': uploader_id,
-            'duration': duration,
-            'view_count': view_count,
-            'release_timestamp': release_timestamp,
-            'timestamp': timestamp,
-            'was_live': was_live,
+            'thumbnail': live_response.get('image_url'),
+            'uploader': traverse_obj(live_response, ('owner', 'name')),
+            'uploader_id': traverse_obj(live_response, ('owner', 'user_id')),
+            'duration': try_get(live_response, lambda x: x['ended_at'] - x['started_at']) if not is_live else None,
+            'view_count': live_response.get('total_viewer_num'),
+            'release_timestamp': live_response.get('started_at'),
+            'timestamp': live_response.get('created_at'),
+            'was_live': bool(live_response.get('is_archive')),
         }
 
 
 class MirrativUserIE(MirrativBaseIE):
     IE_NAME = 'mirrativ:user'
     _VALID_URL = r'https?://(?:www\.)?mirrativ\.com/user/(?P<id>\d+)'
-    LIVE_HISTORY_API_URL = 'https://www.mirrativ.com/api/live/live_history?user_id=%s&page=%d'
-    USER_INFO_API_URL = 'https://www.mirrativ.com/api/user/profile?user_id=%s'
 
     _TESTS = [{
         # Live archive is available up to 3 days
@@ -104,8 +93,8 @@ def _entries(self, user_id):
         page = 1
         while page is not None:
             api_response = self._download_json(
-                self.LIVE_HISTORY_API_URL % (user_id, page), user_id,
-                note='Downloading page %d' % page)
+                f'https://www.mirrativ.com/api/live/live_history?user_id={user_id}&page={page}', user_id,
+                note=f'Downloading page {page}')
             self.assert_error(api_response)
             lives = api_response.get('lives')
             if not lives:
@@ -123,12 +112,10 @@ def _entries(self, user_id):
     def _real_extract(self, url):
         user_id = self._match_id(url)
         user_info = self._download_json(
-            self.USER_INFO_API_URL % user_id, user_id,
+            f'https://www.mirrativ.com/api/user/profile?user_id={user_id}', user_id,
             note='Downloading user info', fatal=False)
         self.assert_error(user_info)
 
-        uploader = user_info.get('name')
-        description = user_info.get('description')
-
-        entries = self._entries(user_id)
-        return self.playlist_result(entries, user_id, uploader, description)
+        return self.playlist_result(
+            self._entries(user_id), user_id,
+            user_info.get('name'), user_info.get('description'))
diff --git a/yt_dlp/extractor/nhk.py b/yt_dlp/extractor/nhk.py
index 4998fed83..626c6379b 100644
--- a/yt_dlp/extractor/nhk.py
+++ b/yt_dlp/extractor/nhk.py
@@ -1,8 +1,15 @@
 from __future__ import unicode_literals
 
+import re
 
 from .common import InfoExtractor
-from ..utils import urljoin
+from ..utils import (
+    parse_duration,
+    traverse_obj,
+    unescapeHTML,
+    unified_timestamp,
+    urljoin
+)
 
 
 class NhkBaseIE(InfoExtractor):
@@ -176,3 +183,143 @@ def _real_extract(self, url):
             program_title = entries[0].get('series')
 
         return self.playlist_result(entries, program_id, program_title)
+
+
+class NhkForSchoolBangumiIE(InfoExtractor):
+    _VALID_URL = r'https?://www2\.nhk\.or\.jp/school/movie/(?P<type>bangumi|clip)\.cgi\?das_id=(?P<id>[a-zA-Z0-9_-]+)'
+    _TESTS = [{
+        'url': 'https://www2.nhk.or.jp/school/movie/bangumi.cgi?das_id=D0005150191_00000',
+        'info_dict': {
+            'id': 'D0005150191_00003',
+            'title': 'にている かな',
+            'duration': 599.999,
+            'timestamp': 1396414800,
+
+            'upload_date': '20140402',
+            'ext': 'mp4',
+
+            'chapters': 'count:12'
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
+    }]
+
+    def _real_extract(self, url):
+        program_type, video_id = self._match_valid_url(url).groups()
+
+        webpage = self._download_webpage(
+            f'https://www2.nhk.or.jp/school/movie/{program_type}.cgi?das_id={video_id}', video_id)
+
+        # searches all variables
+        base_values = {g.group(1): g.group(2) for g in re.finditer(r'var\s+([a-zA-Z_]+)\s*=\s*"([^"]+?)";', webpage)}
+        # and programObj values too
+        program_values = {g.group(1): g.group(3) for g in re.finditer(r'(?:program|clip)Obj\.([a-zA-Z_]+)\s*=\s*(["\'])([^"]+?)\2;', webpage)}
+        # extract all chapters
+        chapter_durations = [parse_duration(g.group(1)) for g in re.finditer(r'chapterTime\.push\(\'([0-9:]+?)\'\);', webpage)]
+        chapter_titles = [' '.join([g.group(1) or '', unescapeHTML(g.group(2))]).strip() for g in re.finditer(r'<div class="cpTitle"><span>(scene\s*\d+)?</span>([^<]+?)</div>', webpage)]
+
+        # this is how player_core.js is actually doing (!)
+        version = base_values.get('r_version') or program_values.get('version')
+        if version:
+            video_id = f'{video_id.split("_")[0]}_{version}'
+
+        formats = self._extract_m3u8_formats(
+            f'https://nhks-vh.akamaihd.net/i/das/{video_id[0:8]}/{video_id}_V_000.f4v/master.m3u8',
+            video_id, ext='mp4', m3u8_id='hls')
+        self._sort_formats(formats)
+
+        duration = parse_duration(base_values.get('r_duration'))
+
+        chapters = None
+        if chapter_durations and chapter_titles and len(chapter_durations) == len(chapter_titles):
+            start_time = chapter_durations
+            end_time = chapter_durations[1:] + [duration]
+            chapters = [{
+                'start_time': s,
+                'end_time': e,
+                'title': t,
+            } for s, e, t in zip(start_time, end_time, chapter_titles)]
+
+        return {
+            'id': video_id,
+            'title': program_values.get('name'),
+            'duration': parse_duration(base_values.get('r_duration')),
+            'timestamp': unified_timestamp(base_values['r_upload']),
+            'formats': formats,
+            'chapters': chapters,
+        }
+
+
+class NhkForSchoolSubjectIE(InfoExtractor):
+    IE_DESC = 'Portal page for each school subjects, like Japanese (kokugo, 国語) or math (sansuu/suugaku or 算数・数学)'
+    KNOWN_SUBJECTS = (
+        'rika', 'syakai', 'kokugo',
+        'sansuu', 'seikatsu', 'doutoku',
+        'ongaku', 'taiiku', 'zukou',
+        'gijutsu', 'katei', 'sougou',
+        'eigo', 'tokkatsu',
+        'tokushi', 'sonota',
+    )
+    _VALID_URL = r'https?://www\.nhk\.or\.jp/school/(?P<id>%s)/?(?:[\?#].*)?$' % '|'.join(re.escape(s) for s in KNOWN_SUBJECTS)
+
+    _TESTS = [{
+        'url': 'https://www.nhk.or.jp/school/sougou/',
+        'info_dict': {
+            'id': 'sougou',
+            'title': '総合的な学習の時間',
+        },
+        'playlist_mincount': 16,
+    }, {
+        'url': 'https://www.nhk.or.jp/school/rika/',
+        'info_dict': {
+            'id': 'rika',
+            'title': '理科',
+        },
+        'playlist_mincount': 15,
+    }]
+
+    def _real_extract(self, url):
+        subject_id = self._match_id(url)
+        webpage = self._download_webpage(url, subject_id)
+
+        return self.playlist_from_matches(
+            re.finditer(rf'href="((?:https?://www\.nhk\.or\.jp)?/school/{re.escape(subject_id)}/[^/]+/)"', webpage),
+            subject_id,
+            self._html_search_regex(r'(?s)<span\s+class="subjectName">\s*<img\s*[^<]+>\s*([^<]+?)</span>', webpage, 'title', fatal=False),
+            lambda g: urljoin(url, g.group(1)))
+
+
+class NhkForSchoolProgramListIE(InfoExtractor):
+    _VALID_URL = r'https?://www\.nhk\.or\.jp/school/(?P<id>(?:%s)/[a-zA-Z0-9_-]+)' % (
+        '|'.join(re.escape(s) for s in NhkForSchoolSubjectIE.KNOWN_SUBJECTS)
+    )
+    _TESTS = [{
+        'url': 'https://www.nhk.or.jp/school/sougou/q/',
+        'info_dict': {
+            'id': 'sougou/q',
+            'title': 'Ｑ～こどものための哲学',
+        },
+        'playlist_mincount': 20,
+    }]
+
+    def _real_extract(self, url):
+        program_id = self._match_id(url)
+
+        webpage = self._download_webpage(f'https://www.nhk.or.jp/school/{program_id}/', program_id)
+
+        title = self._og_search_title(webpage, fatal=False) or self._html_extract_title(webpage, fatal=False) or self._html_search_regex(r'<h3>([^<]+?)とは？\s*</h3>', webpage, 'title', fatal=False)
+        title = re.sub(r'\s*\|\s*NHK\s+for\s+School\s*$', '', title) if title else None
+        description = self._html_search_regex(
+            r'(?s)<div\s+class="programDetail\s*">\s*<p>[^<]+</p>',
+            webpage, 'description', fatal=False, group=0)
+
+        bangumi_list = self._download_json(
+            f'https://www.nhk.or.jp/school/{program_id}/meta/program.json', program_id)
+        # they're always bangumi
+        bangumis = [
+            self.url_result(f'https://www2.nhk.or.jp/school/movie/bangumi.cgi?das_id={x}')
+            for x in traverse_obj(bangumi_list, ('part', ..., 'part-video-dasid')) or []]
+
+        return self.playlist_result(bangumis, program_id, title, description)
diff --git a/yt_dlp/extractor/niconico.py b/yt_dlp/extractor/niconico.py
index 6e561bee5..8f56fc95b 100644
--- a/yt_dlp/extractor/niconico.py
+++ b/yt_dlp/extractor/niconico.py
@@ -3,6 +3,7 @@
 
 import datetime
 import itertools
+import functools
 import json
 import re
 
@@ -12,6 +13,7 @@
     compat_str,
     compat_parse_qs,
     compat_urllib_parse_urlparse,
+    compat_HTTPError,
 )
 from ..utils import (
     ExtractorError,
@@ -24,7 +26,9 @@
     PostProcessingError,
     remove_start,
     str_or_none,
+    traverse_obj,
     try_get,
+    unescapeHTML,
     unified_timestamp,
     urlencode_postdata,
     xpath_text,
@@ -606,8 +610,61 @@ def get_video_info_xml(items):
         }
 
 
-class NiconicoPlaylistIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?nicovideo\.jp/(?:user/\d+/|my/)?mylist/(?P<id>\d+)'
+class NiconicoPlaylistBaseIE(InfoExtractor):
+    _PAGE_SIZE = 100
+
+    _API_HEADERS = {
+        'X-Frontend-ID': '6',
+        'X-Frontend-Version': '0',
+        'X-Niconico-Language': 'en-us'
+    }
+
+    def _call_api(self, list_id, resource, query):
+        "Implement this in child class"
+        pass
+
+    @staticmethod
+    def _parse_owner(item):
+        return {
+            'uploader': traverse_obj(item, ('owner', 'name')),
+            'uploader_id': traverse_obj(item, ('owner', 'id')),
+        }
+
+    def _fetch_page(self, list_id, page):
+        page += 1
+        resp = self._call_api(list_id, 'page %d' % page, {
+            'page': page,
+            'pageSize': self._PAGE_SIZE,
+        })
+        # this is needed to support both mylist and user
+        for video in traverse_obj(resp, ('items', ..., ('video', None))) or []:
+            video_id = video.get('id')
+            if not video_id:
+                # skip {"video": {"id": "blablabla", ...}}
+                continue
+            count = video.get('count') or {}
+            get_count = lambda x: int_or_none(count.get(x))
+            yield {
+                '_type': 'url',
+                'id': video_id,
+                'title': video.get('title'),
+                'url': f'https://www.nicovideo.jp/watch/{video_id}',
+                'description': video.get('shortDescription'),
+                'duration': int_or_none(video.get('duration')),
+                'view_count': get_count('view'),
+                'comment_count': get_count('comment'),
+                'thumbnail': traverse_obj(video, ('thumbnail', ('nHdUrl', 'largeUrl', 'listingUrl', 'url'))),
+                'ie_key': NiconicoIE.ie_key(),
+                **self._parse_owner(video),
+            }
+
+    def _entries(self, list_id):
+        return OnDemandPagedList(functools.partial(self._fetch_page, list_id), self._PAGE_SIZE)
+
+
+class NiconicoPlaylistIE(NiconicoPlaylistBaseIE):
+    IE_NAME = 'niconico:playlist'
+    _VALID_URL = r'https?://(?:(?:www\.|sp\.)?nicovideo\.jp|nico\.ms)/(?:user/\d+/)?(?:my/)?mylist/(?:#/)?(?P<id>\d+)'
 
     _TESTS = [{
         'url': 'http://www.nicovideo.jp/mylist/27411728',
@@ -618,48 +675,110 @@ class NiconicoPlaylistIE(InfoExtractor):
             'uploader': 'のっく',
             'uploader_id': '805442',
         },
-        'playlist_mincount': 225,
+        'playlist_mincount': 291,
     }, {
         'url': 'https://www.nicovideo.jp/user/805442/mylist/27411728',
         'only_matching': True,
+    }, {
+        'url': 'https://www.nicovideo.jp/my/mylist/#/68048635',
+        'only_matching': True,
     }]
 
-    _API_HEADERS = {
-        'X-Frontend-ID': '6',
-        'X-Frontend-Version': '0'
-    }
+    def _call_api(self, list_id, resource, query):
+        return self._download_json(
+            f'https://nvapi.nicovideo.jp/v2/mylists/{list_id}', list_id,
+            f'Downloading {resource}', query=query,
+            headers=self._API_HEADERS)['data']['mylist']
 
     def _real_extract(self, url):
         list_id = self._match_id(url)
+        mylist = self._call_api(list_id, 'list', {
+            'pageSize': 1,
+        })
+        return self.playlist_result(
+            self._entries(list_id), list_id,
+            mylist.get('name'), mylist.get('description'), **self._parse_owner(mylist))
 
-        def get_page_data(pagenum, pagesize):
-            return self._download_json(
-                'http://nvapi.nicovideo.jp/v2/mylists/' + list_id, list_id,
-                query={'page': 1 + pagenum, 'pageSize': pagesize},
-                headers=self._API_HEADERS).get('data').get('mylist')
-
-        data = get_page_data(0, 1)
-        title = data.get('name')
-        description = data.get('description')
-        uploader = data.get('owner').get('name')
-        uploader_id = data.get('owner').get('id')
-
-        def pagefunc(pagenum):
-            data = get_page_data(pagenum, 25)
-            return ({
-                '_type': 'url',
-                'url': 'http://www.nicovideo.jp/watch/' + item.get('watchId'),
-            } for item in data.get('items'))
 
-        return {
-            '_type': 'playlist',
-            'id': list_id,
-            'title': title,
-            'description': description,
-            'uploader': uploader,
-            'uploader_id': uploader_id,
-            'entries': OnDemandPagedList(pagefunc, 25),
-        }
+class NiconicoSeriesIE(InfoExtractor):
+    IE_NAME = 'niconico:series'
+    _VALID_URL = r'https?://(?:(?:www\.|sp\.)?nicovideo\.jp|nico\.ms)/series/(?P<id>\d+)'
+
+    _TESTS = [{
+        'url': 'https://www.nicovideo.jp/series/110226',
+        'info_dict': {
+            'id': '110226',
+            'title': 'ご立派ァ！のシリーズ',
+        },
+        'playlist_mincount': 10,  # as of 2021/03/17
+    }, {
+        'url': 'https://www.nicovideo.jp/series/12312/',
+        'info_dict': {
+            'id': '12312',
+            'title': 'バトルスピリッツ　お勧めカード紹介(調整中)',
+        },
+        'playlist_mincount': 97,  # as of 2021/03/17
+    }, {
+        'url': 'https://nico.ms/series/203559',
+        'only_matching': True,
+    }]
+
+    def _real_extract(self, url):
+        list_id = self._match_id(url)
+        webpage = self._download_webpage(f'https://www.nicovideo.jp/series/{list_id}', list_id)
+
+        title = self._search_regex(
+            (r'<title>「(.+)（全',
+             r'<div class="TwitterShareButton"\s+data-text="(.+)\s+https:'),
+            webpage, 'title', fatal=False)
+        if title:
+            title = unescapeHTML(title)
+        playlist = [
+            self.url_result(f'https://www.nicovideo.jp/watch/{v_id}', video_id=v_id)
+            for v_id in re.findall(r'href="/watch/([a-z0-9]+)" data-href="/watch/\1', webpage)]
+        return self.playlist_result(playlist, list_id, title)
+
+
+class NiconicoHistoryIE(NiconicoPlaylistBaseIE):
+    IE_NAME = 'niconico:history'
+    IE_DESC = 'NicoNico user history. Requires cookies.'
+    _VALID_URL = r'https?://(?:www\.|sp\.)?nicovideo\.jp/my/history'
+
+    _TESTS = [{
+        'note': 'PC page, with /video',
+        'url': 'https://www.nicovideo.jp/my/history/video',
+        'only_matching': True,
+    }, {
+        'note': 'PC page, without /video',
+        'url': 'https://www.nicovideo.jp/my/history',
+        'only_matching': True,
+    }, {
+        'note': 'mobile page, with /video',
+        'url': 'https://sp.nicovideo.jp/my/history/video',
+        'only_matching': True,
+    }, {
+        'note': 'mobile page, without /video',
+        'url': 'https://sp.nicovideo.jp/my/history',
+        'only_matching': True,
+    }]
+
+    def _call_api(self, list_id, resource, query):
+        return self._download_json(
+            'https://nvapi.nicovideo.jp/v1/users/me/watch/history', 'history',
+            f'Downloading {resource}', query=query,
+            headers=self._API_HEADERS)['data']
+
+    def _real_extract(self, url):
+        list_id = 'history'
+        try:
+            mylist = self._call_api(list_id, 'list', {
+                'pageSize': 1,
+            })
+        except ExtractorError as e:
+            if isinstance(e.cause, compat_HTTPError) and e.cause.code == 401:
+                self.raise_login_required('You have to be logged in to get your watch history')
+            raise
+        return self.playlist_result(self._entries(list_id), list_id, **self._parse_owner(mylist))
 
 
 class NicovideoSearchBaseIE(InfoExtractor):
diff --git a/yt_dlp/extractor/rokfin.py b/yt_dlp/extractor/rokfin.py
new file mode 100644
index 000000000..79a5b2336
--- /dev/null
+++ b/yt_dlp/extractor/rokfin.py
@@ -0,0 +1,256 @@
+# coding: utf-8
+import itertools
+from datetime import datetime
+
+from .common import InfoExtractor
+from ..utils import (
+    determine_ext,
+    ExtractorError,
+    float_or_none,
+    format_field,
+    int_or_none,
+    str_or_none,
+    traverse_obj,
+    unified_timestamp,
+    url_or_none,
+)
+
+
+_API_BASE_URL = 'https://prod-api-v2.production.rokfin.com/api/v2/public/'
+
+
+class RokfinIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?rokfin\.com/(?P<id>(?P<type>post|stream)/\d+)'
+    _TESTS = [{
+        'url': 'https://www.rokfin.com/post/57548/Mitt-Romneys-Crazy-Solution-To-Climate-Change',
+        'info_dict': {
+            'id': 'post/57548',
+            'ext': 'mp4',
+            'title': 'Mitt Romney\'s Crazy Solution To Climate Change',
+            'thumbnail': r're:https://img\.production\.rokfin\.com/.+',
+            'upload_date': '20211023',
+            'timestamp': 1634998029,
+            'channel': 'Jimmy Dore',
+            'channel_id': 65429,
+            'channel_url': 'https://rokfin.com/TheJimmyDoreShow',
+            'duration': 213.0,
+            'availability': 'public',
+            'live_status': 'not_live',
+            'dislike_count': int,
+            'like_count': int,
+        }
+    }, {
+        'url': 'https://rokfin.com/post/223/Julian-Assange-Arrested-Streaming-In-Real-Time',
+        'info_dict': {
+            'id': 'post/223',
+            'ext': 'mp4',
+            'title': 'Julian Assange Arrested: Streaming In Real Time',
+            'thumbnail': r're:https://img\.production\.rokfin\.com/.+',
+            'upload_date': '20190412',
+            'timestamp': 1555052644,
+            'channel': 'Ron Placone',
+            'channel_id': 10,
+            'channel_url': 'https://rokfin.com/RonPlacone',
+            'availability': 'public',
+            'live_status': 'not_live',
+            'dislike_count': int,
+            'like_count': int,
+            'tags': ['FreeThinkingMedia^', 'RealProgressives^'],
+        }
+    }, {
+        'url': 'https://www.rokfin.com/stream/10543/Its-A-Crazy-Mess-Regional-Director-Blows-Whistle-On-Pfizers-Vaccine-Trial-Data',
+        'info_dict': {
+            'id': 'stream/10543',
+            'ext': 'mp4',
+            'title': '"It\'s A Crazy Mess" Regional Director Blows Whistle On Pfizer\'s Vaccine Trial Data',
+            'thumbnail': r're:https://img\.production\.rokfin\.com/.+',
+            'description': 'md5:324ce2d3e3b62e659506409e458b9d8e',
+            'channel': 'Ryan Cristián',
+            'channel_id': 53856,
+            'channel_url': 'https://rokfin.com/TLAVagabond',
+            'availability': 'public',
+            'is_live': False,
+            'was_live': True,
+            'live_status': 'was_live',
+            'timestamp': 1635874720,
+            'release_timestamp': 1635874720,
+            'release_date': '20211102',
+            'upload_date': '20211102',
+            'dislike_count': int,
+            'like_count': int,
+            'tags': ['FreeThinkingMedia^'],
+        }
+    }]
+
+    def _real_extract(self, url):
+        video_id, video_type = self._match_valid_url(url).group('id', 'type')
+
+        metadata = self._download_json(f'{_API_BASE_URL}{video_id}', video_id)
+
+        scheduled = unified_timestamp(metadata.get('scheduledAt'))
+        live_status = ('was_live' if metadata.get('stoppedAt')
+                       else 'is_upcoming' if scheduled
+                       else 'is_live' if video_type == 'stream'
+                       else 'not_live')
+
+        video_url = traverse_obj(metadata, 'url', ('content', 'contentUrl'), expected_type=url_or_none)
+        formats, subtitles = [{'url': video_url}] if video_url else [], {}
+        if determine_ext(video_url) == 'm3u8':
+            formats, subtitles = self._extract_m3u8_formats_and_subtitles(
+                video_url, video_id, fatal=False, live=live_status == 'is_live')
+
+        if not formats:
+            if metadata.get('premiumPlan'):
+                self.raise_login_required('This video is only available to premium users', True, method='cookies')
+            elif scheduled:
+                self.raise_no_formats(
+                    f'Stream is offline; sheduled for {datetime.fromtimestamp(scheduled).strftime("%Y-%m-%d %H:%M:%S")}',
+                    video_id=video_id, expected=True)
+        self._sort_formats(formats)
+
+        uploader = traverse_obj(metadata, ('createdBy', 'username'), ('creator', 'username'))
+        timestamp = (scheduled or float_or_none(metadata.get('postedAtMilli'), 1000)
+                     or unified_timestamp(metadata.get('creationDateTime')))
+        return {
+            'id': video_id,
+            'formats': formats,
+            'subtitles': subtitles,
+            'title': str_or_none(traverse_obj(metadata, 'title', ('content', 'contentTitle'))),
+            'duration': float_or_none(traverse_obj(metadata, ('content', 'duration'))),
+            'thumbnail': url_or_none(traverse_obj(metadata, 'thumbnail', ('content', 'thumbnailUrl1'))),
+            'description': str_or_none(traverse_obj(metadata, 'description', ('content', 'contentDescription'))),
+            'like_count': int_or_none(metadata.get('likeCount')),
+            'dislike_count': int_or_none(metadata.get('dislikeCount')),
+            'channel': str_or_none(traverse_obj(metadata, ('createdBy', 'name'), ('creator', 'name'))),
+            'channel_id': traverse_obj(metadata, ('createdBy', 'id'), ('creator', 'id')),
+            'channel_url': url_or_none(f'https://rokfin.com/{uploader}') if uploader else None,
+            'timestamp': timestamp,
+            'release_timestamp': timestamp if live_status != 'not_live' else None,
+            'tags': traverse_obj(metadata, ('tags', ..., 'title'), expected_type=str_or_none),
+            'live_status': live_status,
+            'availability': self._availability(
+                needs_premium=bool(metadata.get('premiumPlan')),
+                is_private=False, needs_subscription=False, needs_auth=False, is_unlisted=False),
+            # 'comment_count': metadata.get('numComments'), # Data provided by website is wrong
+            '__post_extractor': self.extract_comments(video_id) if video_type == 'post' else None,
+        }
+
+    def _get_comments(self, video_id):
+        pages_total = None
+        for page_n in itertools.count():
+            raw_comments = self._download_json(
+                f'{_API_BASE_URL}comment?postId={video_id[5:]}&page={page_n}&size=50',
+                video_id, note=f'Downloading viewer comments page {page_n + 1}{format_field(pages_total, template=" of %s")}',
+                fatal=False) or {}
+
+            for comment in raw_comments.get('content') or []:
+                yield {
+                    'text': str_or_none(comment.get('comment')),
+                    'author': str_or_none(comment.get('name')),
+                    'id': comment.get('commentId'),
+                    'author_id': comment.get('userId'),
+                    'parent': 'root',
+                    'like_count': int_or_none(comment.get('numLikes')),
+                    'dislike_count': int_or_none(comment.get('numDislikes')),
+                    'timestamp': unified_timestamp(comment.get('postedAt'))
+                }
+
+            pages_total = int_or_none(raw_comments.get('totalPages')) or None
+            is_last = raw_comments.get('last')
+            if not raw_comments.get('content') or is_last or (page_n > pages_total if pages_total else is_last is not False):
+                return
+
+
+class RokfinPlaylistBaseIE(InfoExtractor):
+    _TYPES = {
+        'video': 'post',
+        'audio': 'post',
+        'stream': 'stream',
+        'dead_stream': 'stream',
+        'stack': 'stack',
+    }
+
+    def _get_video_data(self, metadata):
+        for content in metadata.get('content') or []:
+            media_type = self._TYPES.get(content.get('mediaType'))
+            video_id = content.get('id') if media_type == 'post' else content.get('mediaId')
+            if not media_type or not video_id:
+                continue
+
+            yield self.url_result(f'https://rokfin.com/{media_type}/{video_id}', video_id=f'{media_type}/{video_id}',
+                                  video_title=str_or_none(traverse_obj(content, ('content', 'contentTitle'))))
+
+
+class RokfinStackIE(RokfinPlaylistBaseIE):
+    IE_NAME = 'rokfin:stack'
+    _VALID_URL = r'https?://(?:www\.)?rokfin\.com/stack/(?P<id>[^/]+)'
+    _TESTS = [{
+        'url': 'https://www.rokfin.com/stack/271/Tulsi-Gabbard-Portsmouth-Townhall-FULL--Feb-9-2020',
+        'playlist_count': 8,
+        'info_dict': {
+            'id': '271',
+        },
+    }]
+
+    def _real_extract(self, url):
+        list_id = self._match_id(url)
+        return self.playlist_result(self._get_video_data(
+            self._download_json(f'{_API_BASE_URL}stack/{list_id}', list_id)), list_id)
+
+
+class RokfinChannelIE(RokfinPlaylistBaseIE):
+    IE_NAME = 'rokfin:channel'
+    _VALID_URL = r'https?://(?:www\.)?rokfin\.com/(?!((feed/?)|(discover/?)|(channels/?))$)(?P<id>[^/]+)/?$'
+    _TESTS = [{
+        'url': 'https://rokfin.com/TheConvoCouch',
+        'playlist_mincount': 100,
+        'info_dict': {
+            'id': '12071-new',
+            'title': 'TheConvoCouch - New',
+            'description': 'md5:bb622b1bca100209b91cd685f7847f06',
+        },
+    }]
+
+    _TABS = {
+        'new': 'posts',
+        'top': 'top',
+        'videos': 'video',
+        'podcasts': 'audio',
+        'streams': 'stream',
+        'stacks': 'stack',
+    }
+
+    def _real_initialize(self):
+        self._validate_extractor_args()
+
+    def _validate_extractor_args(self):
+        requested_tabs = self._configuration_arg('tab', None)
+        if requested_tabs is not None and (len(requested_tabs) > 1 or requested_tabs[0] not in self._TABS):
+            raise ExtractorError(f'Invalid extractor-arg "tab". Must be one of {", ".join(self._TABS)}', expected=True)
+
+    def _entries(self, channel_id, channel_name, tab):
+        pages_total = None
+        for page_n in itertools.count(0):
+            if tab in ('posts', 'top'):
+                data_url = f'{_API_BASE_URL}user/{channel_name}/{tab}?page={page_n}&size=50'
+            else:
+                data_url = f'{_API_BASE_URL}post/search/{tab}?page={page_n}&size=50&creator={channel_id}'
+            metadata = self._download_json(
+                data_url, channel_name,
+                note=f'Downloading video metadata page {page_n + 1}{format_field(pages_total, template=" of %s")}')
+
+            yield from self._get_video_data(metadata)
+            pages_total = int_or_none(metadata.get('totalPages')) or None
+            is_last = metadata.get('last')
+            if is_last or (page_n > pages_total if pages_total else is_last is not False):
+                return
+
+    def _real_extract(self, url):
+        channel_name = self._match_id(url)
+        channel_info = self._download_json(f'{_API_BASE_URL}user/{channel_name}', channel_name)
+        channel_id = channel_info['id']
+        tab = self._configuration_arg('tab', default=['new'])[0]
+
+        return self.playlist_result(
+            self._entries(channel_id, channel_name, self._TABS[tab]),
+            f'{channel_id}-{tab}', f'{channel_name} - {tab.title()}', str_or_none(channel_info.get('description')))
diff --git a/yt_dlp/extractor/spiegel.py b/yt_dlp/extractor/spiegel.py
index 2da32b9b2..58f2ed353 100644
--- a/yt_dlp/extractor/spiegel.py
+++ b/yt_dlp/extractor/spiegel.py
@@ -7,7 +7,7 @@
 
 class SpiegelIE(InfoExtractor):
     _UUID_RE = r'[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12}'
-    _VALID_URL = r'https?://(?:www\.)?(?:spiegel|manager-magazin)\.de(?:/[^/]+)+/[^/]*-(?P<id>[0-9]+|%s)(?:-embed|-iframe)?(?:\.html)?(?:#.*)?$' % _UUID_RE
+    _VALID_URL = r'https?://(?:www\.)?(?:spiegel|manager-magazin)\.de(?:/[^/]+)+/[^/]*-(?P<id>[0-9]+|%s)(?:-embed|-iframe)?(?:\.html)?(?:$|[#?])' % _UUID_RE
     _TESTS = [{
         'url': 'http://www.spiegel.de/video/vulkan-tungurahua-in-ecuador-ist-wieder-aktiv-video-1259285.html',
         'md5': '50c7948883ec85a3e431a0a44b7ad1d6',
diff --git a/yt_dlp/extractor/threeqsdn.py b/yt_dlp/extractor/threeqsdn.py
index 22b4fe7c8..00a51dccd 100644
--- a/yt_dlp/extractor/threeqsdn.py
+++ b/yt_dlp/extractor/threeqsdn.py
@@ -111,8 +111,7 @@ def _real_extract(self, url):
                 subtitles = self._merge_subtitles(subtitles, subs)
             elif source_type == 'hls':
                 fmts, subs = self._extract_m3u8_formats_and_subtitles(
-                    source, video_id, 'mp4', 'm3u8' if live else 'm3u8_native',
-                    m3u8_id='hls', fatal=False)
+                    source, video_id, 'mp4', live=live, m3u8_id='hls', fatal=False)
                 formats.extend(fmts)
                 subtitles = self._merge_subtitles(subtitles, subs)
             elif source_type == 'progressive':
diff --git a/yt_dlp/extractor/tubitv.py b/yt_dlp/extractor/tubitv.py
index 2e9b325ba..e9b66ec77 100644
--- a/yt_dlp/extractor/tubitv.py
+++ b/yt_dlp/extractor/tubitv.py
@@ -107,6 +107,9 @@ def _real_extract(self, url):
                 'url': self._proto_relative_url(sub_url),
             })
 
+        season_number, episode_number, episode_title = self._search_regex(
+            r'^S(\d+):E(\d+) - (.+)', title, 'episode info', fatal=False, group=(1, 2, 3), default=(None, None, None))
+
         return {
             'id': video_id,
             'title': title,
@@ -117,6 +120,9 @@ def _real_extract(self, url):
             'duration': int_or_none(video_data.get('duration')),
             'uploader_id': video_data.get('publisher_id'),
             'release_year': int_or_none(video_data.get('year')),
+            'season_number': int_or_none(season_number),
+            'episode_number': int_or_none(episode_number),
+            'episode_title': episode_title
         }
 
 
@@ -132,9 +138,11 @@ class TubiTvShowIE(InfoExtractor):
 
     def _entries(self, show_url, show_name):
         show_webpage = self._download_webpage(show_url, show_name)
+
         show_json = self._parse_json(self._search_regex(
-            r"window\.__data\s*=\s*({.+?});\s*</script>",
-            show_webpage, 'data',), show_name, transform_source=js_to_json)['video']
+            r'window\.__data\s*=\s*({[^<]+});\s*</script>',
+            show_webpage, 'data'), show_name, transform_source=js_to_json)['video']
+
         for episode_id in show_json['fullContentById'].keys():
             yield self.url_result(
                 'tubitv:%s' % episode_id,
diff --git a/yt_dlp/extractor/tv2.py b/yt_dlp/extractor/tv2.py
index b48dfe389..977da30fe 100644
--- a/yt_dlp/extractor/tv2.py
+++ b/yt_dlp/extractor/tv2.py
@@ -81,9 +81,7 @@ def _real_extract(self, url):
                 elif ext == 'm3u8':
                     if not data.get('drmProtected'):
                         formats.extend(self._extract_m3u8_formats(
-                            video_url, video_id, 'mp4',
-                            'm3u8' if is_live else 'm3u8_native',
-                            m3u8_id=format_id, fatal=False))
+                            video_url, video_id, 'mp4', live=is_live, m3u8_id=format_id, fatal=False))
                 elif ext == 'mpd':
                     formats.extend(self._extract_mpd_formats(
                         video_url, video_id, format_id, fatal=False))
@@ -244,9 +242,7 @@ def _real_extract(self, url):
                 elif ext == 'm3u8':
                     if not data.get('drmProtected'):
                         formats.extend(self._extract_m3u8_formats(
-                            video_url, video_id, 'mp4',
-                            'm3u8' if is_live else 'm3u8_native',
-                            m3u8_id=format_id, fatal=False))
+                            video_url, video_id, 'mp4', live=is_live, m3u8_id=format_id, fatal=False))
                 elif ext == 'mpd':
                     formats.extend(self._extract_mpd_formats(
                         video_url, video_id, format_id, fatal=False))
diff --git a/yt_dlp/extractor/tvnet.py b/yt_dlp/extractor/tvnet.py
index 4fe8dfb6c..aa1e9d923 100644
--- a/yt_dlp/extractor/tvnet.py
+++ b/yt_dlp/extractor/tvnet.py
@@ -111,9 +111,7 @@ def _real_extract(self, url):
                 continue
             stream_urls.add(stream_url)
             formats.extend(self._extract_m3u8_formats(
-                stream_url, video_id, 'mp4',
-                entry_protocol='m3u8' if is_live else 'm3u8_native',
-                m3u8_id='hls', fatal=False))
+                stream_url, video_id, 'mp4', live=is_live, m3u8_id='hls', fatal=False))
         self._sort_formats(formats)
 
         # better support for radio streams
diff --git a/yt_dlp/extractor/twitcasting.py b/yt_dlp/extractor/twitcasting.py
index 08222df95..af911de98 100644
--- a/yt_dlp/extractor/twitcasting.py
+++ b/yt_dlp/extractor/twitcasting.py
@@ -94,6 +94,9 @@ def _real_extract(self, url):
                 urlh.geturl(), video_id, data=request_data,
                 headers={'Origin': 'https://twitcasting.tv'},
                 note='Retrying authentication')
+        # has to check here as the first request can contain password input form even if the password is correct
+        if re.search(r'<form\s+method="POST">\s*<input\s+[^>]+?name="password"', webpage):
+            raise ExtractorError('This video is protected by a password, use the --video-password option', expected=True)
 
         title = (clean_html(get_element_by_id('movietitle', webpage))
                  or self._html_search_meta(['og:title', 'twitter:title'], webpage, fatal=True))
diff --git a/yt_dlp/extractor/vgtv.py b/yt_dlp/extractor/vgtv.py
index 10083cd24..9d6090b08 100644
--- a/yt_dlp/extractor/vgtv.py
+++ b/yt_dlp/extractor/vgtv.py
@@ -195,9 +195,7 @@ def _real_extract(self, url):
         hls_url = streams.get('hls')
         if hls_url:
             formats.extend(self._extract_m3u8_formats(
-                hls_url, video_id, 'mp4',
-                entry_protocol='m3u8' if is_live else 'm3u8_native',
-                m3u8_id='hls', fatal=False))
+                hls_url, video_id, 'mp4', live=is_live, m3u8_id='hls', fatal=False))
 
         hds_url = streams.get('hds')
         if hds_url:
diff --git a/yt_dlp/extractor/vimeo.py b/yt_dlp/extractor/vimeo.py
index c2dec244f..1a9fd00e4 100644
--- a/yt_dlp/extractor/vimeo.py
+++ b/yt_dlp/extractor/vimeo.py
@@ -166,8 +166,7 @@ def _parse_config(self, config, video_id):
                 for f_id, m_url in sep_manifest_urls:
                     if files_type == 'hls':
                         fmts, subs = self._extract_m3u8_formats_and_subtitles(
-                            m_url, video_id, 'mp4',
-                            'm3u8' if is_live else 'm3u8_native', m3u8_id=f_id,
+                            m_url, video_id, 'mp4', live=is_live, m3u8_id=f_id,
                             note='Downloading %s m3u8 information' % cdn_name,
                             fatal=False)
                         formats.extend(fmts)
diff --git a/yt_dlp/extractor/youtube.py b/yt_dlp/extractor/youtube.py
index d10dc20e4..602d48e3c 100644
--- a/yt_dlp/extractor/youtube.py
+++ b/yt_dlp/extractor/youtube.py
@@ -844,7 +844,7 @@ def _extract_video(self, renderer):
             'uploader': uploader,
             'channel_id': channel_id,
             'thumbnails': thumbnails,
-            #  'upload_date': strftime_or_none(timestamp, '%Y%m%d'),
+            'upload_date': strftime_or_none(timestamp, '%Y%m%d') if self._configuration_arg('approximate_date', ie_key='youtubetab') else None,
             'live_status': ('is_upcoming' if scheduled_timestamp is not None
                             else 'was_live' if 'streamed' in time_text.lower()
                             else 'is_live' if overlay_style is not None and overlay_style == 'LIVE' or 'live now' in badges
@@ -2135,6 +2135,7 @@ def mpd_feed(format_id, delay):
             return f['manifest_url'], f['manifest_stream_number'], is_live
 
         for f in formats:
+            f['is_live'] = True
             f['protocol'] = 'http_dash_segments_generator'
             f['fragments'] = functools.partial(
                 self._live_dash_fragments, f['format_id'], live_start_time, mpd_feed)
@@ -2157,12 +2158,12 @@ def _live_dash_fragments(self, format_id, live_start_time, mpd_feed, ctx):
         known_idx, no_fragment_score, last_segment_url = begin_index, 0, None
         fragments, fragment_base_url = None, None
 
-        def _extract_sequence_from_mpd(refresh_sequence):
+        def _extract_sequence_from_mpd(refresh_sequence, immediate):
             nonlocal mpd_url, stream_number, is_live, no_fragment_score, fragments, fragment_base_url
             # Obtain from MPD's maximum seq value
             old_mpd_url = mpd_url
             last_error = ctx.pop('last_error', None)
-            expire_fast = last_error and isinstance(last_error, compat_HTTPError) and last_error.code == 403
+            expire_fast = immediate or last_error and isinstance(last_error, compat_HTTPError) and last_error.code == 403
             mpd_url, stream_number, is_live = (mpd_feed(format_id, 5 if expire_fast else 18000)
                                                or (mpd_url, stream_number, False))
             if not refresh_sequence:
@@ -2176,7 +2177,7 @@ def _extract_sequence_from_mpd(refresh_sequence):
             except ExtractorError:
                 fmts = None
             if not fmts:
-                no_fragment_score += 1
+                no_fragment_score += 2
                 return False, last_seq
             fmt_info = next(x for x in fmts if x['manifest_stream_number'] == stream_number)
             fragments = fmt_info['fragments']
@@ -2199,11 +2200,12 @@ def _extract_sequence_from_mpd(refresh_sequence):
                     urlh = None
                 last_seq = try_get(urlh, lambda x: int_or_none(x.headers['X-Head-Seqnum']))
                 if last_seq is None:
-                    no_fragment_score += 1
+                    no_fragment_score += 2
                     last_segment_url = None
                     continue
             else:
-                should_continue, last_seq = _extract_sequence_from_mpd(True)
+                should_continue, last_seq = _extract_sequence_from_mpd(True, no_fragment_score > 15)
+                no_fragment_score += 2
                 if not should_continue:
                     continue
 
@@ -2221,7 +2223,7 @@ def _extract_sequence_from_mpd(refresh_sequence):
             try:
                 for idx in range(known_idx, last_seq):
                     # do not update sequence here or you'll get skipped some part of it
-                    should_continue, _ = _extract_sequence_from_mpd(False)
+                    should_continue, _ = _extract_sequence_from_mpd(False, False)
                     if not should_continue:
                         known_idx = idx - 1
                         raise ExtractorError('breaking out of outer loop')
@@ -2418,7 +2420,7 @@ def _extract_n_function_name(self, jscode):
         if not idx:
             return nfunc
         return json.loads(js_to_json(self._search_regex(
-            rf'var {nfunc}\s*=\s*(\[.+?\]);', jscode,
+            rf'var {re.escape(nfunc)}\s*=\s*(\[.+?\]);', jscode,
             f'Initial JS player n function list ({nfunc}.{idx})')))[int(idx)]
 
     def _extract_n_function(self, video_id, player_url):
@@ -2936,6 +2938,7 @@ def _extract_formats(self, streaming_data, video_id, player_url, is_live):
             'small', 'medium', 'large', 'hd720', 'hd1080', 'hd1440', 'hd2160', 'hd2880', 'highres'
         ])
         streaming_formats = traverse_obj(streaming_data, (..., ('formats', 'adaptiveFormats'), ...), default=[])
+        approx_duration = max(traverse_obj(streaming_formats, (..., 'approxDurationMs'), expected_type=float_or_none) or [0]) or None
 
         for fmt in streaming_formats:
             if fmt.get('targetDurationSec') or fmt.get('drmFamilies'):
@@ -2995,12 +2998,14 @@ def _extract_formats(self, streaming_data, video_id, player_url, is_live):
                 itags[itag] = 'https'
                 stream_ids.append(stream_id)
 
-            tbr = float_or_none(
-                fmt.get('averageBitrate') or fmt.get('bitrate'), 1000)
+            tbr = float_or_none(fmt.get('averageBitrate') or fmt.get('bitrate'), 1000)
             language_preference = (
                 10 if audio_track.get('audioIsDefault') and 10
                 else -10 if 'descriptive' in (audio_track.get('displayName') or '').lower() and -10
                 else -1)
+            # Some formats may have much smaller duration than others (possibly damaged during encoding)
+            # Eg: 2-nOtRESiUc Ref: https://github.com/yt-dlp/yt-dlp/issues/2823
+            is_damaged = try_get(fmt, lambda x: float(x['approxDurationMs']) < approx_duration - 10000)
             dct = {
                 'asr': int_or_none(fmt.get('audioSampleRate')),
                 'filesize': int_or_none(fmt.get('contentLength')),
@@ -3009,7 +3014,7 @@ def _extract_formats(self, streaming_data, video_id, player_url, is_live):
                     '%s%s' % (audio_track.get('displayName') or '',
                               ' (default)' if language_preference > 0 else ''),
                     fmt.get('qualityLabel') or quality.replace('audio_quality_', ''),
-                    throttled and 'THROTTLED', delim=', '),
+                    throttled and 'THROTTLED', is_damaged and 'DAMAGED', delim=', '),
                 'source_preference': -10 if throttled else -1,
                 'fps': int_or_none(fmt.get('fps')) or None,
                 'height': height,
@@ -3020,6 +3025,7 @@ def _extract_formats(self, streaming_data, video_id, player_url, is_live):
                 'language': join_nonempty(audio_track.get('id', '').split('.')[0],
                                           'desc' if language_preference < -1 else ''),
                 'language_preference': language_preference,
+                'preference': -10 if is_damaged else None,
             }
             mime_mobj = re.match(
                 r'((?:[^/]+)/(?:[^;]+))(?:;\s*codecs="([^"]+)")?', fmt.get('mimeType') or '')
@@ -3944,13 +3950,14 @@ def _entries(self, tab, item_id, ytcfg, account_syncid, visitor_data):
             break
 
     @staticmethod
-    def _extract_selected_tab(tabs):
+    def _extract_selected_tab(tabs, fatal=True):
         for tab in tabs:
             renderer = dict_get(tab, ('tabRenderer', 'expandableTabRenderer')) or {}
             if renderer.get('selected') is True:
                 return renderer
         else:
-            raise ExtractorError('Unable to find selected tab')
+            if fatal:
+                raise ExtractorError('Unable to find selected tab')
 
     @classmethod
     def _extract_uploader(cls, data):
@@ -4223,7 +4230,7 @@ def _extract_webpage(self, url, item_id, fatal=True):
                     self.report_warning(error_to_compat_str(e))
                     break
 
-                if dict_get(data, ('contents', 'currentVideoEndpoint')):
+                if dict_get(data, ('contents', 'currentVideoEndpoint', 'onResponseReceivedActions')):
                     break
 
                 last_error = 'Incomplete yt initial data received'
@@ -4240,6 +4247,16 @@ def _extract_data(self, url, item_id, ytcfg=None, fatal=True, webpage_fatal=Fals
         if 'webpage' not in self._configuration_arg('skip'):
             webpage, data = self._extract_webpage(url, item_id, fatal=webpage_fatal)
             ytcfg = ytcfg or self.extract_ytcfg(item_id, webpage)
+            # Reject webpage data if redirected to home page without explicitly requesting
+            selected_tab = self._extract_selected_tab(traverse_obj(
+                data, ('contents', 'twoColumnBrowseResultsRenderer', 'tabs'), expected_type=list, default=[]), fatal=False) or {}
+            if (url != 'https://www.youtube.com/feed/recommended'
+                    and selected_tab.get('tabIdentifier') == 'FEwhat_to_watch'  # Home page
+                    and 'no-youtube-channel-redirect' not in self.get_param('compat_opts', [])):
+                msg = 'The channel/playlist does not exist and the URL redirected to youtube.com home page'
+                if fatal:
+                    raise ExtractorError(msg, expected=True)
+                self.report_warning(msg, only_once=True)
         if not data:
             if not ytcfg and self.is_authenticated:
                 msg = 'Playlists that require authentication may not extract correctly without a successful webpage download.'
@@ -4264,7 +4281,7 @@ def _extract_tab_endpoint(self, url, item_id, ytcfg=None, fatal=True, default_cl
                 return self._extract_response(
                     item_id=item_id, query=params, ep=ep, headers=headers,
                     ytcfg=ytcfg, fatal=fatal, default_client=default_client,
-                    check_get_keys=('contents', 'currentVideoEndpoint'))
+                    check_get_keys=('contents', 'currentVideoEndpoint', 'onResponseReceivedActions'))
         err_note = 'Failed to resolve url (does the playlist exist?)'
         if fatal:
             raise ExtractorError(err_note, expected=True)
@@ -4965,6 +4982,10 @@ class YoutubeTabIE(YoutubeTabBaseInfoExtractor):
             'skip_download': True,
             'extractor_args': {'youtubetab': {'skip': ['webpage']}}
         },
+    }, {
+        'note': 'non-standard redirect to regional channel',
+        'url': 'https://www.youtube.com/channel/UCwVVpHQ2Cs9iGJfpdFngePQ',
+        'only_matching': True
     }]
 
     @classmethod
@@ -5037,6 +5058,16 @@ def get_mobj(url):
 
         data, ytcfg = self._extract_data(url, item_id)
 
+        # YouTube may provide a non-standard redirect to the regional channel
+        # See: https://github.com/yt-dlp/yt-dlp/issues/2694
+        redirect_url = traverse_obj(
+            data, ('onResponseReceivedActions', ..., 'navigateAction', 'endpoint', 'commandMetadata', 'webCommandMetadata', 'url'), get_all=False)
+        if redirect_url and 'no-youtube-channel-redirect' not in compat_opts:
+            redirect_url = ''.join((
+                urljoin('https://www.youtube.com', redirect_url), mobj['tab'], mobj['post']))
+            self.to_screen(f'This playlist is likely not available in your region. Following redirect to regional playlist {redirect_url}')
+            return self.url_result(redirect_url, ie=YoutubeTabIE.ie_key())
+
         tabs = traverse_obj(data, ('contents', 'twoColumnBrowseResultsRenderer', 'tabs'), expected_type=list)
         if tabs:
             selected_tab = self._extract_selected_tab(tabs)
diff --git a/yt_dlp/extractor/zee5.py b/yt_dlp/extractor/zee5.py
index 55c225d85..ebe393ec7 100644
--- a/yt_dlp/extractor/zee5.py
+++ b/yt_dlp/extractor/zee5.py
@@ -23,7 +23,7 @@ class Zee5IE(InfoExtractor):
                         zee5:|
                         https?://(?:www\.)?zee5\.com/(?:[^#?]+/)?
                         (?:
-                            (?:tv-shows|kids|zee5originals)(?:/[^#/?]+){3}
+                            (?:tv-shows|kids|web-series|zee5originals)(?:/[^#/?]+){3}
                             |movies/[^#/?]+
                         )/(?P<display_id>[^#/?]+)/
                      )
@@ -82,6 +82,9 @@ class Zee5IE(InfoExtractor):
     }, {
         'url': 'https://www.zee5.com/global/hi/tv-shows/details/kundali-bhagya/0-6-366/kundali-bhagya-march-08-2021/0-1-manual_7g9jv1os7730',
         'only_matching': True
+    }, {
+        'url': 'https://www.zee5.com/web-series/details/mithya/0-6-4z587408/maine-dekhi-hai-uski-mrityu/0-1-6z587412',
+        'only_matching': True
     }]
     _DETAIL_API_URL = 'https://spapi.zee5.com/singlePlayback/getDetails?content_id={}&device_id={}&platform_name=desktop_web&country=IN&check_parental_control=false'
     _DEVICE_ID = 'iIxsxYf40cqO3koIkwzKHZhnJzHN13zb'
@@ -179,7 +182,7 @@ class Zee5SeriesIE(InfoExtractor):
                      (?:
                         zee5:series:|
                         https?://(?:www\.)?zee5\.com/(?:[^#?]+/)?
-                        (?:tv-shows|kids|zee5originals)(?:/[^#/?]+){2}/
+                        (?:tv-shows|web-series|kids|zee5originals)(?:/[^#/?]+){2}/
                      )
                      (?P<id>[^#/?]+)(?:/episodes)?/?(?:$|[?#])
                      '''
@@ -216,6 +219,9 @@ class Zee5SeriesIE(InfoExtractor):
     }, {
         'url': 'https://www.zee5.com/tv-shows/details/chala-hawa-yeu-dya-ladies-zindabaad/0-6-2943/episodes',
         'only_matching': True,
+    }, {
+        'url': 'https://www.zee5.com/web-series/details/mithya/0-6-4z587408',
+        'only_matching': True,
     }]
 
     def _entries(self, show_id):
diff --git a/yt_dlp/options.py b/yt_dlp/options.py
index 2ba7d2601..6fcef98cd 100644
--- a/yt_dlp/options.py
+++ b/yt_dlp/options.py
@@ -1550,11 +1550,11 @@ def _dict_from_options_callback(
             'and (optionally) arguments to be passed to it, separated by a colon ":". '
             'ARGS are a semicolon ";" delimited list of NAME=VALUE. '
             'The "when" argument determines when the postprocessor is invoked. '
-            'It can be one of "pre_process" (after extraction), '
-            '"before_dl" (before video download), "post_process" (after video download; default), '
-            '"after_move" (after moving file to their final locations), '
+            'It can be one of "pre_process" (after video extraction), "after_filter" (after video passes filter), '
+            '"before_dl" (before each video download), "post_process" (after each video download; default), '
+            '"after_move" (after moving video file to it\'s final locations), '
             '"after_video" (after downloading and processing all formats of a video), '
-            'or "playlist" (end of playlist). '
+            'or "playlist" (at end of playlist). '
             'This option can be used multiple times to add different postprocessors'))
 
     sponsorblock = optparse.OptionGroup(parser, 'SponsorBlock Options', description=(
diff --git a/yt_dlp/postprocessor/common.py b/yt_dlp/postprocessor/common.py
index f2467c542..d761c9303 100644
--- a/yt_dlp/postprocessor/common.py
+++ b/yt_dlp/postprocessor/common.py
@@ -103,12 +103,14 @@ def _copy_infodict(self, info_dict):
         return getattr(self._downloader, '_copy_infodict', dict)(info_dict)
 
     @staticmethod
-    def _restrict_to(*, video=True, audio=True, images=True):
+    def _restrict_to(*, video=True, audio=True, images=True, simulated=True):
         allowed = {'video': video, 'audio': audio, 'images': images}
 
         def decorator(func):
             @functools.wraps(func)
             def wrapper(self, info):
+                if not simulated and (self.get_param('simulate') or self.get_param('skip_download')):
+                    return [], info
                 format_type = (
                     'video' if info.get('vcodec') != 'none'
                     else 'audio' if info.get('acodec') != 'none'
diff --git a/yt_dlp/postprocessor/ffmpeg.py b/yt_dlp/postprocessor/ffmpeg.py
index 05eeee2d7..907627381 100644
--- a/yt_dlp/postprocessor/ffmpeg.py
+++ b/yt_dlp/postprocessor/ffmpeg.py
@@ -1055,7 +1055,7 @@ def run(self, info):
 
 
 class FFmpegThumbnailsConvertorPP(FFmpegPostProcessor):
-    SUPPORTED_EXTS = ('jpg', 'png')
+    SUPPORTED_EXTS = ('jpg', 'png', 'webp')
 
     def __init__(self, downloader=None, format=None):
         super(FFmpegThumbnailsConvertorPP, self).__init__(downloader)
@@ -1145,16 +1145,15 @@ def concat_files(self, in_files, out_file):
         super().concat_files(in_files, out_file)
         return in_files
 
-    @PostProcessor._restrict_to(images=False)
+    @PostProcessor._restrict_to(images=False, simulated=False)
     def run(self, info):
         entries = info.get('entries') or []
-        if (self.get_param('skip_download') or not any(entries)
-                or self._only_multi_video and info['_type'] != 'multi_video'):
+        if not any(entries) or (self._only_multi_video and info['_type'] != 'multi_video'):
             return [], info
         elif any(len(entry) > 1 for entry in traverse_obj(entries, (..., 'requested_downloads')) or []):
             raise PostProcessingError('Concatenation is not supported when downloading multiple separate formats')
 
-        in_files = traverse_obj(entries, (..., 'requested_downloads', 0, 'filepath'))
+        in_files = traverse_obj(entries, (..., 'requested_downloads', 0, 'filepath')) or []
         if len(in_files) < len(entries):
             raise PostProcessingError('Aborting concatenation because some downloads failed')
 
diff --git a/yt_dlp/postprocessor/sponsorblock.py b/yt_dlp/postprocessor/sponsorblock.py
index cd48b15ae..e7e04e86e 100644
--- a/yt_dlp/postprocessor/sponsorblock.py
+++ b/yt_dlp/postprocessor/sponsorblock.py
@@ -49,6 +49,9 @@ def _get_sponsor_chapters(self, info, duration):
 
         def duration_filter(s):
             start_end = s['segment']
+            # Ignore entire video segments (https://wiki.sponsor.ajay.app/w/Types).
+            if start_end == (0, 0):
+                return False
             # Ignore milliseconds difference at the start.
             if start_end[0] <= 1:
                 start_end[0] = 0
@@ -89,6 +92,7 @@ def _get_sponsor_segments(self, video_id, service):
         url = f'{self._API_URL}/api/skipSegments/{hash[:4]}?' + compat_urllib_parse_urlencode({
             'service': service,
             'categories': json.dumps(self._categories),
+            'actionTypes': json.dumps(['skip', 'poi'])
         })
         self.write_debug(f'SponsorBlock query: {url}')
         for d in self._get_json(url):
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
index c5489d494..8e9a7dbc8 100644
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -4,6 +4,7 @@
 from __future__ import unicode_literals
 
 import asyncio
+import atexit
 import base64
 import binascii
 import calendar
@@ -2140,18 +2141,28 @@ def _unlock_file(f):
             raise OSError('Unlocking file failed: %r' % ctypes.FormatError())
 
 else:
-    # Some platforms, such as Jython, is missing fcntl
     try:
         import fcntl
 
         def _lock_file(f, exclusive, block):
-            fcntl.flock(f,
-                        fcntl.LOCK_SH if not exclusive
-                        else fcntl.LOCK_EX if block
-                        else fcntl.LOCK_EX | fcntl.LOCK_NB)
+            try:
+                fcntl.flock(f,
+                            fcntl.LOCK_SH if not exclusive
+                            else fcntl.LOCK_EX if block
+                            else fcntl.LOCK_EX | fcntl.LOCK_NB)
+            except BlockingIOError:
+                raise
+            except OSError:  # AOSP does not have flock()
+                fcntl.lockf(f,
+                            fcntl.LOCK_SH if not exclusive
+                            else fcntl.LOCK_EX if block
+                            else fcntl.LOCK_EX | fcntl.LOCK_NB)
 
         def _unlock_file(f):
-            fcntl.flock(f, fcntl.LOCK_UN)
+            try:
+                fcntl.flock(f, fcntl.LOCK_UN)
+            except OSError:
+                fcntl.lockf(f, fcntl.LOCK_UN)
 
     except ImportError:
         UNSUPPORTED_MSG = 'file locking is not supported on this platform'
@@ -2554,6 +2565,13 @@ def url_or_none(url):
     return url if re.match(r'^(?:(?:https?|rt(?:m(?:pt?[es]?|fp)|sp[su]?)|mms|ftps?):)?//', url) else None
 
 
+def request_to_url(req):
+    if isinstance(req, compat_urllib_request.Request):
+        return req.get_full_url()
+    else:
+        return req
+
+
 def strftime_or_none(timestamp, date_format, default=None):
     datetime_object = None
     try:
@@ -2791,13 +2809,14 @@ def __len__(self):
     def __init__(self, pagefunc, pagesize, use_cache=True):
         self._pagefunc = pagefunc
         self._pagesize = pagesize
+        self._pagecount = float('inf')
         self._use_cache = use_cache
         self._cache = {}
 
     def getpage(self, pagenum):
         page_results = self._cache.get(pagenum)
         if page_results is None:
-            page_results = list(self._pagefunc(pagenum))
+            page_results = [] if pagenum > self._pagecount else list(self._pagefunc(pagenum))
         if self._use_cache:
             self._cache[pagenum] = page_results
         return page_results
@@ -2809,7 +2828,7 @@ def _getslice(self, start, end):
         raise NotImplementedError('This method must be implemented by subclasses')
 
     def __getitem__(self, idx):
-        # NOTE: cache must be enabled if this is used
+        assert self._use_cache, 'Indexing PagedList requires cache'
         if not isinstance(idx, int) or idx < 0:
             raise TypeError('indices must be non-negative integers')
         entries = self.getslice(idx, idx + 1)
@@ -2835,7 +2854,11 @@ def _getslice(self, start, end):
                 if (end is not None and firstid <= end <= nextfirstid)
                 else None)
 
-            page_results = self.getpage(pagenum)
+            try:
+                page_results = self.getpage(pagenum)
+            except Exception:
+                self._pagecount = pagenum - 1
+                raise
             if startv != 0 or endv is not None:
                 page_results = page_results[startv:endv]
             yield from page_results
@@ -2855,8 +2878,8 @@ def _getslice(self, start, end):
 
 class InAdvancePagedList(PagedList):
     def __init__(self, pagefunc, pagecount, pagesize):
-        self._pagecount = pagecount
         PagedList.__init__(self, pagefunc, pagesize, True)
+        self._pagecount = pagecount
 
     def _getslice(self, start, end):
         start_page = start // self._pagesize
@@ -3143,6 +3166,8 @@ def fix_kv(m):
 
         return '"%s"' % v
 
+    code = re.sub(r'new Date\((".+")\)', r'\g<1>', code)
+
     return re.sub(r'''(?sx)
         "(?:[^"\\]*(?:\\\\|\\['"nurtbfx/\n]))*[^"\\]*"|
         '(?:[^'\\]*(?:\\\\|\\['"nurtbfx/\n]))*[^'\\]*'|
@@ -3164,7 +3189,7 @@ def q(qid):
     return q
 
 
-POSTPROCESS_WHEN = {'pre_process', 'before_dl', 'after_move', 'post_process', 'after_video', 'playlist'}
+POSTPROCESS_WHEN = {'pre_process', 'after_filter', 'before_dl', 'after_move', 'post_process', 'after_video', 'playlist'}
 
 
 DEFAULT_OUTTMPL = {
@@ -5170,6 +5195,22 @@ def variadic(x, allowed_types=(str, bytes, dict)):
     return x if isinstance(x, collections.abc.Iterable) and not isinstance(x, allowed_types) else (x,)
 
 
+def decode_base(value, digits):
+    # This will convert given base-x string to scalar (long or int)
+    table = {char: index for index, char in enumerate(digits)}
+    result = 0
+    base = len(digits)
+    for chr in value:
+        result *= base
+        result += table[chr]
+    return result
+
+
+def time_seconds(**kwargs):
+    t = datetime.datetime.now(datetime.timezone(datetime.timedelta(**kwargs)))
+    return t.timestamp()
+
+
 # create a JSON Web Signature (jws) with HS256 algorithm
 # the resulting format is in JWS Compact Serialization
 # implemented following JWT https://www.rfc-editor.org/rfc/rfc7519.html
@@ -5226,6 +5267,16 @@ def join_nonempty(*values, delim='-', from_dict=None):
     return delim.join(map(str, filter(None, values)))
 
 
+def parse_http_range(range):
+    """ Parse value of "Range" or "Content-Range" HTTP header into tuple. """
+    if not range:
+        return None, None, None
+    crg = re.search(r'bytes[ =](\d+)-(\d+)?(?:/(\d+))?', range)
+    if not crg:
+        return None, None, None
+    return int(crg.group(1)), int_or_none(crg.group(2)), int_or_none(crg.group(3))
+
+
 class Config:
     own_args = None
     filename = None
@@ -5323,6 +5374,7 @@ def __init__(self, url, headers=None):
         self.conn = compat_websockets.connect(
             url, extra_headers=headers, ping_interval=None,
             close_timeout=float('inf'), loop=self.loop, ping_timeout=float('inf'))
+        atexit.register(self.__exit__, None, None, None)
 
     def __enter__(self):
         self.pool = self.run_with_loop(self.conn.__aenter__(), self.loop)
@@ -5339,7 +5391,7 @@ def __exit__(self, type, value, traceback):
             return self.run_with_loop(self.conn.__aexit__(type, value, traceback), self.loop)
         finally:
             self.loop.close()
-            self.r_cancel_all_tasks(self.loop)
+            self._cancel_all_tasks(self.loop)
 
     # taken from https://github.com/python/cpython/blob/3.9/Lib/asyncio/runners.py with modifications
     # for contributors: If there's any new library using asyncio needs to be run in non-async, move these function out of this class


Test patch:
diff --git a/test/test_YoutubeDL.py b/test/test_YoutubeDL.py
index 34ed814b4..7637297be 100644
--- a/test/test_YoutubeDL.py
+++ b/test/test_YoutubeDL.py
@@ -30,9 +30,7 @@ def __init__(self, *args, **kwargs):
         self.msgs = []
 
     def process_info(self, info_dict):
-        info_dict = info_dict.copy()
-        info_dict.pop('__original_infodict', None)
-        self.downloaded_info_dicts.append(info_dict)
+        self.downloaded_info_dicts.append(info_dict.copy())
 
     def to_screen(self, msg):
         self.msgs.append(msg)
@@ -898,20 +896,6 @@ def run(self, info):
         os.unlink(filename)
 
     def test_match_filter(self):
-        class FilterYDL(YDL):
-            def __init__(self, *args, **kwargs):
-                super(FilterYDL, self).__init__(*args, **kwargs)
-                self.params['simulate'] = True
-
-            def process_info(self, info_dict):
-                super(YDL, self).process_info(info_dict)
-
-            def _match_entry(self, info_dict, incomplete=False):
-                res = super(FilterYDL, self)._match_entry(info_dict, incomplete)
-                if res is None:
-                    self.downloaded_info_dicts.append(info_dict.copy())
-                return res
-
         first = {
             'id': '1',
             'url': TEST_URL,
@@ -939,7 +923,7 @@ def _match_entry(self, info_dict, incomplete=False):
         videos = [first, second]
 
         def get_videos(filter_=None):
-            ydl = FilterYDL({'match_filter': filter_})
+            ydl = YDL({'match_filter': filter_, 'simulate': True})
             for v in videos:
                 ydl.process_ie_result(v, download=True)
             return [v['id'] for v in ydl.downloaded_info_dicts]