Repository: yt-dlp/yt-dlp
Pull request #3775 (2022-05-18 05:56:38)

[youtube] Fix :ytnotifications extractor
<!--
# Please follow the guide below

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like

-->

### Before submitting a *pull request* make sure you have:
- [X] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [X] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [X] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [X] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [X] Fix or improvement to an extractor (Make sure to add/update tests)
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [ ] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))

---

### Description of your *pull request* and other information

Was crashing when channel is None (this occurs on notifications such as premieres)

Also soft hyphens should be removed from the notification title before title extraction since the channel name may contain them (e.g. long channel name). 

This fixes the extractor from crashing, but there are still some issues:
- title for recommended notifications is not extracted. 
   - channel is extracted, but title is in the form "Recommended: \<video title\>"
- no title or channel are extracted for YouTube feature(?) notifications
    - but they may link to a video e.g. https://www.youtube.com/watch?v=EwY6doa-0Bg




pukkandan: What's "YouTube feature notifications"?
coletdjnz: > What's "YouTube feature notifications"?

![image](https://user-images.githubusercontent.com/27879174/168980706-55c3fd96-99aa-4fb6-95cf-b984fa4fb775.png)

krichbanana: For me, premieres work fine like before, but "liked comment" notifications seem to break it.

For more thorough testing, try to get an example of each of these: https://www.youtube.com/account_notifications , and make sure desktop push notifications are enabled.
- Will removing the soft hyphens before extracting the title break the title regex?
- Should we be trying to extract every video id if one can be found? In my initial PR there was "nothing to be done about comments" per feedback, but since the notification should link to a video (and comment) we should be able add that video to our results. (see also the next main point)
  - (you can extract videos from comment notifications with `('navigationEndpoint', 'getCommentsFromInboxCommand', 'videoId')`)
- Since most (all?) notifications can be turned off, we could iterate the menu entries for the type of notification that is received. Notably, you can turn off both channel notifications and recommendation notifications for recommended videos.   We *could* use this information to tag the entries in the result so that we can e.g. filter only for posts. Some examples:
  - "Turn off all from ", "<channel name>"
  - "Turn off all recommendation notifications"
  - "Turn off comment activity"
  - "Turn off members-only posts from ", "<channel name>"
- Before my code was cleaned up, I had this regex which extracts the title on a simpler basis:
`        title = self._search_regex(r'.*?: (.*)', status_text, 'status text', default=None, fatal=False)`
   we could use this regex as a fallback if the initial title extraction failed. The "titles" won't necessarily match the video, though.


Patch:
diff --git a/Makefile b/Makefile
index f12282edc..4c16e88ad 100644
--- a/Makefile
+++ b/Makefile
@@ -129,7 +129,7 @@ completions/fish/yt-dlp.fish: yt_dlp/*.py yt_dlp/*/*.py devscripts/fish-completi
 	mkdir -p completions/fish
 	$(PYTHON) devscripts/fish-completion.py
 
-_EXTRACTOR_FILES = $(shell find yt_dlp/extractor -iname '*.py' -and -not -iname 'lazy_extractors.py')
+_EXTRACTOR_FILES = $(shell find yt_dlp/extractor -name '*.py' -and -not -name 'lazy_extractors.py')
 yt_dlp/extractor/lazy_extractors.py: devscripts/make_lazy_extractors.py devscripts/lazy_load_template.py $(_EXTRACTOR_FILES)
 	$(PYTHON) devscripts/make_lazy_extractors.py $@
 
diff --git a/README.md b/README.md
index e71a150fd..912a42f72 100644
--- a/README.md
+++ b/README.md
@@ -111,7 +111,7 @@ # NEW FEATURES
 
 * **Output template improvements**: Output templates can now have date-time formatting, numeric offsets, object traversal etc. See [output template](#output-template) for details. Even more advanced operations can also be done with the help of `--parse-metadata` and `--replace-in-metadata`
 
-* **Other new options**: Many new options have been added such as `--concat-playlist`, `--print`, `--wait-for-video`, `--sleep-requests`, `--convert-thumbnails`, `--write-link`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc
+* **Other new options**: Many new options have been added such as `--alias`, `--print`, `--concat-playlist`, `--wait-for-video`, `--retry-sleep`, `--sleep-requests`, `--convert-thumbnails`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc
 
 * **Improvements**: Regex and other operators in `--format`/`--match-filter`, multiple `--postprocessor-args` and `--downloader-args`, faster archive checking, more [format selection options](#format-selection), merge multi-video/audio, multiple `--config-locations`, `--exec` at different stages, etc
 
@@ -146,12 +146,12 @@ ### Differences in default behavior
 * Thumbnail embedding in `mp4` is done with mutagen if possible. Use `--compat-options embed-thumbnail-atomicparsley` to force the use of AtomicParsley instead
 * Some private fields such as filenames are removed by default from the infojson. Use `--no-clean-infojson` or `--compat-options no-clean-infojson` to revert this
 * When `--embed-subs` and `--write-subs` are used together, the subtitles are written to disk and also embedded in the media file. You can use just `--embed-subs` to embed the subs and automatically delete the separate file. See [#630 (comment)](https://github.com/yt-dlp/yt-dlp/issues/630#issuecomment-893659460) for more info. `--compat-options no-keep-subs` can be used to revert this
-* `certifi` will be used for SSL root certificates, if installed. If you want to use system certificates (e.g. self-signed), use `--compat-options no-certifi`
+* `certifi` will be used for SSL root certificates, if installed. If you want to use only system certificates, use `--compat-options no-certifi`
 * youtube-dl tries to remove some superfluous punctuations from filenames. While this can sometimes be helpfull, it is often undesirable. So yt-dlp tries to keep the fields in the filenames as close to their original values as possible. You can use `--compat-options filename-sanitization` to revert to youtube-dl's behavior
 
 For ease of use, a few more compat options are available:
 
-* `--compat-options all`: Use all compat options
+* `--compat-options all`: Use all compat options (Do NOT use)
 * `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams`
 * `--compat-options youtube-dlc`: Same as `--compat-options all,-no-live-chat,-no-youtube-channel-redirect`
 
@@ -358,8 +358,9 @@ ## General Options:
                                     defined in the current file
     --config-locations PATH         Location of the main configuration file;
                                     either the path to the config or its
-                                    containing directory. Can be used multiple
-                                    times and inside other configuration files
+                                    containing directory ("-" for stdin). Can be
+                                    used multiple times and inside other
+                                    configuration files
     --flat-playlist                 Do not extract the videos of a playlist,
                                     only list them
     --no-flat-playlist              Extract the videos of a playlist
@@ -1002,9 +1003,9 @@ ## Post-Processing Options:
     --no-remove-chapters            Do not remove any chapters from the file
                                     (default)
     --force-keyframes-at-cuts       Force keyframes around chapters when
-                                    removing/splitting them. The resulting video
-                                    may have fewer artifacts around the cuts,
-                                    but is very slow due to needing a re-encode
+                                    removing/splitting them. This is slow due to
+                                    needing a re-encode, but the resulting video
+                                    may have fewer artifacts around the cuts
     --no-force-keyframes-at-cuts    Do not force keyframes around the chapters
                                     when cutting/splitting (default)
     --use-postprocessor NAME[:ARGS]
@@ -2021,6 +2022,7 @@ #### Developer options
 These options are not intended to be used by the end-user
 
     --test                           Download only part of video for testing extractors
+    --load-pages                     Load pages dumped by --write-pages
     --youtube-print-sig-code         For testing youtube signatures
     --allow-unplayable-formats       List unplayable formats also
     --no-allow-unplayable-formats    Default
diff --git a/devscripts/make_readme.py b/devscripts/make_readme.py
index 15c4a7c7d..42578cb0a 100755
--- a/devscripts/make_readme.py
+++ b/devscripts/make_readme.py
@@ -12,6 +12,8 @@
 OPTIONS_END = 'CONFIGURATION'
 EPILOG_START = 'See full documentation'
 
+DISABLE_PATCH = object()
+
 
 def take_section(text, start=None, end=None, *, shift=0):
     return text[
@@ -21,7 +23,7 @@ def take_section(text, start=None, end=None, *, shift=0):
 
 
 def apply_patch(text, patch):
-    return re.sub(*patch, text)
+    return text if patch[0] is DISABLE_PATCH else re.sub(*patch, text)
 
 
 options = take_section(sys.stdin.read(), f'\n  {OPTIONS_START}', f'\n{EPILOG_START}', shift=1)
@@ -38,11 +40,15 @@ def apply_patch(text, patch):
         rf'({delim[:-1]})? (?P<label>\[\S+\] )?(?P<url>https?({delim})?:({delim})?/({delim})?/(({delim})?\S+)+)\s',
         lambda mobj: ''.join((delim, mobj.group('label') or '', re.sub(r'\s+', '', mobj.group('url')), '\n'))
     ),
-    # This creates issues with prepare_manpage
-    # (  # Avoid newline when a space is available b/w switch and description
-    #     r'(?m)^(\s{4}-.{%d})(%s)' % (switch_col_width - 6, delim),
-    #     r'\1 '
-    # ),
+    (  # Do not split "words"
+        rf'(?m)({delim}\S+)+$',
+        lambda mobj: ''.join((delim, mobj.group(0).replace(delim, '')))
+    ),
+    (  # Avoid newline when a space is available b/w switch and description
+        DISABLE_PATCH,  # This creates issues with prepare_manpage
+        r'(?m)^(\s{4}-.{%d})(%s)' % (switch_col_width - 6, delim),
+        r'\1 '
+    ),
 )
 
 with open(README_FILE, encoding='utf-8') as f:
diff --git a/setup.cfg b/setup.cfg
index 4780ee95b..946d7b122 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -1,17 +1,21 @@
 [wheel]
 universal = true
 
+
 [flake8]
-exclude = build,venv,.tox,.git
+exclude = build,venv,.tox,.git,.pytest_cache
 ignore = E402,E501,E731,E741,W503
+max_line_length = 120
 per_file_ignores =
-    ./devscripts/lazy_load_template.py: F401
+    devscripts/lazy_load_template.py: F401
+
 
 [tool:pytest]
 addopts = -ra -v --strict-markers
 markers =
     download
 
+
 [tox:tox]
 skipsdist = true
 envlist = py{36,37,38,39,310},pypy{36,37,38,39}
@@ -25,6 +29,7 @@ passenv = HOME  # For test_compat_expanduser
 setenv =
     # PYTHONWARNINGS = error  # Catches PIP's warnings too
 
+
 [isort]
 py_version = 36
 multi_line_output = VERTICAL_HANGING_INDENT
diff --git a/yt-dlp.sh b/yt-dlp.sh
index 0321a3362..22a69250c 100755
--- a/yt-dlp.sh
+++ b/yt-dlp.sh
@@ -1,2 +1,2 @@
-#!/bin/sh
+#!/usr/bin/env sh
 exec "${PYTHON:-python3}" -bb -Werror -Xdev "$(dirname "$(realpath "$0")")/yt_dlp/__main__.py" "$@"
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
index b1753241d..5aae25707 100644
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -27,6 +27,7 @@
 
 from .cache import Cache
 from .compat import (
+    HAS_LEGACY as compat_has_legacy,
     compat_get_terminal_size,
     compat_os_name,
     compat_shlex_quote,
@@ -591,7 +592,10 @@ def check_deprecated(param, option, suggestion):
         for msg in self.params.get('_deprecation_warnings', []):
             self.deprecation_warning(msg)
 
-        if 'list-formats' in self.params.get('compat_opts', []):
+        self.params['compat_opts'] = set(self.params.get('compat_opts', ()))
+        if not compat_has_legacy:
+            self.params['compat_opts'].add('no-compat-legacy')
+        if 'list-formats' in self.params['compat_opts']:
             self.params['listformats_table'] = False
 
         if 'overwrites' not in self.params and self.params.get('nooverwrites') is not None:
@@ -788,9 +792,9 @@ def to_stdout(self, message, skip_eol=False, quiet=None):
         """Print message to stdout"""
         if quiet is not None:
             self.deprecation_warning('"YoutubeDL.to_stdout" no longer accepts the argument quiet. Use "YoutubeDL.to_screen" instead')
-        self._write_string(
-            '%s%s' % (self._bidi_workaround(message), ('' if skip_eol else '\n')),
-            self._out_files.out)
+        if skip_eol is not False:
+            self.deprecation_warning('"YoutubeDL.to_stdout" no longer accepts the argument skip_eol. Use "YoutubeDL.to_screen" instead')
+        self._write_string(f'{self._bidi_workaround(message)}\n', self._out_files.out)
 
     def to_screen(self, message, skip_eol=False, quiet=None):
         """Print message to screen if not in quiet mode"""
@@ -942,7 +946,7 @@ def write_debug(self, message, only_once=False):
         '''Log debug message or Print message to stderr'''
         if not self.params.get('verbose', False):
             return
-        message = '[debug] %s' % message
+        message = f'[debug] {message}'
         if self.params.get('logger'):
             self.params['logger'].debug(message)
         else:
@@ -1136,7 +1140,7 @@ def get_value(mdict):
         def filename_sanitizer(key, value, restricted=self.params.get('restrictfilenames')):
             return sanitize_filename(str(value), restricted=restricted, is_id=(
                 bool(re.search(r'(^|[_.])id(\.|$)', key))
-                if 'filename-sanitization' in self.params.get('compat_opts', [])
+                if 'filename-sanitization' in self.params['compat_opts']
                 else NO_DEFAULT))
 
         sanitizer = sanitize if callable(sanitize) else filename_sanitizer
@@ -1775,7 +1779,7 @@ def get_entry(i):
         max_failures = self.params.get('skip_playlist_after_errors') or float('inf')
         for i, entry_tuple in enumerate(entries, 1):
             playlist_index, entry = entry_tuple
-            if 'playlist-index' in self.params.get('compat_opts', []):
+            if 'playlist-index' in self.params['compat_opts']:
                 playlist_index = playlistitems[i - 1] if playlistitems else i + playliststart - 1
             self.to_screen('[download] Downloading video %s of %s' % (
                 self._format_screen(i, self.Styles.ID), self._format_screen(n_entries, self.Styles.EMPHASIS)))
@@ -1906,7 +1910,7 @@ def _check_formats(self, formats):
             temp_file.close()
             try:
                 success, _ = self.dl(temp_file.name, f, test=True)
-            except (DownloadError, IOError, OSError, ValueError) + network_exceptions:
+            except (DownloadError, OSError, ValueError) + network_exceptions:
                 success = False
             finally:
                 if os.path.exists(temp_file.name):
@@ -1935,7 +1939,7 @@ def can_merge():
         compat = (
             prefer_best
             or self.params.get('allow_multiple_audio_streams', False)
-            or 'format-spec' in self.params.get('compat_opts', []))
+            or 'format-spec' in self.params['compat_opts'])
 
         return (
             'best/bestvideo+bestaudio' if prefer_best
@@ -3652,8 +3656,8 @@ def get_encoding(stream):
             write_debug('Plugins: %s' % [
                 '%s%s' % (klass.__name__, '' if klass.__name__ == name else f' as {name}')
                 for name, klass in itertools.chain(plugin_extractors.items(), plugin_postprocessors.items())])
-        if self.params.get('compat_opts'):
-            write_debug('Compatibility options: %s' % ', '.join(self.params.get('compat_opts')))
+        if self.params['compat_opts']:
+            write_debug('Compatibility options: %s' % ', '.join(self.params['compat_opts']))
 
         if source == 'source':
             try:
diff --git a/yt_dlp/__init__.py b/yt_dlp/__init__.py
index 66fee95cd..d0abc395a 100644
--- a/yt_dlp/__init__.py
+++ b/yt_dlp/__init__.py
@@ -9,7 +9,7 @@
 import re
 import sys
 
-from .compat import compat_getpass, compat_os_name, compat_shlex_quote
+from .compat import compat_getpass, compat_shlex_quote
 from .cookies import SUPPORTED_BROWSERS, SUPPORTED_KEYRINGS
 from .downloader import FileDownloader
 from .extractor import GenericIE, list_extractor_classes
@@ -42,6 +42,7 @@
     parse_duration,
     preferredencoding,
     read_batch_urls,
+    read_stdin,
     render_table,
     setproctitle,
     std_headers,
@@ -63,14 +64,9 @@ def get_urls(urls, batchfile, verbose):
     batch_urls = []
     if batchfile is not None:
         try:
-            if batchfile == '-':
-                write_string('Reading URLs from stdin - EOF (%s) to end:\n' % (
-                    'Ctrl+Z' if compat_os_name == 'nt' else 'Ctrl+D'))
-                batchfd = sys.stdin
-            else:
-                batchfd = open(
-                    expand_path(batchfile), encoding='utf-8', errors='ignore')
-            batch_urls = read_batch_urls(batchfd)
+            batch_urls = read_batch_urls(
+                read_stdin('URLs') if batchfile == '-'
+                else open(expand_path(batchfile), encoding='utf-8', errors='ignore'))
             if verbose:
                 write_string('[debug] Batch file urls: ' + repr(batch_urls) + '\n')
         except OSError:
@@ -762,6 +758,7 @@ def parse_options(argv=None):
         'verbose': opts.verbose,
         'dump_intermediate_pages': opts.dump_intermediate_pages,
         'write_pages': opts.write_pages,
+        'load_pages': opts.load_pages,
         'test': opts.test,
         'keepvideo': opts.keepvideo,
         'min_filesize': opts.min_filesize,
diff --git a/yt_dlp/compat/__init__.py b/yt_dlp/compat/__init__.py
index c02e843d4..35875ed20 100644
--- a/yt_dlp/compat/__init__.py
+++ b/yt_dlp/compat/__init__.py
@@ -9,8 +9,13 @@
 
 
 # XXX: Implement this the same way as other DeprecationWarnings without circular import
-passthrough_module(__name__, '._legacy', callback=lambda attr: warnings.warn(
-    DeprecationWarning(f'{__name__}.{attr} is deprecated'), stacklevel=2))
+try:
+    passthrough_module(__name__, '._legacy', callback=lambda attr: warnings.warn(
+        DeprecationWarning(f'{__name__}.{attr} is deprecated'), stacklevel=2))
+    HAS_LEGACY = True
+except ModuleNotFoundError:
+    # Keep working even without _legacy module
+    HAS_LEGACY = False
 del passthrough_module
 
 
diff --git a/yt_dlp/downloader/__init__.py b/yt_dlp/downloader/__init__.py
index 5aba303dd..3b4a82635 100644
--- a/yt_dlp/downloader/__init__.py
+++ b/yt_dlp/downloader/__init__.py
@@ -1,4 +1,3 @@
-from ..compat import compat_str
 from ..utils import NO_DEFAULT, determine_protocol
 
 
@@ -91,7 +90,7 @@ def _get_suitable_downloader(info_dict, protocol, params, default):
     info_dict['protocol'] = protocol
     downloaders = params.get('external_downloader')
     external_downloader = (
-        downloaders if isinstance(downloaders, compat_str) or downloaders is None
+        downloaders if isinstance(downloaders, str) or downloaders is None
         else downloaders.get(shorten_protocol_name(protocol, True), downloaders.get('default')))
 
     if external_downloader is None:
diff --git a/yt_dlp/downloader/common.py b/yt_dlp/downloader/common.py
index e2266cfb0..cd30d1eff 100644
--- a/yt_dlp/downloader/common.py
+++ b/yt_dlp/downloader/common.py
@@ -283,9 +283,9 @@ def _prepare_multiline_status(self, lines=1):
         elif self.ydl.params.get('logger'):
             self._multiline = MultilineLogger(self.ydl.params['logger'], lines)
         elif self.params.get('progress_with_newline'):
-            self._multiline = BreaklineStatusPrinter(self.ydl._out_files.screen, lines)
+            self._multiline = BreaklineStatusPrinter(self.ydl._out_files.out, lines)
         else:
-            self._multiline = MultilinePrinter(self.ydl._out_files.screen, lines, not self.params.get('quiet'))
+            self._multiline = MultilinePrinter(self.ydl._out_files.out, lines, not self.params.get('quiet'))
         self._multiline.allow_colors = self._multiline._HAVE_FULLCAP and not self.params.get('no_color')
 
     def _finish_multiline_status(self):
diff --git a/yt_dlp/extractor/animelab.py b/yt_dlp/extractor/animelab.py
deleted file mode 100644
index fe2b70aed..000000000
--- a/yt_dlp/extractor/animelab.py
+++ /dev/null
@@ -1,270 +0,0 @@
-from .common import InfoExtractor
-
-from ..utils import (
-    ExtractorError,
-    urlencode_postdata,
-    int_or_none,
-    str_or_none,
-    determine_ext,
-)
-
-from ..compat import compat_HTTPError
-
-
-class AnimeLabBaseIE(InfoExtractor):
-    _LOGIN_URL = 'https://www.animelab.com/login'
-    _NETRC_MACHINE = 'animelab'
-    _LOGGED_IN = False
-
-    def _is_logged_in(self, login_page=None):
-        if not self._LOGGED_IN:
-            if not login_page:
-                login_page = self._download_webpage(self._LOGIN_URL, None, 'Downloading login page')
-            AnimeLabBaseIE._LOGGED_IN = 'Sign In' not in login_page
-        return self._LOGGED_IN
-
-    def _perform_login(self, username, password):
-        if self._is_logged_in():
-            return
-
-        login_form = {
-            'email': username,
-            'password': password,
-        }
-
-        try:
-            response = self._download_webpage(
-                self._LOGIN_URL, None, 'Logging in', 'Wrong login info',
-                data=urlencode_postdata(login_form),
-                headers={'Content-Type': 'application/x-www-form-urlencoded'})
-        except ExtractorError as e:
-            if isinstance(e.cause, compat_HTTPError) and e.cause.code == 400:
-                raise ExtractorError('Unable to log in (wrong credentials?)', expected=True)
-            raise
-
-        if not self._is_logged_in(response):
-            raise ExtractorError('Unable to login (cannot verify if logged in)')
-
-    def _real_initialize(self):
-        if not self._is_logged_in():
-            self.raise_login_required('Login is required to access any AnimeLab content')
-
-
-class AnimeLabIE(AnimeLabBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?animelab\.com/player/(?P<id>[^/]+)'
-
-    _TEST = {
-        'url': 'https://www.animelab.com/player/fullmetal-alchemist-brotherhood-episode-42',
-        'md5': '05bde4b91a5d1ff46ef5b94df05b0f7f',
-        'info_dict': {
-            'id': '383',
-            'ext': 'mp4',
-            'display_id': 'fullmetal-alchemist-brotherhood-episode-42',
-            'title': 'Fullmetal Alchemist: Brotherhood - Episode 42 - Signs of a Counteroffensive',
-            'description': 'md5:103eb61dd0a56d3dfc5dbf748e5e83f4',
-            'series': 'Fullmetal Alchemist: Brotherhood',
-            'episode': 'Signs of a Counteroffensive',
-            'episode_number': 42,
-            'duration': 1469,
-            'season': 'Season 1',
-            'season_number': 1,
-            'season_id': '38',
-        },
-        'params': {
-            # Ensure the same video is downloaded whether the user is premium or not
-            'format': '[format_id=21711_yeshardsubbed_ja-JP][height=480]',
-        },
-    }
-
-    def _real_extract(self, url):
-        display_id = self._match_id(url)
-
-        # unfortunately we can get different URLs for the same formats
-        # e.g. if we are using a "free" account so no dubs available
-        # (so _remove_duplicate_formats is not effective)
-        # so we use a dictionary as a workaround
-        formats = {}
-        for language_option_url in ('https://www.animelab.com/player/%s/subtitles',
-                                    'https://www.animelab.com/player/%s/dubbed'):
-            actual_url = language_option_url % display_id
-            webpage = self._download_webpage(actual_url, display_id, 'Downloading URL ' + actual_url)
-
-            video_collection = self._parse_json(self._search_regex(r'new\s+?AnimeLabApp\.VideoCollection\s*?\((.*?)\);', webpage, 'AnimeLab VideoCollection'), display_id)
-            position = int_or_none(self._search_regex(r'playlistPosition\s*?=\s*?(\d+)', webpage, 'Playlist Position'))
-
-            raw_data = video_collection[position]['videoEntry']
-
-            video_id = str_or_none(raw_data['id'])
-
-            # create a title from many sources (while grabbing other info)
-            # TODO use more fallback sources to get some of these
-            series = raw_data.get('showTitle')
-            video_type = raw_data.get('videoEntryType', {}).get('name')
-            episode_number = raw_data.get('episodeNumber')
-            episode_name = raw_data.get('name')
-
-            title_parts = (series, video_type, episode_number, episode_name)
-            if None not in title_parts:
-                title = '%s - %s %s - %s' % title_parts
-            else:
-                title = episode_name
-
-            description = raw_data.get('synopsis') or self._og_search_description(webpage, default=None)
-
-            duration = int_or_none(raw_data.get('duration'))
-
-            thumbnail_data = raw_data.get('images', [])
-            thumbnails = []
-            for thumbnail in thumbnail_data:
-                for instance in thumbnail['imageInstances']:
-                    image_data = instance.get('imageInfo', {})
-                    thumbnails.append({
-                        'id': str_or_none(image_data.get('id')),
-                        'url': image_data.get('fullPath'),
-                        'width': image_data.get('width'),
-                        'height': image_data.get('height'),
-                    })
-
-            season_data = raw_data.get('season', {}) or {}
-            season = str_or_none(season_data.get('name'))
-            season_number = int_or_none(season_data.get('seasonNumber'))
-            season_id = str_or_none(season_data.get('id'))
-
-            for video_data in raw_data['videoList']:
-                current_video_list = {}
-                current_video_list['language'] = video_data.get('language', {}).get('languageCode')
-
-                is_hardsubbed = video_data.get('hardSubbed')
-
-                for video_instance in video_data['videoInstances']:
-                    httpurl = video_instance.get('httpUrl')
-                    url = httpurl if httpurl else video_instance.get('rtmpUrl')
-                    if url is None:
-                        # this video format is unavailable to the user (not premium etc.)
-                        continue
-
-                    current_format = current_video_list.copy()
-
-                    format_id_parts = []
-
-                    format_id_parts.append(str_or_none(video_instance.get('id')))
-
-                    if is_hardsubbed is not None:
-                        if is_hardsubbed:
-                            format_id_parts.append('yeshardsubbed')
-                        else:
-                            format_id_parts.append('nothardsubbed')
-
-                    format_id_parts.append(current_format['language'])
-
-                    format_id = '_'.join([x for x in format_id_parts if x is not None])
-
-                    ext = determine_ext(url)
-                    if ext == 'm3u8':
-                        for format_ in self._extract_m3u8_formats(
-                                url, video_id, m3u8_id=format_id, fatal=False):
-                            formats[format_['format_id']] = format_
-                        continue
-                    elif ext == 'mpd':
-                        for format_ in self._extract_mpd_formats(
-                                url, video_id, mpd_id=format_id, fatal=False):
-                            formats[format_['format_id']] = format_
-                        continue
-
-                    current_format['url'] = url
-                    quality_data = video_instance.get('videoQuality')
-                    if quality_data:
-                        quality = quality_data.get('name') or quality_data.get('description')
-                    else:
-                        quality = None
-
-                    height = None
-                    if quality:
-                        height = int_or_none(self._search_regex(r'(\d+)p?$', quality, 'Video format height', default=None))
-
-                    if height is None:
-                        self.report_warning('Could not get height of video')
-                    else:
-                        current_format['height'] = height
-                    current_format['format_id'] = format_id
-
-                    formats[current_format['format_id']] = current_format
-
-        formats = list(formats.values())
-        self._sort_formats(formats)
-
-        return {
-            'id': video_id,
-            'display_id': display_id,
-            'title': title,
-            'description': description,
-            'series': series,
-            'episode': episode_name,
-            'episode_number': int_or_none(episode_number),
-            'thumbnails': thumbnails,
-            'duration': duration,
-            'formats': formats,
-            'season': season,
-            'season_number': season_number,
-            'season_id': season_id,
-        }
-
-
-class AnimeLabShowsIE(AnimeLabBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?animelab\.com/shows/(?P<id>[^/]+)'
-
-    _TEST = {
-        'url': 'https://www.animelab.com/shows/attack-on-titan',
-        'info_dict': {
-            'id': '45',
-            'title': 'Attack on Titan',
-            'description': 'md5:989d95a2677e9309368d5cf39ba91469',
-        },
-        'playlist_count': 59,
-        'skip': 'All AnimeLab content requires authentication',
-    }
-
-    def _real_extract(self, url):
-        _BASE_URL = 'http://www.animelab.com'
-        _SHOWS_API_URL = '/api/videoentries/show/videos/'
-        display_id = self._match_id(url)
-
-        webpage = self._download_webpage(url, display_id, 'Downloading requested URL')
-
-        show_data_str = self._search_regex(r'({"id":.*}),\svideoEntry', webpage, 'AnimeLab show data')
-        show_data = self._parse_json(show_data_str, display_id)
-
-        show_id = str_or_none(show_data.get('id'))
-        title = show_data.get('name')
-        description = show_data.get('shortSynopsis') or show_data.get('longSynopsis')
-
-        entries = []
-        for season in show_data['seasons']:
-            season_id = season['id']
-            get_data = urlencode_postdata({
-                'seasonId': season_id,
-                'limit': 1000,
-            })
-            # despite using urlencode_postdata, we are sending a GET request
-            target_url = _BASE_URL + _SHOWS_API_URL + show_id + "?" + get_data.decode('utf-8')
-            response = self._download_webpage(
-                target_url,
-                None, 'Season id %s' % season_id)
-
-            season_data = self._parse_json(response, display_id)
-
-            for video_data in season_data['list']:
-                entries.append(self.url_result(
-                    _BASE_URL + '/player/' + video_data['slug'], 'AnimeLab',
-                    str_or_none(video_data.get('id')), video_data.get('name')
-                ))
-
-        return {
-            '_type': 'playlist',
-            'id': show_id,
-            'title': title,
-            'description': description,
-            'entries': entries,
-        }
-
-# TODO implement myqueue
diff --git a/yt_dlp/extractor/bloomberg.py b/yt_dlp/extractor/bloomberg.py
index c0aaeae02..c842c342c 100644
--- a/yt_dlp/extractor/bloomberg.py
+++ b/yt_dlp/extractor/bloomberg.py
@@ -7,13 +7,11 @@ class BloombergIE(InfoExtractor):
     _VALID_URL = r'https?://(?:www\.)?bloomberg\.com/(?:[^/]+/)*(?P<id>[^/?#]+)'
 
     _TESTS = [{
-        'url': 'http://www.bloomberg.com/news/videos/b/aaeae121-5949-481e-a1ce-4562db6f5df2',
-        # The md5 checksum changes
+        'url': 'https://www.bloomberg.com/news/videos/2021-09-14/apple-unveils-the-new-iphone-13-stock-doesn-t-move-much-video',
         'info_dict': {
-            'id': 'qurhIVlJSB6hzkVi229d8g',
+            'id': 'V8cFcYMxTHaMcEiiYVr39A',
             'ext': 'flv',
-            'title': 'Shah\'s Presentation on Foreign-Exchange Strategies',
-            'description': 'md5:a8ba0302912d03d246979735c17d2761',
+            'title': 'Apple Unveils the New IPhone 13, Stock Doesn\'t Move Much',
         },
         'params': {
             'format': 'best[format_id^=hds]',
@@ -57,7 +55,7 @@ def _real_extract(self, url):
         title = re.sub(': Video$', '', self._og_search_title(webpage))
 
         embed_info = self._download_json(
-            'http://www.bloomberg.com/api/embed?id=%s' % video_id, video_id)
+            'http://www.bloomberg.com/multimedia/api/embed?id=%s' % video_id, video_id)
         formats = []
         for stream in embed_info['streams']:
             stream_url = stream.get('url')
diff --git a/yt_dlp/extractor/common.py b/yt_dlp/extractor/common.py
index eee908089..71e982f02 100644
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -610,8 +610,7 @@ def _initialize_geo_bypass(self, geo_bypass_context):
 
             if ip_block:
                 self._x_forwarded_for_ip = GeoUtils.random_ipv4(ip_block)
-                self._downloader.write_debug(
-                    '[debug] Using fake IP %s as X-Forwarded-For' % self._x_forwarded_for_ip)
+                self.write_debug(f'Using fake IP {self._x_forwarded_for_ip} as X-Forwarded-For')
                 return
 
             # Path 2: bypassing based on country code
@@ -725,6 +724,13 @@ def __can_accept_status_code(err, expected_status):
         else:
             return err.code in variadic(expected_status)
 
+    def _create_request(self, url_or_request, data=None, headers={}, query={}):
+        if isinstance(url_or_request, compat_urllib_request.Request):
+            return update_Request(url_or_request, data=data, headers=headers, query=query)
+        if query:
+            url_or_request = update_url_query(url_or_request, query)
+        return sanitized_Request(url_or_request, data, headers)
+
     def _request_webpage(self, url_or_request, video_id, note=None, errnote=None, fatal=True, data=None, headers={}, query={}, expected_status=None):
         """
         Return the response handle.
@@ -756,16 +762,8 @@ def _request_webpage(self, url_or_request, video_id, note=None, errnote=None, fa
             if 'X-Forwarded-For' not in headers:
                 headers['X-Forwarded-For'] = self._x_forwarded_for_ip
 
-        if isinstance(url_or_request, compat_urllib_request.Request):
-            url_or_request = update_Request(
-                url_or_request, data=data, headers=headers, query=query)
-        else:
-            if query:
-                url_or_request = update_url_query(url_or_request, query)
-            if data is not None or headers:
-                url_or_request = sanitized_Request(url_or_request, data, headers)
         try:
-            return self._downloader.urlopen(url_or_request)
+            return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
         except network_exceptions as err:
             if isinstance(err, compat_urllib_error.HTTPError):
                 if self.__can_accept_status_code(err, expected_status):
@@ -792,8 +790,35 @@ def _download_webpage_handle(self, url_or_request, video_id, note=None, errnote=
         """
         Return a tuple (page content as string, URL handle).
 
-        See _download_webpage docstring for arguments specification.
+        Arguments:
+        url_or_request -- plain text URL as a string or
+            a compat_urllib_request.Requestobject
+        video_id -- Video/playlist/item identifier (string)
+
+        Keyword arguments:
+        note -- note printed before downloading (string)
+        errnote -- note printed in case of an error (string)
+        fatal -- flag denoting whether error should be considered fatal,
+            i.e. whether it should cause ExtractionError to be raised,
+            otherwise a warning will be reported and extraction continued
+        encoding -- encoding for a page content decoding, guessed automatically
+            when not explicitly specified
+        data -- POST data (bytes)
+        headers -- HTTP headers (dict)
+        query -- URL query (dict)
+        expected_status -- allows to accept failed HTTP requests (non 2xx
+            status code) by explicitly specifying a set of accepted status
+            codes. Can be any of the following entities:
+                - an integer type specifying an exact failed status code to
+                  accept
+                - a list or a tuple of integer types specifying a list of
+                  failed status codes to accept
+                - a callable accepting an actual failed status code and
+                  returning True if it should be accepted
+            Note that this argument does not affect success status codes (2xx)
+            which are always accepted.
         """
+
         # Strip hashes from the URL (#1038)
         if isinstance(url_or_request, (compat_str, str)):
             url_or_request = url_or_request.partition('#')[0]
@@ -850,140 +875,48 @@ def __check_blocked(self, content):
                 'Visit http://blocklist.rkn.gov.ru/ for a block reason.',
                 expected=True)
 
+    def _request_dump_filename(self, url, video_id):
+        basen = f'{video_id}_{url}'
+        trim_length = self.get_param('trim_file_name') or 240
+        if len(basen) > trim_length:
+            h = '___' + hashlib.md5(basen.encode('utf-8')).hexdigest()
+            basen = basen[:trim_length - len(h)] + h
+        filename = sanitize_filename(f'{basen}.dump', restricted=True)
+        # Working around MAX_PATH limitation on Windows (see
+        # http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx)
+        if compat_os_name == 'nt':
+            absfilepath = os.path.abspath(filename)
+            if len(absfilepath) > 259:
+                filename = fR'\\?\{absfilepath}'
+        return filename
+
+    def __decode_webpage(self, webpage_bytes, encoding, headers):
+        if not encoding:
+            encoding = self._guess_encoding_from_content(headers.get('Content-Type', ''), webpage_bytes)
+        try:
+            return webpage_bytes.decode(encoding, 'replace')
+        except LookupError:
+            return webpage_bytes.decode('utf-8', 'replace')
+
     def _webpage_read_content(self, urlh, url_or_request, video_id, note=None, errnote=None, fatal=True, prefix=None, encoding=None):
-        content_type = urlh.headers.get('Content-Type', '')
         webpage_bytes = urlh.read()
         if prefix is not None:
             webpage_bytes = prefix + webpage_bytes
-        if not encoding:
-            encoding = self._guess_encoding_from_content(content_type, webpage_bytes)
         if self.get_param('dump_intermediate_pages', False):
             self.to_screen('Dumping request to ' + urlh.geturl())
             dump = base64.b64encode(webpage_bytes).decode('ascii')
             self._downloader.to_screen(dump)
-        if self.get_param('write_pages', False):
-            basen = f'{video_id}_{urlh.geturl()}'
-            trim_length = self.get_param('trim_file_name') or 240
-            if len(basen) > trim_length:
-                h = '___' + hashlib.md5(basen.encode('utf-8')).hexdigest()
-                basen = basen[:trim_length - len(h)] + h
-            raw_filename = basen + '.dump'
-            filename = sanitize_filename(raw_filename, restricted=True)
-            self.to_screen('Saving request to ' + filename)
-            # Working around MAX_PATH limitation on Windows (see
-            # http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx)
-            if compat_os_name == 'nt':
-                absfilepath = os.path.abspath(filename)
-                if len(absfilepath) > 259:
-                    filename = '\\\\?\\' + absfilepath
+        if self.get_param('write_pages'):
+            filename = self._request_dump_filename(video_id, urlh.geturl())
+            self.to_screen(f'Saving request to {filename}')
             with open(filename, 'wb') as outf:
                 outf.write(webpage_bytes)
 
-        try:
-            content = webpage_bytes.decode(encoding, 'replace')
-        except LookupError:
-            content = webpage_bytes.decode('utf-8', 'replace')
-
+        content = self.__decode_webpage(webpage_bytes, encoding, urlh.headers)
         self.__check_blocked(content)
 
         return content
 
-    def _download_webpage(
-            self, url_or_request, video_id, note=None, errnote=None,
-            fatal=True, tries=1, timeout=5, encoding=None, data=None,
-            headers={}, query={}, expected_status=None):
-        """
-        Return the data of the page as a string.
-
-        Arguments:
-        url_or_request -- plain text URL as a string or
-            a compat_urllib_request.Requestobject
-        video_id -- Video/playlist/item identifier (string)
-
-        Keyword arguments:
-        note -- note printed before downloading (string)
-        errnote -- note printed in case of an error (string)
-        fatal -- flag denoting whether error should be considered fatal,
-            i.e. whether it should cause ExtractionError to be raised,
-            otherwise a warning will be reported and extraction continued
-        tries -- number of tries
-        timeout -- sleep interval between tries
-        encoding -- encoding for a page content decoding, guessed automatically
-            when not explicitly specified
-        data -- POST data (bytes)
-        headers -- HTTP headers (dict)
-        query -- URL query (dict)
-        expected_status -- allows to accept failed HTTP requests (non 2xx
-            status code) by explicitly specifying a set of accepted status
-            codes. Can be any of the following entities:
-                - an integer type specifying an exact failed status code to
-                  accept
-                - a list or a tuple of integer types specifying a list of
-                  failed status codes to accept
-                - a callable accepting an actual failed status code and
-                  returning True if it should be accepted
-            Note that this argument does not affect success status codes (2xx)
-            which are always accepted.
-        """
-
-        success = False
-        try_count = 0
-        while success is False:
-            try:
-                res = self._download_webpage_handle(
-                    url_or_request, video_id, note, errnote, fatal,
-                    encoding=encoding, data=data, headers=headers, query=query,
-                    expected_status=expected_status)
-                success = True
-            except compat_http_client.IncompleteRead as e:
-                try_count += 1
-                if try_count >= tries:
-                    raise e
-                self._sleep(timeout, video_id)
-        if res is False:
-            return res
-        else:
-            content, _ = res
-            return content
-
-    def _download_xml_handle(
-            self, url_or_request, video_id, note='Downloading XML',
-            errnote='Unable to download XML', transform_source=None,
-            fatal=True, encoding=None, data=None, headers={}, query={},
-            expected_status=None):
-        """
-        Return a tuple (xml as an xml.etree.ElementTree.Element, URL handle).
-
-        See _download_webpage docstring for arguments specification.
-        """
-        res = self._download_webpage_handle(
-            url_or_request, video_id, note, errnote, fatal=fatal,
-            encoding=encoding, data=data, headers=headers, query=query,
-            expected_status=expected_status)
-        if res is False:
-            return res
-        xml_string, urlh = res
-        return self._parse_xml(
-            xml_string, video_id, transform_source=transform_source,
-            fatal=fatal), urlh
-
-    def _download_xml(
-            self, url_or_request, video_id,
-            note='Downloading XML', errnote='Unable to download XML',
-            transform_source=None, fatal=True, encoding=None,
-            data=None, headers={}, query={}, expected_status=None):
-        """
-        Return the xml as an xml.etree.ElementTree.Element.
-
-        See _download_webpage docstring for arguments specification.
-        """
-        res = self._download_xml_handle(
-            url_or_request, video_id, note=note, errnote=errnote,
-            transform_source=transform_source, fatal=fatal, encoding=encoding,
-            data=data, headers=headers, query=query,
-            expected_status=expected_status)
-        return res if res is False else res[0]
-
     def _parse_xml(self, xml_string, video_id, transform_source=None, fatal=True):
         if transform_source:
             xml_string = transform_source(xml_string)
@@ -996,49 +929,19 @@ def _parse_xml(self, xml_string, video_id, transform_source=None, fatal=True):
             else:
                 self.report_warning(errmsg + str(ve))
 
-    def _download_json_handle(
-            self, url_or_request, video_id, note='Downloading JSON metadata',
-            errnote='Unable to download JSON metadata', transform_source=None,
-            fatal=True, encoding=None, data=None, headers={}, query={},
-            expected_status=None):
-        """
-        Return a tuple (JSON object, URL handle).
-
-        See _download_webpage docstring for arguments specification.
-        """
-        res = self._download_webpage_handle(
-            url_or_request, video_id, note, errnote, fatal=fatal,
-            encoding=encoding, data=data, headers=headers, query=query,
-            expected_status=expected_status)
-        if res is False:
-            return res
-        json_string, urlh = res
-        return self._parse_json(
-            json_string, video_id, transform_source=transform_source,
-            fatal=fatal), urlh
-
-    def _download_json(
-            self, url_or_request, video_id, note='Downloading JSON metadata',
-            errnote='Unable to download JSON metadata', transform_source=None,
-            fatal=True, encoding=None, data=None, headers={}, query={},
-            expected_status=None):
-        """
-        Return the JSON object as a dict.
-
-        See _download_webpage docstring for arguments specification.
-        """
-        res = self._download_json_handle(
-            url_or_request, video_id, note=note, errnote=errnote,
-            transform_source=transform_source, fatal=fatal, encoding=encoding,
-            data=data, headers=headers, query=query,
-            expected_status=expected_status)
-        return res if res is False else res[0]
-
-    def _parse_json(self, json_string, video_id, transform_source=None, fatal=True):
+    def _parse_json(self, json_string, video_id, transform_source=None, fatal=True, lenient=False):
         if transform_source:
             json_string = transform_source(json_string)
         try:
-            return json.loads(json_string, strict=False)
+            try:
+                return json.loads(json_string, strict=False)
+            except json.JSONDecodeError as e:
+                if not lenient:
+                    raise
+                try:
+                    return json.loads(json_string[:e.pos], strict=False)
+                except ValueError:
+                    raise e
         except ValueError as ve:
             errmsg = '%s: Failed to parse JSON ' % video_id
             if fatal:
@@ -1051,43 +954,95 @@ def _parse_socket_response_as_json(self, data, video_id, transform_source=None,
             data[data.find('{'):data.rfind('}') + 1],
             video_id, transform_source, fatal)
 
-    def _download_socket_json_handle(
-            self, url_or_request, video_id, note='Polling socket',
-            errnote='Unable to poll socket', transform_source=None,
-            fatal=True, encoding=None, data=None, headers={}, query={},
-            expected_status=None):
-        """
-        Return a tuple (JSON object, URL handle).
+    def __create_download_methods(name, parser, note, errnote, return_value):
+
+        def parse(ie, content, *args, **kwargs):
+            if parser is None:
+                return content
+            # parser is fetched by name so subclasses can override it
+            return getattr(ie, parser)(content, *args, **kwargs)
+
+        def download_handle(self, url_or_request, video_id, note=note, errnote=errnote,
+                            transform_source=None, fatal=True, *args, **kwargs):
+            res = self._download_webpage_handle(url_or_request, video_id, note, errnote, fatal, *args, **kwargs)
+            if res is False:
+                return res
+            content, urlh = res
+            return parse(self, content, video_id, transform_source, fatal), urlh
+
+        def download_content(self, url_or_request, video_id, note=note, errnote=errnote, transform_source=None,
+                             fatal=True, encoding=None, data=None, headers={}, query={}, *args, **kwargs):
+            if self.get_param('load_pages'):
+                url_or_request = self._create_request(url_or_request, data, headers, query)
+                filename = self._request_dump_filename(url_or_request.full_url, video_id)
+                self.to_screen(f'Loading request from {filename}')
+                try:
+                    with open(filename, 'rb') as dumpf:
+                        webpage_bytes = dumpf.read()
+                except OSError as e:
+                    self.report_warning(f'Unable to load request from disk: {e}')
+                else:
+                    content = self.__decode_webpage(webpage_bytes, encoding, url_or_request.headers)
+                    return parse(self, content, video_id, transform_source, fatal)
+            args = [url_or_request, video_id, note, errnote, transform_source, fatal, encoding, data, headers, query, *args]
+            if parser is None:
+                args.pop(4)  # transform_source
+            # The method is fetched by name so subclasses can override _download_..._handle
+            res = getattr(self, download_handle.__name__)(*args, **kwargs)
+            return res if res is False else res[0]
+
+        def impersonate(func, name, return_value):
+            func.__name__, func.__qualname__ = name, f'InfoExtractor.{name}'
+            func.__doc__ = f'''
+                @param transform_source     Apply this transformation before parsing
+                @returns                    {return_value}
+
+                See _download_webpage_handle docstring for other arguments specification
+            '''
+
+        impersonate(download_handle, f'_download_{name}_handle', f'({return_value}, URL handle)')
+        impersonate(download_content, f'_download_{name}', f'{return_value}')
+        return download_handle, download_content
+
+    _download_xml_handle, _download_xml = __create_download_methods(
+        'xml', '_parse_xml', 'Downloading XML', 'Unable to download XML', 'xml as an xml.etree.ElementTree.Element')
+    _download_json_handle, _download_json = __create_download_methods(
+        'json', '_parse_json', 'Downloading JSON metadata', 'Unable to download JSON metadata', 'JSON object as a dict')
+    _download_socket_json_handle, _download_socket_json = __create_download_methods(
+        'socket_json', '_parse_socket_response_as_json', 'Polling socket', 'Unable to poll socket', 'JSON object as a dict')
+    __download_webpage = __create_download_methods('webpage', None, None, None, 'data of the page as a string')[1]
 
-        See _download_webpage docstring for arguments specification.
-        """
-        res = self._download_webpage_handle(
-            url_or_request, video_id, note, errnote, fatal=fatal,
-            encoding=encoding, data=data, headers=headers, query=query,
-            expected_status=expected_status)
-        if res is False:
-            return res
-        webpage, urlh = res
-        return self._parse_socket_response_as_json(
-            webpage, video_id, transform_source=transform_source,
-            fatal=fatal), urlh
-
-    def _download_socket_json(
-            self, url_or_request, video_id, note='Polling socket',
-            errnote='Unable to poll socket', transform_source=None,
-            fatal=True, encoding=None, data=None, headers={}, query={},
-            expected_status=None):
+    def _download_webpage(
+            self, url_or_request, video_id, note=None, errnote=None,
+            fatal=True, tries=1, timeout=NO_DEFAULT, *args, **kwargs):
         """
-        Return the JSON object as a dict.
+        Return the data of the page as a string.
 
-        See _download_webpage docstring for arguments specification.
+        Keyword arguments:
+        tries -- number of tries
+        timeout -- sleep interval between tries
+
+        See _download_webpage_handle docstring for other arguments specification.
         """
-        res = self._download_socket_json_handle(
-            url_or_request, video_id, note=note, errnote=errnote,
-            transform_source=transform_source, fatal=fatal, encoding=encoding,
-            data=data, headers=headers, query=query,
-            expected_status=expected_status)
-        return res if res is False else res[0]
+
+        R''' # NB: These are unused; should they be deprecated?
+        if tries != 1:
+            self._downloader.deprecation_warning('tries argument is deprecated in InfoExtractor._download_webpage')
+        if timeout is NO_DEFAULT:
+            timeout = 5
+        else:
+            self._downloader.deprecation_warning('timeout argument is deprecated in InfoExtractor._download_webpage')
+        '''
+
+        try_count = 0
+        while True:
+            try:
+                return self.__download_webpage(url_or_request, video_id, note, errnote, None, fatal, *args, **kwargs)
+            except compat_http_client.IncompleteRead as e:
+                try_count += 1
+                if try_count >= tries:
+                    raise e
+                self._sleep(timeout, video_id)
 
     def report_warning(self, msg, video_id=None, *args, only_once=False, **kwargs):
         idstr = format_field(video_id, template='%s: ')
@@ -1506,7 +1461,7 @@ def extract_video_object(e):
             assert e['@type'] == 'VideoObject'
             author = e.get('author')
             info.update({
-                'url': url_or_none(e.get('contentUrl')),
+                'url': traverse_obj(e, 'contentUrl', 'embedUrl', expected_type=url_or_none),
                 'title': unescapeHTML(e.get('name')),
                 'description': unescapeHTML(e.get('description')),
                 'thumbnails': [{'url': url}
@@ -1574,6 +1529,8 @@ def traverse_json_ld(json_ld, at_top_level=True):
                     })
                     if traverse_obj(e, ('video', 0, '@type')) == 'VideoObject':
                         extract_video_object(e['video'][0])
+                    elif traverse_obj(e, ('subjectOf', 0, '@type')) == 'VideoObject':
+                        extract_video_object(e['subjectOf'][0])
                 elif item_type == 'VideoObject':
                     extract_video_object(e)
                     if expected_type is None:
@@ -3190,7 +3147,8 @@ def parse_content_type(content_type):
                 return f
             return {}
 
-        def _media_formats(src, cur_media_type, type_info={}):
+        def _media_formats(src, cur_media_type, type_info=None):
+            type_info = type_info or {}
             full_url = absolute_url(src)
             ext = type_info.get('ext') or determine_ext(full_url)
             if ext == 'm3u8':
@@ -3208,6 +3166,7 @@ def _media_formats(src, cur_media_type, type_info={}):
                 formats = [{
                     'url': full_url,
                     'vcodec': 'none' if cur_media_type == 'audio' else None,
+                    'ext': ext,
                 }]
             return is_plain_url, formats
 
@@ -3234,7 +3193,8 @@ def _media_formats(src, cur_media_type, type_info={}):
             media_attributes = extract_attributes(media_tag)
             src = strip_or_none(media_attributes.get('src'))
             if src:
-                _, formats = _media_formats(src, media_type)
+                f = parse_content_type(media_attributes.get('type'))
+                _, formats = _media_formats(src, media_type, f)
                 media_info['formats'].extend(formats)
             media_info['thumbnail'] = absolute_url(media_attributes.get('poster'))
             if media_content:
diff --git a/yt_dlp/extractor/curiositystream.py b/yt_dlp/extractor/curiositystream.py
index 5b76b29ff..be4e53e44 100644
--- a/yt_dlp/extractor/curiositystream.py
+++ b/yt_dlp/extractor/curiositystream.py
@@ -23,6 +23,11 @@ def _handle_errors(self, result):
 
     def _call_api(self, path, video_id, query=None):
         headers = {}
+        if not self._auth_token:
+            auth_cookie = self._get_cookies('https://curiositystream.com').get('auth_token')
+            if auth_cookie:
+                self.write_debug('Obtained auth_token cookie')
+                self._auth_token = auth_cookie.value
         if self._auth_token:
             headers['X-Auth-Token'] = self._auth_token
         result = self._download_json(
diff --git a/yt_dlp/extractor/dropout.py b/yt_dlp/extractor/dropout.py
index 096216418..039ac4b2c 100644
--- a/yt_dlp/extractor/dropout.py
+++ b/yt_dlp/extractor/dropout.py
@@ -1,8 +1,8 @@
 from .common import InfoExtractor
 from .vimeo import VHXEmbedIE
 from ..utils import (
-    clean_html,
     ExtractorError,
+    clean_html,
     get_element_by_class,
     get_element_by_id,
     get_elements_by_class,
@@ -96,11 +96,12 @@ def _get_authenticity_token(self, display_id):
 
     def _login(self, display_id):
         username, password = self._get_login_info()
-        if not (username and password):
-            self.raise_login_required(method='password')
+        if not username:
+            return True
 
         response = self._download_webpage(
-            self._LOGIN_URL, display_id, note='Logging in', data=urlencode_postdata({
+            self._LOGIN_URL, display_id, note='Logging in', fatal=False,
+            data=urlencode_postdata({
                 'email': username,
                 'password': password,
                 'authenticity_token': self._get_authenticity_token(display_id),
@@ -110,19 +111,25 @@ def _login(self, display_id):
         user_has_subscription = self._search_regex(
             r'user_has_subscription:\s*["\'](.+?)["\']', response, 'subscription status', default='none')
         if user_has_subscription.lower() == 'true':
-            return response
+            return
         elif user_has_subscription.lower() == 'false':
-            raise ExtractorError('Account is not subscribed')
+            return 'Account is not subscribed'
         else:
-            raise ExtractorError('Incorrect username/password')
+            return 'Incorrect username/password'
 
     def _real_extract(self, url):
         display_id = self._match_id(url)
+        login_err, webpage = False, ''
         try:
-            self._login(display_id)
-            webpage = self._download_webpage(url, display_id, note='Downloading video webpage')
+            login_err = self._login(display_id)
+            webpage = self._download_webpage(url, display_id)
         finally:
-            self._download_webpage('https://www.dropout.tv/logout', display_id, note='Logging out', fatal=False)
+            if not login_err:
+                self._download_webpage('https://www.dropout.tv/logout', display_id, note='Logging out', fatal=False)
+            elif '<div id="watch-unauthorized"' in webpage:
+                if login_err is True:
+                    self.raise_login_required(method='password')
+                raise ExtractorError(login_err, expected=True)
 
         embed_url = self._search_regex(r'embed_url:\s*["\'](.+?)["\']', webpage, 'embed url')
         thumbnail = self._og_search_thumbnail(webpage)
diff --git a/yt_dlp/extractor/duboku.py b/yt_dlp/extractor/duboku.py
index 24403842d..fb0546cae 100644
--- a/yt_dlp/extractor/duboku.py
+++ b/yt_dlp/extractor/duboku.py
@@ -51,31 +51,39 @@ def _get_element_by_tag_and_attrib(html, tag=None, attribute=None, value=None, e
 
 class DubokuIE(InfoExtractor):
     IE_NAME = 'duboku'
-    IE_DESC = 'www.duboku.co'
+    IE_DESC = 'www.duboku.io'
 
-    _VALID_URL = r'(?:https?://[^/]+\.duboku\.co/vodplay/)(?P<id>[0-9]+-[0-9-]+)\.html.*'
+    _VALID_URL = r'(?:https?://[^/]+\.duboku\.io/vodplay/)(?P<id>[0-9]+-[0-9-]+)\.html.*'
     _TESTS = [{
-        'url': 'https://www.duboku.co/vodplay/1575-1-1.html',
+        'url': 'https://w.duboku.io/vodplay/1575-1-1.html',
         'info_dict': {
             'id': '1575-1-1',
-            'ext': 'ts',
+            'ext': 'mp4',
             'series': '白色月光',
             'title': 'contains:白色月光',
             'season_number': 1,
             'episode_number': 1,
+            'season': 'Season 1',
+            'episode_id': '1',
+            'season_id': '1',
+            'episode': 'Episode 1',
         },
         'params': {
             'skip_download': 'm3u8 download',
         },
     }, {
-        'url': 'https://www.duboku.co/vodplay/1588-1-1.html',
+        'url': 'https://w.duboku.io/vodplay/1588-1-1.html',
         'info_dict': {
             'id': '1588-1-1',
-            'ext': 'ts',
+            'ext': 'mp4',
             'series': '亲爱的自己',
-            'title': 'contains:预告片',
+            'title': 'contains:第1集',
             'season_number': 1,
             'episode_number': 1,
+            'episode': 'Episode 1',
+            'season': 'Season 1',
+            'episode_id': '1',
+            'season_id': '1',
         },
         'params': {
             'skip_download': 'm3u8 download',
@@ -91,7 +99,7 @@ def _real_extract(self, url):
         season_id = temp[1]
         episode_id = temp[2]
 
-        webpage_url = 'https://www.duboku.co/vodplay/%s.html' % video_id
+        webpage_url = 'https://w.duboku.io/vodplay/%s.html' % video_id
         webpage_html = self._download_webpage(webpage_url, video_id)
 
         # extract video url
@@ -124,12 +132,13 @@ def _real_extract(self, url):
         data_from = player_data.get('from')
 
         # if it is an embedded iframe, maybe it's an external source
+        headers = {'Referer': webpage_url}
         if data_from == 'iframe':
             # use _type url_transparent to retain the meaningful details
             # of the video.
             return {
                 '_type': 'url_transparent',
-                'url': smuggle_url(data_url, {'http_headers': {'Referer': webpage_url}}),
+                'url': smuggle_url(data_url, {'http_headers': headers}),
                 'id': video_id,
                 'title': title,
                 'series': series_title,
@@ -139,7 +148,7 @@ def _real_extract(self, url):
                 'episode_id': episode_id,
             }
 
-        formats = self._extract_m3u8_formats(data_url, video_id, 'mp4')
+        formats = self._extract_m3u8_formats(data_url, video_id, 'mp4', headers=headers)
 
         return {
             'id': video_id,
@@ -150,36 +159,29 @@ def _real_extract(self, url):
             'episode_number': int_or_none(episode_id),
             'episode_id': episode_id,
             'formats': formats,
-            'http_headers': {'Referer': 'https://www.duboku.co/static/player/videojs.html'}
+            'http_headers': headers
         }
 
 
 class DubokuPlaylistIE(InfoExtractor):
     IE_NAME = 'duboku:list'
-    IE_DESC = 'www.duboku.co entire series'
+    IE_DESC = 'www.duboku.io entire series'
 
-    _VALID_URL = r'(?:https?://[^/]+\.duboku\.co/voddetail/)(?P<id>[0-9]+)\.html.*'
+    _VALID_URL = r'(?:https?://[^/]+\.duboku\.io/voddetail/)(?P<id>[0-9]+)\.html.*'
     _TESTS = [{
-        'url': 'https://www.duboku.co/voddetail/1575.html',
+        'url': 'https://w.duboku.io/voddetail/1575.html',
         'info_dict': {
             'id': 'startswith:1575',
             'title': '白色月光',
         },
         'playlist_count': 12,
     }, {
-        'url': 'https://www.duboku.co/voddetail/1554.html',
+        'url': 'https://w.duboku.io/voddetail/1554.html',
         'info_dict': {
             'id': 'startswith:1554',
             'title': '以家人之名',
         },
         'playlist_mincount': 30,
-    }, {
-        'url': 'https://www.duboku.co/voddetail/1554.html#playlist2',
-        'info_dict': {
-            'id': '1554#playlist2',
-            'title': '以家人之名',
-        },
-        'playlist_mincount': 27,
     }]
 
     def _real_extract(self, url):
@@ -189,7 +191,7 @@ def _real_extract(self, url):
         series_id = mobj.group('id')
         fragment = compat_urlparse.urlparse(url).fragment
 
-        webpage_url = 'https://www.duboku.co/voddetail/%s.html' % series_id
+        webpage_url = 'https://w.duboku.io/voddetail/%s.html' % series_id
         webpage_html = self._download_webpage(webpage_url, series_id)
 
         # extract title
@@ -234,6 +236,6 @@ def _real_extract(self, url):
         # return url results
         return self.playlist_result([
             self.url_result(
-                compat_urlparse.urljoin('https://www.duboku.co', x['href']),
+                compat_urlparse.urljoin('https://w.duboku.io', x['href']),
                 ie=DubokuIE.ie_key(), video_title=x.get('title'))
             for x in playlist], series_id + '#' + playlist_id, title)
diff --git a/yt_dlp/extractor/extractors.py b/yt_dlp/extractor/extractors.py
index 9c5a5f482..c3392899d 100644
--- a/yt_dlp/extractor/extractors.py
+++ b/yt_dlp/extractor/extractors.py
@@ -54,10 +54,6 @@
     AluraCourseIE
 )
 from .amcnetworks import AMCNetworksIE
-from .animelab import (
-    AnimeLabIE,
-    AnimeLabShowsIE,
-)
 from .amazon import AmazonStoreIE
 from .americastestkitchen import (
     AmericasTestKitchenIE,
@@ -1261,6 +1257,7 @@
 from .playplustv import PlayPlusTVIE
 from .plays import PlaysTVIE
 from .playstuff import PlayStuffIE
+from .playsuisse import PlaySuisseIE
 from .playtvak import PlaytvakIE
 from .playvid import PlayvidIE
 from .playwire import PlaywireIE
@@ -1274,7 +1271,6 @@
 from .pokemon import (
     PokemonIE,
     PokemonWatchIE,
-    PokemonSoundLibraryIE,
 )
 from .pokergo import (
     PokerGoIE,
diff --git a/yt_dlp/extractor/fc2.py b/yt_dlp/extractor/fc2.py
index 54b136ec7..8983a16fd 100644
--- a/yt_dlp/extractor/fc2.py
+++ b/yt_dlp/extractor/fc2.py
@@ -1,9 +1,7 @@
 import re
 
 from .common import InfoExtractor
-from ..compat import (
-    compat_parse_qs,
-)
+from ..compat import compat_parse_qs
 from ..dependencies import websockets
 from ..utils import (
     ExtractorError,
@@ -209,7 +207,7 @@ def _real_extract(self, url):
             'User-Agent': self.get_param('http_headers')['User-Agent'],
         })
 
-        self.write_debug('[debug] Sending HLS server request')
+        self.write_debug('Sending HLS server request')
 
         while True:
             recv = ws.recv()
@@ -231,13 +229,10 @@ def _real_extract(self, url):
             if not data or not isinstance(data, dict):
                 continue
             if data.get('name') == '_response_' and data.get('id') == 1:
-                self.write_debug('[debug] Goodbye.')
+                self.write_debug('Goodbye')
                 playlist_data = data
                 break
-            elif self._downloader.params.get('verbose', False):
-                if len(recv) > 100:
-                    recv = recv[:100] + '...'
-                self.to_screen('[debug] Server said: %s' % recv)
+            self.write_debug('Server said: %s%s' % (recv[:100], '...' if len(recv) > 100 else ''))
 
         if not playlist_data:
             raise ExtractorError('Unable to fetch HLS playlist info via WebSocket')
diff --git a/yt_dlp/extractor/iwara.py b/yt_dlp/extractor/iwara.py
index 4b88da35f..f99e22fab 100644
--- a/yt_dlp/extractor/iwara.py
+++ b/yt_dlp/extractor/iwara.py
@@ -1,3 +1,4 @@
+import itertools
 import re
 import urllib
 
@@ -171,37 +172,70 @@ class IwaraUserIE(IwaraBaseIE):
     IE_NAME = 'iwara:user'
 
     _TESTS = [{
-        'url': 'https://ecchi.iwara.tv/users/CuteMMD',
+        'note': 'number of all videos page is just 1 page. less than 40 videos',
+        'url': 'https://ecchi.iwara.tv/users/infinityyukarip',
         'info_dict': {
-            'id': 'CuteMMD',
+            'title': 'Uploaded videos from Infinity_YukariP',
+            'id': 'infinityyukarip',
+            'uploader': 'Infinity_YukariP',
+            'uploader_id': 'infinityyukarip',
         },
-        'playlist_mincount': 198,
+        'playlist_mincount': 39,
     }, {
-        # urlencoded
-        'url': 'https://ecchi.iwara.tv/users/%E5%92%95%E5%98%BF%E5%98%BF',
+        'note': 'no even all videos page. probably less than 10 videos',
+        'url': 'https://ecchi.iwara.tv/users/mmd-quintet',
+        'info_dict': {
+            'title': 'Uploaded videos from mmd quintet',
+            'id': 'mmd-quintet',
+            'uploader': 'mmd quintet',
+            'uploader_id': 'mmd-quintet',
+        },
+        'playlist_mincount': 6,
+    }, {
+        'note': 'has paging. more than 40 videos',
+        'url': 'https://ecchi.iwara.tv/users/theblackbirdcalls',
+        'info_dict': {
+            'title': 'Uploaded videos from TheBlackbirdCalls',
+            'id': 'theblackbirdcalls',
+            'uploader': 'TheBlackbirdCalls',
+            'uploader_id': 'theblackbirdcalls',
+        },
+        'playlist_mincount': 420,
+    }, {
+        'note': 'foreign chars in URL. there must be foreign characters in URL',
+        'url': 'https://ecchi.iwara.tv/users/ぶた丼',
         'info_dict': {
-            'id': '咕嘿嘿',
+            'title': 'Uploaded videos from ぶた丼',
+            'id': 'ぶた丼',
+            'uploader': 'ぶた丼',
+            'uploader_id': 'ぶた丼',
         },
-        'playlist_mincount': 141,
+        'playlist_mincount': 170,
     }]
 
-    def _entries(self, playlist_id, base_url, webpage):
-        yield from self._extract_playlist(base_url, webpage)
-
-        page_urls = re.findall(
-            r'class="pager-item"[^>]*>\s*<a[^<]+href="([^"]+)', webpage)
-
-        for n, path in enumerate(page_urls, 2):
+    def _entries(self, playlist_id, base_url):
+        webpage = self._download_webpage(
+            f'{base_url}/users/{playlist_id}', playlist_id)
+        videos_url = self._search_regex(r'<a href="(/users/[^/]+/videos)(?:\?[^"]+)?">', webpage, 'all videos url', default=None)
+        if not videos_url:
+            yield from self._extract_playlist(base_url, webpage)
+            return
+
+        videos_url = urljoin(base_url, videos_url)
+
+        for n in itertools.count(1):
+            page = self._download_webpage(
+                videos_url, playlist_id, note=f'Downloading playlist page {n}',
+                query={'page': str(n - 1)} if n > 1 else {})
             yield from self._extract_playlist(
-                base_url, self._download_webpage(
-                    urljoin(base_url, path), playlist_id, note=f'Downloading playlist page {n}'))
+                base_url, page)
+
+            if f'page={n}' not in page:
+                break
 
     def _real_extract(self, url):
         playlist_id, base_url = self._match_valid_url(url).group('id', 'base_url')
         playlist_id = urllib.parse.unquote(playlist_id)
 
-        webpage = self._download_webpage(
-            f'{base_url}/users/{playlist_id}/videos', playlist_id)
-
         return self.playlist_result(
-            self._entries(playlist_id, base_url, webpage), playlist_id)
+            self._entries(playlist_id, base_url), playlist_id)
diff --git a/yt_dlp/extractor/niconico.py b/yt_dlp/extractor/niconico.py
index a80b544f8..82fb27631 100644
--- a/yt_dlp/extractor/niconico.py
+++ b/yt_dlp/extractor/niconico.py
@@ -647,14 +647,14 @@ class NiconicoSeriesIE(InfoExtractor):
             'id': '110226',
             'title': 'ご立派ァ！のシリーズ',
         },
-        'playlist_mincount': 10,  # as of 2021/03/17
+        'playlist_mincount': 10,
     }, {
         'url': 'https://www.nicovideo.jp/series/12312/',
         'info_dict': {
             'id': '12312',
             'title': 'バトルスピリッツ　お勧めカード紹介(調整中)',
         },
-        'playlist_mincount': 97,  # as of 2021/03/17
+        'playlist_mincount': 103,
     }, {
         'url': 'https://nico.ms/series/203559',
         'only_matching': True,
@@ -672,7 +672,7 @@ def _real_extract(self, url):
             title = unescapeHTML(title)
         playlist = [
             self.url_result(f'https://www.nicovideo.jp/watch/{v_id}', video_id=v_id)
-            for v_id in re.findall(r'href="/watch/([a-z0-9]+)" data-href="/watch/\1', webpage)]
+            for v_id in re.findall(r'data-href=[\'"](?:https://www\.nicovideo\.jp)?/watch/([a-z0-9]+)', webpage)]
         return self.playlist_result(playlist, list_id, title)
 
 
diff --git a/yt_dlp/extractor/npr.py b/yt_dlp/extractor/npr.py
index 6d93f154c..e4ff8d6c2 100644
--- a/yt_dlp/extractor/npr.py
+++ b/yt_dlp/extractor/npr.py
@@ -51,6 +51,15 @@ class NprIE(InfoExtractor):
         # multimedia, no formats, stream
         'url': 'https://www.npr.org/2020/02/14/805476846/laura-stevenson-tiny-desk-concert',
         'only_matching': True,
+    }, {
+        'url': 'https://www.npr.org/2022/03/15/1084896560/bonobo-tiny-desk-home-concert',
+        'info_dict': {
+            'id': '1086468851',
+            'ext': 'mp4',
+            'title': 'Bonobo: Tiny Desk (Home) Concert',
+            'duration': 1061,
+            'thumbnail': r're:^https?://media.npr.org/assets/img/.*\.jpg$',
+        },
     }]
 
     def _real_extract(self, url):
@@ -65,6 +74,10 @@ def _real_extract(self, url):
             })['list']['story'][0]
         playlist_title = story.get('title', {}).get('$text')
 
+        # Fetch the JSON-LD from the npr page.
+        json_ld = self._search_json_ld(
+            self._download_webpage(url, playlist_id), playlist_id, 'NewsArticle', fatal=False)
+
         KNOWN_FORMATS = ('threegp', 'm3u8', 'smil', 'mp4', 'mp3')
         quality = qualities(KNOWN_FORMATS)
 
@@ -110,6 +123,10 @@ def _real_extract(self, url):
                 formats.extend(self._extract_m3u8_formats(
                     stream_url, stream_id, 'mp4', 'm3u8_native',
                     m3u8_id='hls', fatal=False))
+
+            if not formats and json_ld.get('url'):
+                formats.extend(self._extract_m3u8_formats(json_ld['url'], media_id, 'mp4', m3u8_id='hls', fatal=False))
+
             self._sort_formats(formats)
 
             entries.append({
diff --git a/yt_dlp/extractor/playsuisse.py b/yt_dlp/extractor/playsuisse.py
new file mode 100644
index 000000000..a635ac92f
--- /dev/null
+++ b/yt_dlp/extractor/playsuisse.py
@@ -0,0 +1,147 @@
+import json
+
+from .common import InfoExtractor
+from ..utils import int_or_none, traverse_obj
+
+
+class PlaySuisseIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?playsuisse\.ch/watch/(?P<id>[0-9]+)'
+    _TESTS = [
+        {
+            'url': 'https://www.playsuisse.ch/watch/763211/0',
+            'md5': '82df2a470b2dfa60c2d33772a8a60cf8',
+            'info_dict': {
+                'id': '763211',
+                'ext': 'mp4',
+                'title': 'Knochen',
+                'description': 'md5:8ea7a8076ba000cd9e8bc132fd0afdd8',
+                'duration': 3344,
+                'series': 'Wilder',
+                'season': 'Season 1',
+                'season_number': 1,
+                'episode': 'Knochen',
+                'episode_number': 1,
+                'thumbnail': 'md5:9260abe0c0ec9b69914d0a10d54c5878'
+            }
+        },
+        {
+            'url': 'https://www.playsuisse.ch/watch/808675/0',
+            'md5': '818b94c1d2d7c4beef953f12cb8f3e75',
+            'info_dict': {
+                'id': '808675',
+                'ext': 'mp4',
+                'title': 'Der Läufer',
+                'description': 'md5:9f61265c7e6dcc3e046137a792b275fd',
+                'duration': 5280,
+                'episode': 'Der Läufer',
+                'thumbnail': 'md5:44af7d65ee02bbba4576b131868bb783'
+            }
+        },
+        {
+            'url': 'https://www.playsuisse.ch/watch/817193/0',
+            'md5': '1d6c066f92cd7fffd8b28a53526d6b59',
+            'info_dict': {
+                'id': '817193',
+                'ext': 'mp4',
+                'title': 'Die Einweihungsparty',
+                'description': 'md5:91ebf04d3a42cb3ab70666acf750a930',
+                'duration': 1380,
+                'series': 'Nr. 47',
+                'season': 'Season 1',
+                'season_number': 1,
+                'episode': 'Die Einweihungsparty',
+                'episode_number': 1,
+                'thumbnail': 'md5:637585fb106e3a4bcd991958924c7e44'
+            }
+        }
+    ]
+
+    _GRAPHQL_QUERY = '''
+        query AssetWatch($assetId: ID!) {
+            assetV2(id: $assetId) {
+                ...Asset
+                episodes {
+                    ...Asset
+                }
+            }
+        }
+        fragment Asset on AssetV2 {
+            id
+            name
+            description
+            duration
+            episodeNumber
+            seasonNumber
+            seriesName
+            medias {
+                type
+                url
+            }
+            thumbnail16x9 {
+                ...ImageDetails
+            }
+            thumbnail2x3 {
+                ...ImageDetails
+            }
+            thumbnail16x9WithTitle {
+                ...ImageDetails
+            }
+            thumbnail2x3WithTitle {
+                ...ImageDetails
+            }
+        }
+        fragment ImageDetails on AssetImage {
+            id
+            url
+        }'''
+
+    def _get_media_data(self, media_id):
+        # NOTE In the web app, the "locale" header is used to switch between languages,
+        # However this doesn't seem to take effect when passing the header here.
+        response = self._download_json(
+            'https://4bbepzm4ef.execute-api.eu-central-1.amazonaws.com/prod/graphql',
+            media_id, data=json.dumps({
+                'operationName': 'AssetWatch',
+                'query': self._GRAPHQL_QUERY,
+                'variables': {'assetId': media_id}
+            }).encode('utf-8'),
+            headers={'Content-Type': 'application/json', 'locale': 'de'})
+
+        return response['data']['assetV2']
+
+    def _real_extract(self, url):
+        media_id = self._match_id(url)
+        media_data = self._get_media_data(media_id)
+        info = self._extract_single(media_data)
+        if media_data.get('episodes'):
+            info.update({
+                '_type': 'playlist',
+                'entries': map(self._extract_single, media_data['episodes']),
+            })
+        return info
+
+    def _extract_single(self, media_data):
+        thumbnails = traverse_obj(media_data, lambda k, _: k.startswith('thumbnail'))
+
+        formats, subtitles = [], {}
+        for media in traverse_obj(media_data, 'medias', default=[]):
+            if not media.get('url') or media.get('type') != 'HLS':
+                continue
+            f, subs = self._extract_m3u8_formats_and_subtitles(
+                media['url'], media_data['id'], 'mp4', m3u8_id='HLS', fatal=False)
+            formats.extend(f)
+            self._merge_subtitles(subs, target=subtitles)
+
+        return {
+            'id': media_data['id'],
+            'title': media_data.get('name'),
+            'description': media_data.get('description'),
+            'thumbnails': thumbnails,
+            'duration': int_or_none(media_data.get('duration')),
+            'formats': formats,
+            'subtitles': subtitles,
+            'series': media_data.get('seriesName'),
+            'season_number': int_or_none(media_data.get('seasonNumber')),
+            'episode': media_data.get('name'),
+            'episode_number': int_or_none(media_data.get('episodeNumber')),
+        }
diff --git a/yt_dlp/extractor/pokemon.py b/yt_dlp/extractor/pokemon.py
index eef0d02ca..0911893d4 100644
--- a/yt_dlp/extractor/pokemon.py
+++ b/yt_dlp/extractor/pokemon.py
@@ -1,5 +1,3 @@
-import re
-
 from .common import InfoExtractor
 from ..utils import (
     ExtractorError,
@@ -136,42 +134,3 @@ def _real_extract(self, url):
             'episode': video_data.get('title'),
             'episode_number': int_or_none(video_data.get('episode')),
         })
-
-
-class PokemonSoundLibraryIE(InfoExtractor):
-    _VALID_URL = r'https?://soundlibrary\.pokemon\.co\.jp'
-
-    _TESTS = [{
-        'url': 'https://soundlibrary.pokemon.co.jp/',
-        'info_dict': {
-            'title': 'Pokémon Diamond and Pearl Sound Tracks',
-        },
-        'playlist_mincount': 149,
-    }]
-
-    def _real_extract(self, url):
-        musicbox_webpage = self._download_webpage(
-            'https://soundlibrary.pokemon.co.jp/musicbox', None,
-            'Downloading list of songs')
-        song_titles = [x.group(1) for x in re.finditer(r'<span>([^>]+?)</span><br/>をてもち曲に加えます。', musicbox_webpage)]
-        song_titles = song_titles[4::2]
-
-        # each songs don't have permalink; instead we return all songs at once
-        song_entries = [{
-            'id': f'pokemon-soundlibrary-{song_id}',
-            'url': f'https://soundlibrary.pokemon.co.jp/api/assets/signing/sounds/wav/{song_id}.wav',
-            # note: the server always serves MP3 files, despite its extension of the URL above
-            'ext': 'mp3',
-            'acodec': 'mp3',
-            'vcodec': 'none',
-            'title': song_title,
-            'track': song_title,
-            'artist': 'Nintendo / Creatures Inc. / GAME FREAK inc.',
-            'uploader': 'Pokémon',
-            'release_year': 2006,
-            'release_date': '20060928',
-            'track_number': song_id,
-            'album': 'Pokémon Diamond and Pearl',
-        } for song_id, song_title in enumerate(song_titles, 1)]
-
-        return self.playlist_result(song_entries, playlist_title='Pokémon Diamond and Pearl Sound Tracks')
diff --git a/yt_dlp/extractor/spotify.py b/yt_dlp/extractor/spotify.py
index a2068a1b6..fef8d8dd2 100644
--- a/yt_dlp/extractor/spotify.py
+++ b/yt_dlp/extractor/spotify.py
@@ -1,12 +1,15 @@
+import functools
 import json
 import re
 
 from .common import InfoExtractor
 from ..utils import (
+    OnDemandPagedList,
     clean_podcast_url,
     float_or_none,
     int_or_none,
     strip_or_none,
+    traverse_obj,
     try_get,
     unified_strdate,
 )
@@ -25,7 +28,7 @@ def _real_initialize(self):
         self._ACCESS_TOKEN = self._download_json(
             'https://open.spotify.com/get_access_token', None)['accessToken']
 
-    def _call_api(self, operation, video_id, variables):
+    def _call_api(self, operation, video_id, variables, **kwargs):
         return self._download_json(
             'https://api-partner.spotify.com/pathfinder/v1/query', video_id, query={
                 'operationName': 'query' + operation,
@@ -35,7 +38,8 @@ def _call_api(self, operation, video_id, variables):
                         'sha256Hash': self._OPERATION_HASHES[operation],
                     },
                 })
-            }, headers={'authorization': 'Bearer ' + self._ACCESS_TOKEN})['data']
+            }, headers={'authorization': 'Bearer ' + self._ACCESS_TOKEN},
+            **kwargs)['data']
 
     def _extract_episode(self, episode, series):
         episode_id = episode['id']
@@ -143,22 +147,25 @@ class SpotifyShowIE(SpotifyBaseIE):
         },
         'playlist_mincount': 36,
     }
+    _PER_PAGE = 100
+
+    def _fetch_page(self, show_id, page=0):
+        return self._call_api('ShowEpisodes', show_id, {
+            'limit': 100,
+            'offset': page * self._PER_PAGE,
+            'uri': f'spotify:show:{show_id}',
+        }, note=f'Downloading page {page + 1} JSON metadata')['podcast']
 
     def _real_extract(self, url):
         show_id = self._match_id(url)
-        podcast = self._call_api('ShowEpisodes', show_id, {
-            'limit': 1000000000,
-            'offset': 0,
-            'uri': 'spotify:show:' + show_id,
-        })['podcast']
-        podcast_name = podcast.get('name')
-
-        entries = []
-        for item in (try_get(podcast, lambda x: x['episodes']['items']) or []):
-            episode = item.get('episode')
-            if not episode:
-                continue
-            entries.append(self._extract_episode(episode, podcast_name))
+        first_page = self._fetch_page(show_id)
+
+        def _entries(page):
+            podcast = self._fetch_page(show_id, page) if page else first_page
+            yield from map(
+                functools.partial(self._extract_episode, series=podcast.get('name')),
+                traverse_obj(podcast, ('episodes', 'items', ..., 'episode')))
 
         return self.playlist_result(
-            entries, show_id, podcast_name, podcast.get('description'))
+            OnDemandPagedList(_entries, self._PER_PAGE),
+            show_id, first_page.get('name'), first_page.get('description'))
diff --git a/yt_dlp/extractor/vevo.py b/yt_dlp/extractor/vevo.py
index bc0187511..825089f47 100644
--- a/yt_dlp/extractor/vevo.py
+++ b/yt_dlp/extractor/vevo.py
@@ -33,10 +33,124 @@ class VevoIE(VevoBaseIE):
            https?://cache\.vevo\.com/m/html/embed\.html\?video=|
            https?://videoplayer\.vevo\.com/embed/embedded\?videoId=|
            https?://embed\.vevo\.com/.*?[?&]isrc=|
+           https?://tv\.vevo\.com/watch/artist/(?:[^/]+)/|
            vevo:)
         (?P<id>[^&?#]+)'''
 
-    _TESTS = []
+    _TESTS = [{
+        'url': 'http://www.vevo.com/watch/hurts/somebody-to-die-for/GB1101300280',
+        'md5': '95ee28ee45e70130e3ab02b0f579ae23',
+        'info_dict': {
+            'id': 'GB1101300280',
+            'ext': 'mp4',
+            'title': 'Hurts - Somebody to Die For',
+            'timestamp': 1372057200,
+            'upload_date': '20130624',
+            'uploader': 'Hurts',
+            'track': 'Somebody to Die For',
+            'artist': 'Hurts',
+            'genre': 'Pop',
+        },
+        'expected_warnings': ['Unable to download SMIL file', 'Unable to download info'],
+    }, {
+        'note': 'v3 SMIL format',
+        'url': 'http://www.vevo.com/watch/cassadee-pope/i-wish-i-could-break-your-heart/USUV71302923',
+        'md5': 'f6ab09b034f8c22969020b042e5ac7fc',
+        'info_dict': {
+            'id': 'USUV71302923',
+            'ext': 'mp4',
+            'title': 'Cassadee Pope - I Wish I Could Break Your Heart',
+            'timestamp': 1392796919,
+            'upload_date': '20140219',
+            'uploader': 'Cassadee Pope',
+            'track': 'I Wish I Could Break Your Heart',
+            'artist': 'Cassadee Pope',
+            'genre': 'Country',
+        },
+        'expected_warnings': ['Unable to download SMIL file', 'Unable to download info'],
+    }, {
+        'note': 'Age-limited video',
+        'url': 'https://www.vevo.com/watch/justin-timberlake/tunnel-vision-explicit/USRV81300282',
+        'info_dict': {
+            'id': 'USRV81300282',
+            'ext': 'mp4',
+            'title': 'Justin Timberlake - Tunnel Vision (Explicit)',
+            'age_limit': 18,
+            'timestamp': 1372888800,
+            'upload_date': '20130703',
+            'uploader': 'Justin Timberlake',
+            'track': 'Tunnel Vision (Explicit)',
+            'artist': 'Justin Timberlake',
+            'genre': 'Pop',
+        },
+        'expected_warnings': ['Unable to download SMIL file', 'Unable to download info'],
+    }, {
+        'note': 'No video_info',
+        'url': 'http://www.vevo.com/watch/k-camp-1/Till-I-Die/USUV71503000',
+        'md5': '8b83cc492d72fc9cf74a02acee7dc1b0',
+        'info_dict': {
+            'id': 'USUV71503000',
+            'ext': 'mp4',
+            'title': 'K Camp ft. T.I. - Till I Die',
+            'age_limit': 18,
+            'timestamp': 1449468000,
+            'upload_date': '20151207',
+            'uploader': 'K Camp',
+            'track': 'Till I Die',
+            'artist': 'K Camp',
+            'genre': 'Hip-Hop',
+        },
+        'expected_warnings': ['Unable to download SMIL file', 'Unable to download info'],
+    }, {
+        'note': 'Featured test',
+        'url': 'https://www.vevo.com/watch/lemaitre/Wait/USUV71402190',
+        'md5': 'd28675e5e8805035d949dc5cf161071d',
+        'info_dict': {
+            'id': 'USUV71402190',
+            'ext': 'mp4',
+            'title': 'Lemaitre ft. LoLo - Wait',
+            'age_limit': 0,
+            'timestamp': 1413432000,
+            'upload_date': '20141016',
+            'uploader': 'Lemaitre',
+            'track': 'Wait',
+            'artist': 'Lemaitre',
+            'genre': 'Electronic',
+        },
+        'expected_warnings': ['Unable to download SMIL file', 'Unable to download info'],
+    }, {
+        'note': 'Only available via webpage',
+        'url': 'http://www.vevo.com/watch/GBUV71600656',
+        'md5': '67e79210613865b66a47c33baa5e37fe',
+        'info_dict': {
+            'id': 'GBUV71600656',
+            'ext': 'mp4',
+            'title': 'ABC - Viva Love',
+            'age_limit': 0,
+            'timestamp': 1461830400,
+            'upload_date': '20160428',
+            'uploader': 'ABC',
+            'track': 'Viva Love',
+            'artist': 'ABC',
+            'genre': 'Pop',
+        },
+        'expected_warnings': ['Failed to download video versions info'],
+    }, {
+        # no genres available
+        'url': 'http://www.vevo.com/watch/INS171400764',
+        'only_matching': True,
+    }, {
+        # Another case available only via the webpage; using streams/streamsV3 formats
+        # Geo-restricted to Netherlands/Germany
+        'url': 'http://www.vevo.com/watch/boostee/pop-corn-clip-officiel/FR1A91600909',
+        'only_matching': True,
+    }, {
+        'url': 'https://embed.vevo.com/?isrc=USH5V1923499&partnerId=4d61b777-8023-4191-9ede-497ed6c24647&partnerAdCode=',
+        'only_matching': True,
+    }, {
+        'url': 'https://tv.vevo.com/watch/artist/janet-jackson/US0450100550',
+        'only_matching': True,
+    }]
     _VERSIONS = {
         0: 'youtube',  # only in AuthenticateVideo videoVersions
         1: 'level3',
@@ -138,6 +252,7 @@ def _real_extract(self, url):
                     fatal=False))
             else:
                 m = re.search(r'''(?xi)
+                    _(?P<quality>[a-z0-9]+)
                     _(?P<width>[0-9]+)x(?P<height>[0-9]+)
                     _(?P<vcodec>[a-z0-9]+)
                     _(?P<vbr>[0-9]+)
@@ -149,7 +264,7 @@ def _real_extract(self, url):
 
                 formats.append({
                     'url': version_url,
-                    'format_id': 'http-%s-%s' % (version, video_version['quality']),
+                    'format_id': f'http-{version}-{video_version.get("quality") or m.group("quality")}',
                     'vcodec': m.group('vcodec'),
                     'acodec': m.group('acodec'),
                     'vbr': int(m.group('vbr')),
diff --git a/yt_dlp/extractor/yahoo.py b/yt_dlp/extractor/yahoo.py
index 3fe6192bf..171fbf585 100644
--- a/yt_dlp/extractor/yahoo.py
+++ b/yt_dlp/extractor/yahoo.py
@@ -434,7 +434,7 @@ def _entries(self, program_id):
         page = 1
         while True:
             playlist = self._download_json(
-                f'https://gyao.yahoo.co.jp/api/programs/{program_id}/videos?page={page}', program_id,
+                f'https://gyao.yahoo.co.jp/api/programs/{program_id}/videos?page={page}&serviceId=gy', program_id,
                 note=f'Downloading JSON metadata page {page}')
             if not playlist:
                 break
diff --git a/yt_dlp/extractor/youtube.py b/yt_dlp/extractor/youtube.py
index 3f6886131..c9bdd309d 100644
--- a/yt_dlp/extractor/youtube.py
+++ b/yt_dlp/extractor/youtube.py
@@ -397,8 +397,8 @@ def _check_login_required(self):
         if self._LOGIN_REQUIRED and not self._cookies_passed:
             self.raise_login_required('Login details are needed to download this content', method='cookies')
 
-    _YT_INITIAL_DATA_RE = r'(?:window\s*\[\s*["\']ytInitialData["\']\s*\]|ytInitialData)\s*=\s*({.+?})\s*;'
-    _YT_INITIAL_PLAYER_RESPONSE_RE = r'ytInitialPlayerResponse\s*=\s*({.+?})\s*;'
+    _YT_INITIAL_DATA_RE = r'(?:window\s*\[\s*["\']ytInitialData["\']\s*\]|ytInitialData)\s*=\s*({.+})\s*;'
+    _YT_INITIAL_PLAYER_RESPONSE_RE = r'ytInitialPlayerResponse\s*=\s*({.+})\s*;'
     _YT_INITIAL_BOUNDARY_RE = r'(?:var\s+meta|</script|\n)'
 
     def _get_default_ytcfg(self, client='web'):
@@ -2212,28 +2212,54 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
         }, {
             # Story. Requires specific player params to work.
             # Note: stories get removed after some period of time
-            'url': 'https://www.youtube.com/watch?v=yN3x1t3sieA',
+            'url': 'https://www.youtube.com/watch?v=vv8qTUWmulI',
             'info_dict': {
-                'id': 'yN3x1t3sieA',
+                'id': 'vv8qTUWmulI',
                 'ext': 'mp4',
-                'uploader': 'Linus Tech Tips',
-                'duration': 13,
-                'channel': 'Linus Tech Tips',
+                'availability': 'unlisted',
+                'view_count': int,
+                'channel_id': 'UCzIZ8HrzDgc-pNQDUG6avBA',
+                'upload_date': '20220526',
+                'categories': ['Education'],
+                'title': 'Story',
+                'channel': 'IT\'S HISTORY',
+                'description': '',
+                'uploader_id': 'BlastfromthePast',
+                'duration': 12,
+                'uploader': 'IT\'S HISTORY',
                 'playable_in_embed': True,
-                'tags': [],
                 'age_limit': 0,
-                'uploader_url': 'http://www.youtube.com/user/LinusTechTips',
-                'upload_date': '20220402',
-                'thumbnail': 'https://i.ytimg.com/vi_webp/yN3x1t3sieA/maxresdefault.webp',
-                'title': 'Story',
                 'live_status': 'not_live',
-                'uploader_id': 'LinusTechTips',
+                'tags': [],
+                'thumbnail': 'https://i.ytimg.com/vi_webp/vv8qTUWmulI/maxresdefault.webp',
+                'uploader_url': 'http://www.youtube.com/user/BlastfromthePast',
+                'channel_url': 'https://www.youtube.com/channel/UCzIZ8HrzDgc-pNQDUG6avBA',
+            }
+        }, {
+            'url': 'https://www.youtube.com/watch?v=tjjjtzRLHvA',
+            'info_dict': {
+                'id': 'tjjjtzRLHvA',
+                'ext': 'mp4',
+                'title': 'ハッシュタグ無し };if window.ytcsi',
+                'upload_date': '20220323',
+                'like_count': int,
+                'availability': 'unlisted',
+                'channel': 'nao20010128nao',
+                'thumbnail': 'https://i.ytimg.com/vi_webp/tjjjtzRLHvA/maxresdefault.webp',
+                'age_limit': 0,
+                'uploader': 'nao20010128nao',
+                'uploader_id': 'nao20010128nao',
+                'categories': ['Music'],
                 'view_count': int,
                 'description': '',
-                'channel_id': 'UCXuqSBlHAE6Xw-yeJA0Tunw',
-                'categories': ['Science & Technology'],
-                'channel_url': 'https://www.youtube.com/channel/UCXuqSBlHAE6Xw-yeJA0Tunw',
-                'availability': 'unlisted',
+                'channel_url': 'https://www.youtube.com/channel/UCdqltm_7iv1Vs6kp6Syke5A',
+                'channel_id': 'UCdqltm_7iv1Vs6kp6Syke5A',
+                'live_status': 'not_live',
+                'playable_in_embed': True,
+                'channel_follower_count': int,
+                'duration': 6,
+                'tags': [],
+                'uploader_url': 'http://www.youtube.com/user/nao20010128nao',
             }
         }
     ]
@@ -2530,22 +2556,16 @@ def _parse_sig_js(self, jscode):
 
     def _decrypt_signature(self, s, video_id, player_url):
         """Turn the encrypted s field into a working signature"""
-
-        if player_url is None:
-            raise ExtractorError('Cannot decrypt signature without player_url')
-
         try:
             player_id = (player_url, self._signature_cache_id(s))
             if player_id not in self._player_cache:
-                func = self._extract_signature_function(
-                    video_id, player_url, s
-                )
+                func = self._extract_signature_function(video_id, player_url, s)
                 self._player_cache[player_id] = func
             func = self._player_cache[player_id]
             self._print_sig_code(func, s)
             return func(s)
         except Exception as e:
-            raise ExtractorError('Signature extraction failed: ' + traceback.format_exc(), cause=e)
+            raise ExtractorError(traceback.format_exc(), cause=e, video_id=video_id)
 
     def _decrypt_nsig(self, s, video_id, player_url):
         """Turn the encrypted n field into a working signature"""
@@ -2721,6 +2741,21 @@ def _extract_chapters_from_engagement_panel(self, data, duration):
                 for contents in content_list
             ))), [])
 
+    @staticmethod
+    def _extract_chapters_from_description(description, duration):
+        chapters = [{'start_time': 0}]
+        for timestamp, title in re.findall(
+                r'(?m)^((?:\d+:)?\d{1,2}:\d{2})\b\W*\s(.+?)\s*$', description or ''):
+            start = parse_duration(timestamp)
+            if start and title and chapters[-1]['start_time'] < start < duration:
+                chapters[-1]['end_time'] = start
+                chapters.append({
+                    'start_time': start,
+                    'title': title,
+                })
+        chapters[-1]['end_time'] = duration
+        return chapters[1:]
+
     def _extract_chapters(self, chapter_list, chapter_time, chapter_title, duration):
         chapters = []
         last_chapter = {'start_time': 0}
@@ -2745,7 +2780,7 @@ def _extract_chapters(self, chapter_list, chapter_time, chapter_title, duration)
     def _extract_yt_initial_variable(self, webpage, regex, video_id, name):
         return self._parse_json(self._search_regex(
             (fr'{regex}\s*{self._YT_INITIAL_BOUNDARY_RE}',
-             regex), webpage, name, default='{}'), video_id, fatal=False)
+             regex), webpage, name, default='{}'), video_id, fatal=False, lenient=True)
 
     def _extract_comment(self, comment_renderer, parent=None):
         comment_id = comment_renderer.get('commentId')
@@ -3147,13 +3182,17 @@ def _extract_formats(self, streaming_data, video_id, player_url, is_live, durati
                 sc = compat_parse_qs(fmt.get('signatureCipher'))
                 fmt_url = url_or_none(try_get(sc, lambda x: x['url'][0]))
                 encrypted_sig = try_get(sc, lambda x: x['s'][0])
-                if not (sc and fmt_url and encrypted_sig):
+                if not all((sc, fmt_url, player_url, encrypted_sig)):
                     continue
-                if not player_url:
+                try:
+                    fmt_url += '&%s=%s' % (
+                        traverse_obj(sc, ('sp', -1)) or 'signature',
+                        self._decrypt_signature(encrypted_sig, video_id, player_url)
+                    )
+                except ExtractorError as e:
+                    self.report_warning('Signature extraction failed: Some formats may be missing', only_once=True)
+                    self.write_debug(e, only_once=True)
                     continue
-                signature = self._decrypt_signature(sc['s'][0], video_id, player_url)
-                sp = try_get(sc, lambda x: x['sp'][0]) or 'signature'
-                fmt_url += '&' + sp + '=' + signature
 
             query = parse_qs(fmt_url)
             throttled = False
@@ -3164,7 +3203,8 @@ def _extract_formats(self, streaming_data, video_id, player_url, is_live, durati
                 except ExtractorError as e:
                     self.report_warning(
                         'nsig extraction failed: You may experience throttling for some formats\n'
-                        f'n = {query["n"][0]} ; player = {player_url}\n{e}', only_once=True)
+                        f'n = {query["n"][0]} ; player = {player_url}', only_once=True)
+                    self.write_debug(e, only_once=True)
                     throttled = True
 
             if itag:
@@ -3414,6 +3454,13 @@ def feed_entry(name):
             or get_first(microformats, 'lengthSeconds')
             or parse_duration(search_meta('duration'))) or None
 
+        if get_first(video_details, 'isPostLiveDvr'):
+            self.write_debug('Video is in Post-Live Manifestless mode')
+            if duration or 0 > 4 * 3600:
+                self.report_warning(
+                    'The livestream has not finished processing. Only 4 hours of the video can be currently downloaded. '
+                    'This is a known issue and patches are welcome')
+
         live_broadcast_details, is_live, streaming_data, formats = self._list_formats(
             video_id, microformats, video_details, player_responses, player_url, duration)
 
@@ -3662,6 +3709,7 @@ def process_language(container, base_url, lang_code, sub_name, query):
             info['chapters'] = (
                 self._extract_chapters_from_json(initial_data, duration)
                 or self._extract_chapters_from_engagement_panel(initial_data, duration)
+                or self._extract_chapters_from_description(video_description, duration)
                 or None)
 
         contents = traverse_obj(
@@ -4015,9 +4063,12 @@ def _post_thread_continuation_entries(self, post_thread_continuation):
             return
         for content in contents:
             renderer = content.get('backstagePostThreadRenderer')
-            if not isinstance(renderer, dict):
+            if isinstance(renderer, dict):
+                yield from self._post_thread_entries(renderer)
                 continue
-            yield from self._post_thread_entries(renderer)
+            renderer = content.get('videoRenderer')
+            if isinstance(renderer, dict):
+                yield self._video_entry(renderer)
 
     r''' # unused
     def _rich_grid_entries(self, contents):
@@ -5622,11 +5673,13 @@ def _extract_notification_renderer(self, notification):
         channel = traverse_obj(
             notification, ('contextualMenu', 'menuRenderer', 'items', 1, 'menuServiceItemRenderer', 'text', 'runs', 1, 'text'),
             expected_type=str)
+        notification_title = self._get_text(notification, 'shortMessage')
+        if notification_title:
+            notification_title = notification_title.replace('\xad', '')  # remove soft hyphens
+        # TODO: handle recommended videos
         title = self._search_regex(
-            rf'{re.escape(channel)} [^:]+: (.+)', self._get_text(notification, 'shortMessage'),
+            rf'{re.escape(channel or "")}[^:]+: (.+)', notification_title,
             'video title', default=None)
-        if title:
-            title = title.replace('\xad', '')  # remove soft hyphens
         upload_date = (strftime_or_none(self._extract_time_text(notification, 'sentTimeText')[0], '%Y%m%d')
                        if self._configuration_arg('approximate_date', ie_key=YoutubeTabIE.ie_key())
                        else None)
diff --git a/yt_dlp/extractor/zattoo.py b/yt_dlp/extractor/zattoo.py
index 16f827a7e..2a7e85472 100644
--- a/yt_dlp/extractor/zattoo.py
+++ b/yt_dlp/extractor/zattoo.py
@@ -220,7 +220,7 @@ def _extract_live(self, channel_name):
             'id': channel_name,
             'title': channel_name,
             'is_live': True,
-            'format': formats,
+            'formats': formats,
             'subtitles': subtitles
         }
 
diff --git a/yt_dlp/options.py b/yt_dlp/options.py
index c0718e007..7cffcecfa 100644
--- a/yt_dlp/options.py
+++ b/yt_dlp/options.py
@@ -366,8 +366,8 @@ def _alias_callback(option, opt_str, value, parser, opts, nargs):
         '--config-locations',
         dest='config_locations', metavar='PATH', action='append',
         help=(
-            'Location of the main configuration file; either the path to the config or its containing directory. '
-            'Can be used multiple times and inside other configuration files'))
+            'Location of the main configuration file; either the path to the config or its containing directory '
+            '("-" for stdin). Can be used multiple times and inside other configuration files'))
     general.add_option(
         '--flat-playlist',
         action='store_const', dest='extract_flat', const='in_playlist', default=False,
@@ -1154,6 +1154,10 @@ def _alias_callback(option, opt_str, value, parser, opts, nargs):
         '--write-pages',
         action='store_true', dest='write_pages', default=False,
         help='Write downloaded intermediary pages to files in the current directory to debug problems')
+    verbosity.add_option(
+        '--load-pages',
+        action='store_true', dest='load_pages', default=False,
+        help=optparse.SUPPRESS_HELP)
     verbosity.add_option(
         '--youtube-print-sig-code',
         action='store_true', dest='youtube_print_sig_code', default=False,
@@ -1634,8 +1638,8 @@ def _alias_callback(option, opt_str, value, parser, opts, nargs):
         action='store_true', dest='force_keyframes_at_cuts', default=False,
         help=(
             'Force keyframes around chapters when removing/splitting them. '
-            'The resulting video may have fewer artifacts around the cuts, '
-            'but is very slow due to needing a re-encode'))
+            'This is slow due to needing a re-encode, but '
+            'the resulting video may have fewer artifacts around the cuts'))
     postproc.add_option(
         '--no-force-keyframes-at-cuts',
         action='store_false', dest='force_keyframes_at_cuts',
diff --git a/yt_dlp/postprocessor/common.py b/yt_dlp/postprocessor/common.py
index addc46e5b..9f22b378d 100644
--- a/yt_dlp/postprocessor/common.py
+++ b/yt_dlp/postprocessor/common.py
@@ -176,6 +176,8 @@ def add_progress_hook(self, ph):
 
     def report_progress(self, s):
         s['_default_template'] = '%(postprocessor)s %(status)s' % s
+        if not self._downloader:
+            return
 
         progress_dict = s.copy()
         progress_dict.pop('info_dict')
@@ -184,7 +186,8 @@ def report_progress(self, s):
         progress_template = self.get_param('progress_template', {})
         tmpl = progress_template.get('postprocess')
         if tmpl:
-            self._downloader.to_stdout(self._downloader.evaluate_outtmpl(tmpl, progress_dict))
+            self._downloader.to_screen(
+                self._downloader.evaluate_outtmpl(tmpl, progress_dict), skip_eol=True, quiet=False)
 
         self._downloader.to_console_title(self._downloader.evaluate_outtmpl(
             progress_template.get('postprocess-title') or 'yt-dlp %(progress._default_template)s',
diff --git a/yt_dlp/postprocessor/ffmpeg.py b/yt_dlp/postprocessor/ffmpeg.py
index 5a1d8561f..2a456e567 100644
--- a/yt_dlp/postprocessor/ffmpeg.py
+++ b/yt_dlp/postprocessor/ffmpeg.py
@@ -66,15 +66,6 @@ def __init__(self, downloader=None):
         self._prefer_ffmpeg = self.get_param('prefer_ffmpeg', True)
         self._paths = self._determine_executables()
 
-    def check_version(self):
-        if not self.available:
-            raise FFmpegPostProcessorError('ffmpeg not found. Please install or provide the path using --ffmpeg-location')
-
-        required_version = '10-0' if self.basename == 'avconv' else '1.0'
-        if is_outdated_version(self._version, required_version):
-            self.report_warning(f'Your copy of {self.basename} is outdated, update {self.basename} '
-                                f'to version {required_version} or newer if you encounter any errors')
-
     @staticmethod
     def get_versions_and_features(downloader=None):
         pp = FFmpegPostProcessor(downloader)
@@ -205,6 +196,15 @@ def stream_copy_opts(copy=True, *, ext=None):
         if ext in ('mp4', 'mov', 'm4a'):
             yield from ('-c:s', 'mov_text')
 
+    def check_version(self):
+        if not self.available:
+            raise FFmpegPostProcessorError('ffmpeg not found. Please install or provide the path using --ffmpeg-location')
+
+        required_version = '10-0' if self.basename == 'avconv' else '1.0'
+        if is_outdated_version(self._version, required_version):
+            self.report_warning(f'Your copy of {self.basename} is outdated, update {self.basename} '
+                                f'to version {required_version} or newer if you encounter any errors')
+
     def get_audio_codec(self, path):
         if not self.probe_available and not self.available:
             raise PostProcessingError('ffprobe and ffmpeg not found. Please install or provide the path using --ffmpeg-location')
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
index 2e3c51562..b0300b724 100644
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -619,9 +619,9 @@ def sanitize_open(filename, open_mode):
                     # Ref: https://github.com/yt-dlp/yt-dlp/issues/3124
                     raise LockingUnsupportedError()
                 stream = locked_file(filename, open_mode, block=False).__enter__()
-            except LockingUnsupportedError:
+            except OSError:
                 stream = open(filename, open_mode)
-            return (stream, filename)
+            return stream, filename
         except OSError as err:
             if attempt or err.errno in (errno.EACCES,):
                 raise
@@ -815,12 +815,9 @@ def escapeHTML(text):
 
 
 def process_communicate_or_kill(p, *args, **kwargs):
-    try:
-        return p.communicate(*args, **kwargs)
-    except BaseException:  # Including KeyboardInterrupt
-        p.kill()
-        p.wait()
-        raise
+    write_string('DeprecationWarning: yt_dlp.utils.process_communicate_or_kill is deprecated '
+                 'and may be removed in a future version. Use yt_dlp.utils.Popen.communicate_or_kill instead')
+    return Popen.communicate_or_kill(p, *args, **kwargs)
 
 
 class Popen(subprocess.Popen):
@@ -834,7 +831,12 @@ def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs, startupinfo=self._startupinfo)
 
     def communicate_or_kill(self, *args, **kwargs):
-        return process_communicate_or_kill(self, *args, **kwargs)
+        try:
+            return self.communicate(*args, **kwargs)
+        except BaseException:  # Including KeyboardInterrupt
+            self.kill()
+            self.wait()
+            raise
 
 
 def get_subprocess_encoding():
@@ -921,22 +923,23 @@ def make_HTTPS_handler(params, **kwargs):
         context.options |= 4  # SSL_OP_LEGACY_SERVER_CONNECT
         # Allow use of weaker ciphers in Python 3.10+. See https://bugs.python.org/issue43998
         context.set_ciphers('DEFAULT')
+
     context.verify_mode = ssl.CERT_REQUIRED if opts_check_certificate else ssl.CERT_NONE
     if opts_check_certificate:
         if has_certifi and 'no-certifi' not in params.get('compat_opts', []):
             context.load_verify_locations(cafile=certifi.where())
-        else:
-            try:
-                context.load_default_certs()
-                # Work around the issue in load_default_certs when there are bad certificates. See:
-                # https://github.com/yt-dlp/yt-dlp/issues/1060,
-                # https://bugs.python.org/issue35665, https://bugs.python.org/issue45312
-            except ssl.SSLError:
-                # enum_certificates is not present in mingw python. See https://github.com/yt-dlp/yt-dlp/issues/1151
-                if sys.platform == 'win32' and hasattr(ssl, 'enum_certificates'):
-                    for storename in ('CA', 'ROOT'):
-                        _ssl_load_windows_store_certs(context, storename)
-                context.set_default_verify_paths()
+        try:
+            context.load_default_certs()
+        # Work around the issue in load_default_certs when there are bad certificates. See:
+        # https://github.com/yt-dlp/yt-dlp/issues/1060,
+        # https://bugs.python.org/issue35665, https://bugs.python.org/issue45312
+        except ssl.SSLError:
+            # enum_certificates is not present in mingw python. See https://github.com/yt-dlp/yt-dlp/issues/1151
+            if sys.platform == 'win32' and hasattr(ssl, 'enum_certificates'):
+                for storename in ('CA', 'ROOT'):
+                    _ssl_load_windows_store_certs(context, storename)
+            context.set_default_verify_paths()
+
     client_certfile = params.get('client_certificate')
     if client_certfile:
         try:
@@ -945,6 +948,13 @@ def make_HTTPS_handler(params, **kwargs):
                 password=params.get('client_certificate_password'))
         except ssl.SSLError:
             raise YoutubeDLError('Unable to load client certificate')
+
+    # Some servers may reject requests if ALPN extension is not sent. See:
+    # https://github.com/python/cpython/issues/85140
+    # https://github.com/yt-dlp/yt-dlp/issues/3878
+    with contextlib.suppress(NotImplementedError):
+        context.set_alpn_protocols(['http/1.1'])
+
     return YoutubeDLHTTPSHandler(params, context=context, **kwargs)
 
 
@@ -1885,11 +1895,11 @@ def platform_name():
 
 @functools.cache
 def get_windows_version():
-    ''' Get Windows version. None if it's not running on Windows '''
+    ''' Get Windows version. returns () if it's not running on Windows '''
     if compat_os_name == 'nt':
         return version_tuple(platform.win32_ver()[1])
     else:
-        return None
+        return ()
 
 
 def write_string(s, out=None, encoding=None):
@@ -1899,14 +1909,14 @@ def write_string(s, out=None, encoding=None):
     if compat_os_name == 'nt' and supports_terminal_sequences(out):
         s = re.sub(r'([\r\n]+)', r' \1', s)
 
-    enc = None
+    enc, buffer = None, out
     if 'b' in getattr(out, 'mode', ''):
         enc = encoding or preferredencoding()
     elif hasattr(out, 'buffer'):
-        out = out.buffer
+        buffer = out.buffer
         enc = encoding or getattr(out, 'encoding', None) or preferredencoding()
 
-    out.write(s.encode(enc, 'ignore') if enc else s)
+    buffer.write(s.encode(enc, 'ignore') if enc else s)
     out.flush()
 
 
@@ -1925,7 +1935,7 @@ def intlist_to_bytes(xs):
     return compat_struct_pack('%dB' % len(xs), *xs)
 
 
-class LockingUnsupportedError(IOError):
+class LockingUnsupportedError(OSError):
     msg = 'File locking is not supported on this platform'
 
     def __init__(self):
@@ -5089,7 +5099,7 @@ def jwt_decode_hs256(jwt):
 @functools.cache
 def supports_terminal_sequences(stream):
     if compat_os_name == 'nt':
-        if not WINDOWS_VT_MODE or get_windows_version() < (10, 0, 10586):
+        if not WINDOWS_VT_MODE:
             return False
     elif not os.getenv('TERM'):
         return False
@@ -5100,7 +5110,7 @@ def supports_terminal_sequences(stream):
 
 
 def windows_enable_vt_mode():  # TODO: Do this the proper way https://bugs.python.org/issue30075
-    if compat_os_name != 'nt':
+    if get_windows_version() < (10, 0, 10586):
         return
     global WINDOWS_VT_MODE
     startupinfo = subprocess.STARTUPINFO()
@@ -5163,6 +5173,12 @@ def parse_http_range(range):
     return int(crg.group(1)), int_or_none(crg.group(2)), int_or_none(crg.group(3))
 
 
+def read_stdin(what):
+    eof = 'Ctrl+Z' if compat_os_name == 'nt' else 'Ctrl+D'
+    write_string(f'Reading {what} from STDIN - EOF ({eof}) to end:\n')
+    return sys.stdin
+
+
 class Config:
     own_args = None
     parsed_args = None
@@ -5188,6 +5204,9 @@ def init(self, args=None, filename=None):
         self.parsed_args, self.filename = args, filename
 
         for location in opts.config_locations or []:
+            if location == '-':
+                self.append_config(shlex.split(read_stdin('options'), comments=True), label='stdin')
+                continue
             location = os.path.join(directory, expand_path(location))
             if os.path.isdir(location):
                 location = os.path.join(location, 'yt-dlp.conf')


Test patch:
diff --git a/devscripts/run_tests.sh b/devscripts/run_tests.sh
index e9904ae35..d496a092b 100755
--- a/devscripts/run_tests.sh
+++ b/devscripts/run_tests.sh
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/usr/bin/env sh
 
 if [ -z $1 ]; then
     test_set='test'
diff --git a/test/test_InfoExtractor.py b/test/test_InfoExtractor.py
index 257ea7dd3..928246668 100644
--- a/test/test_InfoExtractor.py
+++ b/test/test_InfoExtractor.py
@@ -502,6 +502,24 @@ def test_parse_html5_media_entries(self):
                 }],
             })
 
+        # from https://0000.studio/
+        # with type attribute but without extension in URL
+        expect_dict(
+            self,
+            self.ie._parse_html5_media_entries(
+                'https://0000.studio',
+                r'''
+                <video src="https://d1ggyt9m8pwf3g.cloudfront.net/protected/ap-northeast-1:1864af40-28d5-492b-b739-b32314b1a527/archive/clip/838db6a7-8973-4cd6-840d-8517e4093c92"
+                    controls="controls" type="video/mp4" preload="metadata" autoplay="autoplay" playsinline class="object-contain">
+                </video>
+                ''', None)[0],
+            {
+                'formats': [{
+                    'url': 'https://d1ggyt9m8pwf3g.cloudfront.net/protected/ap-northeast-1:1864af40-28d5-492b-b739-b32314b1a527/archive/clip/838db6a7-8973-4cd6-840d-8517e4093c92',
+                    'ext': 'mp4',
+                }],
+            })
+
     def test_extract_jwplayer_data_realworld(self):
         # from http://www.suffolk.edu/sjc/
         expect_dict(