Repository: yt-dlp/yt-dlp
Pull request #4965 (2022-09-18 23:46:53)

[soundcloud] Enriched search results
### Description of your *pull request* and other information

</details>

<!--

Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible

-->

Moved _extract_info_dict, _IMAGE_REPL_RE, _ARTWORK_MAP from SoundCloudIE to SoundcloudBaseIE so that they could be used in any subextractor. Modified _extract_info_dict to take an 'extract_flat' argument. When false, it will still extract info from a SoundCloud track, but it will not make any network requests. Used _extract_info_dict(extract_flat=False) to then fatten up search results of SoundcloudSearchIE. I also fixed SoundcloudSearchIE's test case by adding the 'id' field.

Preview:
```json
{
  "id": "hello",
  "title": "hello",
  "_type": "playlist",
  "entries": [
    {
      "id": "230155983",
      "uploader": "Grace Davies",
      "uploader_id": "44817748",
      "uploader_url": "https://soundcloud.com/gracedaviesofficial",
      "timestamp": 1445871069,
      "title": "Hello - Adele",
      "description": "Vocals/Piano/Recording/Production by me, in my bedroom!\n\nFollow me on:\nWebsite: www.gracedaviesofficial.com\nFacebook: gracedaviesofficial\nYouTube: gracedaviesoffical\nInstagram: gracedaviesofficial\nSnapchat: gracedofficial\nTwitter: gracedavies",
      "thumbnails": [
        {
          "id": "mini",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-mini.jpg",
          "width": 16,
          "height": 16
        },
        {
          "id": "tiny",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-tiny.jpg",
          "width": 20,
          "height": 20
        },
        {
          "id": "small",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-small.jpg",
          "width": 32,
          "height": 32
        },
        {
          "id": "badge",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-badge.jpg",
          "width": 47,
          "height": 47
        },
        {
          "id": "t67x67",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-t67x67.jpg",
          "width": 67,
          "height": 67
        },
        {
          "id": "large",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-large.jpg",
          "width": 100,
          "height": 100
        },
        {
          "id": "t300x300",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-t300x300.jpg",
          "width": 300,
          "height": 300
        },
        {
          "id": "crop",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-crop.jpg",
          "width": 400,
          "height": 400
        },
        {
          "id": "t500x500",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-t500x500.jpg",
          "width": 500,
          "height": 500
        },
        {
          "id": "original",
          "url": "https://i1.sndcdn.com/artworks-000134001026-tugyco-original.jpg",
          "preference": 10
        }
      ],
      "duration": 300.88,
      "webpage_url": "https://soundcloud.com/gracedaviesofficial/hello-adele",
      "license": "all-rights-reserved",
      "view_count": 67911929,
      "like_count": 1074562,
      "comment_count": 8909,
      "repost_count": 53298,
      "genre": "hello",
      "formats": [],
      "ie_key": "Soundcloud",
      "_type": "url",
      "url": "https://api.soundcloud.com/tracks/230155983",
      "__x_forwarded_for_ip": null
    }
  ],
  "webpage_url": "scsearch1:hello",
  "original_url": "scsearch1:hello",
  "webpage_url_basename": "hello",
  "webpage_url_domain": null,
  "extractor": "soundcloud:search",
  "extractor_key": "SoundcloudSearch",
  "requested_entries": [
    1
  ],
  "playlist_count": 1,
  "__files_to_move": {},
  "epoch": 1663544663,
  "_version": {
    "version": "2022.09.01",
    "current_git_head": "b2d5beeaa",
    "release_git_head": "5d7c7d656",
    "repository": "yt-dlp/yt-dlp"
  }
}
```

<details open><summary>Template</summary> <!-- OPEN is intentional -->

<!--

# PLEASE FOLLOW THE GUIDE BELOW

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like

-->

### Before submitting a *pull request* make sure you have:
- [X] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [X] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [X] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [X] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [X] Fix or improvement to an extractor (Make sure to add/update tests)
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [ ] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))



Patch:
diff --git a/README.md b/README.md
index 9f331663d..07ed04061 100644
--- a/README.md
+++ b/README.md
@@ -3,7 +3,7 @@
 
 [![YT-DLP](https://raw.githubusercontent.com/yt-dlp/yt-dlp/master/.github/banner.svg)](#readme)
 
-[![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=brightgreen&label=Download&style=for-the-badge)](##installation "Installation")
+[![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=brightgreen&label=Download&style=for-the-badge)](#installation "Installation")
 [![PyPi](https://img.shields.io/badge/-PyPi-blue.svg?logo=pypi&labelColor=555555&style=for-the-badge)](https://pypi.org/project/yt-dlp "PyPi")
 [![Donate](https://img.shields.io/badge/_-Donate-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](Collaborators.md#collaborators "Donate")
 [![Matrix](https://img.shields.io/matrix/yt-dlp:matrix.org?color=brightgreen&labelColor=555555&label=&logo=element&style=for-the-badge)](https://matrix.to/#/#yt-dlp:matrix.org "Matrix")
@@ -562,7 +562,7 @@ ## Download Options:
                                     Needs ffmpeg. This option can be used
                                     multiple times to download multiple
                                     sections, e.g. --download-sections
-                                    "*10:15-15:00" --download-sections "intro"
+                                    "*10:15-inf" --download-sections "intro"
     --downloader [PROTO:]NAME       Name or path of the external downloader to
                                     use (optionally) prefixed by the protocols
                                     (http, ftp, m3u8, dash, rstp, rtmp, mms) to
@@ -1079,9 +1079,9 @@ ## Extractor Options:
     --no-hls-split-discontinuity    Do not split HLS playlists to different
                                     formats at discontinuities such as ad breaks
                                     (default)
-    --extractor-args KEY:ARGS       Pass these arguments to the extractor. See
-                                    "EXTRACTOR ARGUMENTS" for details. You can
-                                    use this option multiple times to give
+    --extractor-args IE_KEY:ARGS    Pass ARGS arguments to the IE_KEY extractor.
+                                    See "EXTRACTOR ARGUMENTS" for details. You
+                                    can use this option multiple times to give
                                     arguments for different extractors
 
 # CONFIGURATION
@@ -1092,14 +1092,14 @@ # CONFIGURATION
 1. **Portable Configuration**: `yt-dlp.conf` in the same directory as the bundled binary. If you are running from source-code (`<root dir>/yt_dlp/__main__.py`), the root directory is used instead.
 1. **Home Configuration**: `yt-dlp.conf` in the home path given by `-P`, or in the current directory if no such path is given
 1. **User Configuration**:
-    * `%XDG_CONFIG_HOME%/yt-dlp/config` (recommended on Linux/macOS)
-    * `%XDG_CONFIG_HOME%/yt-dlp.conf`
-    * `%APPDATA%/yt-dlp/config` (recommended on Windows)
-    * `%APPDATA%/yt-dlp/config.txt`
+    * `$XDG_CONFIG_HOME/yt-dlp/config` (recommended on Linux/macOS)
+    * `$XDG_CONFIG_HOME/yt-dlp.conf`
+    * `$APPDATA/yt-dlp/config` (recommended on Windows)
+    * `$APPDATA/yt-dlp/config.txt`
     * `~/yt-dlp.conf`
     * `~/yt-dlp.conf.txt`
     
-    `%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined. On windows, `%APPDATA%` generally points to `C:\Users\<user name>\AppData\Roaming` and `~` points to `%HOME%` if present, `%USERPROFILE%` (generally `C:\Users\<user name>`), or `%HOMEDRIVE%%HOMEPATH%`
+    `$XDG_CONFIG_HOME` defaults to `~/.config` if undefined. On windows, `$APPDATA` generally points to `C:\Users\<user name>\AppData\Roaming` and `~` points to `$HOME` if present, `$USERPROFILE` (generally `C:\Users\<user name>`), or `${HOMEDRIVE}${HOMEPATH}`
 
 1. **System Configuration**: `/etc/yt-dlp.conf`
 
@@ -1120,7 +1120,7 @@ # Save all videos under YouTube directory in your home directory
 -o ~/YouTube/%(title)s.%(ext)s
 ```
 
-Note that options in configuration file are just the same options aka switches used in regular command line calls; thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`.
+Note that options in configuration file are just the same options aka switches used in regular command line calls; thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`. They must also be quoted when necessary as-if it were a UNIX shell.
 
 You can use `--ignore-config` if you want to disable all configuration files for a particular yt-dlp run. If `--ignore-config` is found inside any configuration file, no further configuration will be loaded. For example, having the option in the portable configuration file prevents loading of home, user, and system configurations. Additionally, (for backward compatibility) if `--ignore-config` is found inside the system configuration file, the user configuration is not loaded.
 
@@ -1148,7 +1148,7 @@ ### Authentication with `.netrc` file
 ```
 To activate authentication with the `.netrc` file you should pass `--netrc` to yt-dlp or place it in the [configuration file](#configuration).
 
-The default location of the .netrc file is `$HOME` (`~`) in UNIX. On Windows, it is `%HOME%` if present, `%USERPROFILE%` (generally `C:\Users\<user name>`) or `%HOMEDRIVE%%HOMEPATH%`
+The default location of the .netrc file is `$HOME` (`~`). On Windows, if `$HOME` is not present, `$USERPROFILE` (generally `C:\Users\<user name>`) or `${HOMEDRIVE}${HOMEPATH}` is used
 
 # OUTPUT TEMPLATE
 
@@ -1627,7 +1627,7 @@ # MODIFYING METADATA
 
 The general syntax of `--parse-metadata FROM:TO` is to give the name of a field or an [output template](#output-template) to extract data from, and the format to interpret it as, separated by a colon `:`. Either a [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax) with named capture groups or a similar syntax to the [output template](#output-template) (only `%(field)s` formatting is supported) can be used for `TO`. The option can be used multiple times to parse and modify various fields.
 
-Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--add-metadata`.
+Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--embed-metadata`.
 
 This option also has a few special uses:
 
@@ -1673,11 +1673,11 @@ # Set title as "Series name S01E05"
 $ yt-dlp --parse-metadata "%(series)s S%(season_number)02dE%(episode_number)02d:%(title)s"
 
 # Prioritize uploader as the "artist" field in video metadata
-$ yt-dlp --parse-metadata "%(uploader|)s:%(meta_artist)s" --add-metadata
+$ yt-dlp --parse-metadata "%(uploader|)s:%(meta_artist)s" --embed-metadata
 
 # Set "comment" field in video metadata using description instead of webpage_url,
 # handling multiple lines correctly
-$ yt-dlp --parse-metadata "description:(?s)(?P<meta_comment>.+)" --add-metadata
+$ yt-dlp --parse-metadata "description:(?s)(?P<meta_comment>.+)" --embed-metadata
 
 # Do not set any "synopsis" in the video metadata
 $ yt-dlp --parse-metadata ":(?P<meta_synopsis>)"
@@ -1697,16 +1697,16 @@ # EXTRACTOR ARGUMENTS
 The following extractors use this feature:
 
 #### youtube
+* `lang`: Language code to prefer translated metadata of this language (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube.py](https://github.com/yt-dlp/yt-dlp/blob/c26f9b991a0681fd3ea548d535919cec1fbbd430/yt_dlp/extractor/youtube.py#L381-L390) for list of supported content language codes
 * `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively
 * `player_client`: Clients to extract video data from. The main clients are `web`, `android` and `ios` with variants `_music`, `_embedded`, `_embedscreen`, `_creator` (e.g. `web_embedded`); and `mweb` and `tv_embedded` (agegate bypass) with no variants. By default, `android,web` is used, but `tv_embedded` and `creator` variants are added as required for age-gated videos. Similarly the music variants are added for `music.youtube.com` urls. You can use `all` to use all the clients, and `default` for the default clients.
 * `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause some issues. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) for more details
-* `include_live_dash`: Include live dash formats even without `--live-from-start` (These formats don't download properly)
 * `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side)
 * `max_comments`: Limit the amount of comments to gather. Comma-separated list of integers representing `max-comments,max-parents,max-replies,max-replies-per-thread`. Default is `all,all,all,all`
     * E.g. `all,all,1000,10` will get a maximum of 1000 replies total, with up to 10 replies per thread. `1000,all,100` will get a maximum of 1000 comments, with a maximum of 100 replies total
+* `include_live_dash`: Include live dash formats even without `--live-from-start` (These formats don't download properly)
 * `innertube_host`: Innertube API host to use for all API requests; e.g. `studio.youtube.com`, `youtubei.googleapis.com`. Note that cookies exported from one subdomain will not work on others
 * `innertube_key`: Innertube API key to use for all API requests
-* `lang`: Language code to prefer translated metadata of this language (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube.py](https://github.com/yt-dlp/yt-dlp/blob/c26f9b991a0681fd3ea548d535919cec1fbbd430/yt_dlp/extractor/youtube.py#L381-L390) for list of supported content language codes
 
 #### youtubetab (YouTube playlists, channels, feeds, etc.)
 * `skip`: One or more of `webpage` (skip initial webpage download), `authcheck` (allow the download of playlists requiring authentication when no initial webpage is downloaded. This may cause unwanted behavior, see [#1122](https://github.com/yt-dlp/yt-dlp/pull/1122) for more details)
diff --git a/devscripts/make_lazy_extractors.py b/devscripts/make_lazy_extractors.py
index 383c7e057..2d4530eb9 100644
--- a/devscripts/make_lazy_extractors.py
+++ b/devscripts/make_lazy_extractors.py
@@ -2,8 +2,8 @@
 
 # Allow direct execution
 import os
-import sys
 import shutil
+import sys
 
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
diff --git a/setup.cfg b/setup.cfg
index d33c7d854..2def390f5 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -10,6 +10,14 @@ per_file_ignores =
     devscripts/lazy_load_template.py: F401
 
 
+[autoflake]
+ignore-init-module-imports = true
+ignore-pass-after-docstring = true
+remove-all-unused-imports = true
+remove-duplicate-keys = true
+remove-unused-variables = true
+
+
 [tool:pytest]
 addopts = -ra -v --strict-markers
 markers =
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
index 0bfc47767..7b0616cba 100644
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -1687,8 +1687,8 @@ def process_ie_result(self, ie_result, download=True, extra_info=None):
         elif result_type in ('playlist', 'multi_video'):
             # Protect from infinite recursion due to recursively nested playlists
             # (see https://github.com/ytdl-org/youtube-dl/issues/27833)
-            webpage_url = ie_result['webpage_url']
-            if webpage_url in self._playlist_urls:
+            webpage_url = ie_result.get('webpage_url')  # Playlists maynot have webpage_url
+            if webpage_url and webpage_url in self._playlist_urls:
                 self.to_screen(
                     '[download] Skipping already downloaded playlist: %s'
                     % ie_result.get('title') or ie_result.get('id'))
@@ -1742,14 +1742,17 @@ def _playlist_infodict(ie_result, strict=False, **kwargs):
         }
         if strict:
             return info
+        if ie_result.get('webpage_url'):
+            info.update({
+                'webpage_url': ie_result['webpage_url'],
+                'webpage_url_basename': url_basename(ie_result['webpage_url']),
+                'webpage_url_domain': get_domain(ie_result['webpage_url']),
+            })
         return {
             **info,
             'playlist_index': 0,
             '__last_playlist_index': max(ie_result['requested_entries'] or (0, 0)),
             'extractor': ie_result['extractor'],
-            'webpage_url': ie_result['webpage_url'],
-            'webpage_url_basename': url_basename(ie_result['webpage_url']),
-            'webpage_url_domain': get_domain(ie_result['webpage_url']),
             'extractor_key': ie_result['extractor_key'],
         }
 
@@ -2697,24 +2700,21 @@ def is_wellformed(f):
             # Process what we can, even without any available formats.
             formats_to_download = [{}]
 
-        requested_ranges = self.params.get('download_ranges')
-        if requested_ranges:
-            requested_ranges = tuple(requested_ranges(info_dict, self))
-
+        requested_ranges = tuple(self.params.get('download_ranges', lambda *_: [{}])(info_dict, self))
         best_format, downloaded_formats = formats_to_download[-1], []
         if download:
-            if best_format:
+            if best_format and requested_ranges:
                 def to_screen(*msg):
                     self.to_screen(f'[info] {info_dict["id"]}: {" ".join(", ".join(variadic(m)) for m in msg)}')
 
                 to_screen(f'Downloading {len(formats_to_download)} format(s):',
                           (f['format_id'] for f in formats_to_download))
-                if requested_ranges:
+                if requested_ranges != ({}, ):
                     to_screen(f'Downloading {len(requested_ranges)} time ranges:',
                               (f'{c["start_time"]:.1f}-{c["end_time"]:.1f}' for c in requested_ranges))
             max_downloads_reached = False
 
-            for fmt, chapter in itertools.product(formats_to_download, requested_ranges or [{}]):
+            for fmt, chapter in itertools.product(formats_to_download, requested_ranges):
                 new_info = self._copy_infodict(info_dict)
                 new_info.update(fmt)
                 offset, duration = info_dict.get('section_start') or 0, info_dict.get('duration') or float('inf')
diff --git a/yt_dlp/__init__.py b/yt_dlp/__init__.py
index 29c467b0e..9382ff43b 100644
--- a/yt_dlp/__init__.py
+++ b/yt_dlp/__init__.py
@@ -489,7 +489,7 @@ def report_conflict(arg1, opt1, arg2='--allow-unplayable-formats', opt2='allow_u
                     val1=opts.sponskrub and opts.sponskrub_cut)
 
     # Conflicts with --allow-unplayable-formats
-    report_conflict('--add-metadata', 'addmetadata')
+    report_conflict('--embed-metadata', 'addmetadata')
     report_conflict('--embed-chapters', 'addchapters')
     report_conflict('--embed-info-json', 'embed_infojson')
     report_conflict('--embed-subs', 'embedsubtitles')
diff --git a/yt_dlp/cookies.py b/yt_dlp/cookies.py
index 24a8250da..3032d0712 100644
--- a/yt_dlp/cookies.py
+++ b/yt_dlp/cookies.py
@@ -845,12 +845,15 @@ def _get_linux_keyring_password(browser_keyring_name, keyring, logger):
 def _get_mac_keyring_password(browser_keyring_name, logger):
     logger.debug('using find-generic-password to obtain password from OSX keychain')
     try:
-        stdout, _, _ = Popen.run(
+        stdout, _, returncode = Popen.run(
             ['security', 'find-generic-password',
              '-w',  # write password to stdout
              '-a', browser_keyring_name,  # match 'account'
              '-s', f'{browser_keyring_name} Safe Storage'],  # match 'service'
             stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
+        if returncode:
+            logger.warning('find-generic-password failed')
+            return None
         return stdout.rstrip(b'\n')
     except Exception as e:
         logger.warning(f'exception running find-generic-password: {error_to_str(e)}')
diff --git a/yt_dlp/extractor/_extractors.py b/yt_dlp/extractor/_extractors.py
index 43e2f93d3..f334b7833 100644
--- a/yt_dlp/extractor/_extractors.py
+++ b/yt_dlp/extractor/_extractors.py
@@ -21,6 +21,7 @@
     YoutubeYtBeIE,
     YoutubeYtUserIE,
     YoutubeWatchLaterIE,
+    YoutubeShortsAudioPivotIE
 )
 
 from .abc import (
@@ -1618,6 +1619,7 @@
 from .slideshare import SlideshareIE
 from .slideslive import SlidesLiveIE
 from .slutload import SlutloadIE
+from .smotrim import SmotrimIE
 from .snotr import SnotrIE
 from .sohu import SohuIE
 from .sonyliv import (
@@ -2141,6 +2143,7 @@
 from .wistia import (
     WistiaIE,
     WistiaPlaylistIE,
+    WistiaChannelIE,
 )
 from .worldstarhiphop import WorldStarHipHopIE
 from .wppilot import (
diff --git a/yt_dlp/extractor/amazon.py b/yt_dlp/extractor/amazon.py
index 56a8d844a..9e9e9772d 100644
--- a/yt_dlp/extractor/amazon.py
+++ b/yt_dlp/extractor/amazon.py
@@ -39,7 +39,7 @@ class AmazonStoreIE(InfoExtractor):
     def _real_extract(self, url):
         id = self._match_id(url)
 
-        for retry in self.RetryManager(fatal=True):
+        for retry in self.RetryManager():
             webpage = self._download_webpage(url, id)
             try:
                 data_json = self._parse_json(self._html_search_regex(r'var\s?obj\s?=\s?jQuery\.parseJSON\(\'(.*)\'\)', webpage, 'data'), id)
diff --git a/yt_dlp/extractor/arte.py b/yt_dlp/extractor/arte.py
index 25ecb4230..d3ec4a66c 100644
--- a/yt_dlp/extractor/arte.py
+++ b/yt_dlp/extractor/arte.py
@@ -135,6 +135,7 @@ def _real_extract(self, url):
                 'Video is not available in this language edition of Arte or broadcast rights expired', expected=True)
 
         formats, subtitles = [], {}
+        secondary_formats = []
         for stream in config['data']['attributes']['streams']:
             # official player contains code like `e.get("versions")[0].eStat.ml5`
             stream_version = stream['versions'][0]
@@ -152,22 +153,26 @@ def _real_extract(self, url):
                     not m.group('sdh_sub'),                 # and we prefer not the hard-of-hearing subtitles if there are subtitles
                 )))
 
+            short_label = traverse_obj(stream_version, 'shortLabel', expected_type=str, default='?')
             if stream['protocol'].startswith('HLS'):
                 fmts, subs = self._extract_m3u8_formats_and_subtitles(
                     stream['url'], video_id=video_id, ext='mp4', m3u8_id=stream_version_code, fatal=False)
                 for fmt in fmts:
                     fmt.update({
-                        'format_note': f'{stream_version.get("label", "unknown")} [{stream_version.get("shortLabel", "?")}]',
+                        'format_note': f'{stream_version.get("label", "unknown")} [{short_label}]',
                         'language_preference': lang_pref,
                     })
-                formats.extend(fmts)
+                if any(map(short_label.startswith, ('cc', 'OGsub'))):
+                    secondary_formats.extend(fmts)
+                else:
+                    formats.extend(fmts)
                 self._merge_subtitles(subs, target=subtitles)
 
             elif stream['protocol'] in ('HTTPS', 'RTMP'):
                 formats.append({
                     'format_id': f'{stream["protocol"]}-{stream_version_code}',
                     'url': stream['url'],
-                    'format_note': f'{stream_version.get("label", "unknown")} [{stream_version.get("shortLabel", "?")}]',
+                    'format_note': f'{stream_version.get("label", "unknown")} [{short_label}]',
                     'language_preference': lang_pref,
                     # 'ext': 'mp4',  # XXX: may or may not be necessary, at least for HTTPS
                 })
@@ -179,6 +184,8 @@ def _real_extract(self, url):
             # The JS also looks for chapters in config['data']['attributes']['chapters'],
             # but I am yet to find a video having those
 
+        formats.extend(secondary_formats)
+        self._remove_duplicate_formats(formats)
         self._sort_formats(formats)
 
         metadata = config['data']['attributes']['metadata']
diff --git a/yt_dlp/extractor/common.py b/yt_dlp/extractor/common.py
index e8fa8fdde..d36f025ab 100644
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -1236,7 +1236,7 @@ def _search_json(self, start_pattern, string, name, video_id, *, end_pattern='',
             fatal, has_default = False, True
 
         json_string = self._search_regex(
-            rf'{start_pattern}\s*(?P<json>{{\s*{contains_pattern}\s*}})\s*{end_pattern}',
+            rf'(?:{start_pattern})\s*(?P<json>{{\s*(?:{contains_pattern})\s*}})\s*(?:{end_pattern})',
             string, name, group='json', fatal=fatal, default=None if has_default else NO_DEFAULT)
         if not json_string:
             return default
@@ -1536,10 +1536,10 @@ def extract_chapter_information(e):
                 info['chapters'] = chapters
 
         def extract_video_object(e):
-            assert is_type(e, 'VideoObject')
             author = e.get('author')
             info.update({
                 'url': url_or_none(e.get('contentUrl')),
+                'ext': mimetype2ext(e.get('encodingFormat')),
                 'title': unescapeHTML(e.get('name')),
                 'description': unescapeHTML(e.get('description')),
                 'thumbnails': [{'url': unescapeHTML(url)}
@@ -1552,12 +1552,19 @@ def extract_video_object(e):
                 # however some websites are using 'Text' type instead.
                 # 1. https://schema.org/VideoObject
                 'uploader': author.get('name') if isinstance(author, dict) else author if isinstance(author, str) else None,
+                'artist': traverse_obj(e, ('byArtist', 'name'), expected_type=str),
                 'filesize': int_or_none(float_or_none(e.get('contentSize'))),
                 'tbr': int_or_none(e.get('bitrate')),
                 'width': int_or_none(e.get('width')),
                 'height': int_or_none(e.get('height')),
                 'view_count': int_or_none(e.get('interactionCount')),
+                'tags': try_call(lambda: e.get('keywords').split(',')),
             })
+            if is_type(e, 'AudioObject'):
+                info.update({
+                    'vcodec': 'none',
+                    'abr': int_or_none(e.get('bitrate')),
+                })
             extract_interaction_statistic(e)
             extract_chapter_information(e)
 
@@ -1608,7 +1615,7 @@ def traverse_json_ld(json_ld, at_top_level=True):
                         extract_video_object(e['video'][0])
                     elif is_type(traverse_obj(e, ('subjectOf', 0)), 'VideoObject'):
                         extract_video_object(e['subjectOf'][0])
-                elif is_type(e, 'VideoObject'):
+                elif is_type(e, 'VideoObject', 'AudioObject'):
                     extract_video_object(e)
                     if expected_type is None:
                         continue
@@ -3857,8 +3864,10 @@ def _yes_playlist(self, playlist_id, video_id, smuggled_data=None, *, playlist_l
         return True
 
     def _error_or_warning(self, err, _count=None, _retries=0, *, fatal=True):
-        RetryManager.report_retry(err, _count or int(fatal), _retries, info=self.to_screen, warn=self.report_warning,
-                                  sleep_func=self.get_param('retry_sleep_functions', {}).get('extractor'))
+        RetryManager.report_retry(
+            err, _count or int(fatal), _retries,
+            info=self.to_screen, warn=self.report_warning, error=None if fatal else self.report_warning,
+            sleep_func=self.get_param('retry_sleep_functions', {}).get('extractor'))
 
     def RetryManager(self, **kwargs):
         return RetryManager(self.get_param('extractor_retries', 3), self._error_or_warning, **kwargs)
diff --git a/yt_dlp/extractor/cybrary.py b/yt_dlp/extractor/cybrary.py
index 7da581828..73f2439b3 100644
--- a/yt_dlp/extractor/cybrary.py
+++ b/yt_dlp/extractor/cybrary.py
@@ -1,11 +1,10 @@
-﻿from .common import InfoExtractor
-
+from .common import InfoExtractor
 from ..utils import (
     ExtractorError,
     smuggle_url,
     str_or_none,
     traverse_obj,
-    urlencode_postdata
+    urlencode_postdata,
 )
 
 
diff --git a/yt_dlp/extractor/generic.py b/yt_dlp/extractor/generic.py
index 55b3addde..73aefc782 100644
--- a/yt_dlp/extractor/generic.py
+++ b/yt_dlp/extractor/generic.py
@@ -873,22 +873,6 @@ class GenericIE(InfoExtractor):
                 'thumbnail': r're:^https?://.*\.jpg$',
             },
         },
-        # Wistia embed
-        {
-            'url': 'http://study.com/academy/lesson/north-american-exploration-failed-colonies-of-spain-france-england.html#lesson',
-            'md5': '1953f3a698ab51cfc948ed3992a0b7ff',
-            'info_dict': {
-                'id': '6e2wtrbdaf',
-                'ext': 'mov',
-                'title': 'paywall_north-american-exploration-failed-colonies-of-spain-france-england',
-                'description': 'a Paywall Videos video from Remilon',
-                'duration': 644.072,
-                'uploader': 'study.com',
-                'timestamp': 1459678540,
-                'upload_date': '20160403',
-                'filesize': 24687186,
-            },
-        },
         # Wistia standard embed (async)
         {
             'url': 'https://www.getdrip.com/university/brennan-dunn-drip-workshop/',
@@ -903,7 +887,8 @@ class GenericIE(InfoExtractor):
             },
             'params': {
                 'skip_download': True,
-            }
+            },
+            'skip': 'webpage 404 not found',
         },
         # Soundcloud embed
         {
@@ -2623,8 +2608,8 @@ def _real_extract(self, url):
 
         url, smuggled_data = unsmuggle_url(url, {})
         force_videoid = None
-        is_intentional = smuggled_data and smuggled_data.get('to_generic')
-        if smuggled_data and 'force_videoid' in smuggled_data:
+        is_intentional = smuggled_data.get('to_generic')
+        if 'force_videoid' in smuggled_data:
             force_videoid = smuggled_data['force_videoid']
             video_id = force_videoid
         else:
@@ -2925,7 +2910,7 @@ def _real_extract(self, url):
         if json_ld.get('url') not in (url, None):
             self.report_detected('JSON LD')
             return merge_dicts({
-                '_type': 'url_transparent',
+                '_type': 'video' if json_ld.get('ext') else 'url_transparent',
                 'url': smuggle_url(json_ld['url'], {
                     'force_videoid': video_id,
                     'to_generic': True,
diff --git a/yt_dlp/extractor/heise.py b/yt_dlp/extractor/heise.py
index 4f689c6e4..86661d75a 100644
--- a/yt_dlp/extractor/heise.py
+++ b/yt_dlp/extractor/heise.py
@@ -1,10 +1,12 @@
+import urllib.parse
+
 from .common import InfoExtractor
 from .kaltura import KalturaIE
 from .youtube import YoutubeIE
 from ..utils import (
+    NO_DEFAULT,
     determine_ext,
     int_or_none,
-    NO_DEFAULT,
     parse_iso8601,
     smuggle_url,
     xpath_text,
@@ -23,6 +25,9 @@ class HeiseIE(InfoExtractor):
             'timestamp': 1512734959,
             'upload_date': '20171208',
             'description': 'md5:c934cbfb326c669c2bcabcbe3d3fcd20',
+            'thumbnail': 're:^https?://.*/thumbnail/.*',
+            'duration': 2845,
+            'view_count': int,
         },
         'params': {
             'skip_download': True,
@@ -34,11 +39,27 @@ class HeiseIE(InfoExtractor):
         'info_dict': {
             'id': '6kmWbXleKW4',
             'ext': 'mp4',
-            'title': 'NEU IM SEPTEMBER | Netflix',
-            'description': 'md5:2131f3c7525e540d5fd841de938bd452',
+            'title': 'Neu im September 2017 | Netflix',
+            'description': 'md5:d6852d1f96bb80760608eed3b907437c',
             'upload_date': '20170830',
             'uploader': 'Netflix Deutschland, Österreich und Schweiz',
             'uploader_id': 'netflixdach',
+            'categories': ['Entertainment'],
+            'tags': 'count:27',
+            'age_limit': 0,
+            'availability': 'public',
+            'comment_count': int,
+            'channel_id': 'UCZqgRlLcvO3Fnx_npQJygcQ',
+            'thumbnail': 'https://i.ytimg.com/vi_webp/6kmWbXleKW4/maxresdefault.webp',
+            'uploader_url': 'http://www.youtube.com/user/netflixdach',
+            'playable_in_embed': True,
+            'live_status': 'not_live',
+            'channel_url': 'https://www.youtube.com/channel/UCZqgRlLcvO3Fnx_npQJygcQ',
+            'view_count': int,
+            'channel': 'Netflix Deutschland, Österreich und Schweiz',
+            'channel_follower_count': int,
+            'like_count': int,
+            'duration': 67,
         },
         'params': {
             'skip_download': True,
@@ -52,11 +73,15 @@ class HeiseIE(InfoExtractor):
             'description': 'md5:47e8ffb6c46d85c92c310a512d6db271',
             'timestamp': 1512470717,
             'upload_date': '20171205',
+            'duration': 786,
+            'view_count': int,
+            'thumbnail': 're:^https?://.*/thumbnail/.*',
         },
         'params': {
             'skip_download': True,
         },
     }, {
+        # FIXME: Video m3u8 fails to download; issue with Kaltura extractor
         'url': 'https://www.heise.de/ct/artikel/c-t-uplink-20-8-Staubsaugerroboter-Xiaomi-Vacuum-2-AR-Brille-Meta-2-und-Android-rooten-3959893.html',
         'info_dict': {
             'id': '1_59mk80sf',
@@ -69,6 +94,18 @@ class HeiseIE(InfoExtractor):
         'params': {
             'skip_download': True,
         },
+    }, {
+        # videout
+        'url': 'https://www.heise.de/ct/artikel/c-t-uplink-3-8-Anonyme-SIM-Karten-G-Sync-Monitore-Citizenfour-2440327.html',
+        'info_dict': {
+            'id': '2440327',
+            'ext': 'mp4',
+            'title': 'c\'t uplink 3.8: Anonyme SIM-Karten, G-Sync-Monitore, Citizenfour',
+            'thumbnail': 'http://www.heise.de/imagine/yxM2qmol0xV3iFB7qFb70dGvXjc/gallery/',
+            'description': 'md5:fa164d8c8707dff124a9626d39205f5d',
+            'timestamp': 1414825200,
+            'upload_date': '20141101',
+        }
     }, {
         'url': 'http://www.heise.de/ct/artikel/c-t-uplink-3-3-Owncloud-Tastaturen-Peilsender-Smartphone-2403911.html',
         'only_matching': True,
@@ -127,20 +164,22 @@ def _make_kaltura_result(kaltura_url):
                 yt_urls, video_id, title, ie=YoutubeIE.ie_key())
 
         title = extract_title()
+        api_params = urllib.parse.parse_qs(
+            self._search_regex(r'/videout/feed\.json\?([^\']+)', webpage, 'feed params', default=None) or '')
+        if not api_params or 'container' not in api_params or 'sequenz' not in api_params:
+            container_id = self._search_regex(
+                r'<div class="videoplayerjw"[^>]+data-container="([0-9]+)"',
+                webpage, 'container ID')
 
-        container_id = self._search_regex(
-            r'<div class="videoplayerjw"[^>]+data-container="([0-9]+)"',
-            webpage, 'container ID')
-
-        sequenz_id = self._search_regex(
-            r'<div class="videoplayerjw"[^>]+data-sequenz="([0-9]+)"',
-            webpage, 'sequenz ID')
-
-        doc = self._download_xml(
-            'http://www.heise.de/videout/feed', video_id, query={
+            sequenz_id = self._search_regex(
+                r'<div class="videoplayerjw"[^>]+data-sequenz="([0-9]+)"',
+                webpage, 'sequenz ID')
+            api_params = {
                 'container': container_id,
                 'sequenz': sequenz_id,
-            })
+            }
+        doc = self._download_xml(
+            'http://www.heise.de/videout/feed', video_id, query=api_params)
 
         formats = []
         for source_node in doc.findall('.//{http://rss.jwpcdn.com/}source'):
diff --git a/yt_dlp/extractor/hungama.py b/yt_dlp/extractor/hungama.py
index 938a24296..717f50a83 100644
--- a/yt_dlp/extractor/hungama.py
+++ b/yt_dlp/extractor/hungama.py
@@ -20,15 +20,17 @@ class HungamaIE(InfoExtractor):
                     '''
     _TESTS = [{
         'url': 'http://www.hungama.com/video/krishna-chants/39349649/',
-        'md5': 'a845a6d1ebd08d80c1035126d49bd6a0',
+        'md5': '687c5f1e9f832f3b59f44ed0eb1f120a',
         'info_dict': {
-            'id': '2931166',
+            'id': '39349649',
             'ext': 'mp4',
-            'title': 'Lucky Ali - Kitni Haseen Zindagi',
-            'track': 'Kitni Haseen Zindagi',
-            'artist': 'Lucky Ali',
-            'album': 'Aks',
-            'release_year': 2000,
+            'title': 'Krishna Chants',
+            'description': 'Watch Krishna Chants video now. You can also watch other latest videos only at Hungama',
+            'upload_date': '20180829',
+            'duration': 264,
+            'timestamp': 1535500800,
+            'view_count': int,
+            'thumbnail': 'https://images.hungama.com/c/1/0dc/2ca/39349649/39349649_700x394.jpg',
         }
     }, {
         'url': 'https://www.hungama.com/movie/kahaani-2/44129919/',
@@ -40,12 +42,7 @@ class HungamaIE(InfoExtractor):
 
     def _real_extract(self, url):
         video_id = self._match_id(url)
-
-        webpage = self._download_webpage(url, video_id)
-
-        info = self._search_json_ld(webpage, video_id)
-
-        m3u8_url = self._download_json(
+        video_json = self._download_json(
             'https://www.hungama.com/index.php', video_id,
             data=urlencode_postdata({'content_id': video_id}), headers={
                 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
@@ -53,18 +50,25 @@ def _real_extract(self, url):
             }, query={
                 'c': 'common',
                 'm': 'get_video_mdn_url',
-            })['stream_url']
+            })
 
-        formats = self._extract_m3u8_formats(
-            m3u8_url, video_id, ext='mp4', entry_protocol='m3u8_native',
-            m3u8_id='hls')
+        formats = self._extract_m3u8_formats(video_json['stream_url'], video_id, ext='mp4', m3u8_id='hls')
         self._sort_formats(formats)
 
-        info.update({
+        json_ld = self._search_json_ld(
+            self._download_webpage(url, video_id, fatal=False) or '', video_id, fatal=False)
+
+        return {
+            **json_ld,
             'id': video_id,
             'formats': formats,
-        })
-        return info
+            'subtitles': {
+                'en': [{
+                    'url': video_json['sub_title'],
+                    'ext': 'vtt',
+                }]
+            } if video_json.get('sub_title') else None,
+        }
 
 
 class HungamaSongIE(InfoExtractor):
diff --git a/yt_dlp/extractor/instagram.py b/yt_dlp/extractor/instagram.py
index e997a3fbb..c9da7e36f 100644
--- a/yt_dlp/extractor/instagram.py
+++ b/yt_dlp/extractor/instagram.py
@@ -173,18 +173,9 @@ def _extract_product(self, product_info):
         if isinstance(product_info, list):
             product_info = product_info[0]
 
-        comment_data = traverse_obj(product_info, ('edge_media_to_parent_comment', 'edges'))
-        comments = [{
-            'author': traverse_obj(comment_dict, ('node', 'owner', 'username')),
-            'author_id': traverse_obj(comment_dict, ('node', 'owner', 'id')),
-            'id': traverse_obj(comment_dict, ('node', 'id')),
-            'text': traverse_obj(comment_dict, ('node', 'text')),
-            'timestamp': traverse_obj(comment_dict, ('node', 'created_at'), expected_type=int_or_none),
-        } for comment_dict in comment_data] if comment_data else None
-
         user_info = product_info.get('user') or {}
         info_dict = {
-            'id': product_info.get('code') or _pk_to_id(product_info.get('pk')),
+            'id': _pk_to_id(traverse_obj(product_info, 'pk', 'id', expected_type=str_or_none)[:19]),
             'title': product_info.get('title') or f'Video by {user_info.get("username")}',
             'description': traverse_obj(product_info, ('caption', 'text'), expected_type=str_or_none),
             'timestamp': int_or_none(product_info.get('taken_at')),
@@ -194,7 +185,7 @@ def _extract_product(self, product_info):
             'view_count': int_or_none(product_info.get('view_count')),
             'like_count': int_or_none(product_info.get('like_count')),
             'comment_count': int_or_none(product_info.get('comment_count')),
-            'comments': comments,
+            '__post_extractor': self.extract_comments(_pk_to_id(product_info.get('pk'))),
             'http_headers': {
                 'Referer': 'https://www.instagram.com/',
             }
@@ -216,6 +207,23 @@ def _extract_product(self, product_info):
             **self._extract_product_media(product_info)
         }
 
+    def _get_comments(self, video_id):
+        comments_info = self._download_json(
+            f'{self._API_BASE_URL}/media/{_id_to_pk(video_id)}/comments/?can_support_threading=true&permalink_enabled=false', video_id,
+            fatal=False, errnote='Comments extraction failed', note='Downloading comments info', headers=self._API_HEADERS) or {}
+
+        comment_data = traverse_obj(comments_info, ('edge_media_to_parent_comment', 'edges'), 'comments')
+        for comment_dict in comment_data or []:
+            yield {
+                'author': traverse_obj(comment_dict, ('node', 'owner', 'username'), ('user', 'username')),
+                'author_id': traverse_obj(comment_dict, ('node', 'owner', 'id'), ('user', 'pk')),
+                'author_thumbnail': traverse_obj(comment_dict, ('node', 'owner', 'profile_pic_url'), ('user', 'profile_pic_url'), expected_type=url_or_none),
+                'id': traverse_obj(comment_dict, ('node', 'id'), 'pk'),
+                'text': traverse_obj(comment_dict, ('node', 'text'), 'text'),
+                'like_count': traverse_obj(comment_dict, ('node', 'edge_liked_by', 'count'), 'comment_like_count', expected_type=int_or_none),
+                'timestamp': traverse_obj(comment_dict, ('node', 'created_at'), 'created_at', expected_type=int_or_none),
+            }
+
 
 class InstagramIOSIE(InfoExtractor):
     IE_DESC = 'IOS instagram:// URL'
@@ -258,7 +266,7 @@ class InstagramIE(InstagramBaseIE):
             'title': 'Video by naomipq',
             'description': 'md5:1f17f0ab29bd6fe2bfad705f58de3cb8',
             'thumbnail': r're:^https?://.*\.jpg',
-            'duration': 0,
+            'duration': 8.747,
             'timestamp': 1371748545,
             'upload_date': '20130620',
             'uploader_id': '2815873',
@@ -268,27 +276,34 @@ class InstagramIE(InstagramBaseIE):
             'comment_count': int,
             'comments': list,
         },
+        'expected_warnings': [
+            'General metadata extraction failed',
+            'Main webpage is locked behind the login page',
+        ],
     }, {
-        # missing description
-        'url': 'https://www.instagram.com/p/BA-pQFBG8HZ/?taken-by=britneyspears',
+        # reel
+        'url': 'https://www.instagram.com/reel/Chunk8-jurw/',
+        'md5': 'f6d8277f74515fa3ff9f5791426e42b1',
         'info_dict': {
-            'id': 'BA-pQFBG8HZ',
+            'id': 'Chunk8-jurw',
             'ext': 'mp4',
-            'title': 'Video by britneyspears',
+            'title': 'Video by instagram',
+            'description': 'md5:c9cde483606ed6f80fbe9283a6a2b290',
             'thumbnail': r're:^https?://.*\.jpg',
-            'duration': 0,
-            'timestamp': 1453760977,
-            'upload_date': '20160125',
-            'uploader_id': '12246775',
-            'uploader': 'Britney Spears',
-            'channel': 'britneyspears',
+            'duration': 5.016,
+            'timestamp': 1661529231,
+            'upload_date': '20220826',
+            'uploader_id': '25025320',
+            'uploader': 'Instagram',
+            'channel': 'instagram',
             'like_count': int,
             'comment_count': int,
             'comments': list,
         },
-        'params': {
-            'skip_download': True,
-        },
+        'expected_warnings': [
+            'General metadata extraction failed',
+            'Main webpage is locked behind the login page',
+        ],
     }, {
         # multi video post
         'url': 'https://www.instagram.com/p/BQ0eAlwhDrw/',
@@ -297,18 +312,24 @@ class InstagramIE(InstagramBaseIE):
                 'id': 'BQ0dSaohpPW',
                 'ext': 'mp4',
                 'title': 'Video 1',
+                'thumbnail': r're:^https?://.*\.jpg',
+                'view_count': int,
             },
         }, {
             'info_dict': {
                 'id': 'BQ0dTpOhuHT',
                 'ext': 'mp4',
                 'title': 'Video 2',
+                'thumbnail': r're:^https?://.*\.jpg',
+                'view_count': int,
             },
         }, {
             'info_dict': {
                 'id': 'BQ0dT7RBFeF',
                 'ext': 'mp4',
                 'title': 'Video 3',
+                'thumbnail': r're:^https?://.*\.jpg',
+                'view_count': int,
             },
         }],
         'info_dict': {
@@ -316,6 +337,10 @@ class InstagramIE(InstagramBaseIE):
             'title': 'Post by instagram',
             'description': 'md5:0f9203fc6a2ce4d228da5754bcf54957',
         },
+        'expected_warnings': [
+            'General metadata extraction failed',
+            'Main webpage is locked behind the login page',
+        ],
     }, {
         # IGTV
         'url': 'https://www.instagram.com/tv/BkfuX9UB-eK/',
@@ -334,7 +359,11 @@ class InstagramIE(InstagramBaseIE):
             'comment_count': int,
             'comments': list,
             'description': 'Meet Cass Hirst (@cass.fb), a fingerboarding pro who can perform tiny ollies and kickflips while blindfolded.',
-        }
+        },
+        'expected_warnings': [
+            'General metadata extraction failed',
+            'Main webpage is locked behind the login page',
+        ],
     }, {
         'url': 'https://instagram.com/p/-Cmh1cukG2/',
         'only_matching': True,
@@ -367,6 +396,15 @@ def _real_extract(self, url):
         video_id, url = self._match_valid_url(url).group('id', 'url')
         media, webpage = {}, ''
 
+        if self._get_cookies(url).get('sessionid'):
+            info = traverse_obj(self._download_json(
+                f'{self._API_BASE_URL}/media/{_id_to_pk(video_id)}/info/', video_id,
+                fatal=False, errnote='Video info extraction failed',
+                note='Downloading video info', headers=self._API_HEADERS), ('items', 0))
+            if info:
+                media.update(info)
+                return self._extract_product(media)
+
         api_check = self._download_json(
             f'{self._API_BASE_URL}/web/get_ruling_for_content/?content_type=MEDIA&target_id={_id_to_pk(video_id)}',
             video_id, headers=self._API_HEADERS, fatal=False, note='Setting up session', errnote=False) or {}
@@ -374,40 +412,32 @@ def _real_extract(self, url):
 
         if not csrf_token:
             self.report_warning('No csrf token set by Instagram API', video_id)
-        elif api_check.get('status') != 'ok':
-            self.report_warning('Instagram API is not granting access', video_id)
         else:
-            if self._get_cookies(url).get('sessionid'):
-                media.update(traverse_obj(self._download_json(
-                    f'{self._API_BASE_URL}/media/{_id_to_pk(video_id)}/info/', video_id,
-                    fatal=False, note='Downloading video info', headers={
-                        **self._API_HEADERS,
-                        'X-CSRFToken': csrf_token.value,
-                    }), ('items', 0)) or {})
-                if media:
-                    return self._extract_product(media)
-
-            variables = {
-                'shortcode': video_id,
-                'child_comment_count': 3,
-                'fetch_comment_count': 40,
-                'parent_comment_count': 24,
-                'has_threaded_comments': True,
-            }
-            general_info = self._download_json(
-                'https://www.instagram.com/graphql/query/', video_id, fatal=False,
-                headers={
-                    **self._API_HEADERS,
-                    'X-CSRFToken': csrf_token.value,
-                    'X-Requested-With': 'XMLHttpRequest',
-                    'Referer': url,
-                }, query={
-                    'query_hash': '9f8827793ef34641b2fb195d4d41151c',
-                    'variables': json.dumps(variables, separators=(',', ':')),
-                })
-            media.update(traverse_obj(general_info, ('data', 'shortcode_media')) or {})
-
-        if not media:
+            csrf_token = csrf_token.value if api_check.get('status') == 'ok' else None
+            if not csrf_token:
+                self.report_warning('Instagram API is not granting access', video_id)
+
+        variables = {
+            'shortcode': video_id,
+            'child_comment_count': 3,
+            'fetch_comment_count': 40,
+            'parent_comment_count': 24,
+            'has_threaded_comments': True,
+        }
+        general_info = self._download_json(
+            'https://www.instagram.com/graphql/query/', video_id, fatal=False, errnote=False,
+            headers={
+                **self._API_HEADERS,
+                'X-CSRFToken': csrf_token or '',
+                'X-Requested-With': 'XMLHttpRequest',
+                'Referer': url,
+            }, query={
+                'query_hash': '9f8827793ef34641b2fb195d4d41151c',
+                'variables': json.dumps(variables, separators=(',', ':')),
+            })
+        media.update(traverse_obj(general_info, ('data', 'shortcode_media')) or {})
+
+        if not general_info:
             self.report_warning('General metadata extraction failed (some metadata might be missing).', video_id)
             webpage, urlh = self._download_webpage_handle(url, video_id)
             shared_data = self._search_json(
@@ -418,12 +448,12 @@ def _real_extract(self, url):
                     shared_data, ('entry_data', 'PostPage', 0, 'graphql', 'shortcode_media'),
                     ('entry_data', 'PostPage', 0, 'media'), expected_type=dict) or {})
             else:
-                self.report_warning('Main webpage is locked behind the login page. Retrying with embed webpage')
+                self.report_warning('Main webpage is locked behind the login page. Retrying with embed webpage (some metadata might be missing).')
                 webpage = self._download_webpage(
                     f'{url}/embed/', video_id, note='Downloading embed webpage', fatal=False)
                 additional_data = self._search_json(
                     r'window\.__additionalDataLoaded\s*\(\s*[^,]+,\s*', webpage, 'additional data', video_id, fatal=False)
-                if not additional_data:
+                if not additional_data and not media:
                     self.raise_login_required('Requested content is not available, rate-limit reached or login required')
 
                 product_item = traverse_obj(additional_data, ('items', 0), expected_type=dict)
diff --git a/yt_dlp/extractor/nebula.py b/yt_dlp/extractor/nebula.py
index 7057b8b26..861fcb164 100644
--- a/yt_dlp/extractor/nebula.py
+++ b/yt_dlp/extractor/nebula.py
@@ -7,6 +7,8 @@
 from .common import InfoExtractor
 from ..utils import ExtractorError, parse_iso8601, try_get
 
+_BASE_URL_RE = r'https?://(?:www\.)?(?:watchnebula\.com|nebula\.app|nebula\.tv)'
+
 
 class NebulaBaseIE(InfoExtractor):
     _NETRC_MACHINE = 'watchnebula'
@@ -148,7 +150,7 @@ def _perform_login(self, username=None, password=None):
 
 
 class NebulaIE(NebulaBaseIE):
-    _VALID_URL = r'https?://(?:www\.)?(?:watchnebula\.com|nebula\.app)/videos/(?P<id>[-\w]+)'
+    _VALID_URL = rf'{_BASE_URL_RE}/videos/(?P<id>[-\w]+)'
     _TESTS = [
         {
             'url': 'https://nebula.app/videos/that-time-disney-remade-beauty-and-the-beast',
@@ -246,7 +248,7 @@ def _real_extract(self, url):
 
 class NebulaSubscriptionsIE(NebulaBaseIE):
     IE_NAME = 'nebula:subscriptions'
-    _VALID_URL = r'https?://(?:www\.)?(?:watchnebula\.com|nebula\.app)/myshows'
+    _VALID_URL = rf'{_BASE_URL_RE}/myshows'
     _TESTS = [
         {
             'url': 'https://nebula.app/myshows',
@@ -274,7 +276,7 @@ def _real_extract(self, url):
 
 class NebulaChannelIE(NebulaBaseIE):
     IE_NAME = 'nebula:channel'
-    _VALID_URL = r'https?://(?:www\.)?(?:watchnebula\.com|nebula\.app)/(?!myshows|videos/)(?P<id>[-\w]+)'
+    _VALID_URL = rf'{_BASE_URL_RE}/(?!myshows|videos/)(?P<id>[-\w]+)'
     _TESTS = [
         {
             'url': 'https://nebula.app/tom-scott-presents-money',
diff --git a/yt_dlp/extractor/niconico.py b/yt_dlp/extractor/niconico.py
index 82fb27631..82b60b476 100644
--- a/yt_dlp/extractor/niconico.py
+++ b/yt_dlp/extractor/niconico.py
@@ -557,8 +557,7 @@ class NiconicoPlaylistBaseIE(InfoExtractor):
     }
 
     def _call_api(self, list_id, resource, query):
-        "Implement this in child class"
-        pass
+        raise NotImplementedError('Must be implemented in subclasses')
 
     @staticmethod
     def _parse_owner(item):
diff --git a/yt_dlp/extractor/patreon.py b/yt_dlp/extractor/patreon.py
index 529aba178..43c90c8f1 100644
--- a/yt_dlp/extractor/patreon.py
+++ b/yt_dlp/extractor/patreon.py
@@ -277,6 +277,7 @@ def _real_extract(self, url):
                 }
             elif name == 'video':
                 formats, subtitles = self._extract_m3u8_formats_and_subtitles(post_file['url'], video_id)
+                self._sort_formats(formats)
                 return {
                     **info,
                     'formats': formats,
diff --git a/yt_dlp/extractor/rutube.py b/yt_dlp/extractor/rutube.py
index 380c5e14e..34af0d594 100644
--- a/yt_dlp/extractor/rutube.py
+++ b/yt_dlp/extractor/rutube.py
@@ -93,7 +93,7 @@ class RutubeIE(RutubeBaseIE):
     IE_NAME = 'rutube'
     IE_DESC = 'Rutube videos'
     _VALID_URL = r'https?://rutube\.ru/(?:video|(?:play/)?embed)/(?P<id>[\da-z]{32})'
-    _EMBED_REGEX = [r'<iframe[^>]+?src=(["\'])(?P<url>(?:https?:)?//rutube\.ru/embed/[\da-z]{32}.*?)\1']
+    _EMBED_REGEX = [r'<iframe[^>]+?src=(["\'])(?P<url>(?:https?:)?//rutube\.ru/(?:play/)?embed/[\da-z]{32}.*?)\1']
 
     _TESTS = [{
         'url': 'http://rutube.ru/video/3eac3b4561676c17df9132a9a1e62e3e/',
diff --git a/yt_dlp/extractor/rutv.py b/yt_dlp/extractor/rutv.py
index 0b07dc5ad..75da01f7d 100644
--- a/yt_dlp/extractor/rutv.py
+++ b/yt_dlp/extractor/rutv.py
@@ -141,7 +141,7 @@ def _real_extract(self, url):
         if media['errors']:
             raise ExtractorError('%s said: %s' % (self.IE_NAME, media['errors']), expected=True)
 
-        view_count = playlist.get('count_views')
+        view_count = int_or_none(playlist.get('count_views'))
         priority_transport = playlist['priority_transport']
 
         thumbnail = media['picture']
@@ -152,6 +152,7 @@ def _real_extract(self, url):
         duration = int_or_none(media.get('duration'))
 
         formats = []
+        subtitles = {}
 
         for transport, links in media['sources'].items():
             for quality, url in links.items():
@@ -171,8 +172,10 @@ def _real_extract(self, url):
                         'vbr': str_to_int(quality),
                     }
                 elif transport == 'm3u8':
-                    formats.extend(self._extract_m3u8_formats(
-                        url, video_id, 'mp4', quality=preference, m3u8_id='hls'))
+                    fmt, subs = self._extract_m3u8_formats_and_subtitles(
+                        url, video_id, 'mp4', quality=preference, m3u8_id='hls')
+                    formats.extend(fmt)
+                    self._merge_subtitles(subs, target=subtitles)
                     continue
                 else:
                     fmt = {
@@ -186,7 +189,7 @@ def _real_extract(self, url):
                 })
                 formats.append(fmt)
 
-        self._sort_formats(formats)
+        self._sort_formats(formats, ('source', ))
 
         return {
             'id': video_id,
@@ -196,5 +199,6 @@ def _real_extract(self, url):
             'view_count': view_count,
             'duration': duration,
             'formats': formats,
+            'subtitles': subtitles,
             'is_live': is_live,
         }
diff --git a/yt_dlp/extractor/smotrim.py b/yt_dlp/extractor/smotrim.py
new file mode 100644
index 000000000..d3f1b695b
--- /dev/null
+++ b/yt_dlp/extractor/smotrim.py
@@ -0,0 +1,65 @@
+from .common import InfoExtractor
+from ..utils import ExtractorError
+
+
+class SmotrimIE(InfoExtractor):
+    _VALID_URL = r'https?://smotrim\.ru/(?P<type>brand|video|article|live)/(?P<id>[0-9]+)'
+    _TESTS = [{  # video
+        'url': 'https://smotrim.ru/video/1539617',
+        'md5': 'b1923a533c8cab09679789d720d0b1c5',
+        'info_dict': {
+            'id': '1539617',
+            'ext': 'mp4',
+            'title': 'Полиглот. Китайский с нуля за 16 часов! Урок №16',
+            'description': '',
+        },
+        'add_ie': ['RUTV'],
+    }, {  # article (geo-restricted? plays fine from the US and JP)
+        'url': 'https://smotrim.ru/article/2813445',
+        'md5': 'e0ac453952afbc6a2742e850b4dc8e77',
+        'info_dict': {
+            'id': '2431846',
+            'ext': 'mp4',
+            'title': 'Новости культуры. Съёмки первой программы "Большие и маленькие"',
+            'description': 'md5:94a4a22472da4252bf5587a4ee441b99',
+        },
+        'add_ie': ['RUTV'],
+    }, {  # brand, redirect
+        'url': 'https://smotrim.ru/brand/64356',
+        'md5': '740472999ccff81d7f6df79cecd91c18',
+        'info_dict': {
+            'id': '2354523',
+            'ext': 'mp4',
+            'title': 'Большие и маленькие. Лучшее. 4-й выпуск',
+            'description': 'md5:84089e834429008371ea41ea3507b989',
+        },
+        'add_ie': ['RUTV'],
+    }, {  # live
+        'url': 'https://smotrim.ru/live/19201',
+        'info_dict': {
+            'id': '19201',
+            'ext': 'mp4',
+            # this looks like a TV channel name
+            'title': 'Россия Культура. Прямой эфир',
+            'description': '',
+        },
+        'add_ie': ['RUTV'],
+    }]
+
+    def _real_extract(self, url):
+        video_id, typ = self._match_valid_url(url).group('id', 'type')
+        rutv_type = 'video'
+        if typ not in ('video', 'live'):
+            webpage = self._download_webpage(url, video_id, f'Resolving {typ} link')
+            # there are two cases matching regex:
+            # 1. "embedUrl" in JSON LD (/brand/)
+            # 2. "src" attribute from iframe (/article/)
+            video_id = self._search_regex(
+                r'"https://player.smotrim.ru/iframe/video/id/(?P<video_id>\d+)/',
+                webpage, 'video_id', default=None)
+            if not video_id:
+                raise ExtractorError('There are no video in this page.', expected=True)
+        elif typ == 'live':
+            rutv_type = 'live'
+
+        return self.url_result(f'https://player.vgtrk.com/iframe/{rutv_type}/id/{video_id}')
diff --git a/yt_dlp/extractor/soundcloud.py b/yt_dlp/extractor/soundcloud.py
index 2730052a0..228e19c3e 100644
--- a/yt_dlp/extractor/soundcloud.py
+++ b/yt_dlp/extractor/soundcloud.py
@@ -60,6 +60,21 @@ class SoundcloudBaseIE(InfoExtractor):
     _access_token = None
     _HEADERS = {}
 
+    _IMAGE_REPL_RE = r'-([0-9a-z]+)\.jpg'
+
+    _ARTWORK_MAP = {
+        'mini': 16,
+        'tiny': 20,
+        'small': 32,
+        'badge': 47,
+        't67x67': 67,
+        'large': 100,
+        't300x300': 300,
+        'crop': 400,
+        't500x500': 500,
+        'original': 0,
+    }
+
     def _store_client_id(self, client_id):
         self.cache.store('soundcloud', 'client_id', client_id)
 
@@ -179,6 +194,158 @@ def sign(self, user, pw, clid):
 
         return out
 
+    def _extract_info_dict(self, info, full_title=None, secret_token=None, extract_flat=False):
+        track_id = compat_str(info['id'])
+        title = info['title']
+
+        format_urls = set()
+        formats = []
+        query = {'client_id': self._CLIENT_ID}
+        if secret_token:
+            query['secret_token'] = secret_token
+
+        if not extract_flat and info.get('downloadable') and info.get('has_downloads_left'):
+            download_url = update_url_query(
+                self._API_V2_BASE + 'tracks/' + track_id + '/download', query)
+            redirect_url = (self._download_json(download_url, track_id, fatal=False) or {}).get('redirectUri')
+            if redirect_url:
+                urlh = self._request_webpage(
+                    HEADRequest(redirect_url), track_id, fatal=False)
+                if urlh:
+                    format_url = urlh.geturl()
+                    format_urls.add(format_url)
+                    formats.append({
+                        'format_id': 'download',
+                        'ext': urlhandle_detect_ext(urlh) or 'mp3',
+                        'filesize': int_or_none(urlh.headers.get('Content-Length')),
+                        'url': format_url,
+                        'quality': 10,
+                    })
+
+        def invalid_url(url):
+            return not url or url in format_urls
+
+        def add_format(f, protocol, is_preview=False):
+            mobj = re.search(r'\.(?P<abr>\d+)\.(?P<ext>[0-9a-z]{3,4})(?=[/?])', stream_url)
+            if mobj:
+                for k, v in mobj.groupdict().items():
+                    if not f.get(k):
+                        f[k] = v
+            format_id_list = []
+            if protocol:
+                format_id_list.append(protocol)
+            ext = f.get('ext')
+            if ext == 'aac':
+                f['abr'] = '256'
+            for k in ('ext', 'abr'):
+                v = f.get(k)
+                if v:
+                    format_id_list.append(v)
+            preview = is_preview or re.search(r'/(?:preview|playlist)/0/30/', f['url'])
+            if preview:
+                format_id_list.append('preview')
+            abr = f.get('abr')
+            if abr:
+                f['abr'] = int(abr)
+            if protocol == 'hls':
+                protocol = 'm3u8' if ext == 'aac' else 'm3u8_native'
+            else:
+                protocol = 'http'
+            f.update({
+                'format_id': '_'.join(format_id_list),
+                'protocol': protocol,
+                'preference': -10 if preview else None,
+            })
+            formats.append(f)
+
+        # New API
+        transcodings = try_get(
+            info, lambda x: x['media']['transcodings'], list) or []
+        for t in transcodings:
+            if not isinstance(t, dict):
+                continue
+            format_url = url_or_none(t.get('url'))
+            if not format_url:
+                continue
+            stream = None if extract_flat else self._download_json(
+                format_url, track_id, query=query, fatal=False, headers=self._HEADERS)
+            if not isinstance(stream, dict):
+                continue
+            stream_url = url_or_none(stream.get('url'))
+            if invalid_url(stream_url):
+                continue
+            format_urls.add(stream_url)
+            stream_format = t.get('format') or {}
+            protocol = stream_format.get('protocol')
+            if protocol != 'hls' and '/hls' in format_url:
+                protocol = 'hls'
+            ext = None
+            preset = str_or_none(t.get('preset'))
+            if preset:
+                ext = preset.split('_')[0]
+            if ext not in KNOWN_EXTENSIONS:
+                ext = mimetype2ext(stream_format.get('mime_type'))
+            add_format({
+                'url': stream_url,
+                'ext': ext,
+            }, 'http' if protocol == 'progressive' else protocol,
+                t.get('snipped') or '/preview/' in format_url)
+
+        for f in formats:
+            f['vcodec'] = 'none'
+
+        if not formats and info.get('policy') == 'BLOCK':
+            self.raise_geo_restricted(metadata_available=True)
+        self._sort_formats(formats)
+
+        user = info.get('user') or {}
+
+        thumbnails = []
+        artwork_url = info.get('artwork_url')
+        thumbnail = artwork_url or user.get('avatar_url')
+        if isinstance(thumbnail, compat_str):
+            if re.search(self._IMAGE_REPL_RE, thumbnail):
+                for image_id, size in self._ARTWORK_MAP.items():
+                    i = {
+                        'id': image_id,
+                        'url': re.sub(self._IMAGE_REPL_RE, '-%s.jpg' % image_id, thumbnail),
+                    }
+                    if image_id == 'tiny' and not artwork_url:
+                        size = 18
+                    elif image_id == 'original':
+                        i['preference'] = 10
+                    if size:
+                        i.update({
+                            'width': size,
+                            'height': size,
+                        })
+                    thumbnails.append(i)
+            else:
+                thumbnails = [{'url': thumbnail}]
+
+        def extract_count(key):
+            return int_or_none(info.get('%s_count' % key))
+
+        return {
+            'id': track_id,
+            'uploader': user.get('username'),
+            'uploader_id': str_or_none(user.get('id')) or user.get('permalink'),
+            'uploader_url': user.get('permalink_url'),
+            'timestamp': unified_timestamp(info.get('created_at')),
+            'title': title,
+            'description': info.get('description'),
+            'thumbnails': thumbnails,
+            'duration': float_or_none(info.get('duration'), 1000),
+            'webpage_url': info.get('permalink_url'),
+            'license': info.get('license'),
+            'view_count': extract_count('playback'),
+            'like_count': extract_count('favoritings') or extract_count('likes'),
+            'comment_count': extract_count('comment'),
+            'repost_count': extract_count('reposts'),
+            'genre': info.get('genre'),
+            'formats': formats if not extract_flat else None
+        }
+
     @classmethod
     def _resolv_url(cls, url):
         return cls._API_V2_BASE + 'resolve?url=' + url
@@ -377,173 +544,6 @@ class SoundcloudIE(SoundcloudBaseIE):
         },
     ]
 
-    _IMAGE_REPL_RE = r'-([0-9a-z]+)\.jpg'
-
-    _ARTWORK_MAP = {
-        'mini': 16,
-        'tiny': 20,
-        'small': 32,
-        'badge': 47,
-        't67x67': 67,
-        'large': 100,
-        't300x300': 300,
-        'crop': 400,
-        't500x500': 500,
-        'original': 0,
-    }
-
-    def _extract_info_dict(self, info, full_title=None, secret_token=None):
-        track_id = compat_str(info['id'])
-        title = info['title']
-
-        format_urls = set()
-        formats = []
-        query = {'client_id': self._CLIENT_ID}
-        if secret_token:
-            query['secret_token'] = secret_token
-
-        if info.get('downloadable') and info.get('has_downloads_left'):
-            download_url = update_url_query(
-                self._API_V2_BASE + 'tracks/' + track_id + '/download', query)
-            redirect_url = (self._download_json(download_url, track_id, fatal=False) or {}).get('redirectUri')
-            if redirect_url:
-                urlh = self._request_webpage(
-                    HEADRequest(redirect_url), track_id, fatal=False)
-                if urlh:
-                    format_url = urlh.geturl()
-                    format_urls.add(format_url)
-                    formats.append({
-                        'format_id': 'download',
-                        'ext': urlhandle_detect_ext(urlh) or 'mp3',
-                        'filesize': int_or_none(urlh.headers.get('Content-Length')),
-                        'url': format_url,
-                        'quality': 10,
-                    })
-
-        def invalid_url(url):
-            return not url or url in format_urls
-
-        def add_format(f, protocol, is_preview=False):
-            mobj = re.search(r'\.(?P<abr>\d+)\.(?P<ext>[0-9a-z]{3,4})(?=[/?])', stream_url)
-            if mobj:
-                for k, v in mobj.groupdict().items():
-                    if not f.get(k):
-                        f[k] = v
-            format_id_list = []
-            if protocol:
-                format_id_list.append(protocol)
-            ext = f.get('ext')
-            if ext == 'aac':
-                f['abr'] = '256'
-            for k in ('ext', 'abr'):
-                v = f.get(k)
-                if v:
-                    format_id_list.append(v)
-            preview = is_preview or re.search(r'/(?:preview|playlist)/0/30/', f['url'])
-            if preview:
-                format_id_list.append('preview')
-            abr = f.get('abr')
-            if abr:
-                f['abr'] = int(abr)
-            if protocol == 'hls':
-                protocol = 'm3u8' if ext == 'aac' else 'm3u8_native'
-            else:
-                protocol = 'http'
-            f.update({
-                'format_id': '_'.join(format_id_list),
-                'protocol': protocol,
-                'preference': -10 if preview else None,
-            })
-            formats.append(f)
-
-        # New API
-        transcodings = try_get(
-            info, lambda x: x['media']['transcodings'], list) or []
-        for t in transcodings:
-            if not isinstance(t, dict):
-                continue
-            format_url = url_or_none(t.get('url'))
-            if not format_url:
-                continue
-            stream = self._download_json(
-                format_url, track_id, query=query, fatal=False, headers=self._HEADERS)
-            if not isinstance(stream, dict):
-                continue
-            stream_url = url_or_none(stream.get('url'))
-            if invalid_url(stream_url):
-                continue
-            format_urls.add(stream_url)
-            stream_format = t.get('format') or {}
-            protocol = stream_format.get('protocol')
-            if protocol != 'hls' and '/hls' in format_url:
-                protocol = 'hls'
-            ext = None
-            preset = str_or_none(t.get('preset'))
-            if preset:
-                ext = preset.split('_')[0]
-            if ext not in KNOWN_EXTENSIONS:
-                ext = mimetype2ext(stream_format.get('mime_type'))
-            add_format({
-                'url': stream_url,
-                'ext': ext,
-            }, 'http' if protocol == 'progressive' else protocol,
-                t.get('snipped') or '/preview/' in format_url)
-
-        for f in formats:
-            f['vcodec'] = 'none'
-
-        if not formats and info.get('policy') == 'BLOCK':
-            self.raise_geo_restricted(metadata_available=True)
-        self._sort_formats(formats)
-
-        user = info.get('user') or {}
-
-        thumbnails = []
-        artwork_url = info.get('artwork_url')
-        thumbnail = artwork_url or user.get('avatar_url')
-        if isinstance(thumbnail, compat_str):
-            if re.search(self._IMAGE_REPL_RE, thumbnail):
-                for image_id, size in self._ARTWORK_MAP.items():
-                    i = {
-                        'id': image_id,
-                        'url': re.sub(self._IMAGE_REPL_RE, '-%s.jpg' % image_id, thumbnail),
-                    }
-                    if image_id == 'tiny' and not artwork_url:
-                        size = 18
-                    elif image_id == 'original':
-                        i['preference'] = 10
-                    if size:
-                        i.update({
-                            'width': size,
-                            'height': size,
-                        })
-                    thumbnails.append(i)
-            else:
-                thumbnails = [{'url': thumbnail}]
-
-        def extract_count(key):
-            return int_or_none(info.get('%s_count' % key))
-
-        return {
-            'id': track_id,
-            'uploader': user.get('username'),
-            'uploader_id': str_or_none(user.get('id')) or user.get('permalink'),
-            'uploader_url': user.get('permalink_url'),
-            'timestamp': unified_timestamp(info.get('created_at')),
-            'title': title,
-            'description': info.get('description'),
-            'thumbnails': thumbnails,
-            'duration': float_or_none(info.get('duration'), 1000),
-            'webpage_url': info.get('permalink_url'),
-            'license': info.get('license'),
-            'view_count': extract_count('playback'),
-            'like_count': extract_count('favoritings') or extract_count('likes'),
-            'comment_count': extract_count('comment'),
-            'repost_count': extract_count('reposts'),
-            'genre': info.get('genre'),
-            'formats': formats
-        }
-
     def _real_extract(self, url):
         mobj = self._match_valid_url(url)
 
@@ -891,6 +891,7 @@ class SoundcloudSearchIE(SoundcloudBaseIE, SearchInfoExtractor):
     _TESTS = [{
         'url': 'scsearch15:post-avant jazzcore',
         'info_dict': {
+            'id': 'post-avant jazzcore',
             'title': 'post-avant jazzcore',
         },
         'playlist_count': 15,
@@ -917,7 +918,8 @@ def _get_collection(self, endpoint, collection_id, **query):
 
             for item in response.get('collection') or []:
                 if item:
-                    yield self.url_result(item['uri'], SoundcloudIE.ie_key())
+                    yield self.url_result(
+                        item['uri'], SoundcloudIE.ie_key(), **self._extract_info_dict(item, extract_flat=True))
 
             next_url = response.get('next_href')
             if not next_url:
diff --git a/yt_dlp/extractor/telegraaf.py b/yt_dlp/extractor/telegraaf.py
index bc9a8d608..6562d122c 100644
--- a/yt_dlp/extractor/telegraaf.py
+++ b/yt_dlp/extractor/telegraaf.py
@@ -31,7 +31,9 @@ def _real_extract(self, url):
         article_id = self._match_id(url)
 
         video_id = self._download_json(
-            'https://www.telegraaf.nl/graphql', article_id, query={
+            'https://app.telegraaf.nl/graphql', article_id,
+            headers={'User-Agent': 'De Telegraaf/6.8.11 (Android 11; en_US)'},
+            query={
                 'query': '''{
   article(uid: %s) {
     videos {
diff --git a/yt_dlp/extractor/tiktok.py b/yt_dlp/extractor/tiktok.py
index c58538394..4a35a241c 100644
--- a/yt_dlp/extractor/tiktok.py
+++ b/yt_dlp/extractor/tiktok.py
@@ -25,7 +25,7 @@
 
 
 class TikTokBaseIE(InfoExtractor):
-    _APP_VERSIONS = [('20.9.3', '293'), ('20.4.3', '243'), ('20.2.1', '221'), ('20.1.2', '212'), ('20.0.4', '204')]
+    _APP_VERSIONS = [('26.1.3', '260103'), ('26.1.2', '260102'), ('26.1.1', '260101'), ('25.6.2', '250602')]
     _WORKING_APP_VERSION = None
     _APP_NAME = 'trill'
     _AID = 1180
@@ -33,7 +33,6 @@ class TikTokBaseIE(InfoExtractor):
     _UPLOADER_URL_FORMAT = 'https://www.tiktok.com/@%s'
     _WEBPAGE_HOST = 'https://www.tiktok.com/'
     QUALITIES = ('360p', '540p', '720p', '1080p')
-    _session_initialized = False
 
     @staticmethod
     def _create_url(user_id, video_id):
@@ -43,12 +42,6 @@ def _get_sigi_state(self, webpage, display_id):
         return self._parse_json(get_element_by_id(
             'SIGI_STATE|sigi-persisted-data', webpage, escape_value=False), display_id)
 
-    def _real_initialize(self):
-        if self._session_initialized:
-            return
-        self._request_webpage(HEADRequest('https://www.tiktok.com'), None, note='Setting up session', fatal=False)
-        TikTokBaseIE._session_initialized = True
-
     def _call_api_impl(self, ep, query, manifest_app_version, video_id, fatal=True,
                        note='Downloading API JSON', errnote='Unable to download API page'):
         self._set_cookie(self._API_HOSTNAME, 'odin_tt', ''.join(random.choice('0123456789abcdef') for _ in range(160)))
@@ -289,7 +282,7 @@ def extract_addr(addr, add_meta={}):
             'uploader_url': user_url,
             'track': music_track,
             'album': str_or_none(music_info.get('album')) or None,
-            'artist': music_author,
+            'artist': music_author or None,
             'timestamp': int_or_none(aweme_detail.get('create_time')),
             'formats': formats,
             'subtitles': self.extract_subtitles(aweme_detail, aweme_id),
@@ -522,7 +515,7 @@ class TikTokIE(TikTokBaseIE):
             'repost_count': int,
             'comment_count': int,
         },
-        'expected_warnings': ['trying feed workaround', 'Unable to find video in feed']
+        'skip': 'This video is unavailable',
     }, {
         # Auto-captions available
         'url': 'https://www.tiktok.com/@hankgreen1/video/7047596209028074758',
@@ -530,18 +523,11 @@ class TikTokIE(TikTokBaseIE):
     }]
 
     def _extract_aweme_app(self, aweme_id):
-        try:
-            aweme_detail = self._call_api('aweme/detail', {'aweme_id': aweme_id}, aweme_id,
-                                          note='Downloading video details', errnote='Unable to download video details').get('aweme_detail')
-            if not aweme_detail:
-                raise ExtractorError('Video not available', video_id=aweme_id)
-        except ExtractorError as e:
-            self.report_warning(f'{e.orig_msg}; trying feed workaround')
-            feed_list = self._call_api('feed', {'aweme_id': aweme_id}, aweme_id,
-                                       note='Downloading video feed', errnote='Unable to download video feed').get('aweme_list') or []
-            aweme_detail = next((aweme for aweme in feed_list if str(aweme.get('aweme_id')) == aweme_id), None)
-            if not aweme_detail:
-                raise ExtractorError('Unable to find video in feed', video_id=aweme_id)
+        feed_list = self._call_api('feed', {'aweme_id': aweme_id}, aweme_id,
+                                   note='Downloading video feed', errnote='Unable to download video feed').get('aweme_list') or []
+        aweme_detail = next((aweme for aweme in feed_list if str(aweme.get('aweme_id')) == aweme_id), None)
+        if not aweme_detail:
+            raise ExtractorError('Unable to find video in feed', video_id=aweme_id)
         return self._parse_aweme_video_app(aweme_detail)
 
     def _real_extract(self, url):
@@ -572,6 +558,7 @@ def _real_extract(self, url):
 class TikTokUserIE(TikTokBaseIE):
     IE_NAME = 'tiktok:user'
     _VALID_URL = r'https?://(?:www\.)?tiktok\.com/@(?P<id>[\w\.-]+)/?(?:$|[#?])'
+    _WORKING = False
     _TESTS = [{
         'url': 'https://tiktok.com/@corgibobaa?lang=en',
         'playlist_mincount': 45,
@@ -708,6 +695,7 @@ def _real_extract(self, url):
 class TikTokSoundIE(TikTokBaseListIE):
     IE_NAME = 'tiktok:sound'
     _VALID_URL = r'https?://(?:www\.)?tiktok\.com/music/[\w\.-]+-(?P<id>[\d]+)[/?#&]?'
+    _WORKING = False
     _QUERY_NAME = 'music_id'
     _API_ENDPOINT = 'music/aweme'
     _TESTS = [{
@@ -731,6 +719,7 @@ class TikTokSoundIE(TikTokBaseListIE):
 class TikTokEffectIE(TikTokBaseListIE):
     IE_NAME = 'tiktok:effect'
     _VALID_URL = r'https?://(?:www\.)?tiktok\.com/sticker/[\w\.-]+-(?P<id>[\d]+)[/?#&]?'
+    _WORKING = False
     _QUERY_NAME = 'sticker_id'
     _API_ENDPOINT = 'sticker/aweme'
     _TESTS = [{
@@ -750,6 +739,7 @@ class TikTokEffectIE(TikTokBaseListIE):
 class TikTokTagIE(TikTokBaseListIE):
     IE_NAME = 'tiktok:tag'
     _VALID_URL = r'https?://(?:www\.)?tiktok\.com/tag/(?P<id>[^/?#&]+)'
+    _WORKING = False
     _QUERY_NAME = 'ch_id'
     _API_ENDPOINT = 'challenge/aweme'
     _TESTS = [{
diff --git a/yt_dlp/extractor/wistia.py b/yt_dlp/extractor/wistia.py
index 438828624..e1e5855c2 100644
--- a/yt_dlp/extractor/wistia.py
+++ b/yt_dlp/extractor/wistia.py
@@ -1,30 +1,36 @@
 import re
+import urllib.error
+import urllib.parse
+from base64 import b64decode
 
 from .common import InfoExtractor
 from ..utils import (
     ExtractorError,
     float_or_none,
     int_or_none,
-    try_call,
+    parse_qs,
+    traverse_obj,
     try_get,
+    update_url_query,
 )
 
 
 class WistiaBaseIE(InfoExtractor):
     _VALID_ID_REGEX = r'(?P<id>[a-z0-9]{10})'
     _VALID_URL_BASE = r'https?://(?:\w+\.)?wistia\.(?:net|com)/(?:embed/)?'
-    _EMBED_BASE_URL = 'http://fast.wistia.com/embed/'
+    _EMBED_BASE_URL = 'http://fast.wistia.net/embed/'
 
     def _download_embed_config(self, config_type, config_id, referer):
-        base_url = self._EMBED_BASE_URL + '%ss/%s' % (config_type, config_id)
+        base_url = self._EMBED_BASE_URL + '%s/%s' % (config_type, config_id)
         embed_config = self._download_json(
             base_url + '.json', config_id, headers={
                 'Referer': referer if referer.startswith('http') else base_url,  # Some videos require this.
             })
 
-        if isinstance(embed_config, dict) and embed_config.get('error'):
+        error = traverse_obj(embed_config, 'error')
+        if error:
             raise ExtractorError(
-                'Error while getting the playlist', expected=True)
+                f'Error while getting the playlist: {error}', expected=True)
 
         return embed_config
 
@@ -114,10 +120,38 @@ def _extract_media(self, embed_config):
             'subtitles': subtitles,
         }
 
+    @classmethod
+    def _extract_from_webpage(cls, url, webpage):
+        from .teachable import TeachableIE
+
+        if list(TeachableIE._extract_embed_urls(url, webpage)):
+            return
+
+        yield from super()._extract_from_webpage(url, webpage)
+
+    @classmethod
+    def _extract_wistia_async_embed(cls, webpage):
+        # https://wistia.com/support/embed-and-share/video-on-your-website
+        # https://wistia.com/support/embed-and-share/channel-embeds
+        yield from re.finditer(
+            r'''(?sx)
+                <(?:div|section)[^>]+class=([\"'])(?:(?!\1).)*?(?P<type>wistia[a-z_0-9]+)\s*\bwistia_async_(?P<id>[a-z0-9]{10})\b(?:(?!\1).)*?\1
+            ''', webpage)
+
+    @classmethod
+    def _extract_url_media_id(cls, url):
+        mobj = re.search(r'(?:wmediaid|wvideo(?:id)?)]?=(?P<id>[a-z0-9]{10})', urllib.parse.unquote_plus(url))
+        if mobj:
+            return mobj.group('id')
+
 
 class WistiaIE(WistiaBaseIE):
     _VALID_URL = r'(?:wistia:|%s(?:iframe|medias)/)%s' % (WistiaBaseIE._VALID_URL_BASE, WistiaBaseIE._VALID_ID_REGEX)
-    _EMBED_REGEX = [r'<(?:meta[^>]+?content|(?:iframe|script)[^>]+?src)=["\'](?P<url>(?:https?:)?//(?:fast\.)?wistia\.(?:net|com)/embed/(?:iframe|medias)/[a-z0-9]{10})']
+    _EMBED_REGEX = [
+        r'''(?x)
+            <(?:meta[^>]+?content|(?:iframe|script)[^>]+?src)=["\']
+            (?P<url>(?:https?:)?//(?:fast\.)?wistia\.(?:net|com)/embed/(?:iframe|medias)/[a-z0-9]{10})
+            ''']
     _TESTS = [{
         # with hls video
         'url': 'wistia:807fafadvk',
@@ -131,6 +165,33 @@ class WistiaIE(WistiaBaseIE):
             'timestamp': 1463607249,
             'duration': 4987.11,
         },
+        'skip': 'video unavailable',
+    }, {
+        'url': 'wistia:a6ndpko1wg',
+        'md5': '10c1ce9c4dde638202513ed17a3767bd',
+        'info_dict': {
+            'id': 'a6ndpko1wg',
+            'ext': 'bin',
+            'title': 'Episode 2: Boxed Water\'s retention is thirsty',
+            'upload_date': '20210324',
+            'description': 'md5:da5994c2c2d254833b412469d9666b7a',
+            'duration': 966.0,
+            'timestamp': 1616614369,
+            'thumbnail': 'https://embed-ssl.wistia.com/deliveries/53dc60239348dc9b9fba3755173ea4c2.bin',
+        }
+    }, {
+        'url': 'wistia:5vd7p4bct5',
+        'md5': 'b9676d24bf30945d97060638fbfe77f0',
+        'info_dict': {
+            'id': '5vd7p4bct5',
+            'ext': 'bin',
+            'title': 'md5:eaa9f64c4efd7b5f098b9b6118597679',
+            'description': 'md5:a9bea0315f0616aa5df2dc413ddcdd0f',
+            'upload_date': '20220915',
+            'timestamp': 1663258727,
+            'duration': 623.019,
+            'thumbnail': r're:https?://embed(?:-ssl)?.wistia.com/.+\.(?:jpg|bin)$',
+        },
     }, {
         'url': 'wistia:sh7fpupwlt',
         'only_matching': True,
@@ -145,39 +206,53 @@ class WistiaIE(WistiaBaseIE):
         'only_matching': True,
     }]
 
-    # https://wistia.com/support/embed-and-share/video-on-your-website
+    _WEBPAGE_TESTS = [{
+        'url': 'https://www.weidert.com/blog/wistia-channels-video-marketing-tool',
+        'info_dict': {
+            'id': 'cqwukac3z1',
+            'ext': 'bin',
+            'title': 'How Wistia Channels Can Help Capture Inbound Value From Your Video Content',
+            'duration': 158.125,
+            'timestamp': 1618974400,
+            'description': 'md5:27abc99a758573560be72600ef95cece',
+            'upload_date': '20210421',
+            'thumbnail': 'https://embed-ssl.wistia.com/deliveries/6c551820ae950cdee2306d6cbe9ef742.bin',
+        }
+    }, {
+        'url': 'https://study.com/academy/lesson/north-american-exploration-failed-colonies-of-spain-france-england.html#lesson',
+        'md5': 'b9676d24bf30945d97060638fbfe77f0',
+        'info_dict': {
+            'id': '5vd7p4bct5',
+            'ext': 'bin',
+            'title': 'paywall_north-american-exploration-failed-colonies-of-spain-france-england',
+            'upload_date': '20220915',
+            'timestamp': 1663258727,
+            'duration': 623.019,
+            'thumbnail': 'https://embed-ssl.wistia.com/deliveries/83e6ec693e2c05a0ce65809cbaead86a.bin',
+            'description': 'a Paywall Videos video',
+        },
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        embed_config = self._download_embed_config('medias', video_id, url)
+        return self._extract_media(embed_config)
+
     @classmethod
     def _extract_embed_urls(cls, url, webpage):
         urls = list(super()._extract_embed_urls(url, webpage))
-
-        for match in re.finditer(
-                r'''(?sx)
-                    <div[^>]+class=(["'])(?:(?!\1).)*?\bwistia_async_(?P<id>[a-z0-9]{10})\b(?:(?!\1).)*?\1
-                ''', webpage):
-            urls.append('wistia:%s' % match.group('id'))
-        for match in re.finditer(r'(?:data-wistia-?id=["\']|Wistia\.embed\(["\']|id=["\']wistia_)(?P<id>[a-z0-9]{10})', webpage):
+        for match in cls._extract_wistia_async_embed(webpage):
+            if match.group('type') != 'wistia_channel':
+                urls.append('wistia:%s' % match.group('id'))
+        for match in re.finditer(r'(?:data-wistia-?id=["\']|Wistia\.embed\(["\']|id=["\']wistia_)(?P<id>[a-z0-9]{10})',
+                                 webpage):
             urls.append('wistia:%s' % match.group('id'))
+        if not WistiaChannelIE._extract_embed_urls(url, webpage):  # Fallback
+            media_id = cls._extract_url_media_id(url)
+            if media_id:
+                urls.append('wistia:%s' % match.group('id'))
         return urls
 
-    @classmethod
-    def _extract_from_webpage(cls, url, webpage):
-        from .teachable import TeachableIE
-
-        if list(TeachableIE._extract_embed_urls(url, webpage)):
-            return
-
-        for entry in super()._extract_from_webpage(url, webpage):
-            yield {
-                **entry,
-                '_type': 'url_transparent',
-                'uploader': try_call(lambda: re.match(r'(?:https?://)?([^/]+)/', url).group(1)),
-            }
-
-    def _real_extract(self, url):
-        video_id = self._match_id(url)
-        embed_config = self._download_embed_config('media', video_id, url)
-        return self._extract_media(embed_config)
-
 
 class WistiaPlaylistIE(WistiaBaseIE):
     _VALID_URL = r'%splaylists/%s' % (WistiaBaseIE._VALID_URL_BASE, WistiaBaseIE._VALID_ID_REGEX)
@@ -192,7 +267,7 @@ class WistiaPlaylistIE(WistiaBaseIE):
 
     def _real_extract(self, url):
         playlist_id = self._match_id(url)
-        playlist = self._download_embed_config('playlist', playlist_id, url)
+        playlist = self._download_embed_config('playlists', playlist_id, url)
 
         entries = []
         for media in (try_get(playlist, lambda x: x[0]['medias']) or []):
@@ -202,3 +277,107 @@ def _real_extract(self, url):
             entries.append(self._extract_media(embed_config))
 
         return self.playlist_result(entries, playlist_id)
+
+
+class WistiaChannelIE(WistiaBaseIE):
+    _VALID_URL = r'(?:wistiachannel:|%schannel/)%s' % (WistiaBaseIE._VALID_URL_BASE, WistiaBaseIE._VALID_ID_REGEX)
+
+    _TESTS = [{
+        # JSON Embed API returns 403, should fall back to webpage
+        'url': 'https://fast.wistia.net/embed/channel/yvyvu7wjbg?wchannelid=yvyvu7wjbg',
+        'info_dict': {
+            'id': 'yvyvu7wjbg',
+            'title': 'Copysmith Tutorials and Education!',
+            'description': 'Learn all things Copysmith via short and informative videos!'
+        },
+        'playlist_mincount': 7,
+        'expected_warnings': ['falling back to webpage'],
+    }, {
+        'url': 'https://fast.wistia.net/embed/channel/3802iirk0l',
+        'info_dict': {
+            'id': '3802iirk0l',
+            'title': 'The Roof',
+        },
+        'playlist_mincount': 20,
+    }, {
+        # link to popup video, follow --no-playlist
+        'url': 'https://fast.wistia.net/embed/channel/3802iirk0l?wchannelid=3802iirk0l&wmediaid=sp5dqjzw3n',
+        'info_dict': {
+            'id': 'sp5dqjzw3n',
+            'ext': 'bin',
+            'title': 'The Roof S2: The Modern CRO',
+            'thumbnail': 'https://embed-ssl.wistia.com/deliveries/dadfa9233eaa505d5e0c85c23ff70741.bin',
+            'duration': 86.487,
+            'description': 'A sales leader on The Roof? Man, they really must be letting anyone up here this season.\n',
+            'timestamp': 1619790290,
+            'upload_date': '20210430',
+        },
+        'params': {'noplaylist': True, 'skip_download': True},
+    }]
+    _WEBPAGE_TESTS = [{
+        'url': 'https://www.profitwell.com/recur/boxed-out',
+        'info_dict': {
+            'id': '6jyvmqz6zs',
+            'title': 'Boxed Out',
+            'description': 'md5:14a8a93a1dbe236718e6a59f8c8c7bae',
+        },
+        'playlist_mincount': 30,
+    }, {
+        # section instead of div
+        'url': 'https://360learning.com/studio/onboarding-joei/',
+        'info_dict': {
+            'id': 'z874k93n2o',
+            'title': 'Onboarding Joei.',
+            'description': 'Coming to you weekly starting Feb 19th.',
+        },
+        'playlist_mincount': 20,
+    }, {
+        'url': 'https://amplitude.com/amplify-sessions?amp%5Bwmediaid%5D=pz0m0l0if3&amp%5Bwvideo%5D=pz0m0l0if3&wchannelid=emyjmwjf79&wmediaid=i8um783bdt',
+        'info_dict': {
+            'id': 'pz0m0l0if3',
+            'title': 'A Framework for Improving Product Team Performance',
+            'ext': 'bin',
+            'timestamp': 1653935275,
+            'upload_date': '20220530',
+            'description': 'Learn how to help your company improve and achieve your product related goals.',
+            'duration': 1854.39,
+            'thumbnail': 'https://embed-ssl.wistia.com/deliveries/12fd19e56413d9d6f04e2185c16a6f8854e25226.bin',
+        },
+        'params': {'noplaylist': True, 'skip_download': True},
+    }]
+
+    def _real_extract(self, url):
+        channel_id = self._match_id(url)
+        media_id = self._extract_url_media_id(url)
+        if not self._yes_playlist(channel_id, media_id, playlist_label='channel'):
+            return self.url_result(f'wistia:{media_id}', 'Wistia')
+
+        try:
+            data = self._download_embed_config('channel', channel_id, url)
+        except (ExtractorError, urllib.error.HTTPError):
+            # Some channels give a 403 from the JSON API
+            self.report_warning('Failed to download channel data from API, falling back to webpage.')
+            webpage = self._download_webpage(f'https://fast.wistia.net/embed/channel/{channel_id}', channel_id)
+            data = self._parse_json(
+                self._search_regex(r'wchanneljsonp-%s\'\]\s*=[^\"]*\"([A-Za-z0-9=/]*)' % channel_id, webpage, 'jsonp', channel_id),
+                channel_id, transform_source=lambda x: urllib.parse.unquote_plus(b64decode(x).decode('utf-8')))
+
+        # XXX: can there be more than one series?
+        series = traverse_obj(data, ('series', 0), default={})
+
+        entries = [
+            self.url_result(f'wistia:{video["hashedId"]}', WistiaIE, title=video.get('name'))
+            for video in traverse_obj(series, ('sections', ..., 'videos', ...)) or []
+            if video.get('hashedId')
+        ]
+
+        return self.playlist_result(
+            entries, channel_id, playlist_title=series.get('title'), playlist_description=series.get('description'))
+
+    @classmethod
+    def _extract_embed_urls(cls, url, webpage):
+        yield from super()._extract_embed_urls(url, webpage)
+        for match in cls._extract_wistia_async_embed(webpage):
+            if match.group('type') == 'wistia_channel':
+                # original url may contain wmediaid query param
+                yield update_url_query(f'wistiachannel:{match.group("id")}', parse_qs(url))
diff --git a/yt_dlp/extractor/youtube.py b/yt_dlp/extractor/youtube.py
index ac1a5f210..f73465ba4 100644
--- a/yt_dlp/extractor/youtube.py
+++ b/yt_dlp/extractor/youtube.py
@@ -30,6 +30,7 @@
     clean_html,
     datetime_from_str,
     dict_get,
+    filter_dict,
     float_or_none,
     format_field,
     get_first,
@@ -291,8 +292,8 @@ class YoutubeBaseInfoExtractor(InfoExtractor):
     _RESERVED_NAMES = (
         r'channel|c|user|playlist|watch|w|v|embed|e|watch_popup|clip|'
         r'shorts|movies|results|search|shared|hashtag|trending|explore|feed|feeds|'
-        r'browse|oembed|get_video_info|iframe_api|s/player|'
-        r'storefront|oops|index|account|reporthistory|t/terms|about|upload|signin|logout')
+        r'browse|oembed|get_video_info|iframe_api|s/player|source|'
+        r'storefront|oops|index|account|t/terms|about|upload|signin|logout')
 
     _PLAYLIST_ID_RE = r'(?:(?:PL|LL|EC|UU|FL|RD|UL|TL|PU|OLAK5uy_)[0-9A-Za-z-_]{10,}|RDMM|WL|LL|LM)'
 
@@ -617,7 +618,7 @@ def generate_api_headers(
         if auth is not None:
             headers['Authorization'] = auth
             headers['X-Origin'] = origin
-        return {h: v for h, v in headers.items() if v is not None}
+        return filter_dict(headers)
 
     def _download_ytcfg(self, client, video_id):
         url = {
@@ -672,20 +673,10 @@ def _extract_continuation(cls, renderer):
         if next_continuation:
             return next_continuation
 
-        contents = []
-        for key in ('contents', 'items'):
-            contents.extend(try_get(renderer, lambda x: x[key], list) or [])
-
-        for content in contents:
-            if not isinstance(content, dict):
-                continue
-            continuation_ep = try_get(
-                content, (lambda x: x['continuationItemRenderer']['continuationEndpoint'],
-                          lambda x: x['continuationItemRenderer']['button']['buttonRenderer']['command']),
-                dict)
-            continuation = cls._extract_continuation_ep_data(continuation_ep)
-            if continuation:
-                return continuation
+        return traverse_obj(renderer, (
+            ('contents', 'items', 'rows'), ..., 'continuationItemRenderer',
+            ('continuationEndpoint', ('button', 'buttonRenderer', 'command'))
+        ), get_all=False, expected_type=cls._extract_continuation_ep_data)
 
     @classmethod
     def _extract_alerts(cls, data):
@@ -1009,7 +1000,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
     _EMBED_REGEX = [
         r'''(?x)
             (?:
-                <iframe[^>]+?src=|
+                <(?:[0-9A-Za-z-]+?)?iframe[^>]+?src=|
                 data-video-url=|
                 <embed[^>]+?src=|
                 embedSWF\(?:\s*|
@@ -3034,8 +3025,9 @@ def _extract_chapters(self, chapter_list, chapter_time, chapter_title, duration,
                 self.report_warning(f'Incomplete chapter {idx}')
             elif chapters[-1]['start_time'] <= chapter['start_time'] <= duration:
                 chapters.append(chapter)
-            else:
-                self.report_warning(f'Invalid start time for chapter "{chapter["title"]}"')
+            elif chapter not in chapters:
+                self.report_warning(
+                    f'Invalid start time ({chapter["start_time"]} < {chapters[-1]["start_time"]}) for chapter "{chapter["title"]}"')
         return chapters[1:]
 
     def _extract_comment(self, comment_renderer, parent=None):
@@ -4327,8 +4319,8 @@ def _playlist_entries(self, video_list_renderer):
             yield self._extract_video(renderer)
 
     def _rich_entries(self, rich_grid_renderer):
-        renderer = try_get(
-            rich_grid_renderer, lambda x: x['content']['videoRenderer'], dict) or {}
+        renderer = traverse_obj(
+            rich_grid_renderer, ('content', ('videoRenderer', 'reelItemRenderer')), get_all=False) or {}
         video_id = renderer.get('videoId')
         if not video_id:
             return
@@ -4405,6 +4397,13 @@ def _rich_grid_entries(self, contents):
                     yield entry
     '''
 
+    def _report_history_entries(self, renderer):
+        for url in traverse_obj(renderer, (
+                'rows', ..., 'reportHistoryTableRowRenderer', 'cells', ...,
+                'reportHistoryTableCellRenderer', 'cell', 'reportHistoryTableTextCellRenderer', 'text', 'runs', ...,
+                'navigationEndpoint', 'commandMetadata', 'webCommandMetadata', 'url')):
+            yield self.url_result(urljoin('https://www.youtube.com', url), YoutubeIE)
+
     def _extract_entries(self, parent_renderer, continuation_list):
         # continuation_list is modified in-place with continuation_list = [continuation_token]
         continuation_list[:] = [None]
@@ -4416,12 +4415,16 @@ def _extract_entries(self, parent_renderer, continuation_list):
                 content, 'itemSectionRenderer', 'musicShelfRenderer', 'musicShelfContinuation',
                 expected_type=dict)
             if not is_renderer:
-                renderer = content.get('richItemRenderer')
-                if renderer:
-                    for entry in self._rich_entries(renderer):
+                if content.get('richItemRenderer'):
+                    for entry in self._rich_entries(content['richItemRenderer']):
                         yield entry
                     continuation_list[0] = self._extract_continuation(parent_renderer)
+                elif content.get('reportHistorySectionRenderer'):  # https://www.youtube.com/reporthistory
+                    table = traverse_obj(content, ('reportHistorySectionRenderer', 'table', 'tableRenderer'))
+                    yield from self._report_history_entries(table)
+                    continuation_list[0] = self._extract_continuation(table)
                 continue
+
             isr_contents = try_get(is_renderer, lambda x: x['contents'], list) or []
             for isr_content in isr_contents:
                 if not isinstance(isr_content, dict):
@@ -4482,26 +4485,6 @@ def _entries(self, tab, item_id, ytcfg, account_syncid, visitor_data):
             # See: https://github.com/ytdl-org/youtube-dl/issues/28702
             visitor_data = self._extract_visitor_data(response) or visitor_data
 
-            known_continuation_renderers = {
-                'playlistVideoListContinuation': self._playlist_entries,
-                'gridContinuation': self._grid_entries,
-                'itemSectionContinuation': self._post_thread_continuation_entries,
-                'sectionListContinuation': extract_entries,  # for feeds
-            }
-            continuation_contents = try_get(
-                response, lambda x: x['continuationContents'], dict) or {}
-            continuation_renderer = None
-            for key, value in continuation_contents.items():
-                if key not in known_continuation_renderers:
-                    continue
-                continuation_renderer = value
-                continuation_list = [None]
-                yield from known_continuation_renderers[key](continuation_renderer)
-                continuation = continuation_list[0] or self._extract_continuation(continuation_renderer)
-                break
-            if continuation_renderer:
-                continue
-
             known_renderers = {
                 'videoRenderer': (self._grid_entries, 'items'),  # for membership tab
                 'gridPlaylistRenderer': (self._grid_entries, 'items'),
@@ -4510,24 +4493,32 @@ def _entries(self, tab, item_id, ytcfg, account_syncid, visitor_data):
                 'playlistVideoRenderer': (self._playlist_entries, 'contents'),
                 'itemSectionRenderer': (extract_entries, 'contents'),  # for feeds
                 'richItemRenderer': (extract_entries, 'contents'),  # for hashtag
-                'backstagePostThreadRenderer': (self._post_thread_continuation_entries, 'contents')
+                'backstagePostThreadRenderer': (self._post_thread_continuation_entries, 'contents'),
+                'reportHistoryTableRowRenderer': (self._report_history_entries, 'rows'),
+                'playlistVideoListContinuation': (self._playlist_entries, None),
+                'gridContinuation': (self._grid_entries, None),
+                'itemSectionContinuation': (self._post_thread_continuation_entries, None),
+                'sectionListContinuation': (extract_entries, None),  # for feeds
             }
-            on_response_received = dict_get(response, ('onResponseReceivedActions', 'onResponseReceivedEndpoints'))
-            continuation_items = try_get(
-                on_response_received, lambda x: x[0]['appendContinuationItemsAction']['continuationItems'], list)
-            continuation_item = try_get(continuation_items, lambda x: x[0], dict) or {}
+
+            continuation_items = traverse_obj(response, (
+                ('onResponseReceivedActions', 'onResponseReceivedEndpoints'), ...,
+                'appendContinuationItemsAction', 'continuationItems'
+            ), 'continuationContents', get_all=False)
+            continuation_item = traverse_obj(continuation_items, 0, None, expected_type=dict, default={})
+
             video_items_renderer = None
-            for key, value in continuation_item.items():
+            for key in continuation_item.keys():
                 if key not in known_renderers:
                     continue
-                video_items_renderer = {known_renderers[key][1]: continuation_items}
+                func, parent_key = known_renderers[key]
+                video_items_renderer = {parent_key: continuation_items} if parent_key else continuation_items
                 continuation_list = [None]
-                yield from known_renderers[key][0](video_items_renderer)
+                yield from func(video_items_renderer)
                 continuation = continuation_list[0] or self._extract_continuation(video_items_renderer)
+
+            if not video_items_renderer:
                 break
-            if video_items_renderer:
-                continue
-            break
 
     @staticmethod
     def _extract_selected_tab(tabs, fatal=True):
@@ -4553,7 +4544,7 @@ def _extract_uploader(self, data):
             uploader['uploader_url'] = urljoin(
                 'https://www.youtube.com/',
                 try_get(owner, lambda x: x['navigationEndpoint']['browseEndpoint']['canonicalBaseUrl'], str))
-        return {k: v for k, v in uploader.items() if v is not None}
+        return filter_dict(uploader)
 
     def _extract_from_tabs(self, item_id, ytcfg, data, tabs):
         playlist_id = title = description = channel_url = channel_name = channel_id = None
@@ -5640,6 +5631,16 @@ class YoutubeTabIE(YoutubeTabBaseInfoExtractor):
         'playlist_mincount': 1,
         'params': {'extractor_args': {'youtube': {'lang': ['ja']}}},
         'expected_warnings': ['Preferring "ja"'],
+    }, {
+        # shorts audio pivot for 2GtVksBMYFM.
+        'url': 'https://www.youtube.com/feed/sfv_audio_pivot?bp=8gUrCikSJwoLMkd0VmtzQk1ZRk0SCzJHdFZrc0JNWUZNGgsyR3RWa3NCTVlGTQ==',
+        'info_dict': {
+            'id': 'sfv_audio_pivot',
+            'title': 'sfv_audio_pivot',
+            'tags': [],
+        },
+        'playlist_mincount': 50,
+
     }]
 
     @classmethod
@@ -6307,6 +6308,30 @@ def _real_extract(self, url):
             ie=YoutubeTabIE, video_id=playlist_id)
 
 
+class YoutubeShortsAudioPivotIE(InfoExtractor):
+    IE_DESC = 'YouTube Shorts audio pivot (Shorts using audio of a given video)'
+    IE_NAME = 'youtube:shorts:pivot:audio'
+    _VALID_URL = r'https?://(?:www\.)?youtube\.com/source/(?P<id>[\w-]{11})/shorts'
+    _TESTS = [{
+        'url': 'https://www.youtube.com/source/Lyj-MZSAA9o/shorts',
+        'only_matching': True,
+    }]
+
+    @staticmethod
+    def _generate_audio_pivot_params(video_id):
+        """
+        Generates sfv_audio_pivot browse params for this video id
+        """
+        pb_params = b'\xf2\x05+\n)\x12\'\n\x0b%b\x12\x0b%b\x1a\x0b%b' % ((video_id.encode(),) * 3)
+        return urllib.parse.quote(base64.b64encode(pb_params).decode())
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        return self.url_result(
+            f'https://www.youtube.com/feed/sfv_audio_pivot?bp={self._generate_audio_pivot_params(video_id)}',
+            ie=YoutubeTabIE)
+
+
 class YoutubeTruncatedURLIE(InfoExtractor):
     IE_NAME = 'youtube:truncated_url'
     IE_DESC = False  # Do not list
diff --git a/yt_dlp/options.py b/yt_dlp/options.py
index 9ad48486e..861bbf786 100644
--- a/yt_dlp/options.py
+++ b/yt_dlp/options.py
@@ -1820,14 +1820,14 @@ def _alias_callback(option, opt_str, value, parser, opts, nargs):
         val.replace(r'\,', ',').strip() for val in re.split(r'(?<!\\),', vals)])
     extractor.add_option(
         '--extractor-args',
-        metavar='KEY:ARGS', dest='extractor_args', default={}, type='str',
+        metavar='IE_KEY:ARGS', dest='extractor_args', default={}, type='str',
         action='callback', callback=_dict_from_options_callback,
         callback_kwargs={
             'multiple_keys': False,
             'process': lambda val: dict(
                 _extractor_arg_parser(*arg.split('=', 1)) for arg in val.split(';'))
         }, help=(
-            'Pass these arguments to the extractor. See "EXTRACTOR ARGUMENTS" for details. '
+            'Pass ARGS arguments to the IE_KEY extractor. See "EXTRACTOR ARGUMENTS" for details. '
             'You can use this option multiple times to give arguments for different extractors'))
     extractor.add_option(
         '--youtube-include-dash-manifest', '--no-youtube-skip-dash-manifest',
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
index 443c49814..3e2ce8434 100644
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -5,6 +5,7 @@
 import calendar
 import codecs
 import collections
+import collections.abc
 import contextlib
 import datetime
 import email.header
@@ -231,7 +232,7 @@ def random_user_agent():
 ])
 
 PACKED_CODES_RE = r"}\('(.+)',(\d+),(\d+),'([^']+)'\.split\('\|'\)"
-JSON_LD_RE = r'(?is)<script[^>]+type=(["\']?)application/ld\+json\1[^>]*>\s*(?P<json_ld>{.+?})\s*</script>'
+JSON_LD_RE = r'(?is)<script[^>]+type=(["\']?)application/ld\+json\1[^>]*>\s*(?P<json_ld>{.+?}|\[.+?\])\s*</script>'
 
 NUMBER_RE = r'\d+(?:\.\d+)?'
 
@@ -591,9 +592,14 @@ def __init__(self, *args, transform_source=None, ignore_extra=False, **kwargs):
     def decode(self, s):
         if self.transform_source:
             s = self.transform_source(s)
-        if self.ignore_extra:
-            return self.raw_decode(s.lstrip())[0]
-        return super().decode(s)
+        try:
+            if self.ignore_extra:
+                return self.raw_decode(s.lstrip())[0]
+            return super().decode(s)
+        except json.JSONDecodeError as e:
+            if e.pos is not None:
+                raise type(e)(f'{e.msg} in {s[e.pos-10:e.pos+10]!r}', s, e.pos)
+            raise
 
 
 def sanitize_open(filename, open_mode):
@@ -762,7 +768,7 @@ def sanitized_Request(url, *args, **kwargs):
 
 
 def expand_path(s):
-    """Expand $ shell variables and ~"""
+    """Expand shell variables and ~"""
     return os.path.expandvars(compat_expanduser(s))
 
 
@@ -885,8 +891,9 @@ def kill(self, *, timeout=0):
     @classmethod
     def run(cls, *args, timeout=None, **kwargs):
         with cls(*args, **kwargs) as proc:
+            default = '' if proc.text_mode else b''
             stdout, stderr = proc.communicate_or_kill(timeout=timeout)
-            return stdout or '', stderr or '', proc.returncode
+            return stdout or default, stderr or default, proc.returncode
 
 
 def get_subprocess_encoding():
@@ -3184,7 +3191,7 @@ def try_call(*funcs, expected_type=None, args=[], kwargs={}):
     for f in funcs:
         try:
             val = f(*args, **kwargs)
-        except (AttributeError, KeyError, TypeError, IndexError, ZeroDivisionError):
+        except (AttributeError, KeyError, TypeError, IndexError, ValueError, ZeroDivisionError):
             pass
         else:
             if expected_type is None or isinstance(val, expected_type):
@@ -3293,7 +3300,7 @@ def fix_kv(m):
                     return '"%d":' % i if v.endswith(':') else '%d' % i
 
             if v in vars:
-                return vars[v]
+                return json.dumps(vars[v])
             if strict:
                 raise ValueError(f'Unknown value: {v}')
 
@@ -3305,6 +3312,7 @@ def create_map(mobj):
     code = re.sub(r'new Map\((\[.*?\])?\)', create_map, code)
     if not strict:
         code = re.sub(r'new Date\((".+")\)', r'\g<1>', code)
+        code = re.sub(r'new \w+\((.*?)\)', lambda m: json.dumps(m.group(0)), code)
 
     return re.sub(r'''(?sx)
         "(?:[^"\\]*(?:\\\\|\\['"nurtbfx/\n]))*[^"\\]*"|
@@ -3785,6 +3793,9 @@ def __init__(self, chapters, ranges):
         self.chapters, self.ranges = chapters, ranges
 
     def __call__(self, info_dict, ydl):
+        if not self.ranges and not self.chapters:
+            yield {}
+
         warning = ('There are no chapters matching the regex' if info_dict.get('chapters')
                    else 'Cannot match chapters since chapter information is unavailable')
         for regex in self.chapters or []:
@@ -5279,107 +5290,149 @@ def load_plugins(name, suffix, namespace):
 
 
 def traverse_obj(
-        obj, *path_list, default=None, expected_type=None, get_all=True,
+        obj, *paths, default=None, expected_type=None, get_all=True,
         casesense=True, is_user_input=False, traverse_string=False):
-    ''' Traverse nested list/dict/tuple
-    @param path_list        A list of paths which are checked one by one.
-                            Each path is a list of keys where each key is a:
-                              - None:     Do nothing
-                              - string:   A dictionary key / regex group
-                              - int:      An index into a list
-                              - tuple:    A list of keys all of which will be traversed
-                              - Ellipsis: Fetch all values in the object
-                              - Function: Takes the key and value as arguments
-                                          and returns whether the key matches or not
-    @param default          Default value to return
-    @param expected_type    Only accept final value of this type (Can also be any callable)
-    @param get_all          Return all the values obtained from a path or only the first one
-    @param casesense        Whether to consider dictionary keys as case sensitive
-
-    The following are only meant to be used by YoutubeDL.prepare_outtmpl and is not part of the API
-
-    @param path_list        In addition to the above,
-                              - dict:     Given {k:v, ...}; return {k: traverse_obj(obj, v), ...}
-    @param is_user_input    Whether the keys are generated from user input. If True,
-                            strings are converted to int/slice if necessary
-    @param traverse_string  Whether to traverse inside strings. If True, any
-                            non-compatible object will also be converted into a string
-    '''  # TODO: Write tests
-    if not casesense:
-        _lower = lambda k: (k.lower() if isinstance(k, str) else k)
-        path_list = (map(_lower, variadic(path)) for path in path_list)
-
-    def _traverse_obj(obj, path, _current_depth=0):
-        nonlocal depth
-        path = tuple(variadic(path))
-        for i, key in enumerate(path):
-            if None in (key, obj):
-                return obj
-            if isinstance(key, (list, tuple)):
-                obj = [_traverse_obj(obj, sub_key, _current_depth) for sub_key in key]
-                key = ...
-
-            if key is ...:
-                obj = (obj.values() if isinstance(obj, dict)
-                       else obj if isinstance(obj, (list, tuple, LazyList))
-                       else str(obj) if traverse_string else [])
-                _current_depth += 1
-                depth = max(depth, _current_depth)
-                return [_traverse_obj(inner_obj, path[i + 1:], _current_depth) for inner_obj in obj]
-            elif isinstance(key, dict):
-                obj = filter_dict({k: _traverse_obj(obj, v, _current_depth) for k, v in key.items()})
-            elif callable(key):
-                if isinstance(obj, (list, tuple, LazyList)):
-                    obj = enumerate(obj)
-                elif isinstance(obj, dict):
-                    obj = obj.items()
-                else:
-                    if not traverse_string:
-                        return None
-                    obj = str(obj)
-                _current_depth += 1
-                depth = max(depth, _current_depth)
-                return [_traverse_obj(v, path[i + 1:], _current_depth) for k, v in obj if try_call(key, args=(k, v))]
-            elif isinstance(obj, dict) and not (is_user_input and key == ':'):
-                obj = (obj.get(key) if casesense or (key in obj)
-                       else next((v for k, v in obj.items() if _lower(k) == key), None))
-            else:
-                if is_user_input:
-                    key = (int_or_none(key) if ':' not in key
-                           else slice(*map(int_or_none, key.split(':'))))
-                    if key == slice(None):
-                        return _traverse_obj(obj, (..., *path[i + 1:]), _current_depth)
-                if not isinstance(key, (int, slice)):
-                    return None
-                if not isinstance(obj, (list, tuple, LazyList)):
-                    if not traverse_string:
-                        return None
-                    obj = str(obj)
-                try:
-                    obj = obj[key]
-                except IndexError:
-                    return None
-        return obj
+    """
+    Safely traverse nested `dict`s and `Sequence`s
+
+    >>> obj = [{}, {"key": "value"}]
+    >>> traverse_obj(obj, (1, "key"))
+    "value"
+
+    Each of the provided `paths` is tested and the first producing a valid result will be returned.
+    A value of None is treated as the absence of a value.
+
+    The paths will be wrapped in `variadic`, so that `'key'` is conveniently the same as `('key', )`.
+
+    The keys in the path can be one of:
+        - `None`:           Return the current object.
+        - `str`/`int`:      Return `obj[key]`.
+        - `slice`:          Branch out and return all values in `obj[key]`.
+        - `Ellipsis`:       Branch out and return a list of all values.
+        - `tuple`/`list`:   Branch out and return a list of all matching values.
+                            Read as: `[traverse_obj(obj, branch) for branch in branches]`.
+        - `function`:       Branch out and return values filtered by the function.
+                            Read as: `[value for key, value in obj if function(key, value)]`.
+                            For `Sequence`s, `key` is the index of the value.
+        - `dict`            Transform the current object and return a matching dict.
+                            Read as: `{key: traverse_obj(obj, path) for key, path in dct.items()}`.
+
+        `tuple`, `list`, and `dict` all support nested paths and branches
+
+    @params paths           Paths which to traverse by.
+    @param default          Value to return if the paths do not match.
+    @param expected_type    If a `type`, only accept final values of this type.
+                            If any other callable, try to call the function on each result.
+    @param get_all          If `False`, return the first matching result, otherwise all matching ones.
+    @param casesense        If `False`, consider string dictionary keys as case insensitive.
+
+    The following are only meant to be used by YoutubeDL.prepare_outtmpl and are not part of the API
+
+    @param is_user_input    Whether the keys are generated from user input.
+                            If `True` strings get converted to `int`/`slice` if needed.
+    @param traverse_string  Whether to traverse into objects as strings.
+                            If `True`, any non-compatible object will first be
+                            converted into a string and then traversed into.
+
+
+    @returns                The result of the object traversal.
+                            If successful, `get_all=True`, and the path branches at least once,
+                            then a list of results is returned instead.
+    """
+    is_sequence = lambda x: isinstance(x, collections.abc.Sequence) and not isinstance(x, (str, bytes))
+    casefold = lambda k: k.casefold() if isinstance(k, str) else k
 
     if isinstance(expected_type, type):
         type_test = lambda val: val if isinstance(val, expected_type) else None
     else:
-        type_test = expected_type or IDENTITY
-
-    for path in path_list:
-        depth = 0
-        val = _traverse_obj(obj, path)
-        if val is not None:
-            if depth:
-                for _ in range(depth - 1):
-                    val = itertools.chain.from_iterable(v for v in val if v is not None)
-                val = [v for v in map(type_test, val) if v is not None]
-                if val:
-                    return val if get_all else val[0]
+        type_test = lambda val: try_call(expected_type or IDENTITY, args=(val,))
+
+    def apply_key(key, obj):
+        if obj is None:
+            return
+
+        elif key is None:
+            yield obj
+
+        elif isinstance(key, (list, tuple)):
+            for branch in key:
+                _, result = apply_path(obj, branch)
+                yield from result
+
+        elif key is ...:
+            if isinstance(obj, collections.abc.Mapping):
+                yield from obj.values()
+            elif is_sequence(obj):
+                yield from obj
+            elif traverse_string:
+                yield from str(obj)
+
+        elif callable(key):
+            if is_sequence(obj):
+                iter_obj = enumerate(obj)
+            elif isinstance(obj, collections.abc.Mapping):
+                iter_obj = obj.items()
+            elif traverse_string:
+                iter_obj = enumerate(str(obj))
             else:
-                val = type_test(val)
-                if val is not None:
-                    return val
+                return
+            yield from (v for k, v in iter_obj if try_call(key, args=(k, v)))
+
+        elif isinstance(key, dict):
+            iter_obj = ((k, _traverse_obj(obj, v)) for k, v in key.items())
+            yield {k: v if v is not None else default for k, v in iter_obj
+                   if v is not None or default is not None}
+
+        elif isinstance(obj, dict):
+            yield (obj.get(key) if casesense or (key in obj)
+                   else next((v for k, v in obj.items() if casefold(k) == key), None))
+
+        else:
+            if is_user_input:
+                key = (int_or_none(key) if ':' not in key
+                       else slice(*map(int_or_none, key.split(':'))))
+
+            if not isinstance(key, (int, slice)):
+                return
+
+            if not is_sequence(obj):
+                if not traverse_string:
+                    return
+                obj = str(obj)
+
+            with contextlib.suppress(IndexError):
+                yield obj[key]
+
+    def apply_path(start_obj, path):
+        objs = (start_obj,)
+        has_branched = False
+
+        for key in variadic(path):
+            if is_user_input and key == ':':
+                key = ...
+
+            if not casesense and isinstance(key, str):
+                key = key.casefold()
+
+            if key is ... or isinstance(key, (list, tuple)) or callable(key):
+                has_branched = True
+
+            key_func = functools.partial(apply_key, key)
+            objs = itertools.chain.from_iterable(map(key_func, objs))
+
+        return has_branched, objs
+
+    def _traverse_obj(obj, path):
+        has_branched, results = apply_path(obj, path)
+        results = LazyList(x for x in map(type_test, results) if x is not None)
+        if results:
+            return results.exhaust() if get_all and has_branched else results[0]
+
+    for path in paths:
+        result = _traverse_obj(obj, path)
+        if result is not None:
+            return result
+
     return default
 
 
diff --git a/yt_dlp/webvtt.py b/yt_dlp/webvtt.py
index 23d67a897..1138865ba 100644
--- a/yt_dlp/webvtt.py
+++ b/yt_dlp/webvtt.py
@@ -140,7 +140,6 @@ class HeaderBlock(Block):
     A WebVTT block that may only appear in the header part of the file,
     i.e. before any cue blocks.
     """
-
     pass
 
 


Test patch:
diff --git a/test/test_utils.py b/test/test_utils.py
index 96477c53f..69313564a 100644
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -109,6 +109,7 @@
     strip_or_none,
     subtitles_filename,
     timeconvert,
+    traverse_obj,
     unescapeHTML,
     unified_strdate,
     unified_timestamp,
@@ -1874,6 +1875,192 @@ def test_get_compatible_ext(self):
         self.assertEqual(get_compatible_ext(
             vcodecs=['av1'], acodecs=['mp4a'], vexts=['webm'], aexts=['m4a'], preferences=('webm', 'mkv')), 'mkv')
 
+    def test_traverse_obj(self):
+        _TEST_DATA = {
+            100: 100,
+            1.2: 1.2,
+            'str': 'str',
+            'None': None,
+            '...': ...,
+            'urls': [
+                {'index': 0, 'url': 'https://www.example.com/0'},
+                {'index': 1, 'url': 'https://www.example.com/1'},
+            ],
+            'data': (
+                {'index': 2},
+                {'index': 3},
+            ),
+        }
+
+        # Test base functionality
+        self.assertEqual(traverse_obj(_TEST_DATA, ('str',)), 'str',
+                         msg='allow tuple path')
+        self.assertEqual(traverse_obj(_TEST_DATA, ['str']), 'str',
+                         msg='allow list path')
+        self.assertEqual(traverse_obj(_TEST_DATA, (value for value in ("str",))), 'str',
+                         msg='allow iterable path')
+        self.assertEqual(traverse_obj(_TEST_DATA, 'str'), 'str',
+                         msg='single items should be treated as a path')
+        self.assertEqual(traverse_obj(_TEST_DATA, None), _TEST_DATA)
+        self.assertEqual(traverse_obj(_TEST_DATA, 100), 100)
+        self.assertEqual(traverse_obj(_TEST_DATA, 1.2), 1.2)
+
+        # Test Ellipsis behavior
+        self.assertCountEqual(traverse_obj(_TEST_DATA, ...),
+                              (item for item in _TEST_DATA.values() if item is not None),
+                              msg='`...` should give all values except `None`')
+        self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', 0, ...)), _TEST_DATA['urls'][0].values(),
+                              msg='`...` selection for dicts should select all values')
+        self.assertEqual(traverse_obj(_TEST_DATA, (..., ..., 'url')),
+                         ['https://www.example.com/0', 'https://www.example.com/1'],
+                         msg='nested `...` queries should work')
+        self.assertCountEqual(traverse_obj(_TEST_DATA, (..., ..., 'index')), range(4),
+                              msg='`...` query result should be flattened')
+
+        # Test function as key
+        self.assertEqual(traverse_obj(_TEST_DATA, lambda x, y: x == 'urls' and isinstance(y, list)),
+                         [_TEST_DATA['urls']],
+                         msg='function as query key should perform a filter based on (key, value)')
+        self.assertCountEqual(traverse_obj(_TEST_DATA, lambda _, x: isinstance(x[0], str)), {'str'},
+                              msg='exceptions in the query function should be catched')
+
+        # Test alternative paths
+        self.assertEqual(traverse_obj(_TEST_DATA, 'fail', 'str'), 'str',
+                         msg='multiple `path_list` should be treated as alternative paths')
+        self.assertEqual(traverse_obj(_TEST_DATA, 'str', 100), 'str',
+                         msg='alternatives should exit early')
+        self.assertEqual(traverse_obj(_TEST_DATA, 'fail', 'fail'), None,
+                         msg='alternatives should return `default` if exhausted')
+
+        # Test branch and path nesting
+        self.assertEqual(traverse_obj(_TEST_DATA, ('urls', (3, 0), 'url')), ['https://www.example.com/0'],
+                         msg='tuple as key should be treated as branches')
+        self.assertEqual(traverse_obj(_TEST_DATA, ('urls', [3, 0], 'url')), ['https://www.example.com/0'],
+                         msg='list as key should be treated as branches')
+        self.assertEqual(traverse_obj(_TEST_DATA, ('urls', ((1, 'fail'), (0, 'url')))), ['https://www.example.com/0'],
+                         msg='double nesting in path should be treated as paths')
+        self.assertEqual(traverse_obj(['0', [1, 2]], [(0, 1), 0]), [1],
+                         msg='do not fail early on branching')
+        self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', ((1, ('fail', 'url')), (0, 'url')))),
+                              ['https://www.example.com/0', 'https://www.example.com/1'],
+                              msg='tripple nesting in path should be treated as branches')
+        self.assertEqual(traverse_obj(_TEST_DATA, ('urls', ('fail', (..., 'url')))),
+                         ['https://www.example.com/0', 'https://www.example.com/1'],
+                         msg='ellipsis as branch path start gets flattened')
+
+        # Test dictionary as key
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: 100, 1: 1.2}), {0: 100, 1: 1.2},
+                         msg='dict key should result in a dict with the same keys')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', 0, 'url')}),
+                         {0: 'https://www.example.com/0'},
+                         msg='dict key should allow paths')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', (3, 0), 'url')}),
+                         {0: ['https://www.example.com/0']},
+                         msg='tuple in dict path should be treated as branches')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', ((1, 'fail'), (0, 'url')))}),
+                         {0: ['https://www.example.com/0']},
+                         msg='double nesting in dict path should be treated as paths')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: ('urls', ((1, ('fail', 'url')), (0, 'url')))}),
+                         {0: ['https://www.example.com/1', 'https://www.example.com/0']},
+                         msg='tripple nesting in dict path should be treated as branches')
+        self.assertEqual(traverse_obj({}, {0: 1}, default=...), {0: ...},
+                         msg='do not remove `None` values when dict key')
+
+        # Testing default parameter behavior
+        _DEFAULT_DATA = {'None': None, 'int': 0, 'list': []}
+        self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail'), None,
+                         msg='default value should be `None`')
+        self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail', 'fail', default=...), ...,
+                         msg='chained fails should result in default')
+        self.assertEqual(traverse_obj(_DEFAULT_DATA, 'None', 'int'), 0,
+                         msg='should not short cirquit on `None`')
+        self.assertEqual(traverse_obj(_DEFAULT_DATA, 'fail', default=1), 1,
+                         msg='invalid dict key should result in `default`')
+        self.assertEqual(traverse_obj(_DEFAULT_DATA, 'None', default=1), 1,
+                         msg='`None` is a deliberate sentinel and should become `default`')
+        self.assertEqual(traverse_obj(_DEFAULT_DATA, ('list', 10)), None,
+                         msg='`IndexError` should result in `default`')
+        self.assertEqual(traverse_obj(_DEFAULT_DATA, (..., 'fail'), default=1), 1,
+                         msg='if branched but not successfull return `default`, not `[]`')
+
+        # Testing expected_type behavior
+        _EXPECTED_TYPE_DATA = {'str': 'str', 'int': 0}
+        self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=str), 'str',
+                         msg='accept matching `expected_type` type')
+        self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str', expected_type=int), None,
+                         msg='reject non matching `expected_type` type')
+        self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'int', expected_type=lambda x: str(x)), '0',
+                         msg='transform type using type function')
+        self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, 'str',
+                                      expected_type=lambda _: 1 / 0), None,
+                         msg='wrap expected_type fuction in try_call')
+        self.assertEqual(traverse_obj(_EXPECTED_TYPE_DATA, ..., expected_type=str), ['str'],
+                         msg='eliminate items that expected_type fails on')
+
+        # Test get_all behavior
+        _GET_ALL_DATA = {'key': [0, 1, 2]}
+        self.assertEqual(traverse_obj(_GET_ALL_DATA, ('key', ...), get_all=False), 0,
+                         msg='if not `get_all`, return only first matching value')
+        self.assertEqual(traverse_obj(_GET_ALL_DATA, ..., get_all=False), [0, 1, 2],
+                         msg='do not overflatten if not `get_all`')
+
+        # Test casesense behavior
+        _CASESENSE_DATA = {
+            'KeY': 'value0',
+            0: {
+                'KeY': 'value1',
+                0: {'KeY': 'value2'},
+            },
+        }
+        self.assertEqual(traverse_obj(_CASESENSE_DATA, 'key'), None,
+                         msg='dict keys should be case sensitive unless `casesense`')
+        self.assertEqual(traverse_obj(_CASESENSE_DATA, 'keY',
+                                      casesense=False), 'value0',
+                         msg='allow non matching key case if `casesense`')
+        self.assertEqual(traverse_obj(_CASESENSE_DATA, (0, ('keY',)),
+                                      casesense=False), ['value1'],
+                         msg='allow non matching key case in branch if `casesense`')
+        self.assertEqual(traverse_obj(_CASESENSE_DATA, (0, ((0, 'keY'),)),
+                                      casesense=False), ['value2'],
+                         msg='allow non matching key case in branch path if `casesense`')
+
+        # Test traverse_string behavior
+        _TRAVERSE_STRING_DATA = {'str': 'str', 1.2: 1.2}
+        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', 0)), None,
+                         msg='do not traverse into string if not `traverse_string`')
+        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', 0),
+                                      traverse_string=True), 's',
+                         msg='traverse into string if `traverse_string`')
+        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, (1.2, 1),
+                                      traverse_string=True), '.',
+                         msg='traverse into converted data if `traverse_string`')
+        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', ...),
+                                      traverse_string=True), list('str'),
+                         msg='`...` branching into string should result in list')
+        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', (0, 2)),
+                                      traverse_string=True), ['s', 'r'],
+                         msg='branching into string should result in list')
+        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', lambda _, x: x),
+                                      traverse_string=True), list('str'),
+                         msg='function branching into string should result in list')
+
+        # Test is_user_input behavior
+        _IS_USER_INPUT_DATA = {'range8': list(range(8))}
+        self.assertEqual(traverse_obj(_IS_USER_INPUT_DATA, ('range8', '3'),
+                                      is_user_input=True), 3,
+                         msg='allow for string indexing if `is_user_input`')
+        self.assertCountEqual(traverse_obj(_IS_USER_INPUT_DATA, ('range8', '3:'),
+                                           is_user_input=True), tuple(range(8))[3:],
+                              msg='allow for string slice if `is_user_input`')
+        self.assertCountEqual(traverse_obj(_IS_USER_INPUT_DATA, ('range8', ':4:2'),
+                                           is_user_input=True), tuple(range(8))[:4:2],
+                              msg='allow step in string slice if `is_user_input`')
+        self.assertCountEqual(traverse_obj(_IS_USER_INPUT_DATA, ('range8', ':'),
+                                           is_user_input=True), range(8),
+                              msg='`:` should be treated as `...` if `is_user_input`')
+        with self.assertRaises(TypeError, msg='too many params should result in error'):
+            traverse_obj(_IS_USER_INPUT_DATA, ('range8', ':::'), is_user_input=True)
+
 
 if __name__ == '__main__':
     unittest.main()