Repository: yt-dlp/yt-dlp
Pull request #981 (2021-09-15 12:59:13)

[CGTN] Add new extractor
## Please follow the guide below

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *pull request* (like that [x])
- Use *Preview* tab to see how your *pull request* will actually look like

---

### Before submitting a *pull request* make sure you have:
- [x] At least skimmed through [adding new extractor tutorial](https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site) and [youtube-dl coding conventions](https://github.com/ytdl-org/youtube-dl#youtube-dl-coding-conventions) sections
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [ ] Bug fix
- [ ] Improvement
- [x] New extractor
- [ ] New feature

---

### Description of your *pull request* and other information

Explanation of your *pull request* in arbitrary form goes here. Please make sure the description explains the purpose and effect of your *pull request* and is worded well enough to be understood. Provide as much context and examples as possible.



Patch:
diff --git a/.gitignore b/.gitignore
index 88a9605f7..bf06c81f0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,6 +3,7 @@
 *.spec
 cookies
 *cookies.txt
+.netrc
 
 # Downloaded
 *.srt
diff --git a/README.md b/README.md
index d9daee69e..44766b76b 100644
--- a/README.md
+++ b/README.md
@@ -695,6 +695,9 @@ ## Authentication Options:
                                      out, yt-dlp will ask interactively
     -2, --twofactor TWOFACTOR        Two-factor authentication code
     -n, --netrc                      Use .netrc authentication data
+    --netrc-location PATH            Location of .netrc authentication data;
+                                     either the path or its containing
+                                     directory. Defaults to ~/.netrc
     --video-password PASSWORD        Video password (vimeo, youku)
     --ap-mso MSO                     Adobe Pass multiple-system operator (TV
                                      provider) identifier, use --ap-list-mso for
@@ -897,7 +900,7 @@ # CONFIGURATION
     * `~/yt-dlp.conf`
     * `~/yt-dlp.conf.txt`
 
-    Note that `~` points to `C:\Users\<user name>` on windows. Also, `%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined
+    `%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined. On windows, `~` points to %HOME% if present, `%USERPROFILE%` (generally `C:\Users\<user name>`) or `%HOMEDRIVE%%HOMEPATH%`.
 1. **System Configuration**: `/etc/yt-dlp.conf`
 
 For example, with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory:
@@ -923,14 +926,14 @@ # Save all videos under YouTube directory in your home directory
 
 ### Authentication with `.netrc` file
 
-You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every yt-dlp execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in your `$HOME` and restrict permissions to read/write by only you:
+You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every yt-dlp execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in `--netrc-location` and restrict permissions to read/write by only you:
 ```
 touch $HOME/.netrc
 chmod a-rwx,u+rw $HOME/.netrc
 ```
 After that you can add credentials for an extractor in the following format, where *extractor* is the name of the extractor in lowercase:
 ```
-machine <extractor> login <login> password <password>
+machine <extractor> login <username> password <password>
 ```
 For example:
 ```
@@ -939,10 +942,7 @@ ### Authentication with `.netrc` file
 ```
 To activate authentication with the `.netrc` file you should pass `--netrc` to yt-dlp or place it in the [configuration file](#configuration).
 
-On Windows you may also need to setup the `%HOME%` environment variable manually. For example:
-```
-set HOME=%USERPROFILE%
-```
+The default location of the .netrc file is `$HOME` (`~`) in UNIX. On Windows, it is `%HOME%` if present, `%USERPROFILE%` (generally `C:\Users\<user name>`) or `%HOMEDRIVE%%HOMEPATH%`
 
 # OUTPUT TEMPLATE
 
@@ -952,18 +952,19 @@ # OUTPUT TEMPLATE
 
 The simplest usage of `-o` is not to set any template arguments when downloading a single file, like in `yt-dlp -o funny_video.flv "https://some/video"` (hard-coding file extension like this is _not_ recommended and could break some post-processing).
 
-It may however also contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations.
+It may however also contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations.
 
 The field names themselves (the part inside the parenthesis) can also have some special formatting:
 1. **Object traversal**: The dictionaries and lists available in metadata can be traversed by using a `.` (dot) separator. You can also do python slicing using `:`. Eg: `%(tags.0)s`, `%(subtitles.en.-1.ext)s`, `%(id.3:7:-1)s`, `%(formats.:.format_id)s`. `%()s` refers to the entire infodict. Note that all the fields that become available using this method are not listed below. Use `-j` to see such fields
 1. **Addition**: Addition and subtraction of numeric fields can be done using `+` and `-` respectively. Eg: `%(playlist_index+10)03d`, `%(n_entries+1-playlist_index)d`
 1. **Date/time Formatting**: Date/time fields can be formatted according to [strftime formatting](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) by specifying it separated from the field name using a `>`. Eg: `%(duration>%H-%M-%S)s`, `%(upload_date>%Y-%m-%d)s`, `%(epoch-3600>%H-%M-%S)s`
-1. **Default**: A default value can be specified for when the field is empty using a `|` seperator. This overrides `--output-na-template`. Eg: `%(uploader|Unknown)s`
-1. **More Conversions**: In addition to the normal format types `diouxXeEfFgGcrs`, `j`, `l`, `q` can be used for converting to **j**son, a comma seperated **l**ist and a string **q**uoted for the terminal respectively
+1. **Alternatives**: Alternate fields can be specified seperated with a `,`. Eg: `%(release_date>%Y,upload_date>%Y|Unknown)s`
+1. **Default**: A literal default value can be specified for when the field is empty using a `|` seperator. This overrides `--output-na-template`. Eg: `%(uploader|Unknown)s`
+1. **More Conversions**: In addition to the normal format types `diouxXeEfFgGcrs`, `B`, `j`, `l`, `q` can be used for converting to **B**ytes, **j**son, a comma seperated **l**ist and a string **q**uoted for the terminal respectively
 
 To summarize, the general syntax for a field is:
 ```
-%(name[.keys][addition][>strf][|default])[flags][width][.precision][length]type
+%(name[.keys][addition][>strf][,alternate][|default])[flags][width][.precision][length]type
 ```
 
 Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation` (deprecated), `infojson`, `pl_thumbnail`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'`  will put the thumbnails in a folder with the same name as the video.
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
index 8432abf1a..50e902c53 100644
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -35,6 +35,7 @@
     compat_kwargs,
     compat_numeric_types,
     compat_os_name,
+    compat_pycrypto_AES,
     compat_shlex_quote,
     compat_str,
     compat_tokenize_tokenize,
@@ -907,7 +908,7 @@ def escape_outtmpl(outtmpl):
     def validate_outtmpl(cls, outtmpl):
         ''' @return None or Exception object '''
         outtmpl = re.sub(
-            STR_FORMAT_RE_TMPL.format('[^)]*', '[ljq]'),
+            STR_FORMAT_RE_TMPL.format('[^)]*', '[ljqB]'),
             lambda mobj: f'{mobj.group(0)[:-1]}s',
             cls._outtmpl_expandpath(outtmpl))
         try:
@@ -939,7 +940,7 @@ def prepare_outtmpl(self, outtmpl, info_dict, sanitize=None):
         }
 
         TMPL_DICT = {}
-        EXTERNAL_FORMAT_RE = re.compile(STR_FORMAT_RE_TMPL.format('[^)]*', f'[{STR_FORMAT_TYPES}ljq]'))
+        EXTERNAL_FORMAT_RE = re.compile(STR_FORMAT_RE_TMPL.format('[^)]*', f'[{STR_FORMAT_TYPES}ljqB]'))
         MATH_FUNCTIONS = {
             '+': float.__add__,
             '-': float.__sub__,
@@ -954,6 +955,7 @@ def prepare_outtmpl(self, outtmpl, info_dict, sanitize=None):
             (?P<fields>{field})
             (?P<maths>(?:{math_op}{math_field})*)
             (?:>(?P<strf_format>.+?))?
+            (?P<alternate>(?<!\\),[^|)]+)?
             (?:\|(?P<default>.*?))?
             $'''.format(field=FIELD_RE, math_op=MATH_OPERATORS_RE, math_field=MATH_FIELD_RE))
 
@@ -995,7 +997,7 @@ def get_value(mdict):
                     operator = None
             # Datetime formatting
             if mdict['strf_format']:
-                value = strftime_or_none(value, mdict['strf_format'])
+                value = strftime_or_none(value, mdict['strf_format'].replace('\\,', ','))
 
             return value
 
@@ -1011,12 +1013,16 @@ def create_key(outer_mobj):
                 return f'%{outer_mobj.group(0)}'
             key = outer_mobj.group('key')
             mobj = re.match(INTERNAL_FORMAT_RE, key)
-            if mobj is None:
-                value, default, mobj = None, na, {'fields': ''}
-            else:
+            initial_field = mobj.group('fields').split('.')[-1] if mobj else ''
+            value, default = None, na
+            while mobj:
                 mobj = mobj.groupdict()
-                default = mobj['default'] if mobj['default'] is not None else na
+                default = mobj['default'] if mobj['default'] is not None else default
                 value = get_value(mobj)
+                if value is None and mobj['alternate']:
+                    mobj = re.match(INTERNAL_FORMAT_RE, mobj['alternate'][1:])
+                else:
+                    break
 
             fmt = outer_mobj.group('format')
             if fmt == 's' and value is not None and key in field_size_compat_map.keys():
@@ -1031,6 +1037,9 @@ def create_key(outer_mobj):
                 value, fmt = json.dumps(value, default=_dumpjson_default), str_fmt
             elif fmt[-1] == 'q':
                 value, fmt = compat_shlex_quote(str(value)), str_fmt
+            elif fmt[-1] == 'B':
+                value = f'%{str_fmt}'.encode('utf-8') % str(value).encode('utf-8')
+                value, fmt = value.decode('utf-8', 'ignore'), 's'
             elif fmt[-1] == 'c':
                 value = str(value)
                 if value is None:
@@ -1048,7 +1057,7 @@ def create_key(outer_mobj):
                     # So we convert it to repr first
                     value, fmt = repr(value), str_fmt
                 if fmt[-1] in 'csr':
-                    value = sanitize(mobj['fields'].split('.')[-1], value)
+                    value = sanitize(initial_field, value)
 
             key = '%s\0%s' % (key.replace('%', '%\0'), outer_mobj.group('format'))
             TMPL_DICT[key] = value
@@ -3292,13 +3301,12 @@ def python_implementation():
         ) or 'none'
         self._write_string('[debug] exe versions: %s\n' % exe_str)
 
-        from .downloader.fragment import can_decrypt_frag
         from .downloader.websocket import has_websockets
         from .postprocessor.embedthumbnail import has_mutagen
         from .cookies import SQLITE_AVAILABLE, KEYRING_AVAILABLE
 
         lib_str = ', '.join(sorted(filter(None, (
-            can_decrypt_frag and 'pycryptodome',
+            compat_pycrypto_AES and compat_pycrypto_AES.__name__.split('.')[0],
             has_websockets and 'websockets',
             has_mutagen and 'mutagen',
             SQLITE_AVAILABLE and 'sqlite',
diff --git a/yt_dlp/__init__.py b/yt_dlp/__init__.py
index f9a7e2f11..5168ed0f7 100644
--- a/yt_dlp/__init__.py
+++ b/yt_dlp/__init__.py
@@ -575,6 +575,7 @@ def report_args_compat(arg, name):
 
     ydl_opts = {
         'usenetrc': opts.usenetrc,
+        'netrc_location': opts.netrc_location,
         'username': opts.username,
         'password': opts.password,
         'twofactor': opts.twofactor,
diff --git a/yt_dlp/aes.py b/yt_dlp/aes.py
index 461bb6d41..57caae069 100644
--- a/yt_dlp/aes.py
+++ b/yt_dlp/aes.py
@@ -2,9 +2,21 @@
 
 from math import ceil
 
-from .compat import compat_b64decode
+from .compat import compat_b64decode, compat_pycrypto_AES
 from .utils import bytes_to_intlist, intlist_to_bytes
 
+
+if compat_pycrypto_AES:
+    def aes_cbc_decrypt_bytes(data, key, iv):
+        """ Decrypt bytes with AES-CBC using pycryptodome """
+        return compat_pycrypto_AES.new(key, compat_pycrypto_AES.MODE_CBC, iv).decrypt(data)
+
+else:
+    def aes_cbc_decrypt_bytes(data, key, iv):
+        """ Decrypt bytes with AES-CBC using native implementation since pycryptodome is unavailable """
+        return intlist_to_bytes(aes_cbc_decrypt(*map(bytes_to_intlist, (data, key, iv))))
+
+
 BLOCK_SIZE_BYTES = 16
 
 
diff --git a/yt_dlp/compat.py b/yt_dlp/compat.py
index ab1a3ba44..7b55b7d9d 100644
--- a/yt_dlp/compat.py
+++ b/yt_dlp/compat.py
@@ -130,6 +130,33 @@ def compat_asyncio_run(coro):
     asyncio.run = compat_asyncio_run
 
 
+# Python 3.8+ does not honor %HOME% on windows, but this breaks compatibility with youtube-dl
+# See https://github.com/yt-dlp/yt-dlp/issues/792
+# https://docs.python.org/3/library/os.path.html#os.path.expanduser
+if compat_os_name in ('nt', 'ce') and 'HOME' in os.environ:
+    _userhome = os.environ['HOME']
+
+    def compat_expanduser(path):
+        if not path.startswith('~'):
+            return path
+        i = path.replace('\\', '/', 1).find('/')  # ~user
+        if i < 0:
+            i = len(path)
+        userhome = os.path.join(os.path.dirname(_userhome), path[1:i]) if i > 1 else _userhome
+        return userhome + path[i:]
+else:
+    compat_expanduser = os.path.expanduser
+
+
+try:
+    from Cryptodome.Cipher import AES as compat_pycrypto_AES
+except ImportError:
+    try:
+        from Crypto.Cipher import AES as compat_pycrypto_AES
+    except ImportError:
+        compat_pycrypto_AES = None
+
+
 #  Deprecated
 
 compat_basestring = str
@@ -152,7 +179,6 @@ def compat_asyncio_run(coro):
 compat_cookies_SimpleCookie = compat_cookies.SimpleCookie
 compat_etree_Element = etree.Element
 compat_etree_register_namespace = etree.register_namespace
-compat_expanduser = os.path.expanduser
 compat_get_terminal_size = shutil.get_terminal_size
 compat_getenv = os.getenv
 compat_getpass = getpass.getpass
@@ -224,6 +250,7 @@ def compat_asyncio_run(coro):
     'compat_os_name',
     'compat_parse_qs',
     'compat_print',
+    'compat_pycrypto_AES',
     'compat_realpath',
     'compat_setenv',
     'compat_shlex_quote',
diff --git a/yt_dlp/cookies.py b/yt_dlp/cookies.py
index b5aff38dd..4f582f4e1 100644
--- a/yt_dlp/cookies.py
+++ b/yt_dlp/cookies.py
@@ -13,6 +13,7 @@
 from yt_dlp.compat import (
     compat_b64decode,
     compat_cookiejar_Cookie,
+    compat_pycrypto_AES
 )
 from yt_dlp.utils import (
     bug_reports_message,
@@ -32,12 +33,6 @@
     SQLITE_AVAILABLE = False
 
 
-try:
-    from Crypto.Cipher import AES
-    CRYPTO_AVAILABLE = True
-except ImportError:
-    CRYPTO_AVAILABLE = False
-
 try:
     import keyring
     KEYRING_AVAILABLE = True
@@ -400,7 +395,7 @@ def decrypt(self, encrypted_value):
             if self._v10_key is None:
                 self._logger.warning('cannot decrypt v10 cookies: no key found', only_once=True)
                 return None
-            elif not CRYPTO_AVAILABLE:
+            elif not compat_pycrypto_AES:
                 self._logger.warning('cannot decrypt cookie as the `pycryptodome` module is not installed. '
                                      'Please install by running `python3 -m pip install pycryptodome`',
                                      only_once=True)
@@ -660,7 +655,7 @@ def _decrypt_aes_cbc(ciphertext, key, logger, initialization_vector=b' ' * 16):
 
 
 def _decrypt_aes_gcm(ciphertext, key, nonce, authentication_tag, logger):
-    cipher = AES.new(key, AES.MODE_GCM, nonce)
+    cipher = compat_pycrypto_AES.new(key, compat_pycrypto_AES.MODE_GCM, nonce)
     try:
         plaintext = cipher.decrypt_and_verify(ciphertext, authentication_tag)
     except ValueError:
diff --git a/yt_dlp/downloader/external.py b/yt_dlp/downloader/external.py
index 9db248df4..1057382e0 100644
--- a/yt_dlp/downloader/external.py
+++ b/yt_dlp/downloader/external.py
@@ -6,13 +6,8 @@
 import sys
 import time
 
-try:
-    from Crypto.Cipher import AES
-    can_decrypt_frag = True
-except ImportError:
-    can_decrypt_frag = False
-
 from .common import FileDownloader
+from ..aes import aes_cbc_decrypt_bytes
 from ..compat import (
     compat_setenv,
     compat_str,
@@ -164,8 +159,7 @@ def _call_downloader(self, tmpfilename, info_dict):
                         decrypt_info['KEY'] = decrypt_info.get('KEY') or self.ydl.urlopen(
                             self._prepare_url(info_dict, info_dict.get('_decryption_key_url') or decrypt_info['URI'])).read()
                         encrypted_data = src.read()
-                        decrypted_data = AES.new(
-                            decrypt_info['KEY'], AES.MODE_CBC, iv).decrypt(encrypted_data)
+                        decrypted_data = aes_cbc_decrypt_bytes(encrypted_data, decrypt_info['KEY'], iv)
                         dest.write(decrypted_data)
                     else:
                         fragment_data = src.read()
diff --git a/yt_dlp/downloader/fragment.py b/yt_dlp/downloader/fragment.py
index e3af140fd..567bf69d3 100644
--- a/yt_dlp/downloader/fragment.py
+++ b/yt_dlp/downloader/fragment.py
@@ -4,12 +4,6 @@
 import time
 import json
 
-try:
-    from Crypto.Cipher import AES
-    can_decrypt_frag = True
-except ImportError:
-    can_decrypt_frag = False
-
 try:
     import concurrent.futures
     can_threaded_download = True
@@ -18,6 +12,7 @@
 
 from .common import FileDownloader
 from .http import HttpFD
+from ..aes import aes_cbc_decrypt_bytes
 from ..compat import (
     compat_urllib_error,
     compat_struct_pack,
@@ -386,7 +381,7 @@ def decrypt_fragment(fragment, frag_content):
             # not what it decrypts to.
             if self.params.get('test', False):
                 return frag_content
-            return AES.new(decrypt_info['KEY'], AES.MODE_CBC, iv).decrypt(frag_content)
+            return aes_cbc_decrypt_bytes(frag_content, decrypt_info['KEY'], iv)
 
         def append_fragment(frag_content, frag_index, ctx):
             if not frag_content:
diff --git a/yt_dlp/downloader/hls.py b/yt_dlp/downloader/hls.py
index 779658b70..e0dc1def7 100644
--- a/yt_dlp/downloader/hls.py
+++ b/yt_dlp/downloader/hls.py
@@ -5,7 +5,7 @@
 import binascii
 
 from ..downloader import get_suitable_downloader
-from .fragment import FragmentFD, can_decrypt_frag
+from .fragment import FragmentFD
 from .external import FFmpegFD
 
 from ..compat import (
@@ -29,7 +29,7 @@ class HlsFD(FragmentFD):
     FD_NAME = 'hlsnative'
 
     @staticmethod
-    def can_download(manifest, info_dict, allow_unplayable_formats=False, with_crypto=can_decrypt_frag):
+    def can_download(manifest, info_dict, allow_unplayable_formats=False):
         UNSUPPORTED_FEATURES = [
             # r'#EXT-X-BYTERANGE',  # playlists composed of byte ranges of media files [2]
 
@@ -56,9 +56,6 @@ def can_download(manifest, info_dict, allow_unplayable_formats=False, with_crypt
 
         def check_results():
             yield not info_dict.get('is_live')
-            is_aes128_enc = '#EXT-X-KEY:METHOD=AES-128' in manifest
-            yield with_crypto or not is_aes128_enc
-            yield not (is_aes128_enc and r'#EXT-X-BYTERANGE' in manifest)
             for feature in UNSUPPORTED_FEATURES:
                 yield not re.search(feature, manifest)
         return all(check_results())
@@ -75,8 +72,6 @@ def real_download(self, filename, info_dict):
             if info_dict.get('extra_param_to_segment_url') or info_dict.get('_decryption_key_url'):
                 self.report_error('pycryptodome not found. Please install')
                 return False
-            if self.can_download(s, info_dict, with_crypto=True):
-                self.report_warning('pycryptodome is needed to download this file natively')
             fd = FFmpegFD(self.ydl, self.params)
             self.report_warning(
                 '%s detected unsupported features; extraction will be delegated to %s' % (self.FD_NAME, fd.get_basename()))
diff --git a/yt_dlp/extractor/cam4.py b/yt_dlp/extractor/cam4.py
new file mode 100644
index 000000000..30daf2be9
--- /dev/null
+++ b/yt_dlp/extractor/cam4.py
@@ -0,0 +1,32 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class CAM4IE(InfoExtractor):
+    _VALID_URL = r'https?://(?:[^/]+\.)?cam4\.com/(?P<id>[a-z0-9_]+)'
+    _TEST = {
+        'url': 'https://www.cam4.com/foxynesss',
+        'info_dict': {
+            'id': 'foxynesss',
+            'ext': 'mp4',
+            'title': 're:^foxynesss [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}$',
+            'age_limit': 18,
+        }
+    }
+
+    def _real_extract(self, url):
+        channel_id = self._match_id(url)
+        m3u8_playlist = self._download_json('https://www.cam4.com/rest/v1.0/profile/{}/streamInfo'.format(channel_id), channel_id).get('cdnURL')
+
+        formats = self._extract_m3u8_formats(m3u8_playlist, channel_id, 'mp4', m3u8_id='hls', live=True)
+        self._sort_formats(formats)
+
+        return {
+            'id': channel_id,
+            'title': self._live_title(channel_id),
+            'is_live': True,
+            'age_limit': 18,
+            'formats': formats,
+        }
diff --git a/yt_dlp/extractor/cgtn.py b/yt_dlp/extractor/cgtn.py
new file mode 100644
index 000000000..89f173887
--- /dev/null
+++ b/yt_dlp/extractor/cgtn.py
@@ -0,0 +1,64 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import (
+    try_get,
+    unified_timestamp,
+)
+
+
+class CGTNIE(InfoExtractor):
+    _VALID_URL = r'https?://news\.cgtn\.com/news/[0-9]{4}-[0-9]{2}-[0-9]{2}/[a-zA-Z0-9-]+-(?P<id>[a-zA-Z0-9-]+)/index\.html'
+    _TESTS = [
+        {
+            'url': 'https://news.cgtn.com/news/2021-03-09/Up-and-Out-of-Poverty-Ep-1-A-solemn-promise-YuOUaOzGQU/index.html',
+            'info_dict': {
+                'id': 'YuOUaOzGQU',
+                'ext': 'mp4',
+                'title': 'Up and Out of Poverty Ep. 1: A solemn promise',
+                'thumbnail': r're:^https?://.*\.jpg$',
+                'timestamp': 1615295940,
+                'upload_date': '20210309',
+            },
+            'params': {
+                'skip_download': True
+            }
+        }, {
+            'url': 'https://news.cgtn.com/news/2021-06-06/China-Indonesia-vow-to-further-deepen-maritime-cooperation-10REvJCewCY/index.html',
+            'info_dict': {
+                'id': '10REvJCewCY',
+                'ext': 'mp4',
+                'title': 'China, Indonesia vow to further deepen maritime cooperation',
+                'thumbnail': r're:^https?://.*\.png$',
+                'description': 'China and Indonesia vowed to upgrade their cooperation into the maritime sector and also for political security, economy, and cultural and people-to-people exchanges.',
+                'author': 'CGTN',
+                'category': 'China',
+                'timestamp': 1622950200,
+                'upload_date': '20210606',
+            },
+            'params': {
+                'skip_download': False
+            }
+        }
+    ]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+
+        download_url = self._html_search_regex(r'data-video ="(?P<url>.+m3u8)"', webpage, 'download_url')
+        datetime_str = self._html_search_regex(r'<span class="date">\s*(.+?)\s*</span>', webpage, 'datetime_str', fatal=False)
+
+        return {
+            'id': video_id,
+            'title': self._og_search_title(webpage),
+            'description': self._og_search_description(webpage, default=None),
+            'thumbnail': self._og_search_thumbnail(webpage),
+            'formats': self._extract_m3u8_formats(download_url, video_id, 'mp4', 'm3u8_native', m3u8_id='hls'),
+            'category': self._html_search_regex(r'<span class="section">\s*(.+?)\s*</span>',
+                                                webpage, 'category', fatal=False),
+            'author': self._html_search_regex(r'<div class="news-author-name">\s*(.+?)\s*</div>',
+                                              webpage, 'author', default=None, fatal=False),
+            'timestamp': try_get(unified_timestamp(datetime_str), lambda x: x - 8 * 3600),
+        }
diff --git a/yt_dlp/extractor/common.py b/yt_dlp/extractor/common.py
index 54a9dc263..f6ca686a3 100644
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -18,6 +18,7 @@
     compat_cookies_SimpleCookie,
     compat_etree_Element,
     compat_etree_fromstring,
+    compat_expanduser,
     compat_getpass,
     compat_http_client,
     compat_os_name,
@@ -1166,7 +1167,10 @@ def _get_netrc_login_info(self, netrc_machine=None):
 
         if self.get_param('usenetrc', False):
             try:
-                info = netrc.netrc().authenticators(netrc_machine)
+                netrc_file = compat_expanduser(self.get_param('netrc_location') or '~')
+                if os.path.isdir(netrc_file):
+                    netrc_file = os.path.join(netrc_file, '.netrc')
+                info = netrc.netrc(file=netrc_file).authenticators(netrc_machine)
                 if info is not None:
                     username = info[0]
                     password = info[2]
@@ -2618,8 +2622,10 @@ def extract_Initialization(source):
                             base_url = base_url_e.text + base_url
                             if re.match(r'^https?://', base_url):
                                 break
-                    if mpd_base_url and not re.match(r'^https?://', base_url):
-                        if not mpd_base_url.endswith('/') and not base_url.startswith('/'):
+                    if mpd_base_url and base_url.startswith('/'):
+                        base_url = compat_urlparse.urljoin(mpd_base_url, base_url)
+                    elif mpd_base_url and not re.match(r'^https?://', base_url):
+                        if not mpd_base_url.endswith('/'):
                             mpd_base_url += '/'
                         base_url = mpd_base_url + base_url
                     representation_id = representation_attrib.get('id')
diff --git a/yt_dlp/extractor/damtomo.py b/yt_dlp/extractor/damtomo.py
new file mode 100644
index 000000000..456cd35a4
--- /dev/null
+++ b/yt_dlp/extractor/damtomo.py
@@ -0,0 +1,113 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .common import InfoExtractor
+from ..utils import ExtractorError, clean_html, int_or_none, try_get, unified_strdate
+from ..compat import compat_str
+
+
+class DamtomoBaseIE(InfoExtractor):
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage, handle = self._download_webpage_handle(self._WEBPAGE_URL_TMPL % video_id, video_id, encoding='sjis')
+
+        if handle.url == 'https://www.clubdam.com/sorry/':
+            raise ExtractorError('You are rate-limited. Try again later.', expected=True)
+        if '<h2>予期せぬエラーが発生しました。</h2>' in webpage:
+            raise ExtractorError('There is an error on server-side. Try again later.', expected=True)
+
+        description = self._search_regex(r'(?m)<div id="public_comment">\s*<p>\s*([^<]*?)\s*</p>', webpage, 'description', default=None)
+        uploader_id = self._search_regex(r'<a href="https://www\.clubdam\.com/app/damtomo/member/info/Profile\.do\?damtomoId=([^"]+)"', webpage, 'uploader_id', default=None)
+
+        data_dict = {
+            mobj.group('class'): re.sub(r'\s+', ' ', clean_html(mobj.group('value')))
+            for mobj in re.finditer(r'(?s)<(p|div)\s+class="(?P<class>[^" ]+?)">(?P<value>.+?)</\1>', webpage)}
+
+        # since videos do not have title, give the name of song instead
+        data_dict['user_name'] = re.sub(r'\s*さん\s*$', '', data_dict['user_name'])
+        title = data_dict.get('song_title')
+
+        stream_tree = self._download_xml(
+            self._DKML_XML_URL % video_id, video_id, note='Requesting stream information', encoding='sjis',
+            # doing this has no problem since there is no character outside ASCII,
+            # and never likely to happen in the future
+            transform_source=lambda x: re.sub(r'\s*encoding="[^"]+?"', '', x))
+        m3u8_url = try_get(stream_tree, lambda x: x.find(
+            './/d:streamingUrl', {'d': self._DKML_XML_NS}).text.strip(), compat_str)
+        if not m3u8_url:
+            raise ExtractorError('Failed to obtain m3u8 URL')
+        formats = self._extract_m3u8_formats(m3u8_url, video_id, ext='mp4')
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': title,
+            'uploader_id': uploader_id,
+            'description': description,
+            'uploader': data_dict.get('user_name'),
+            'upload_date': unified_strdate(self._search_regex(r'(\d{4}/\d{2}/\d{2})', data_dict.get('date'), 'upload_date', default=None)),
+            'view_count': int_or_none(self._search_regex(r'(\d+)', data_dict['audience'], 'view_count', default=None)),
+            'like_count': int_or_none(self._search_regex(r'(\d+)', data_dict['nice'], 'like_count', default=None)),
+            'track': title,
+            'artist': data_dict.get('song_artist'),
+            'formats': formats,
+        }
+
+
+class DamtomoVideoIE(DamtomoBaseIE):
+    IE_NAME = 'damtomo:video'
+    _VALID_URL = r'https?://(?:www\.)?clubdam\.com/app/damtomo/(?:SP/)?karaokeMovie/StreamingDkm\.do\?karaokeMovieId=(?P<id>\d+)'
+    _WEBPAGE_URL_TMPL = 'https://www.clubdam.com/app/damtomo/karaokeMovie/StreamingDkm.do?karaokeMovieId=%s'
+    _DKML_XML_URL = 'https://www.clubdam.com/app/damtomo/karaokeMovie/GetStreamingDkmUrlXML.do?movieSelectFlg=2&karaokeMovieId=%s'
+    _DKML_XML_NS = 'https://www.clubdam.com/app/damtomo/karaokeMovie/GetStreamingDkmUrlXML'
+    _TESTS = [{
+        'url': 'https://www.clubdam.com/app/damtomo/karaokeMovie/StreamingDkm.do?karaokeMovieId=2414316',
+        'info_dict': {
+            'id': '2414316',
+            'title': 'Get Wild',
+            'uploader': 'Ｋドロン',
+            'uploader_id': 'ODk5NTQwMzQ',
+            'track': 'Get Wild',
+            'artist': 'TM NETWORK(TMN)',
+            'upload_date': '20201226',
+        }
+    }]
+
+
+class DamtomoRecordIE(DamtomoBaseIE):
+    IE_NAME = 'damtomo:record'
+    _VALID_URL = r'https?://(?:www\.)?clubdam\.com/app/damtomo/(?:SP/)?karaokePost/StreamingKrk\.do\?karaokeContributeId=(?P<id>\d+)'
+    _WEBPAGE_URL_TMPL = 'https://www.clubdam.com/app/damtomo/karaokePost/StreamingKrk.do?karaokeContributeId=%s'
+    _DKML_XML_URL = 'https://www.clubdam.com/app/damtomo/karaokePost/GetStreamingKrkUrlXML.do?karaokeContributeId=%s'
+    _DKML_XML_NS = 'https://www.clubdam.com/app/damtomo/karaokePost/GetStreamingKrkUrlXML'
+    _TESTS = [{
+        'url': 'https://www.clubdam.com/app/damtomo/karaokePost/StreamingKrk.do?karaokeContributeId=27376862',
+        'info_dict': {
+            'id': '27376862',
+            'title': 'イカSUMMER [良音]',
+            'description': None,
+            'uploader': 'ＮＡＮＡ',
+            'uploader_id': 'MzAyMDExNTY',
+            'upload_date': '20210721',
+            'view_count': 4,
+            'like_count': 1,
+            'track': 'イカSUMMER [良音]',
+            'artist': 'ORANGE RANGE',
+        }
+    }, {
+        'url': 'https://www.clubdam.com/app/damtomo/karaokePost/StreamingKrk.do?karaokeContributeId=27489418',
+        'info_dict': {
+            'id': '27489418',
+            'title': '心みだれて〜say it with flowers〜(生音)',
+            'uploader_id': 'NjI1MjI2MjU',
+            'description': 'やっぱりキーを下げて正解だった感じ。リベンジ成功ということで。',
+            'uploader': '箱の「中の人」',
+            'upload_date': '20210815',
+            'view_count': 5,
+            'like_count': 3,
+            'track': '心みだれて〜say it with flowers〜(生音)',
+            'artist': '小林明子',
+        }
+    }]
diff --git a/yt_dlp/extractor/extractors.py b/yt_dlp/extractor/extractors.py
index bb1e21a07..ddae1d7cc 100644
--- a/yt_dlp/extractor/extractors.py
+++ b/yt_dlp/extractor/extractors.py
@@ -185,6 +185,7 @@
 from .buzzfeed import BuzzFeedIE
 from .byutv import BYUtvIE
 from .c56 import C56IE
+from .cam4 import CAM4IE
 from .camdemy import (
     CamdemyIE,
     CamdemyFolderIE
@@ -238,6 +239,7 @@
     CeskaTelevizeIE,
     CeskaTelevizePoradyIE,
 )
+from .cgtn import CGTNIE
 from .channel9 import Channel9IE
 from .charlierose import CharlieRoseIE
 from .chaturbate import ChaturbateIE
@@ -309,6 +311,10 @@
     DailymotionPlaylistIE,
     DailymotionUserIE,
 )
+from .damtomo import (
+    DamtomoRecordIE,
+    DamtomoVideoIE,
+)
 from .daum import (
     DaumIE,
     DaumClipIE,
@@ -729,6 +735,7 @@
 from .matchtv import MatchTVIE
 from .mdr import MDRIE
 from .medaltv import MedalTVIE
+from .mediaite import MediaiteIE
 from .mediaklikk import MediaKlikkIE
 from .mediaset import MediasetIE
 from .mediasite import (
@@ -1346,6 +1353,7 @@
     StoryFireSeriesIE,
 )
 from .streamable import StreamableIE
+from .streamanity import StreamanityIE
 from .streamcloud import StreamcloudIE
 from .streamcz import StreamCZIE
 from .streetvoice import StreetVoiceIE
diff --git a/yt_dlp/extractor/generic.py b/yt_dlp/extractor/generic.py
index d08f8f30d..b9c5772e0 100644
--- a/yt_dlp/extractor/generic.py
+++ b/yt_dlp/extractor/generic.py
@@ -1215,14 +1215,13 @@ class GenericIE(InfoExtractor):
         },
         {
             # JWPlatform iframe
-            'url': 'https://www.mediaite.com/tv/dem-senator-claims-gary-cohn-faked-a-bad-connection-during-trump-call-to-get-him-off-the-phone/',
-            'md5': 'ca00a040364b5b439230e7ebfd02c4e9',
+            'url': 'https://www.covermagazine.co.uk/feature/2465255/business-protection-involved',
             'info_dict': {
-                'id': 'O0c5JcKT',
+                'id': 'AG26UQXM',
                 'ext': 'mp4',
-                'upload_date': '20171122',
-                'timestamp': 1511366290,
-                'title': 'Dem Senator Claims Gary Cohn Faked a Bad Connection During Trump Call to Get Him Off the Phone',
+                'upload_date': '20160719',
+                'timestamp': 468923808,
+                'title': '2016_05_18 Cover L&G Business Protection V1 FINAL.mp4',
             },
             'add_ie': [JWPlatformIE.ie_key()],
         },
diff --git a/yt_dlp/extractor/globo.py b/yt_dlp/extractor/globo.py
index 0cb3aa31b..a3f024157 100644
--- a/yt_dlp/extractor/globo.py
+++ b/yt_dlp/extractor/globo.py
@@ -9,15 +9,14 @@
 
 from .common import InfoExtractor
 from ..compat import (
-    compat_HTTPError,
     compat_str,
 )
 from ..utils import (
     ExtractorError,
     float_or_none,
-    int_or_none,
     orderedSet,
     str_or_none,
+    try_get,
 )
 
 
@@ -26,18 +25,19 @@ class GloboIE(InfoExtractor):
     _NETRC_MACHINE = 'globo'
     _TESTS = [{
         'url': 'http://g1.globo.com/carros/autoesporte/videos/t/exclusivos-do-g1/v/mercedes-benz-gla-passa-por-teste-de-colisao-na-europa/3607726/',
-        'md5': 'b3ccc801f75cd04a914d51dadb83a78d',
         'info_dict': {
             'id': '3607726',
             'ext': 'mp4',
             'title': 'Mercedes-Benz GLA passa por teste de colisão na Europa',
             'duration': 103.204,
-            'uploader': 'Globo.com',
-            'uploader_id': '265',
+            'uploader': 'G1',
+            'uploader_id': '2015',
+        },
+        'params': {
+            'skip_download': True,
         },
     }, {
         'url': 'http://globoplay.globo.com/v/4581987/',
-        'md5': 'f36a1ecd6a50da1577eee6dd17f67eff',
         'info_dict': {
             'id': '4581987',
             'ext': 'mp4',
@@ -46,6 +46,9 @@ class GloboIE(InfoExtractor):
             'uploader': 'Rede Globo',
             'uploader_id': '196',
         },
+        'params': {
+            'skip_download': True,
+        },
     }, {
         'url': 'http://canalbrasil.globo.com/programas/sangue-latino/videos/3928201.html',
         'only_matching': True,
@@ -66,30 +69,6 @@ class GloboIE(InfoExtractor):
         'only_matching': True,
     }]
 
-    def _real_initialize(self):
-        email, password = self._get_login_info()
-        if email is None:
-            return
-
-        try:
-            glb_id = (self._download_json(
-                'https://login.globo.com/api/authentication', None, data=json.dumps({
-                    'payload': {
-                        'email': email,
-                        'password': password,
-                        'serviceId': 4654,
-                    },
-                }).encode(), headers={
-                    'Content-Type': 'application/json; charset=utf-8',
-                }) or {}).get('glbId')
-            if glb_id:
-                self._set_cookie('.globo.com', 'GLBID', glb_id)
-        except ExtractorError as e:
-            if isinstance(e.cause, compat_HTTPError) and e.cause.code == 401:
-                resp = self._parse_json(e.cause.read(), None)
-                raise ExtractorError(resp.get('userMessage') or resp['id'], expected=True)
-            raise
-
     def _real_extract(self, url):
         video_id = self._match_id(url)
 
@@ -102,73 +81,67 @@ def _real_extract(self, url):
         title = video['title']
 
         formats = []
+        security = self._download_json(
+            'https://playback.video.globo.com/v1/video-session', video_id, 'Downloading security hash for %s' % video_id,
+            headers={'content-type': 'application/json'}, data=json.dumps({
+                "player_type": "desktop",
+                "video_id": video_id,
+                "quality": "max",
+                "content_protection": "widevine",
+                "vsid": "581b986b-4c40-71f0-5a58-803e579d5fa2",
+                "tz": "-3.0:00"
+            }).encode())
+
+        security_hash = security['source']['token']
+        if not security_hash:
+            message = security.get('message')
+            if message:
+                raise ExtractorError(
+                    '%s returned error: %s' % (self.IE_NAME, message), expected=True)
+
+        hash_code = security_hash[:2]
+        padding = '%010d' % random.randint(1, 10000000000)
+        if hash_code in ('04', '14'):
+            received_time = security_hash[3:13]
+            received_md5 = security_hash[24:]
+            hash_prefix = security_hash[:23]
+        elif hash_code in ('02', '12', '03', '13'):
+            received_time = security_hash[2:12]
+            received_md5 = security_hash[22:]
+            padding += '1'
+            hash_prefix = '05' + security_hash[:22]
+
+        padded_sign_time = compat_str(int(received_time) + 86400) + padding
+        md5_data = (received_md5 + padded_sign_time + '0xAC10FD').encode()
+        signed_md5 = base64.urlsafe_b64encode(hashlib.md5(md5_data).digest()).decode().strip('=')
+        signed_hash = hash_prefix + padded_sign_time + signed_md5
+        source = security['source']['url_parts']
+        resource_url = source['scheme'] + '://' + source['domain'] + source['path']
+        signed_url = '%s?h=%s&k=html5&a=%s' % (resource_url, signed_hash, 'F' if video.get('subscriber_only') else 'A')
+
+        formats.extend(self._extract_m3u8_formats(
+            signed_url, video_id, 'mp4', entry_protocol='m3u8_native', m3u8_id='hls', fatal=False))
+        self._sort_formats(formats)
+
         subtitles = {}
         for resource in video['resources']:
-            resource_id = resource.get('_id')
-            resource_url = resource.get('url')
-            resource_type = resource.get('type')
-            if not resource_url or (resource_type == 'media' and not resource_id) or resource_type not in ('subtitle', 'media'):
-                continue
-
-            if resource_type == 'subtitle':
+            if resource.get('type') == 'subtitle':
                 subtitles.setdefault(resource.get('language') or 'por', []).append({
-                    'url': resource_url,
+                    'url': resource.get('url'),
                 })
-                continue
-
-            security = self._download_json(
-                'http://security.video.globo.com/videos/%s/hash' % video_id,
-                video_id, 'Downloading security hash for %s' % resource_id, query={
-                    'player': 'desktop',
-                    'version': '5.19.1',
-                    'resource_id': resource_id,
+        subs = try_get(security, lambda x: x['source']['subtitles'], expected_type=dict) or {}
+        for sub_lang, sub_url in subs.items():
+            if sub_url:
+                subtitles.setdefault(sub_lang or 'por', []).append({
+                    'url': sub_url,
                 })
-
-            security_hash = security.get('hash')
-            if not security_hash:
-                message = security.get('message')
-                if message:
-                    raise ExtractorError(
-                        '%s returned error: %s' % (self.IE_NAME, message), expected=True)
-                continue
-
-            hash_code = security_hash[:2]
-            padding = '%010d' % random.randint(1, 10000000000)
-            if hash_code in ('04', '14'):
-                received_time = security_hash[3:13]
-                received_md5 = security_hash[24:]
-                hash_prefix = security_hash[:23]
-            elif hash_code in ('02', '12', '03', '13'):
-                received_time = security_hash[2:12]
-                received_md5 = security_hash[22:]
-                padding += '1'
-                hash_prefix = '05' + security_hash[:22]
-
-            padded_sign_time = compat_str(int(received_time) + 86400) + padding
-            md5_data = (received_md5 + padded_sign_time + '0xAC10FD').encode()
-            signed_md5 = base64.urlsafe_b64encode(hashlib.md5(md5_data).digest()).decode().strip('=')
-            signed_hash = hash_prefix + padded_sign_time + signed_md5
-            signed_url = '%s?h=%s&k=html5&a=%s&u=%s' % (resource_url, signed_hash, 'F' if video.get('subscriber_only') else 'A', security.get('user') or '')
-
-            if resource_id.endswith('m3u8') or resource_url.endswith('.m3u8'):
-                formats.extend(self._extract_m3u8_formats(
-                    signed_url, resource_id, 'mp4', entry_protocol='m3u8_native',
-                    m3u8_id='hls', fatal=False))
-            elif resource_id.endswith('mpd') or resource_url.endswith('.mpd'):
-                formats.extend(self._extract_mpd_formats(
-                    signed_url, resource_id, mpd_id='dash', fatal=False))
-            elif resource_id.endswith('manifest') or resource_url.endswith('/manifest'):
-                formats.extend(self._extract_ism_formats(
-                    signed_url, resource_id, ism_id='mss', fatal=False))
-            else:
-                formats.append({
-                    'url': signed_url,
-                    'format_id': 'http-%s' % resource_id,
-                    'height': int_or_none(resource.get('height')),
+        subs = try_get(security, lambda x: x['source']['subtitles_webvtt'], expected_type=dict) or {}
+        for sub_lang, sub_url in subs.items():
+            if sub_url:
+                subtitles.setdefault(sub_lang or 'por', []).append({
+                    'url': sub_url,
                 })
 
-        self._sort_formats(formats)
-
         duration = float_or_none(video.get('duration'), 1000)
         uploader = video.get('channel')
         uploader_id = str_or_none(video.get('channel_id'))
diff --git a/yt_dlp/extractor/itv.py b/yt_dlp/extractor/itv.py
index 3418689d6..d69782b78 100644
--- a/yt_dlp/extractor/itv.py
+++ b/yt_dlp/extractor/itv.py
@@ -113,12 +113,15 @@ def _call_api(self, video_id, playlist_url, headers, platform_tag, featureset, f
 
     def _get_subtitles(self, video_id, variants, ios_playlist_url, headers, *args, **kwargs):
         subtitles = {}
+        # Prefer last matching featureset
+        # See: https://github.com/yt-dlp/yt-dlp/issues/986
         platform_tag_subs, featureset_subs = next(
             ((platform_tag, featureset)
-             for platform_tag, featuresets in variants.items() for featureset in featuresets
+             for platform_tag, featuresets in reversed(variants.items()) for featureset in featuresets
              if try_get(featureset, lambda x: x[2]) == 'outband-webvtt'),
             (None, None))
-        if platform_tag_subs or featureset_subs:
+
+        if platform_tag_subs and featureset_subs:
             subs_playlist = self._call_api(
                 video_id, ios_playlist_url, headers, platform_tag_subs, featureset_subs, fatal=False)
             subs = try_get(subs_playlist, lambda x: x['Playlist']['Video']['Subtitles'], list) or []
@@ -139,9 +142,11 @@ def _real_extract(self, url):
         variants = self._parse_json(
             try_get(params, lambda x: x['data-video-variants'], compat_str) or '{}',
             video_id, fatal=False)
+        # Prefer last matching featureset
+        # See: https://github.com/yt-dlp/yt-dlp/issues/986
         platform_tag_video, featureset_video = next(
             ((platform_tag, featureset)
-             for platform_tag, featuresets in variants.items() for featureset in featuresets
+             for platform_tag, featuresets in reversed(variants.items()) for featureset in featuresets
              if try_get(featureset, lambda x: x[:2]) == ['hls', 'aes']),
             (None, None))
         if not platform_tag_video or not featureset_video:
diff --git a/yt_dlp/extractor/ivi.py b/yt_dlp/extractor/ivi.py
index 5e1d89c9b..098ab6665 100644
--- a/yt_dlp/extractor/ivi.py
+++ b/yt_dlp/extractor/ivi.py
@@ -3,7 +3,6 @@
 
 import json
 import re
-import sys
 
 from .common import InfoExtractor
 from ..utils import (
@@ -94,20 +93,21 @@ def _real_extract(self, url):
             ]
         })
 
-        bundled = hasattr(sys, 'frozen')
-
         for site in (353, 183):
             content_data = (data % site).encode()
             if site == 353:
-                if bundled:
-                    continue
                 try:
                     from Cryptodome.Cipher import Blowfish
                     from Cryptodome.Hash import CMAC
-                    pycryptodomex_found = True
+                    pycryptodome_found = True
                 except ImportError:
-                    pycryptodomex_found = False
-                    continue
+                    try:
+                        from Crypto.Cipher import Blowfish
+                        from Crypto.Hash import CMAC
+                        pycryptodome_found = True
+                    except ImportError:
+                        pycryptodome_found = False
+                        continue
 
                 timestamp = (self._download_json(
                     self._LIGHT_URL, video_id,
@@ -140,14 +140,8 @@ def _real_extract(self, url):
                     extractor_msg = 'Video %s does not exist'
                 elif site == 353:
                     continue
-                elif bundled:
-                    raise ExtractorError(
-                        'This feature does not work from bundled exe. Run yt-dlp from sources.',
-                        expected=True)
-                elif not pycryptodomex_found:
-                    raise ExtractorError(
-                        'pycryptodomex not found. Please install',
-                        expected=True)
+                elif not pycryptodome_found:
+                    raise ExtractorError('pycryptodome not found. Please install', expected=True)
                 elif message:
                     extractor_msg += ': ' + message
                 raise ExtractorError(extractor_msg % video_id, expected=True)
diff --git a/yt_dlp/extractor/mediaite.py b/yt_dlp/extractor/mediaite.py
new file mode 100644
index 000000000..646c92223
--- /dev/null
+++ b/yt_dlp/extractor/mediaite.py
@@ -0,0 +1,80 @@
+from __future__ import unicode_literals
+
+
+from .common import InfoExtractor
+
+
+class MediaiteIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?mediaite.com/(?:tv|sports|politics|podcasts|opinion)/[\w-]+/'
+    _TESTS = [{
+        'url': 'https://www.mediaite.com/sports/bill-burr-roasts-nfl-for-promoting-black-lives-matter-while-scheduling-more-games-after-all-the-sht-they-know-about-cte/',
+        'info_dict': {
+            'id': 'vPHKITzy',
+            'ext': 'm4a',
+            'title': 'Bill Burr On NFL And Black Lives Matter',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/vPHKITzy/poster.jpg?width=720',
+            'duration': 55,
+            'timestamp': 1631630185,
+            'upload_date': '20210914',
+        },
+        'params': {'skip_download': True}
+    }, {
+        'url': 'https://www.mediaite.com/tv/joe-scarborough-goes-off-on-tax-breaks-for-super-wealthy-largest-income-redistribution-scam-in-american-history/',
+        'info_dict': {
+            'id': 'eeFcK4Xm',
+            'ext': 'mp4',
+            'title': 'Morning Joe-6_16_52 am - 6_21_10 am-2021-09-14.mp4',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/eeFcK4Xm/poster.jpg?width=720',
+            'duration': 258,
+            'timestamp': 1631618057,
+            'upload_date': '20210914',
+        },
+        'params': {'skip_download': True}
+    }, {
+        'url': 'https://www.mediaite.com/politics/watch-rudy-giuliani-impersonates-queen-elizabeth-calls-mark-milley-an-asshle-in-bizarre-9-11-speech/',
+        'info_dict': {
+            'id': 'EiyiXKcr',
+            'ext': 'mp4',
+            'title': 'Giuliani 1',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/EiyiXKcr/poster.jpg?width=720',
+            'duration': 39,
+            'timestamp': 1631536476,
+            'upload_date': '20210913',
+        },
+        'params': {'skip_download': True}
+    }, {
+        'url': 'https://www.mediaite.com/podcasts/clarissa-ward-says-she-decided-to-become-a-journalist-on-9-11/',
+        'info_dict': {
+            'id': 'TxavoRTx',
+            'ext': 'mp4',
+            'title': 'clarissa-ward-3.mp4',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/TxavoRTx/poster.jpg?width=720',
+            'duration': 83,
+            'timestamp': 1631311188,
+            'upload_date': '20210910',
+        },
+        'params': {'skip_download': True}
+    }, {
+        'url': 'https://www.mediaite.com/opinion/mainstream-media-ignores-rose-mcgowans-bombshell-allegation-that-newsoms-wife-tried-to-silence-her-on-weinstein/',
+        'info_dict': {
+            'id': 'sEIWvKR7',
+            'ext': 'mp4',
+            'title': 'KTTV_09-13-2021_05.34.21',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/sEIWvKR7/poster.jpg?width=720',
+            'duration': 52,
+            'timestamp': 1631553328,
+            'upload_date': '20210913',
+        },
+        'params': {'skip_download': True}
+    }]
+
+    def _real_extract(self, url):
+        webpage = self._download_webpage(url, None)
+        id = self._search_regex(r'data-video-id\s?=\s?\"([^\"]+)\"', webpage, 'id')
+        data_json = self._download_json(f'https://cdn.jwplayer.com/v2/media/{id}', id)
+        return self._parse_jwplayer_data(data_json)
diff --git a/yt_dlp/extractor/ndr.py b/yt_dlp/extractor/ndr.py
index 45aa106c8..f2bae2c1a 100644
--- a/yt_dlp/extractor/ndr.py
+++ b/yt_dlp/extractor/ndr.py
@@ -1,15 +1,14 @@
 # coding: utf-8
 from __future__ import unicode_literals
 
-
 from .common import InfoExtractor
 from ..utils import (
     determine_ext,
     int_or_none,
-    merge_dicts,
-    parse_iso8601,
+    parse_duration,
     qualities,
     try_get,
+    unified_strdate,
     urljoin,
 )
 
@@ -28,110 +27,110 @@ class NDRIE(NDRBaseIE):
     IE_DESC = 'NDR.de - Norddeutscher Rundfunk'
     _VALID_URL = r'https?://(?:www\.)?(?:daserste\.)?ndr\.de/(?:[^/]+/)*(?P<display_id>[^/?#]+),(?P<id>[\da-z]+)\.html'
     _TESTS = [{
-        # httpVideo, same content id
         'url': 'http://www.ndr.de/fernsehen/Party-Poette-und-Parade,hafengeburtstag988.html',
-        'md5': '6515bc255dc5c5f8c85bbc38e035a659',
         'info_dict': {
             'id': 'hafengeburtstag988',
-            'display_id': 'Party-Poette-und-Parade',
             'ext': 'mp4',
             'title': 'Party, Pötte und Parade',
+            'thumbnail': 'https://www.ndr.de/fernsehen/hafengeburtstag990_v-contentxl.jpg',
             'description': 'md5:ad14f9d2f91d3040b6930c697e5f6b4c',
-            'uploader': 'ndrtv',
-            'timestamp': 1431108900,
-            'upload_date': '20150510',
+            'series': None,
+            'channel': 'NDR Fernsehen',
+            'upload_date': '20150508',
             'duration': 3498,
         },
-        'params': {
-            'skip_download': True,
-        },
     }, {
-        # httpVideo, different content id
-        'url': 'http://www.ndr.de/sport/fussball/40-Osnabrueck-spielt-sich-in-einen-Rausch,osna270.html',
-        'md5': '1043ff203eab307f0c51702ec49e9a71',
+        'url': 'https://www.ndr.de/sport/fussball/Rostocks-Matchwinner-Froede-Ein-Hansa-Debuet-wie-im-Maerchen,hansa10312.html',
+        'only_matching': True
+    }, {
+        'url': 'https://www.ndr.de/nachrichten/niedersachsen/kommunalwahl_niedersachsen_2021/Grosse-Parteien-zufrieden-mit-Ergebnissen-der-Kommunalwahl,kommunalwahl1296.html',
         'info_dict': {
-            'id': 'osna272',
-            'display_id': '40-Osnabrueck-spielt-sich-in-einen-Rausch',
+            'id': 'kommunalwahl1296',
             'ext': 'mp4',
-            'title': 'Osnabrück - Wehen Wiesbaden: Die Highlights',
-            'description': 'md5:32e9b800b3d2d4008103752682d5dc01',
-            'uploader': 'ndrtv',
-            'timestamp': 1442059200,
-            'upload_date': '20150912',
-            'duration': 510,
-        },
-        'params': {
-            'skip_download': True,
+            'title': 'Die Spitzenrunde: Die Wahl aus Sicht der Landespolitik',
+            'thumbnail': 'https://www.ndr.de/fernsehen/screenshot1194912_v-contentxl.jpg',
+            'description': 'md5:5c6e2ad744cef499135735a1036d7aa7',
+            'series': 'Hallo Niedersachsen',
+            'channel': 'NDR Fernsehen',
+            'upload_date': '20210913',
+            'duration': 438,
         },
     }, {
-        # httpAudio, same content id
+        'url': 'https://www.ndr.de/fernsehen/sendungen/extra_3/extra-3-Satiremagazin-mit-Christian-Ehring,sendung1091858.html',
+        'info_dict': {
+            'id': 'sendung1091858',
+            'ext': 'mp4',
+            'title': 'Extra 3 vom 11.11.2020 mit Christian Ehring',
+            'thumbnail': 'https://www.ndr.de/fernsehen/screenshot983938_v-contentxl.jpg',
+            'description': 'md5:700f6de264010585012a72f97b0ac0c9',
+            'series': 'extra 3',
+            'channel': 'NDR Fernsehen',
+            'upload_date': '20201111',
+            'duration': 1749,
+        }
+    }, {
         'url': 'http://www.ndr.de/info/La-Valette-entgeht-der-Hinrichtung,audio51535.html',
-        'md5': 'bb3cd38e24fbcc866d13b50ca59307b8',
         'info_dict': {
             'id': 'audio51535',
-            'display_id': 'La-Valette-entgeht-der-Hinrichtung',
             'ext': 'mp3',
             'title': 'La Valette entgeht der Hinrichtung',
+            'thumbnail': 'https://www.ndr.de/mediathek/mediathekbild140_v-podcast.jpg',
             'description': 'md5:22f9541913a40fe50091d5cdd7c9f536',
-            'uploader': 'ndrinfo',
-            'timestamp': 1290626100,
             'upload_date': '20140729',
-            'duration': 884,
-        },
-        'params': {
-            'skip_download': True,
-        },
-    }, {
-        # with subtitles
-        'url': 'https://www.ndr.de/fernsehen/sendungen/extra_3/extra-3-Satiremagazin-mit-Christian-Ehring,sendung1091858.html',
-        'info_dict': {
-            'id': 'extra18674',
-            'display_id': 'extra-3-Satiremagazin-mit-Christian-Ehring',
-            'ext': 'mp4',
-            'title': 'Extra 3 vom 11.11.2020 mit Christian Ehring',
-            'description': 'md5:42ee53990a715eaaf4dc7f13a3bd56c6',
-            'uploader': 'ndrtv',
-            'upload_date': '20201113',
-            'duration': 1749,
-            'subtitles': {
-                'de': [{
-                    'ext': 'ttml',
-                    'url': r're:^https://www\.ndr\.de.+',
-                }],
-            },
+            'duration': 884.0,
         },
-        'params': {
-            'skip_download': True,
-        },
-        'expected_warnings': ['Unable to download f4m manifest'],
-    }, {
-        'url': 'https://www.ndr.de/Fettes-Brot-Ferris-MC-und-Thees-Uhlmann-live-on-stage,festivalsommer116.html',
-        'only_matching': True,
+        'expected_warnings': ['unable to extract json url'],
     }]
 
     def _extract_embed(self, webpage, display_id, id):
-        embed_url = self._html_search_meta(
-            'embedURL', webpage, 'embed URL',
-            default=None) or self._search_regex(
-            r'\bembedUrl["\']\s*:\s*(["\'])(?P<url>(?:(?!\1).)+)\1', webpage,
-            'embed URL', fatal=False, group='url')
-        if embed_url is None:
-            return self.url_result('ndr:%s' % id, ie=NDREmbedBaseIE.ie_key())
-        description = self._search_regex(
-            r'<p[^>]+itemprop="description">([^<]+)</p>',
-            webpage, 'description', default=None) or self._og_search_description(webpage)
-        timestamp = parse_iso8601(
-            self._search_regex(
-                r'<span[^>]+itemprop="(?:datePublished|uploadDate)"[^>]+content="([^"]+)"',
-                webpage, 'upload date', default=None))
-        info = self._search_json_ld(webpage, display_id, default={})
-        return merge_dicts({
-            '_type': 'url_transparent',
-            'url': embed_url,
-            'display_id': display_id,
-            'description': description,
-            'timestamp': timestamp,
-        }, info)
+        formats = []
+        base_url = 'https://www.ndr.de'
+        json_url = self._search_regex(r'<iframe[^>]+src=\"([^\"]+)_theme-ndrde[^\.]*\.html\"', webpage,
+                                      'json url', fatal=False)
+        if json_url:
+            data_json = self._download_json(base_url + json_url.replace('ardplayer_image', 'ardjson_image') + '.json',
+                                            id, fatal=False)
+            info_json = data_json.get('_info', {})
+            media_json = try_get(data_json, lambda x: x['_mediaArray'][0]['_mediaStreamArray'])
+            for media in media_json:
+                if media.get('_quality') == 'auto':
+                    formats.extend(self._extract_m3u8_formats(media['_stream'], id))
+            subtitles = {}
+            sub_url = data_json.get('_subtitleUrl')
+            if sub_url:
+                subtitles.setdefault('de', []).append({
+                    'url': base_url + sub_url,
+                })
+            self._sort_formats(formats)
+            return {
+                'id': id,
+                'title': info_json.get('clipTitle'),
+                'thumbnail': base_url + data_json.get('_previewImage'),
+                'description': info_json.get('clipDescription'),
+                'series': info_json.get('seriesTitle') or None,
+                'channel': info_json.get('channelTitle'),
+                'upload_date': unified_strdate(info_json.get('clipDate')),
+                'duration': data_json.get('_duration'),
+                'formats': formats,
+                'subtitles': subtitles,
+            }
+        else:
+            json_url = base_url + self._search_regex(r'apiUrl\s?=\s?\'([^\']+)\'', webpage, 'json url').replace(
+                '_belongsToPodcast-', '')
+            data_json = self._download_json(json_url, id, fatal=False)
+            return {
+                'id': id,
+                'title': data_json.get('title'),
+                'thumbnail': base_url + data_json.get('poster'),
+                'description': data_json.get('summary'),
+                'upload_date': unified_strdate(data_json.get('publicationDate')),
+                'duration': parse_duration(data_json.get('duration')),
+                'formats': [{
+                    'url': try_get(data_json, (lambda x: x['audio'][0]['url'], lambda x: x['files'][0]['url'])),
+                    'vcodec': 'none',
+                    'ext': 'mp3',
+                }],
+            }
 
 
 class NJoyIE(NDRBaseIE):
diff --git a/yt_dlp/extractor/safari.py b/yt_dlp/extractor/safari.py
index c92e8849b..cca4464ca 100644
--- a/yt_dlp/extractor/safari.py
+++ b/yt_dlp/extractor/safari.py
@@ -193,7 +193,12 @@ def _real_extract(self, url):
         part = self._download_json(
             url, '%s/%s' % (mobj.group('course_id'), mobj.group('part')),
             'Downloading part JSON')
-        return self.url_result(part['web_url'], SafariIE.ie_key())
+        web_url = part['web_url']
+        if 'library/view' in web_url:
+            web_url = web_url.replace('library/view', 'videos')
+            natural_keys = part['natural_key']
+            web_url = f'{web_url.rsplit("/", 1)[0]}/{natural_keys[0]}-{natural_keys[1][:-5]}'
+        return self.url_result(web_url, SafariIE.ie_key())
 
 
 class SafariCourseIE(SafariBaseIE):
diff --git a/yt_dlp/extractor/soundcloud.py b/yt_dlp/extractor/soundcloud.py
index 77e248a47..1503ae586 100644
--- a/yt_dlp/extractor/soundcloud.py
+++ b/yt_dlp/extractor/soundcloud.py
@@ -305,7 +305,7 @@ def _download_json(self, *args, **kwargs):
                 raise
 
     def _real_initialize(self):
-        self._CLIENT_ID = self._downloader.cache.load('soundcloud', 'client_id') or 'fSSdm5yTnDka1g0Fz1CO5Yx6z0NbeHAj'
+        self._CLIENT_ID = self._downloader.cache.load('soundcloud', 'client_id') or 'a3e059563d7fd3372b49b37f00a00bcf'
         self._login()
 
     _USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36'
diff --git a/yt_dlp/extractor/streamanity.py b/yt_dlp/extractor/streamanity.py
new file mode 100644
index 000000000..2e2d5eedf
--- /dev/null
+++ b/yt_dlp/extractor/streamanity.py
@@ -0,0 +1,51 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class StreamanityIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?streamanity\.com/video/(?P<id>[A-Za-z0-9]+)'
+    _TESTS = [{
+        'url': 'https://streamanity.com/video/9DFPTnuYi8f2',
+        'md5': '6ab171e8d4a02ad5dcbff6bea44cf5a1',
+        'info_dict': {
+            'id': '9DFPTnuYi8f2',
+            'ext': 'mp4',
+            'title': 'Bitcoin vs The Lighting Network',
+            'thumbnail': r're:https://res\.cloudinary\.com/.+\.png',
+            'description': '',
+            'uploader': 'Tom Bombadil (Freddy78)',
+        }
+    }, {
+        'url': 'https://streamanity.com/video/JktOUjSlfzTD',
+        'md5': '31f131e28abd3377c38be586a59532dc',
+        'info_dict': {
+            'id': 'JktOUjSlfzTD',
+            'ext': 'mp4',
+            'title': 'Share data when you see it',
+            'thumbnail': r're:https://res\.cloudinary\.com/.+\.png',
+            'description': 'Reposting as data should be public and stored on blockchain',
+            'uploader': 'digitalcurrencydaily',
+        }
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        video_info = self._download_json(
+            f'https://app.streamanity.com/api/video/{video_id}', video_id)['data']['video']
+
+        formats = self._extract_m3u8_formats(
+            f'https://stream.mux.com/{video_info["play_id"]}.m3u8?token={video_info["token"]}',
+            video_id, ext='mp4', m3u8_id='hls')
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': video_info['title'],
+            'description': video_info.get('description'),
+            'uploader': video_info.get('author_name'),
+            'is_live': False,
+            'thumbnail': video_info.get('thumb'),
+            'formats': formats,
+        }
diff --git a/yt_dlp/extractor/videa.py b/yt_dlp/extractor/videa.py
index da0212bb2..512ade7af 100644
--- a/yt_dlp/extractor/videa.py
+++ b/yt_dlp/extractor/videa.py
@@ -47,10 +47,24 @@ class VideaIE(InfoExtractor):
         },
     }, {
         'url': 'http://videa.hu/videok/origo/jarmuvek/supercars-elozes-jAHDWfWSJH5XuFhH',
-        'only_matching': True,
+        'md5': 'd57ccd8812c7fd491d33b1eab8c99975',
+        'info_dict': {
+            'id': 'jAHDWfWSJH5XuFhH',
+            'ext': 'mp4',
+            'title': 'Supercars előzés',
+            'thumbnail': r're:^https?://.*',
+            'duration': 64,
+        },
     }, {
         'url': 'http://videa.hu/player?v=8YfIAjxwWGwT8HVQ',
-        'only_matching': True,
+        'md5': '97a7af41faeaffd9f1fc864a7c7e7603',
+        'info_dict': {
+            'id': '8YfIAjxwWGwT8HVQ',
+            'ext': 'mp4',
+            'title': 'Az őrült kígyász 285 kígyót enged szabadon',
+            'thumbnail': r're:^https?://.*',
+            'duration': 21,
+        },
     }, {
         'url': 'http://videa.hu/player/v/8YfIAjxwWGwT8HVQ?autoplay=1',
         'only_matching': True,
@@ -100,10 +114,14 @@ def _real_extract(self, url):
 
         video_page = self._download_webpage(url, video_id)
 
-        player_url = self._search_regex(
-            r'<iframe.*?src="(/player\?[^"]+)"', video_page, 'player url')
-        player_url = urljoin(url, player_url)
-        player_page = self._download_webpage(player_url, video_id)
+        if 'videa.hu/player' in url:
+            player_url = url
+            player_page = video_page
+        else:
+            player_url = self._search_regex(
+                r'<iframe.*?src="(/player\?[^"]+)"', video_page, 'player url')
+            player_url = urljoin(url, player_url)
+            player_page = self._download_webpage(player_url, video_id)
 
         nonce = self._search_regex(
             r'_xt\s*=\s*"([^"]+)"', player_page, 'nonce')
@@ -134,7 +152,7 @@ def _real_extract(self, url):
         sources = xpath_element(
             info, './video_sources', 'sources', fatal=True)
         hash_values = xpath_element(
-            info, './hash_values', 'hash values', fatal=True)
+            info, './hash_values', 'hash values', fatal=False)
 
         title = xpath_text(video, './title', fatal=True)
 
@@ -143,15 +161,16 @@ def _real_extract(self, url):
             source_url = source.text
             source_name = source.get('name')
             source_exp = source.get('exp')
-            if not (source_url and source_name and source_exp):
+            if not (source_url and source_name):
                 continue
-            hash_value = xpath_text(hash_values, 'hash_value_' + source_name)
-            if not hash_value:
-                continue
-            source_url = update_url_query(source_url, {
-                'md5': hash_value,
-                'expires': source_exp,
-            })
+            hash_value = None
+            if hash_values:
+                hash_value = xpath_text(hash_values, 'hash_value_' + source_name)
+            if hash_value and source_exp:
+                source_url = update_url_query(source_url, {
+                    'md5': hash_value,
+                    'expires': source_exp,
+                })
             f = parse_codecs(source.get('codecs'))
             f.update({
                 'url': self._proto_relative_url(source_url),
diff --git a/yt_dlp/extractor/vrv.py b/yt_dlp/extractor/vrv.py
index 44f90acbd..419602148 100644
--- a/yt_dlp/extractor/vrv.py
+++ b/yt_dlp/extractor/vrv.py
@@ -218,7 +218,7 @@ def _real_extract(self, url):
                 })
 
         thumbnails = []
-        for thumbnail in traverse_obj(video_data, ('images', 'thumbnail', ..., ...)):
+        for thumbnail in traverse_obj(video_data, ('images', 'thumbnail', ..., ...)) or []:
             thumbnail_url = thumbnail.get('source')
             if not thumbnail_url:
                 continue
diff --git a/yt_dlp/options.py b/yt_dlp/options.py
index 7cabc35ae..74c810471 100644
--- a/yt_dlp/options.py
+++ b/yt_dlp/options.py
@@ -116,19 +116,19 @@ def _format_option_string(option):
 
         return ''.join(opts)
 
-    def _list_from_options_callback(option, opt_str, value, parser, append=True, delim=','):
+    def _list_from_options_callback(option, opt_str, value, parser, append=True, delim=',', process=str.strip):
         # append can be True, False or -1 (prepend)
         current = getattr(parser.values, option.dest) if append else []
-        value = [value] if delim is None else value.split(delim)
+        value = [process(value)] if delim is None else list(map(process, value.split(delim)))
         setattr(
             parser.values, option.dest,
             current + value if append is True else value + current)
 
     def _set_from_options_callback(
-            option, opt_str, value, parser,
-            delim=',', allowed_values=None, process=str.lower, aliases={}):
+            option, opt_str, value, parser, delim=',', allowed_values=None, aliases={},
+            process=lambda x: x.lower().strip()):
         current = getattr(parser.values, option.dest)
-        values = [process(value)] if delim is None else process(value).split(delim)[::-1]
+        values = [process(value)] if delim is None else list(map(process, value.split(delim)[::-1]))
         while values:
             actual_val = val = values.pop()
             if val == 'all':
@@ -235,7 +235,7 @@ def _dict_from_options_callback(
         help='Use this prefix for unqualified URLs. For example "gvsearch2:" downloads two videos from google videos for the search term "large apple". Use the value "auto" to let yt-dlp guess ("auto_warning" to emit a warning when guessing). "error" just throws an error. The default value "fixup_error" repairs broken URLs, but emits an error if this is not possible instead of searching')
     general.add_option(
         '--ignore-config', '--no-config',
-        action='store_true',
+        action='store_true', dest='ignoreconfig',
         help=(
             'Disable loading any configuration files except the one provided by --config-location. '
             'When given inside a configuration file, no further configuration files are loaded. '
@@ -275,8 +275,7 @@ def _dict_from_options_callback(
                 'multistreams', 'no-live-chat', 'playlist-index', 'list-formats', 'no-direct-merge',
                 'no-youtube-channel-redirect', 'no-youtube-unavailable-videos', 'no-attach-info-json',
                 'embed-thumbnail-atomicparsley', 'seperate-video-versions', 'no-clean-infojson', 'no-keep-subs',
-            },
-            'aliases': {
+            }, 'aliases': {
                 'youtube-dl': ['-multistreams', 'all'],
                 'youtube-dlc': ['-no-youtube-channel-redirect', '-no-live-chat', 'all'],
             }
@@ -479,6 +478,10 @@ def _dict_from_options_callback(
         '-n', '--netrc',
         action='store_true', dest='usenetrc', default=False,
         help='Use .netrc authentication data')
+    authentication.add_option(
+        '--netrc-location',
+        dest='netrc_location', metavar='PATH',
+        help='Location of .netrc authentication data; either the path or its containing directory. Defaults to ~/.netrc')
     authentication.add_option(
         '--video-password',
         dest='videopassword', metavar='PASSWORD',
@@ -1533,57 +1536,47 @@ def compat_conf(conf):
             'command-line': compat_conf(sys.argv[1:]),
             'custom': [], 'home': [], 'portable': [], 'user': [], 'system': []}
         paths = {'command-line': False}
-        opts, args = parser.parse_args(configs['command-line'])
+
+        def read_options(name, path, user=False):
+            ''' loads config files and returns ignoreconfig '''
+            # Multiple package names can be given here
+            # Eg: ('yt-dlp', 'youtube-dlc', 'youtube-dl') will look for
+            # the configuration file of any of these three packages
+            for package in ('yt-dlp',):
+                if user:
+                    config, current_path = _readUserConf(package, default=None)
+                else:
+                    current_path = os.path.join(path, '%s.conf' % package)
+                    config = _readOptions(current_path, default=None)
+                if config is not None:
+                    configs[name], paths[name] = config, current_path
+                    return parser.parse_args(config)[0].ignoreconfig
+            return False
 
         def get_configs():
-            if '--config-location' in configs['command-line']:
+            opts, _ = parser.parse_args(configs['command-line'])
+            if opts.config_location is not None:
                 location = compat_expanduser(opts.config_location)
                 if os.path.isdir(location):
                     location = os.path.join(location, 'yt-dlp.conf')
                 if not os.path.exists(location):
                     parser.error('config-location %s does not exist.' % location)
-                configs['custom'] = _readOptions(location, default=None)
-                if configs['custom'] is None:
-                    configs['custom'] = []
-                else:
-                    paths['custom'] = location
-            if '--ignore-config' in configs['command-line']:
+                config = _readOptions(location, default=None)
+                if config:
+                    configs['custom'], paths['config'] = config, location
+
+            if opts.ignoreconfig:
                 return
-            if '--ignore-config' in configs['custom']:
+            if parser.parse_args(configs['custom'])[0].ignoreconfig:
                 return
-
-            def read_options(path, user=False):
-                # Multiple package names can be given here
-                # Eg: ('yt-dlp', 'youtube-dlc', 'youtube-dl') will look for
-                # the configuration file of any of these three packages
-                for package in ('yt-dlp',):
-                    if user:
-                        config, current_path = _readUserConf(package, default=None)
-                    else:
-                        current_path = os.path.join(path, '%s.conf' % package)
-                        config = _readOptions(current_path, default=None)
-                    if config is not None:
-                        return config, current_path
-                return [], None
-
-            configs['portable'], paths['portable'] = read_options(get_executable_path())
-            if '--ignore-config' in configs['portable']:
+            if read_options('portable', get_executable_path()):
                 return
-
-            def get_home_path():
-                opts = parser.parse_args(configs['portable'] + configs['custom'] + configs['command-line'])[0]
-                return expand_path(opts.paths.get('home', '')).strip()
-
-            configs['home'], paths['home'] = read_options(get_home_path())
-            if '--ignore-config' in configs['home']:
+            opts, _ = parser.parse_args(configs['portable'] + configs['custom'] + configs['command-line'])
+            if read_options('home', expand_path(opts.paths.get('home', '')).strip()):
                 return
-
-            configs['system'], paths['system'] = read_options('/etc')
-            if '--ignore-config' in configs['system']:
+            if read_options('system', '/etc'):
                 return
-
-            configs['user'], paths['user'] = read_options('', True)
-            if '--ignore-config' in configs['user']:
+            if read_options('user', None, user=True):
                 configs['system'], paths['system'] = [], None
 
         get_configs()
@@ -1592,10 +1585,9 @@ def get_home_path():
         if opts.verbose:
             for label in ('System', 'User', 'Portable', 'Home', 'Custom', 'Command-line'):
                 key = label.lower()
-                if paths.get(key) is None:
-                    continue
-                if paths[key]:
-                    write_string('[debug] %s config file: %s\n' % (label, paths[key]))
-                write_string('[debug] %s config: %s\n' % (label, repr(_hide_login_info(configs[key]))))
+                if paths.get(key):
+                    write_string(f'[debug] {label} config file: {paths[key]}\n')
+                if paths.get(key) is not None:
+                    write_string(f'[debug] {label} config: {_hide_login_info(configs[key])!r}\n')
 
     return parser, opts, args
diff --git a/yt_dlp/postprocessor/modify_chapters.py b/yt_dlp/postprocessor/modify_chapters.py
index 9a7ba8eff..2871e16d5 100644
--- a/yt_dlp/postprocessor/modify_chapters.py
+++ b/yt_dlp/postprocessor/modify_chapters.py
@@ -15,7 +15,7 @@
 )
 
 
-_TINY_SPONSOR_OVERLAP_DURATION = 1
+_TINY_CHAPTER_DURATION = 1
 DEFAULT_SPONSORBLOCK_CHAPTER_TITLE = '[SponsorBlock]: %(category_names)l'
 
 
@@ -50,7 +50,6 @@ def run(self, info):
             if not info.get('__real_download'):
                 raise PostProcessingError('Cannot cut video since the real and expected durations mismatch. '
                                           'Different chapters may have already been removed')
-                return [], info
             else:
                 self.write_debug('Expected and actual durations mismatch')
 
@@ -145,38 +144,15 @@ def excess_duration(c):
 
         new_chapters = []
 
-        def chapter_length(c):
-            return c['end_time'] - c['start_time']
-
-        def original_uncut_chapter(c):
-            return '_was_cut' not in c and '_categories' not in c
-
         def append_chapter(c):
             assert 'remove' not in c
-            length = chapter_length(c) - excess_duration(c)
+            length = c['end_time'] - c['start_time'] - excess_duration(c)
             # Chapter is completely covered by cuts or sponsors.
             if length <= 0:
                 return
             start = new_chapters[-1]['end_time'] if new_chapters else 0
             c.update(start_time=start, end_time=start + length)
-            # Append without checking for tininess to prevent having
-            # a completely empty chapter list.
-            if not new_chapters:
-                new_chapters.append(c)
-                return
-            old_c = new_chapters[-1]
-            # Merge with the previous if the chapter is tiny.
-            # Only tiny chapters resulting from a cut can be skipped.
-            # Chapters that were already tiny in the original list will be preserved.
-            if not original_uncut_chapter(c) and length < _TINY_SPONSOR_OVERLAP_DURATION:
-                old_c['end_time'] = c['end_time']
-            # Previous tiny chapter was appended for the sake of preventing an empty chapter list.
-            # Replace it with the current one.
-            elif not original_uncut_chapter(old_c) and chapter_length(old_c) < _TINY_SPONSOR_OVERLAP_DURATION:
-                c['start_time'] = old_c['start_time']
-                new_chapters[-1] = c
-            else:
-                new_chapters.append(c)
+            new_chapters.append(c)
 
         # Turn into a priority queue, index is a tie breaker.
         # Plain stack sorted by start_time is not enough: after splitting the chapter,
@@ -275,10 +251,36 @@ def append_chapter(c):
                 append_chapter(cur_chapter)
                 cur_i, cur_chapter = i, c
         (append_chapter if 'remove' not in cur_chapter else append_cut)(cur_chapter)
+        return self._remove_tiny_rename_sponsors(new_chapters), cuts
+
+    def _remove_tiny_rename_sponsors(self, chapters):
+        new_chapters = []
+        for i, c in enumerate(chapters):
+            # Merge with the previous/next if the chapter is tiny.
+            # Only tiny chapters resulting from a cut can be skipped.
+            # Chapters that were already tiny in the original list will be preserved.
+            if (('_was_cut' in c or '_categories' in c)
+                    and c['end_time'] - c['start_time'] < _TINY_CHAPTER_DURATION):
+                if not new_chapters:
+                    # Prepend tiny chapter to the next one if possible.
+                    if i < len(chapters) - 1:
+                        chapters[i + 1]['start_time'] = c['start_time']
+                        continue
+                else:
+                    old_c = new_chapters[-1]
+                    if i < len(chapters) - 1:
+                        next_c = chapters[i + 1]
+                        # Not a typo: key names in old_c and next_c are really different.
+                        prev_is_sponsor = 'categories' in old_c
+                        next_is_sponsor = '_categories' in next_c
+                        # Preferentially prepend tiny normals to normals and sponsors to sponsors.
+                        if (('_categories' not in c and prev_is_sponsor and not next_is_sponsor)
+                                or ('_categories' in c and not prev_is_sponsor and next_is_sponsor)):
+                            next_c['start_time'] = c['start_time']
+                            continue
+                    old_c['end_time'] = c['end_time']
+                    continue
 
-        i = -1
-        for c in new_chapters.copy():
-            i += 1
             c.pop('_was_cut', None)
             cats = c.pop('_categories', None)
             if cats:
@@ -292,12 +294,13 @@ def append_chapter(c):
                 })
                 outtmpl, tmpl_dict = self._downloader.prepare_outtmpl(self._sponsorblock_chapter_title, c)
                 c['title'] = self._downloader.escape_outtmpl(outtmpl) % tmpl_dict
-            if i > 0 and c['title'] == new_chapters[i - 1]['title']:
-                new_chapters[i - 1]['end_time'] = c['end_time']
-                new_chapters.pop(i)
-                i -= 1
-
-        return new_chapters, cuts
+                # Merge identically named sponsors.
+                if (new_chapters and 'categories' in new_chapters[-1]
+                        and new_chapters[-1]['title'] == c['title']):
+                    new_chapters[-1]['end_time'] = c['end_time']
+                    continue
+            new_chapters.append(c)
+        return new_chapters
 
     def remove_chapters(self, filename, ranges_to_cut, concat_opts, force_keyframes=False):
         in_file = filename
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
index 983ca6ced..de0213b14 100644
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -1762,6 +1762,7 @@ def random_user_agent():
     '%b %d %Y at %H:%M:%S',
     '%B %d %Y at %H:%M',
     '%B %d %Y at %H:%M:%S',
+    '%H:%M %d-%b-%Y',
 )
 
 DATE_FORMATS_DAY_FIRST = list(DATE_FORMATS)
@@ -3034,8 +3035,16 @@ def redirect_request(self, req, fp, code, msg, headers, newurl):
 
 def extract_timezone(date_str):
     m = re.search(
-        r'^.{8,}?(?P<tz>Z$| ?(?P<sign>\+|-)(?P<hours>[0-9]{2}):?(?P<minutes>[0-9]{2})$)',
-        date_str)
+        r'''(?x)
+            ^.{8,}?                                              # >=8 char non-TZ prefix, if present
+            (?P<tz>Z|                                            # just the UTC Z, or
+                (?:(?<=.\b\d{4}|\b\d{2}:\d\d)|                   # preceded by 4 digits or hh:mm or
+                   (?<!.\b[a-zA-Z]{3}|[a-zA-Z]{4}|..\b\d\d))     # not preceded by 3 alpha word or >= 4 alpha or 2 digits
+                   [ ]?                                          # optional space
+                (?P<sign>\+|-)                                   # +/-
+                (?P<hours>[0-9]{2}):?(?P<minutes>[0-9]{2})       # hh[:]mm
+            $)
+        ''', date_str)
     if not m:
         timezone = datetime.timedelta()
     else:


Test patch:
diff --git a/test/test_YoutubeDL.py b/test/test_YoutubeDL.py
index e689978fd..210bf441c 100644
--- a/test/test_YoutubeDL.py
+++ b/test/test_YoutubeDL.py
@@ -649,6 +649,7 @@ def test_add_extra_info(self):
         'title2': '%PATH%',
         'title3': 'foo/bar\\test',
         'title4': 'foo "bar" test',
+        'title5': 'áéí',
         'timestamp': 1618488000,
         'duration': 100000,
         'playlist_index': 1,
@@ -767,6 +768,7 @@ def expect_same_infodict(out):
         test('%(ext)l', 'mp4')
         test('%(formats.:.id) 15l', '  id1, id2, id3')
         test('%(formats)j', (json.dumps(FORMATS), sanitize(json.dumps(FORMATS))))
+        test('%(title5).3B', 'á')
         if compat_os_name == 'nt':
             test('%(title4)q', ('"foo \\"bar\\" test"', "'foo _'bar_' test'"))
         else:
@@ -788,6 +790,12 @@ def expect_same_infodict(out):
         test('%(formats.0.id.-1+id)f', '1235.000000')
         test('%(formats.0.id.-1+formats.1.id.-1)d', '3')
 
+        # Alternates
+        test('%(title,id)s', '1234')
+        test('%(width-100,height+20|def)d', '1100')
+        test('%(width-100,height+width|def)s', 'def')
+        test('%(timestamp-x>%H\\,%M\\,%S,timestamp>%H\\,%M\\,%S)s', '12,00,00')
+
         # Laziness
         def gen():
             yield from range(5)
diff --git a/test/test_cookies.py b/test/test_cookies.py
index 6faaaa0c9..6053ebb4e 100644
--- a/test/test_cookies.py
+++ b/test/test_cookies.py
@@ -2,8 +2,8 @@
 from datetime import datetime, timezone
 
 from yt_dlp import cookies
+from yt_dlp.compat import compat_pycrypto_AES
 from yt_dlp.cookies import (
-    CRYPTO_AVAILABLE,
     LinuxChromeCookieDecryptor,
     MacChromeCookieDecryptor,
     WindowsChromeCookieDecryptor,
@@ -53,7 +53,7 @@ def test_chrome_cookie_decryptor_linux_v11(self):
             decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
             self.assertEqual(decryptor.decrypt(encrypted_value), value)
 
-    @unittest.skipIf(not CRYPTO_AVAILABLE, 'cryptography library not available')
+    @unittest.skipIf(not compat_pycrypto_AES, 'cryptography library not available')
     def test_chrome_cookie_decryptor_windows_v10(self):
         with MonkeyPatch(cookies, {
             '_get_windows_v10_key': lambda *args, **kwargs: b'Y\xef\xad\xad\xeerp\xf0Y\xe6\x9b\x12\xc2<z\x16]\n\xbb\xb8\xcb\xd7\x9bA\xc3\x14e\x99{\xd6\xf4&'
diff --git a/test/test_postprocessors.py b/test/test_postprocessors.py
index 7d1368769..090c7b47b 100644
--- a/test/test_postprocessors.py
+++ b/test/test_postprocessors.py
@@ -461,11 +461,23 @@ def test_remove_marked_arrange_sponsors_TinyChaptersResultingFromCutsAreIgnored(
         self._remove_marked_arrange_sponsors_test_impl(
             chapters, self._chapters([2, 2.5], ['c1', 'c3']), cuts)
 
+    def test_remove_marked_arrange_sponsors_SingleTinyChapterIsPreserved(self):
+        cuts = [self._chapter(0.5, 2, remove=True)]
+        chapters = self._chapters([2], ['c']) + cuts
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([0.5], ['c']), cuts)
+
+    def test_remove_marked_arrange_sponsors_TinyChapterAtTheStartPrependedToTheNext(self):
+        cuts = [self._chapter(0.5, 2, remove=True)]
+        chapters = self._chapters([2, 4], ['c1', 'c2']) + cuts
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([2.5], ['c2']), cuts)
+
     def test_remove_marked_arrange_sponsors_TinyChaptersResultingFromSponsorOverlapAreIgnored(self):
         chapters = self._chapters([1, 3, 4], ['c1', 'c2', 'c3']) + [
             self._sponsor_chapter(1.5, 2.5, 'sponsor')]
         self._remove_marked_arrange_sponsors_test_impl(
-            chapters, self._chapters([1.5, 3, 4], ['c1', '[SponsorBlock]: Sponsor', 'c3']), [])
+            chapters, self._chapters([1.5, 2.5, 4], ['c1', '[SponsorBlock]: Sponsor', 'c3']), [])
 
     def test_remove_marked_arrange_sponsors_TinySponsorsOverlapsAreIgnored(self):
         chapters = self._chapters([2, 3, 5], ['c1', 'c2', 'c3']) + [
@@ -476,6 +488,26 @@ def test_remove_marked_arrange_sponsors_TinySponsorsOverlapsAreIgnored(self):
             chapters, self._chapters([1, 3, 4, 5], [
                 'c1', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Unpaid/Self Promotion', 'c3']), [])
 
+    def test_remove_marked_arrange_sponsors_TinySponsorsPrependedToTheNextSponsor(self):
+        chapters = self._chapters([4], ['c']) + [
+            self._sponsor_chapter(1.5, 2, 'sponsor'),
+            self._sponsor_chapter(2, 4, 'selfpromo')
+        ]
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([1.5, 4], ['c', '[SponsorBlock]: Unpaid/Self Promotion']), [])
+
+    def test_remove_marked_arrange_sponsors_SmallestSponsorInTheOverlapGetsNamed(self):
+        self._pp._sponsorblock_chapter_title = '[SponsorBlock]: %(name)s'
+        chapters = self._chapters([10], ['c']) + [
+            self._sponsor_chapter(2, 8, 'sponsor'),
+            self._sponsor_chapter(4, 6, 'selfpromo')
+        ]
+        self._remove_marked_arrange_sponsors_test_impl(
+            chapters, self._chapters([2, 4, 6, 8, 10], [
+                'c', '[SponsorBlock]: Sponsor', '[SponsorBlock]: Unpaid/Self Promotion',
+                '[SponsorBlock]: Sponsor', 'c'
+            ]), [])
+
     def test_make_concat_opts_CommonCase(self):
         sponsor_chapters = [self._chapter(1, 2, 's1'), self._chapter(10, 20, 's2')]
         expected = '''ffconcat version 1.0