Repository: yt-dlp/yt-dlp
Pull request #992 (2021-09-16 14:28:31)

[damtomo] add support for Damtomo service
## Please follow the guide below

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes [ ] relevant to your *pull request* (like that [x])
- Use *Preview* tab to see how your *pull request* will actually look like

---

### Before submitting a *pull request* make sure you have:
- [x] At least skimmed through [adding new extractor tutorial](https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site) and [youtube-dl coding conventions](https://github.com/ytdl-org/youtube-dl#youtube-dl-coding-conventions) sections
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [ ] Bug fix
- [ ] Improvement
- [x] New extractor
- [ ] New feature

---

### Description of your *pull request* and other information

Damtomo is a service to expose users' singing scenes.

example video url: https://www.clubdam.com/app/damtomo/karaokeMovie/StreamingDkm.do?karaokeMovieId=2414316
example record url: https://www.clubdam.com/app/damtomo/karaokePost/StreamingKrk.do?karaokeContributeId=27376862



Patch:
diff --git a/README.md b/README.md
index a2c1cbd82..c4f996834 100644
--- a/README.md
+++ b/README.md
@@ -952,14 +952,14 @@ # OUTPUT TEMPLATE
 
 The simplest usage of `-o` is not to set any template arguments when downloading a single file, like in `yt-dlp -o funny_video.flv "https://some/video"` (hard-coding file extension like this is _not_ recommended and could break some post-processing).
 
-It may however also contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations.
+It may however also contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations.
 
 The field names themselves (the part inside the parenthesis) can also have some special formatting:
 1. **Object traversal**: The dictionaries and lists available in metadata can be traversed by using a `.` (dot) separator. You can also do python slicing using `:`. Eg: `%(tags.0)s`, `%(subtitles.en.-1.ext)s`, `%(id.3:7:-1)s`, `%(formats.:.format_id)s`. `%()s` refers to the entire infodict. Note that all the fields that become available using this method are not listed below. Use `-j` to see such fields
 1. **Addition**: Addition and subtraction of numeric fields can be done using `+` and `-` respectively. Eg: `%(playlist_index+10)03d`, `%(n_entries+1-playlist_index)d`
 1. **Date/time Formatting**: Date/time fields can be formatted according to [strftime formatting](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) by specifying it separated from the field name using a `>`. Eg: `%(duration>%H-%M-%S)s`, `%(upload_date>%Y-%m-%d)s`, `%(epoch-3600>%H-%M-%S)s`
 1. **Default**: A default value can be specified for when the field is empty using a `|` seperator. This overrides `--output-na-template`. Eg: `%(uploader|Unknown)s`
-1. **More Conversions**: In addition to the normal format types `diouxXeEfFgGcrs`, `j`, `l`, `q` can be used for converting to **j**son, a comma seperated **l**ist and a string **q**uoted for the terminal respectively
+1. **More Conversions**: In addition to the normal format types `diouxXeEfFgGcrs`, `B`, `j`, `l`, `q` can be used for converting to **B**ytes, **j**son, a comma seperated **l**ist and a string **q**uoted for the terminal respectively
 
 To summarize, the general syntax for a field is:
 ```
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
index 8432abf1a..c53c7ec38 100644
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -35,6 +35,7 @@
     compat_kwargs,
     compat_numeric_types,
     compat_os_name,
+    compat_pycrypto_AES,
     compat_shlex_quote,
     compat_str,
     compat_tokenize_tokenize,
@@ -907,7 +908,7 @@ def escape_outtmpl(outtmpl):
     def validate_outtmpl(cls, outtmpl):
         ''' @return None or Exception object '''
         outtmpl = re.sub(
-            STR_FORMAT_RE_TMPL.format('[^)]*', '[ljq]'),
+            STR_FORMAT_RE_TMPL.format('[^)]*', '[ljqB]'),
             lambda mobj: f'{mobj.group(0)[:-1]}s',
             cls._outtmpl_expandpath(outtmpl))
         try:
@@ -939,7 +940,7 @@ def prepare_outtmpl(self, outtmpl, info_dict, sanitize=None):
         }
 
         TMPL_DICT = {}
-        EXTERNAL_FORMAT_RE = re.compile(STR_FORMAT_RE_TMPL.format('[^)]*', f'[{STR_FORMAT_TYPES}ljq]'))
+        EXTERNAL_FORMAT_RE = re.compile(STR_FORMAT_RE_TMPL.format('[^)]*', f'[{STR_FORMAT_TYPES}ljqB]'))
         MATH_FUNCTIONS = {
             '+': float.__add__,
             '-': float.__sub__,
@@ -1031,6 +1032,9 @@ def create_key(outer_mobj):
                 value, fmt = json.dumps(value, default=_dumpjson_default), str_fmt
             elif fmt[-1] == 'q':
                 value, fmt = compat_shlex_quote(str(value)), str_fmt
+            elif fmt[-1] == 'B':
+                value = f'%{str_fmt}'.encode('utf-8') % str(value).encode('utf-8')
+                value, fmt = value.decode('utf-8', 'ignore'), 's'
             elif fmt[-1] == 'c':
                 value = str(value)
                 if value is None:
@@ -3292,13 +3296,12 @@ def python_implementation():
         ) or 'none'
         self._write_string('[debug] exe versions: %s\n' % exe_str)
 
-        from .downloader.fragment import can_decrypt_frag
         from .downloader.websocket import has_websockets
         from .postprocessor.embedthumbnail import has_mutagen
         from .cookies import SQLITE_AVAILABLE, KEYRING_AVAILABLE
 
         lib_str = ', '.join(sorted(filter(None, (
-            can_decrypt_frag and 'pycryptodome',
+            compat_pycrypto_AES and compat_pycrypto_AES.__name__.split('.')[0],
             has_websockets and 'websockets',
             has_mutagen and 'mutagen',
             SQLITE_AVAILABLE and 'sqlite',
diff --git a/yt_dlp/aes.py b/yt_dlp/aes.py
index 461bb6d41..57caae069 100644
--- a/yt_dlp/aes.py
+++ b/yt_dlp/aes.py
@@ -2,9 +2,21 @@
 
 from math import ceil
 
-from .compat import compat_b64decode
+from .compat import compat_b64decode, compat_pycrypto_AES
 from .utils import bytes_to_intlist, intlist_to_bytes
 
+
+if compat_pycrypto_AES:
+    def aes_cbc_decrypt_bytes(data, key, iv):
+        """ Decrypt bytes with AES-CBC using pycryptodome """
+        return compat_pycrypto_AES.new(key, compat_pycrypto_AES.MODE_CBC, iv).decrypt(data)
+
+else:
+    def aes_cbc_decrypt_bytes(data, key, iv):
+        """ Decrypt bytes with AES-CBC using native implementation since pycryptodome is unavailable """
+        return intlist_to_bytes(aes_cbc_decrypt(*map(bytes_to_intlist, (data, key, iv))))
+
+
 BLOCK_SIZE_BYTES = 16
 
 
diff --git a/yt_dlp/compat.py b/yt_dlp/compat.py
index 363c2d57a..7b55b7d9d 100644
--- a/yt_dlp/compat.py
+++ b/yt_dlp/compat.py
@@ -148,6 +148,15 @@ def compat_expanduser(path):
     compat_expanduser = os.path.expanduser
 
 
+try:
+    from Cryptodome.Cipher import AES as compat_pycrypto_AES
+except ImportError:
+    try:
+        from Crypto.Cipher import AES as compat_pycrypto_AES
+    except ImportError:
+        compat_pycrypto_AES = None
+
+
 #  Deprecated
 
 compat_basestring = str
@@ -241,6 +250,7 @@ def compat_expanduser(path):
     'compat_os_name',
     'compat_parse_qs',
     'compat_print',
+    'compat_pycrypto_AES',
     'compat_realpath',
     'compat_setenv',
     'compat_shlex_quote',
diff --git a/yt_dlp/cookies.py b/yt_dlp/cookies.py
index b5aff38dd..4f582f4e1 100644
--- a/yt_dlp/cookies.py
+++ b/yt_dlp/cookies.py
@@ -13,6 +13,7 @@
 from yt_dlp.compat import (
     compat_b64decode,
     compat_cookiejar_Cookie,
+    compat_pycrypto_AES
 )
 from yt_dlp.utils import (
     bug_reports_message,
@@ -32,12 +33,6 @@
     SQLITE_AVAILABLE = False
 
 
-try:
-    from Crypto.Cipher import AES
-    CRYPTO_AVAILABLE = True
-except ImportError:
-    CRYPTO_AVAILABLE = False
-
 try:
     import keyring
     KEYRING_AVAILABLE = True
@@ -400,7 +395,7 @@ def decrypt(self, encrypted_value):
             if self._v10_key is None:
                 self._logger.warning('cannot decrypt v10 cookies: no key found', only_once=True)
                 return None
-            elif not CRYPTO_AVAILABLE:
+            elif not compat_pycrypto_AES:
                 self._logger.warning('cannot decrypt cookie as the `pycryptodome` module is not installed. '
                                      'Please install by running `python3 -m pip install pycryptodome`',
                                      only_once=True)
@@ -660,7 +655,7 @@ def _decrypt_aes_cbc(ciphertext, key, logger, initialization_vector=b' ' * 16):
 
 
 def _decrypt_aes_gcm(ciphertext, key, nonce, authentication_tag, logger):
-    cipher = AES.new(key, AES.MODE_GCM, nonce)
+    cipher = compat_pycrypto_AES.new(key, compat_pycrypto_AES.MODE_GCM, nonce)
     try:
         plaintext = cipher.decrypt_and_verify(ciphertext, authentication_tag)
     except ValueError:
diff --git a/yt_dlp/downloader/external.py b/yt_dlp/downloader/external.py
index 9db248df4..1057382e0 100644
--- a/yt_dlp/downloader/external.py
+++ b/yt_dlp/downloader/external.py
@@ -6,13 +6,8 @@
 import sys
 import time
 
-try:
-    from Crypto.Cipher import AES
-    can_decrypt_frag = True
-except ImportError:
-    can_decrypt_frag = False
-
 from .common import FileDownloader
+from ..aes import aes_cbc_decrypt_bytes
 from ..compat import (
     compat_setenv,
     compat_str,
@@ -164,8 +159,7 @@ def _call_downloader(self, tmpfilename, info_dict):
                         decrypt_info['KEY'] = decrypt_info.get('KEY') or self.ydl.urlopen(
                             self._prepare_url(info_dict, info_dict.get('_decryption_key_url') or decrypt_info['URI'])).read()
                         encrypted_data = src.read()
-                        decrypted_data = AES.new(
-                            decrypt_info['KEY'], AES.MODE_CBC, iv).decrypt(encrypted_data)
+                        decrypted_data = aes_cbc_decrypt_bytes(encrypted_data, decrypt_info['KEY'], iv)
                         dest.write(decrypted_data)
                     else:
                         fragment_data = src.read()
diff --git a/yt_dlp/downloader/fragment.py b/yt_dlp/downloader/fragment.py
index e3af140fd..567bf69d3 100644
--- a/yt_dlp/downloader/fragment.py
+++ b/yt_dlp/downloader/fragment.py
@@ -4,12 +4,6 @@
 import time
 import json
 
-try:
-    from Crypto.Cipher import AES
-    can_decrypt_frag = True
-except ImportError:
-    can_decrypt_frag = False
-
 try:
     import concurrent.futures
     can_threaded_download = True
@@ -18,6 +12,7 @@
 
 from .common import FileDownloader
 from .http import HttpFD
+from ..aes import aes_cbc_decrypt_bytes
 from ..compat import (
     compat_urllib_error,
     compat_struct_pack,
@@ -386,7 +381,7 @@ def decrypt_fragment(fragment, frag_content):
             # not what it decrypts to.
             if self.params.get('test', False):
                 return frag_content
-            return AES.new(decrypt_info['KEY'], AES.MODE_CBC, iv).decrypt(frag_content)
+            return aes_cbc_decrypt_bytes(frag_content, decrypt_info['KEY'], iv)
 
         def append_fragment(frag_content, frag_index, ctx):
             if not frag_content:
diff --git a/yt_dlp/downloader/hls.py b/yt_dlp/downloader/hls.py
index 779658b70..bea286604 100644
--- a/yt_dlp/downloader/hls.py
+++ b/yt_dlp/downloader/hls.py
@@ -5,7 +5,7 @@
 import binascii
 
 from ..downloader import get_suitable_downloader
-from .fragment import FragmentFD, can_decrypt_frag
+from .fragment import FragmentFD
 from .external import FFmpegFD
 
 from ..compat import (
@@ -29,7 +29,7 @@ class HlsFD(FragmentFD):
     FD_NAME = 'hlsnative'
 
     @staticmethod
-    def can_download(manifest, info_dict, allow_unplayable_formats=False, with_crypto=can_decrypt_frag):
+    def can_download(manifest, info_dict, allow_unplayable_formats=False):
         UNSUPPORTED_FEATURES = [
             # r'#EXT-X-BYTERANGE',  # playlists composed of byte ranges of media files [2]
 
@@ -57,7 +57,6 @@ def can_download(manifest, info_dict, allow_unplayable_formats=False, with_crypt
         def check_results():
             yield not info_dict.get('is_live')
             is_aes128_enc = '#EXT-X-KEY:METHOD=AES-128' in manifest
-            yield with_crypto or not is_aes128_enc
             yield not (is_aes128_enc and r'#EXT-X-BYTERANGE' in manifest)
             for feature in UNSUPPORTED_FEATURES:
                 yield not re.search(feature, manifest)
@@ -75,8 +74,6 @@ def real_download(self, filename, info_dict):
             if info_dict.get('extra_param_to_segment_url') or info_dict.get('_decryption_key_url'):
                 self.report_error('pycryptodome not found. Please install')
                 return False
-            if self.can_download(s, info_dict, with_crypto=True):
-                self.report_warning('pycryptodome is needed to download this file natively')
             fd = FFmpegFD(self.ydl, self.params)
             self.report_warning(
                 '%s detected unsupported features; extraction will be delegated to %s' % (self.FD_NAME, fd.get_basename()))
diff --git a/yt_dlp/extractor/cam4.py b/yt_dlp/extractor/cam4.py
new file mode 100644
index 000000000..30daf2be9
--- /dev/null
+++ b/yt_dlp/extractor/cam4.py
@@ -0,0 +1,32 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class CAM4IE(InfoExtractor):
+    _VALID_URL = r'https?://(?:[^/]+\.)?cam4\.com/(?P<id>[a-z0-9_]+)'
+    _TEST = {
+        'url': 'https://www.cam4.com/foxynesss',
+        'info_dict': {
+            'id': 'foxynesss',
+            'ext': 'mp4',
+            'title': 're:^foxynesss [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}$',
+            'age_limit': 18,
+        }
+    }
+
+    def _real_extract(self, url):
+        channel_id = self._match_id(url)
+        m3u8_playlist = self._download_json('https://www.cam4.com/rest/v1.0/profile/{}/streamInfo'.format(channel_id), channel_id).get('cdnURL')
+
+        formats = self._extract_m3u8_formats(m3u8_playlist, channel_id, 'mp4', m3u8_id='hls', live=True)
+        self._sort_formats(formats)
+
+        return {
+            'id': channel_id,
+            'title': self._live_title(channel_id),
+            'is_live': True,
+            'age_limit': 18,
+            'formats': formats,
+        }
diff --git a/yt_dlp/extractor/damtomo.py b/yt_dlp/extractor/damtomo.py
new file mode 100644
index 000000000..456cd35a4
--- /dev/null
+++ b/yt_dlp/extractor/damtomo.py
@@ -0,0 +1,113 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+import re
+
+from .common import InfoExtractor
+from ..utils import ExtractorError, clean_html, int_or_none, try_get, unified_strdate
+from ..compat import compat_str
+
+
+class DamtomoBaseIE(InfoExtractor):
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage, handle = self._download_webpage_handle(self._WEBPAGE_URL_TMPL % video_id, video_id, encoding='sjis')
+
+        if handle.url == 'https://www.clubdam.com/sorry/':
+            raise ExtractorError('You are rate-limited. Try again later.', expected=True)
+        if '<h2>予期せぬエラーが発生しました。</h2>' in webpage:
+            raise ExtractorError('There is an error on server-side. Try again later.', expected=True)
+
+        description = self._search_regex(r'(?m)<div id="public_comment">\s*<p>\s*([^<]*?)\s*</p>', webpage, 'description', default=None)
+        uploader_id = self._search_regex(r'<a href="https://www\.clubdam\.com/app/damtomo/member/info/Profile\.do\?damtomoId=([^"]+)"', webpage, 'uploader_id', default=None)
+
+        data_dict = {
+            mobj.group('class'): re.sub(r'\s+', ' ', clean_html(mobj.group('value')))
+            for mobj in re.finditer(r'(?s)<(p|div)\s+class="(?P<class>[^" ]+?)">(?P<value>.+?)</\1>', webpage)}
+
+        # since videos do not have title, give the name of song instead
+        data_dict['user_name'] = re.sub(r'\s*さん\s*$', '', data_dict['user_name'])
+        title = data_dict.get('song_title')
+
+        stream_tree = self._download_xml(
+            self._DKML_XML_URL % video_id, video_id, note='Requesting stream information', encoding='sjis',
+            # doing this has no problem since there is no character outside ASCII,
+            # and never likely to happen in the future
+            transform_source=lambda x: re.sub(r'\s*encoding="[^"]+?"', '', x))
+        m3u8_url = try_get(stream_tree, lambda x: x.find(
+            './/d:streamingUrl', {'d': self._DKML_XML_NS}).text.strip(), compat_str)
+        if not m3u8_url:
+            raise ExtractorError('Failed to obtain m3u8 URL')
+        formats = self._extract_m3u8_formats(m3u8_url, video_id, ext='mp4')
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': title,
+            'uploader_id': uploader_id,
+            'description': description,
+            'uploader': data_dict.get('user_name'),
+            'upload_date': unified_strdate(self._search_regex(r'(\d{4}/\d{2}/\d{2})', data_dict.get('date'), 'upload_date', default=None)),
+            'view_count': int_or_none(self._search_regex(r'(\d+)', data_dict['audience'], 'view_count', default=None)),
+            'like_count': int_or_none(self._search_regex(r'(\d+)', data_dict['nice'], 'like_count', default=None)),
+            'track': title,
+            'artist': data_dict.get('song_artist'),
+            'formats': formats,
+        }
+
+
+class DamtomoVideoIE(DamtomoBaseIE):
+    IE_NAME = 'damtomo:video'
+    _VALID_URL = r'https?://(?:www\.)?clubdam\.com/app/damtomo/(?:SP/)?karaokeMovie/StreamingDkm\.do\?karaokeMovieId=(?P<id>\d+)'
+    _WEBPAGE_URL_TMPL = 'https://www.clubdam.com/app/damtomo/karaokeMovie/StreamingDkm.do?karaokeMovieId=%s'
+    _DKML_XML_URL = 'https://www.clubdam.com/app/damtomo/karaokeMovie/GetStreamingDkmUrlXML.do?movieSelectFlg=2&karaokeMovieId=%s'
+    _DKML_XML_NS = 'https://www.clubdam.com/app/damtomo/karaokeMovie/GetStreamingDkmUrlXML'
+    _TESTS = [{
+        'url': 'https://www.clubdam.com/app/damtomo/karaokeMovie/StreamingDkm.do?karaokeMovieId=2414316',
+        'info_dict': {
+            'id': '2414316',
+            'title': 'Get Wild',
+            'uploader': 'Ｋドロン',
+            'uploader_id': 'ODk5NTQwMzQ',
+            'track': 'Get Wild',
+            'artist': 'TM NETWORK(TMN)',
+            'upload_date': '20201226',
+        }
+    }]
+
+
+class DamtomoRecordIE(DamtomoBaseIE):
+    IE_NAME = 'damtomo:record'
+    _VALID_URL = r'https?://(?:www\.)?clubdam\.com/app/damtomo/(?:SP/)?karaokePost/StreamingKrk\.do\?karaokeContributeId=(?P<id>\d+)'
+    _WEBPAGE_URL_TMPL = 'https://www.clubdam.com/app/damtomo/karaokePost/StreamingKrk.do?karaokeContributeId=%s'
+    _DKML_XML_URL = 'https://www.clubdam.com/app/damtomo/karaokePost/GetStreamingKrkUrlXML.do?karaokeContributeId=%s'
+    _DKML_XML_NS = 'https://www.clubdam.com/app/damtomo/karaokePost/GetStreamingKrkUrlXML'
+    _TESTS = [{
+        'url': 'https://www.clubdam.com/app/damtomo/karaokePost/StreamingKrk.do?karaokeContributeId=27376862',
+        'info_dict': {
+            'id': '27376862',
+            'title': 'イカSUMMER [良音]',
+            'description': None,
+            'uploader': 'ＮＡＮＡ',
+            'uploader_id': 'MzAyMDExNTY',
+            'upload_date': '20210721',
+            'view_count': 4,
+            'like_count': 1,
+            'track': 'イカSUMMER [良音]',
+            'artist': 'ORANGE RANGE',
+        }
+    }, {
+        'url': 'https://www.clubdam.com/app/damtomo/karaokePost/StreamingKrk.do?karaokeContributeId=27489418',
+        'info_dict': {
+            'id': '27489418',
+            'title': '心みだれて〜say it with flowers〜(生音)',
+            'uploader_id': 'NjI1MjI2MjU',
+            'description': 'やっぱりキーを下げて正解だった感じ。リベンジ成功ということで。',
+            'uploader': '箱の「中の人」',
+            'upload_date': '20210815',
+            'view_count': 5,
+            'like_count': 3,
+            'track': '心みだれて〜say it with flowers〜(生音)',
+            'artist': '小林明子',
+        }
+    }]
diff --git a/yt_dlp/extractor/extractors.py b/yt_dlp/extractor/extractors.py
index bb1e21a07..3dd56c65a 100644
--- a/yt_dlp/extractor/extractors.py
+++ b/yt_dlp/extractor/extractors.py
@@ -185,6 +185,7 @@
 from .buzzfeed import BuzzFeedIE
 from .byutv import BYUtvIE
 from .c56 import C56IE
+from .cam4 import CAM4IE
 from .camdemy import (
     CamdemyIE,
     CamdemyFolderIE
@@ -309,6 +310,10 @@
     DailymotionPlaylistIE,
     DailymotionUserIE,
 )
+from .damtomo import (
+    DamtomoRecordIE,
+    DamtomoVideoIE,
+)
 from .daum import (
     DaumIE,
     DaumClipIE,
@@ -729,6 +734,7 @@
 from .matchtv import MatchTVIE
 from .mdr import MDRIE
 from .medaltv import MedalTVIE
+from .mediaite import MediaiteIE
 from .mediaklikk import MediaKlikkIE
 from .mediaset import MediasetIE
 from .mediasite import (
@@ -1346,6 +1352,7 @@
     StoryFireSeriesIE,
 )
 from .streamable import StreamableIE
+from .streamanity import StreamanityIE
 from .streamcloud import StreamcloudIE
 from .streamcz import StreamCZIE
 from .streetvoice import StreetVoiceIE
diff --git a/yt_dlp/extractor/generic.py b/yt_dlp/extractor/generic.py
index d08f8f30d..b9c5772e0 100644
--- a/yt_dlp/extractor/generic.py
+++ b/yt_dlp/extractor/generic.py
@@ -1215,14 +1215,13 @@ class GenericIE(InfoExtractor):
         },
         {
             # JWPlatform iframe
-            'url': 'https://www.mediaite.com/tv/dem-senator-claims-gary-cohn-faked-a-bad-connection-during-trump-call-to-get-him-off-the-phone/',
-            'md5': 'ca00a040364b5b439230e7ebfd02c4e9',
+            'url': 'https://www.covermagazine.co.uk/feature/2465255/business-protection-involved',
             'info_dict': {
-                'id': 'O0c5JcKT',
+                'id': 'AG26UQXM',
                 'ext': 'mp4',
-                'upload_date': '20171122',
-                'timestamp': 1511366290,
-                'title': 'Dem Senator Claims Gary Cohn Faked a Bad Connection During Trump Call to Get Him Off the Phone',
+                'upload_date': '20160719',
+                'timestamp': 468923808,
+                'title': '2016_05_18 Cover L&G Business Protection V1 FINAL.mp4',
             },
             'add_ie': [JWPlatformIE.ie_key()],
         },
diff --git a/yt_dlp/extractor/globo.py b/yt_dlp/extractor/globo.py
index 0cb3aa31b..a3f024157 100644
--- a/yt_dlp/extractor/globo.py
+++ b/yt_dlp/extractor/globo.py
@@ -9,15 +9,14 @@
 
 from .common import InfoExtractor
 from ..compat import (
-    compat_HTTPError,
     compat_str,
 )
 from ..utils import (
     ExtractorError,
     float_or_none,
-    int_or_none,
     orderedSet,
     str_or_none,
+    try_get,
 )
 
 
@@ -26,18 +25,19 @@ class GloboIE(InfoExtractor):
     _NETRC_MACHINE = 'globo'
     _TESTS = [{
         'url': 'http://g1.globo.com/carros/autoesporte/videos/t/exclusivos-do-g1/v/mercedes-benz-gla-passa-por-teste-de-colisao-na-europa/3607726/',
-        'md5': 'b3ccc801f75cd04a914d51dadb83a78d',
         'info_dict': {
             'id': '3607726',
             'ext': 'mp4',
             'title': 'Mercedes-Benz GLA passa por teste de colisão na Europa',
             'duration': 103.204,
-            'uploader': 'Globo.com',
-            'uploader_id': '265',
+            'uploader': 'G1',
+            'uploader_id': '2015',
+        },
+        'params': {
+            'skip_download': True,
         },
     }, {
         'url': 'http://globoplay.globo.com/v/4581987/',
-        'md5': 'f36a1ecd6a50da1577eee6dd17f67eff',
         'info_dict': {
             'id': '4581987',
             'ext': 'mp4',
@@ -46,6 +46,9 @@ class GloboIE(InfoExtractor):
             'uploader': 'Rede Globo',
             'uploader_id': '196',
         },
+        'params': {
+            'skip_download': True,
+        },
     }, {
         'url': 'http://canalbrasil.globo.com/programas/sangue-latino/videos/3928201.html',
         'only_matching': True,
@@ -66,30 +69,6 @@ class GloboIE(InfoExtractor):
         'only_matching': True,
     }]
 
-    def _real_initialize(self):
-        email, password = self._get_login_info()
-        if email is None:
-            return
-
-        try:
-            glb_id = (self._download_json(
-                'https://login.globo.com/api/authentication', None, data=json.dumps({
-                    'payload': {
-                        'email': email,
-                        'password': password,
-                        'serviceId': 4654,
-                    },
-                }).encode(), headers={
-                    'Content-Type': 'application/json; charset=utf-8',
-                }) or {}).get('glbId')
-            if glb_id:
-                self._set_cookie('.globo.com', 'GLBID', glb_id)
-        except ExtractorError as e:
-            if isinstance(e.cause, compat_HTTPError) and e.cause.code == 401:
-                resp = self._parse_json(e.cause.read(), None)
-                raise ExtractorError(resp.get('userMessage') or resp['id'], expected=True)
-            raise
-
     def _real_extract(self, url):
         video_id = self._match_id(url)
 
@@ -102,73 +81,67 @@ def _real_extract(self, url):
         title = video['title']
 
         formats = []
+        security = self._download_json(
+            'https://playback.video.globo.com/v1/video-session', video_id, 'Downloading security hash for %s' % video_id,
+            headers={'content-type': 'application/json'}, data=json.dumps({
+                "player_type": "desktop",
+                "video_id": video_id,
+                "quality": "max",
+                "content_protection": "widevine",
+                "vsid": "581b986b-4c40-71f0-5a58-803e579d5fa2",
+                "tz": "-3.0:00"
+            }).encode())
+
+        security_hash = security['source']['token']
+        if not security_hash:
+            message = security.get('message')
+            if message:
+                raise ExtractorError(
+                    '%s returned error: %s' % (self.IE_NAME, message), expected=True)
+
+        hash_code = security_hash[:2]
+        padding = '%010d' % random.randint(1, 10000000000)
+        if hash_code in ('04', '14'):
+            received_time = security_hash[3:13]
+            received_md5 = security_hash[24:]
+            hash_prefix = security_hash[:23]
+        elif hash_code in ('02', '12', '03', '13'):
+            received_time = security_hash[2:12]
+            received_md5 = security_hash[22:]
+            padding += '1'
+            hash_prefix = '05' + security_hash[:22]
+
+        padded_sign_time = compat_str(int(received_time) + 86400) + padding
+        md5_data = (received_md5 + padded_sign_time + '0xAC10FD').encode()
+        signed_md5 = base64.urlsafe_b64encode(hashlib.md5(md5_data).digest()).decode().strip('=')
+        signed_hash = hash_prefix + padded_sign_time + signed_md5
+        source = security['source']['url_parts']
+        resource_url = source['scheme'] + '://' + source['domain'] + source['path']
+        signed_url = '%s?h=%s&k=html5&a=%s' % (resource_url, signed_hash, 'F' if video.get('subscriber_only') else 'A')
+
+        formats.extend(self._extract_m3u8_formats(
+            signed_url, video_id, 'mp4', entry_protocol='m3u8_native', m3u8_id='hls', fatal=False))
+        self._sort_formats(formats)
+
         subtitles = {}
         for resource in video['resources']:
-            resource_id = resource.get('_id')
-            resource_url = resource.get('url')
-            resource_type = resource.get('type')
-            if not resource_url or (resource_type == 'media' and not resource_id) or resource_type not in ('subtitle', 'media'):
-                continue
-
-            if resource_type == 'subtitle':
+            if resource.get('type') == 'subtitle':
                 subtitles.setdefault(resource.get('language') or 'por', []).append({
-                    'url': resource_url,
+                    'url': resource.get('url'),
                 })
-                continue
-
-            security = self._download_json(
-                'http://security.video.globo.com/videos/%s/hash' % video_id,
-                video_id, 'Downloading security hash for %s' % resource_id, query={
-                    'player': 'desktop',
-                    'version': '5.19.1',
-                    'resource_id': resource_id,
+        subs = try_get(security, lambda x: x['source']['subtitles'], expected_type=dict) or {}
+        for sub_lang, sub_url in subs.items():
+            if sub_url:
+                subtitles.setdefault(sub_lang or 'por', []).append({
+                    'url': sub_url,
                 })
-
-            security_hash = security.get('hash')
-            if not security_hash:
-                message = security.get('message')
-                if message:
-                    raise ExtractorError(
-                        '%s returned error: %s' % (self.IE_NAME, message), expected=True)
-                continue
-
-            hash_code = security_hash[:2]
-            padding = '%010d' % random.randint(1, 10000000000)
-            if hash_code in ('04', '14'):
-                received_time = security_hash[3:13]
-                received_md5 = security_hash[24:]
-                hash_prefix = security_hash[:23]
-            elif hash_code in ('02', '12', '03', '13'):
-                received_time = security_hash[2:12]
-                received_md5 = security_hash[22:]
-                padding += '1'
-                hash_prefix = '05' + security_hash[:22]
-
-            padded_sign_time = compat_str(int(received_time) + 86400) + padding
-            md5_data = (received_md5 + padded_sign_time + '0xAC10FD').encode()
-            signed_md5 = base64.urlsafe_b64encode(hashlib.md5(md5_data).digest()).decode().strip('=')
-            signed_hash = hash_prefix + padded_sign_time + signed_md5
-            signed_url = '%s?h=%s&k=html5&a=%s&u=%s' % (resource_url, signed_hash, 'F' if video.get('subscriber_only') else 'A', security.get('user') or '')
-
-            if resource_id.endswith('m3u8') or resource_url.endswith('.m3u8'):
-                formats.extend(self._extract_m3u8_formats(
-                    signed_url, resource_id, 'mp4', entry_protocol='m3u8_native',
-                    m3u8_id='hls', fatal=False))
-            elif resource_id.endswith('mpd') or resource_url.endswith('.mpd'):
-                formats.extend(self._extract_mpd_formats(
-                    signed_url, resource_id, mpd_id='dash', fatal=False))
-            elif resource_id.endswith('manifest') or resource_url.endswith('/manifest'):
-                formats.extend(self._extract_ism_formats(
-                    signed_url, resource_id, ism_id='mss', fatal=False))
-            else:
-                formats.append({
-                    'url': signed_url,
-                    'format_id': 'http-%s' % resource_id,
-                    'height': int_or_none(resource.get('height')),
+        subs = try_get(security, lambda x: x['source']['subtitles_webvtt'], expected_type=dict) or {}
+        for sub_lang, sub_url in subs.items():
+            if sub_url:
+                subtitles.setdefault(sub_lang or 'por', []).append({
+                    'url': sub_url,
                 })
 
-        self._sort_formats(formats)
-
         duration = float_or_none(video.get('duration'), 1000)
         uploader = video.get('channel')
         uploader_id = str_or_none(video.get('channel_id'))
diff --git a/yt_dlp/extractor/itv.py b/yt_dlp/extractor/itv.py
index 3418689d6..d69782b78 100644
--- a/yt_dlp/extractor/itv.py
+++ b/yt_dlp/extractor/itv.py
@@ -113,12 +113,15 @@ def _call_api(self, video_id, playlist_url, headers, platform_tag, featureset, f
 
     def _get_subtitles(self, video_id, variants, ios_playlist_url, headers, *args, **kwargs):
         subtitles = {}
+        # Prefer last matching featureset
+        # See: https://github.com/yt-dlp/yt-dlp/issues/986
         platform_tag_subs, featureset_subs = next(
             ((platform_tag, featureset)
-             for platform_tag, featuresets in variants.items() for featureset in featuresets
+             for platform_tag, featuresets in reversed(variants.items()) for featureset in featuresets
              if try_get(featureset, lambda x: x[2]) == 'outband-webvtt'),
             (None, None))
-        if platform_tag_subs or featureset_subs:
+
+        if platform_tag_subs and featureset_subs:
             subs_playlist = self._call_api(
                 video_id, ios_playlist_url, headers, platform_tag_subs, featureset_subs, fatal=False)
             subs = try_get(subs_playlist, lambda x: x['Playlist']['Video']['Subtitles'], list) or []
@@ -139,9 +142,11 @@ def _real_extract(self, url):
         variants = self._parse_json(
             try_get(params, lambda x: x['data-video-variants'], compat_str) or '{}',
             video_id, fatal=False)
+        # Prefer last matching featureset
+        # See: https://github.com/yt-dlp/yt-dlp/issues/986
         platform_tag_video, featureset_video = next(
             ((platform_tag, featureset)
-             for platform_tag, featuresets in variants.items() for featureset in featuresets
+             for platform_tag, featuresets in reversed(variants.items()) for featureset in featuresets
              if try_get(featureset, lambda x: x[:2]) == ['hls', 'aes']),
             (None, None))
         if not platform_tag_video or not featureset_video:
diff --git a/yt_dlp/extractor/ivi.py b/yt_dlp/extractor/ivi.py
index 5e1d89c9b..098ab6665 100644
--- a/yt_dlp/extractor/ivi.py
+++ b/yt_dlp/extractor/ivi.py
@@ -3,7 +3,6 @@
 
 import json
 import re
-import sys
 
 from .common import InfoExtractor
 from ..utils import (
@@ -94,20 +93,21 @@ def _real_extract(self, url):
             ]
         })
 
-        bundled = hasattr(sys, 'frozen')
-
         for site in (353, 183):
             content_data = (data % site).encode()
             if site == 353:
-                if bundled:
-                    continue
                 try:
                     from Cryptodome.Cipher import Blowfish
                     from Cryptodome.Hash import CMAC
-                    pycryptodomex_found = True
+                    pycryptodome_found = True
                 except ImportError:
-                    pycryptodomex_found = False
-                    continue
+                    try:
+                        from Crypto.Cipher import Blowfish
+                        from Crypto.Hash import CMAC
+                        pycryptodome_found = True
+                    except ImportError:
+                        pycryptodome_found = False
+                        continue
 
                 timestamp = (self._download_json(
                     self._LIGHT_URL, video_id,
@@ -140,14 +140,8 @@ def _real_extract(self, url):
                     extractor_msg = 'Video %s does not exist'
                 elif site == 353:
                     continue
-                elif bundled:
-                    raise ExtractorError(
-                        'This feature does not work from bundled exe. Run yt-dlp from sources.',
-                        expected=True)
-                elif not pycryptodomex_found:
-                    raise ExtractorError(
-                        'pycryptodomex not found. Please install',
-                        expected=True)
+                elif not pycryptodome_found:
+                    raise ExtractorError('pycryptodome not found. Please install', expected=True)
                 elif message:
                     extractor_msg += ': ' + message
                 raise ExtractorError(extractor_msg % video_id, expected=True)
diff --git a/yt_dlp/extractor/mediaite.py b/yt_dlp/extractor/mediaite.py
new file mode 100644
index 000000000..646c92223
--- /dev/null
+++ b/yt_dlp/extractor/mediaite.py
@@ -0,0 +1,80 @@
+from __future__ import unicode_literals
+
+
+from .common import InfoExtractor
+
+
+class MediaiteIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?mediaite.com/(?:tv|sports|politics|podcasts|opinion)/[\w-]+/'
+    _TESTS = [{
+        'url': 'https://www.mediaite.com/sports/bill-burr-roasts-nfl-for-promoting-black-lives-matter-while-scheduling-more-games-after-all-the-sht-they-know-about-cte/',
+        'info_dict': {
+            'id': 'vPHKITzy',
+            'ext': 'm4a',
+            'title': 'Bill Burr On NFL And Black Lives Matter',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/vPHKITzy/poster.jpg?width=720',
+            'duration': 55,
+            'timestamp': 1631630185,
+            'upload_date': '20210914',
+        },
+        'params': {'skip_download': True}
+    }, {
+        'url': 'https://www.mediaite.com/tv/joe-scarborough-goes-off-on-tax-breaks-for-super-wealthy-largest-income-redistribution-scam-in-american-history/',
+        'info_dict': {
+            'id': 'eeFcK4Xm',
+            'ext': 'mp4',
+            'title': 'Morning Joe-6_16_52 am - 6_21_10 am-2021-09-14.mp4',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/eeFcK4Xm/poster.jpg?width=720',
+            'duration': 258,
+            'timestamp': 1631618057,
+            'upload_date': '20210914',
+        },
+        'params': {'skip_download': True}
+    }, {
+        'url': 'https://www.mediaite.com/politics/watch-rudy-giuliani-impersonates-queen-elizabeth-calls-mark-milley-an-asshle-in-bizarre-9-11-speech/',
+        'info_dict': {
+            'id': 'EiyiXKcr',
+            'ext': 'mp4',
+            'title': 'Giuliani 1',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/EiyiXKcr/poster.jpg?width=720',
+            'duration': 39,
+            'timestamp': 1631536476,
+            'upload_date': '20210913',
+        },
+        'params': {'skip_download': True}
+    }, {
+        'url': 'https://www.mediaite.com/podcasts/clarissa-ward-says-she-decided-to-become-a-journalist-on-9-11/',
+        'info_dict': {
+            'id': 'TxavoRTx',
+            'ext': 'mp4',
+            'title': 'clarissa-ward-3.mp4',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/TxavoRTx/poster.jpg?width=720',
+            'duration': 83,
+            'timestamp': 1631311188,
+            'upload_date': '20210910',
+        },
+        'params': {'skip_download': True}
+    }, {
+        'url': 'https://www.mediaite.com/opinion/mainstream-media-ignores-rose-mcgowans-bombshell-allegation-that-newsoms-wife-tried-to-silence-her-on-weinstein/',
+        'info_dict': {
+            'id': 'sEIWvKR7',
+            'ext': 'mp4',
+            'title': 'KTTV_09-13-2021_05.34.21',
+            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
+            'thumbnail': 'https://cdn.jwplayer.com/v2/media/sEIWvKR7/poster.jpg?width=720',
+            'duration': 52,
+            'timestamp': 1631553328,
+            'upload_date': '20210913',
+        },
+        'params': {'skip_download': True}
+    }]
+
+    def _real_extract(self, url):
+        webpage = self._download_webpage(url, None)
+        id = self._search_regex(r'data-video-id\s?=\s?\"([^\"]+)\"', webpage, 'id')
+        data_json = self._download_json(f'https://cdn.jwplayer.com/v2/media/{id}', id)
+        return self._parse_jwplayer_data(data_json)
diff --git a/yt_dlp/extractor/ndr.py b/yt_dlp/extractor/ndr.py
index 45aa106c8..f2bae2c1a 100644
--- a/yt_dlp/extractor/ndr.py
+++ b/yt_dlp/extractor/ndr.py
@@ -1,15 +1,14 @@
 # coding: utf-8
 from __future__ import unicode_literals
 
-
 from .common import InfoExtractor
 from ..utils import (
     determine_ext,
     int_or_none,
-    merge_dicts,
-    parse_iso8601,
+    parse_duration,
     qualities,
     try_get,
+    unified_strdate,
     urljoin,
 )
 
@@ -28,110 +27,110 @@ class NDRIE(NDRBaseIE):
     IE_DESC = 'NDR.de - Norddeutscher Rundfunk'
     _VALID_URL = r'https?://(?:www\.)?(?:daserste\.)?ndr\.de/(?:[^/]+/)*(?P<display_id>[^/?#]+),(?P<id>[\da-z]+)\.html'
     _TESTS = [{
-        # httpVideo, same content id
         'url': 'http://www.ndr.de/fernsehen/Party-Poette-und-Parade,hafengeburtstag988.html',
-        'md5': '6515bc255dc5c5f8c85bbc38e035a659',
         'info_dict': {
             'id': 'hafengeburtstag988',
-            'display_id': 'Party-Poette-und-Parade',
             'ext': 'mp4',
             'title': 'Party, Pötte und Parade',
+            'thumbnail': 'https://www.ndr.de/fernsehen/hafengeburtstag990_v-contentxl.jpg',
             'description': 'md5:ad14f9d2f91d3040b6930c697e5f6b4c',
-            'uploader': 'ndrtv',
-            'timestamp': 1431108900,
-            'upload_date': '20150510',
+            'series': None,
+            'channel': 'NDR Fernsehen',
+            'upload_date': '20150508',
             'duration': 3498,
         },
-        'params': {
-            'skip_download': True,
-        },
     }, {
-        # httpVideo, different content id
-        'url': 'http://www.ndr.de/sport/fussball/40-Osnabrueck-spielt-sich-in-einen-Rausch,osna270.html',
-        'md5': '1043ff203eab307f0c51702ec49e9a71',
+        'url': 'https://www.ndr.de/sport/fussball/Rostocks-Matchwinner-Froede-Ein-Hansa-Debuet-wie-im-Maerchen,hansa10312.html',
+        'only_matching': True
+    }, {
+        'url': 'https://www.ndr.de/nachrichten/niedersachsen/kommunalwahl_niedersachsen_2021/Grosse-Parteien-zufrieden-mit-Ergebnissen-der-Kommunalwahl,kommunalwahl1296.html',
         'info_dict': {
-            'id': 'osna272',
-            'display_id': '40-Osnabrueck-spielt-sich-in-einen-Rausch',
+            'id': 'kommunalwahl1296',
             'ext': 'mp4',
-            'title': 'Osnabrück - Wehen Wiesbaden: Die Highlights',
-            'description': 'md5:32e9b800b3d2d4008103752682d5dc01',
-            'uploader': 'ndrtv',
-            'timestamp': 1442059200,
-            'upload_date': '20150912',
-            'duration': 510,
-        },
-        'params': {
-            'skip_download': True,
+            'title': 'Die Spitzenrunde: Die Wahl aus Sicht der Landespolitik',
+            'thumbnail': 'https://www.ndr.de/fernsehen/screenshot1194912_v-contentxl.jpg',
+            'description': 'md5:5c6e2ad744cef499135735a1036d7aa7',
+            'series': 'Hallo Niedersachsen',
+            'channel': 'NDR Fernsehen',
+            'upload_date': '20210913',
+            'duration': 438,
         },
     }, {
-        # httpAudio, same content id
+        'url': 'https://www.ndr.de/fernsehen/sendungen/extra_3/extra-3-Satiremagazin-mit-Christian-Ehring,sendung1091858.html',
+        'info_dict': {
+            'id': 'sendung1091858',
+            'ext': 'mp4',
+            'title': 'Extra 3 vom 11.11.2020 mit Christian Ehring',
+            'thumbnail': 'https://www.ndr.de/fernsehen/screenshot983938_v-contentxl.jpg',
+            'description': 'md5:700f6de264010585012a72f97b0ac0c9',
+            'series': 'extra 3',
+            'channel': 'NDR Fernsehen',
+            'upload_date': '20201111',
+            'duration': 1749,
+        }
+    }, {
         'url': 'http://www.ndr.de/info/La-Valette-entgeht-der-Hinrichtung,audio51535.html',
-        'md5': 'bb3cd38e24fbcc866d13b50ca59307b8',
         'info_dict': {
             'id': 'audio51535',
-            'display_id': 'La-Valette-entgeht-der-Hinrichtung',
             'ext': 'mp3',
             'title': 'La Valette entgeht der Hinrichtung',
+            'thumbnail': 'https://www.ndr.de/mediathek/mediathekbild140_v-podcast.jpg',
             'description': 'md5:22f9541913a40fe50091d5cdd7c9f536',
-            'uploader': 'ndrinfo',
-            'timestamp': 1290626100,
             'upload_date': '20140729',
-            'duration': 884,
-        },
-        'params': {
-            'skip_download': True,
-        },
-    }, {
-        # with subtitles
-        'url': 'https://www.ndr.de/fernsehen/sendungen/extra_3/extra-3-Satiremagazin-mit-Christian-Ehring,sendung1091858.html',
-        'info_dict': {
-            'id': 'extra18674',
-            'display_id': 'extra-3-Satiremagazin-mit-Christian-Ehring',
-            'ext': 'mp4',
-            'title': 'Extra 3 vom 11.11.2020 mit Christian Ehring',
-            'description': 'md5:42ee53990a715eaaf4dc7f13a3bd56c6',
-            'uploader': 'ndrtv',
-            'upload_date': '20201113',
-            'duration': 1749,
-            'subtitles': {
-                'de': [{
-                    'ext': 'ttml',
-                    'url': r're:^https://www\.ndr\.de.+',
-                }],
-            },
+            'duration': 884.0,
         },
-        'params': {
-            'skip_download': True,
-        },
-        'expected_warnings': ['Unable to download f4m manifest'],
-    }, {
-        'url': 'https://www.ndr.de/Fettes-Brot-Ferris-MC-und-Thees-Uhlmann-live-on-stage,festivalsommer116.html',
-        'only_matching': True,
+        'expected_warnings': ['unable to extract json url'],
     }]
 
     def _extract_embed(self, webpage, display_id, id):
-        embed_url = self._html_search_meta(
-            'embedURL', webpage, 'embed URL',
-            default=None) or self._search_regex(
-            r'\bembedUrl["\']\s*:\s*(["\'])(?P<url>(?:(?!\1).)+)\1', webpage,
-            'embed URL', fatal=False, group='url')
-        if embed_url is None:
-            return self.url_result('ndr:%s' % id, ie=NDREmbedBaseIE.ie_key())
-        description = self._search_regex(
-            r'<p[^>]+itemprop="description">([^<]+)</p>',
-            webpage, 'description', default=None) or self._og_search_description(webpage)
-        timestamp = parse_iso8601(
-            self._search_regex(
-                r'<span[^>]+itemprop="(?:datePublished|uploadDate)"[^>]+content="([^"]+)"',
-                webpage, 'upload date', default=None))
-        info = self._search_json_ld(webpage, display_id, default={})
-        return merge_dicts({
-            '_type': 'url_transparent',
-            'url': embed_url,
-            'display_id': display_id,
-            'description': description,
-            'timestamp': timestamp,
-        }, info)
+        formats = []
+        base_url = 'https://www.ndr.de'
+        json_url = self._search_regex(r'<iframe[^>]+src=\"([^\"]+)_theme-ndrde[^\.]*\.html\"', webpage,
+                                      'json url', fatal=False)
+        if json_url:
+            data_json = self._download_json(base_url + json_url.replace('ardplayer_image', 'ardjson_image') + '.json',
+                                            id, fatal=False)
+            info_json = data_json.get('_info', {})
+            media_json = try_get(data_json, lambda x: x['_mediaArray'][0]['_mediaStreamArray'])
+            for media in media_json:
+                if media.get('_quality') == 'auto':
+                    formats.extend(self._extract_m3u8_formats(media['_stream'], id))
+            subtitles = {}
+            sub_url = data_json.get('_subtitleUrl')
+            if sub_url:
+                subtitles.setdefault('de', []).append({
+                    'url': base_url + sub_url,
+                })
+            self._sort_formats(formats)
+            return {
+                'id': id,
+                'title': info_json.get('clipTitle'),
+                'thumbnail': base_url + data_json.get('_previewImage'),
+                'description': info_json.get('clipDescription'),
+                'series': info_json.get('seriesTitle') or None,
+                'channel': info_json.get('channelTitle'),
+                'upload_date': unified_strdate(info_json.get('clipDate')),
+                'duration': data_json.get('_duration'),
+                'formats': formats,
+                'subtitles': subtitles,
+            }
+        else:
+            json_url = base_url + self._search_regex(r'apiUrl\s?=\s?\'([^\']+)\'', webpage, 'json url').replace(
+                '_belongsToPodcast-', '')
+            data_json = self._download_json(json_url, id, fatal=False)
+            return {
+                'id': id,
+                'title': data_json.get('title'),
+                'thumbnail': base_url + data_json.get('poster'),
+                'description': data_json.get('summary'),
+                'upload_date': unified_strdate(data_json.get('publicationDate')),
+                'duration': parse_duration(data_json.get('duration')),
+                'formats': [{
+                    'url': try_get(data_json, (lambda x: x['audio'][0]['url'], lambda x: x['files'][0]['url'])),
+                    'vcodec': 'none',
+                    'ext': 'mp3',
+                }],
+            }
 
 
 class NJoyIE(NDRBaseIE):
diff --git a/yt_dlp/extractor/streamanity.py b/yt_dlp/extractor/streamanity.py
new file mode 100644
index 000000000..2e2d5eedf
--- /dev/null
+++ b/yt_dlp/extractor/streamanity.py
@@ -0,0 +1,51 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+
+
+class StreamanityIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?streamanity\.com/video/(?P<id>[A-Za-z0-9]+)'
+    _TESTS = [{
+        'url': 'https://streamanity.com/video/9DFPTnuYi8f2',
+        'md5': '6ab171e8d4a02ad5dcbff6bea44cf5a1',
+        'info_dict': {
+            'id': '9DFPTnuYi8f2',
+            'ext': 'mp4',
+            'title': 'Bitcoin vs The Lighting Network',
+            'thumbnail': r're:https://res\.cloudinary\.com/.+\.png',
+            'description': '',
+            'uploader': 'Tom Bombadil (Freddy78)',
+        }
+    }, {
+        'url': 'https://streamanity.com/video/JktOUjSlfzTD',
+        'md5': '31f131e28abd3377c38be586a59532dc',
+        'info_dict': {
+            'id': 'JktOUjSlfzTD',
+            'ext': 'mp4',
+            'title': 'Share data when you see it',
+            'thumbnail': r're:https://res\.cloudinary\.com/.+\.png',
+            'description': 'Reposting as data should be public and stored on blockchain',
+            'uploader': 'digitalcurrencydaily',
+        }
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        video_info = self._download_json(
+            f'https://app.streamanity.com/api/video/{video_id}', video_id)['data']['video']
+
+        formats = self._extract_m3u8_formats(
+            f'https://stream.mux.com/{video_info["play_id"]}.m3u8?token={video_info["token"]}',
+            video_id, ext='mp4', m3u8_id='hls')
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': video_info['title'],
+            'description': video_info.get('description'),
+            'uploader': video_info.get('author_name'),
+            'is_live': False,
+            'thumbnail': video_info.get('thumb'),
+            'formats': formats,
+        }
diff --git a/yt_dlp/options.py b/yt_dlp/options.py
index 099b151c6..74c810471 100644
--- a/yt_dlp/options.py
+++ b/yt_dlp/options.py
@@ -235,7 +235,7 @@ def _dict_from_options_callback(
         help='Use this prefix for unqualified URLs. For example "gvsearch2:" downloads two videos from google videos for the search term "large apple". Use the value "auto" to let yt-dlp guess ("auto_warning" to emit a warning when guessing). "error" just throws an error. The default value "fixup_error" repairs broken URLs, but emits an error if this is not possible instead of searching')
     general.add_option(
         '--ignore-config', '--no-config',
-        action='store_true',
+        action='store_true', dest='ignoreconfig',
         help=(
             'Disable loading any configuration files except the one provided by --config-location. '
             'When given inside a configuration file, no further configuration files are loaded. '
@@ -1536,57 +1536,47 @@ def compat_conf(conf):
             'command-line': compat_conf(sys.argv[1:]),
             'custom': [], 'home': [], 'portable': [], 'user': [], 'system': []}
         paths = {'command-line': False}
-        opts, args = parser.parse_args(configs['command-line'])
+
+        def read_options(name, path, user=False):
+            ''' loads config files and returns ignoreconfig '''
+            # Multiple package names can be given here
+            # Eg: ('yt-dlp', 'youtube-dlc', 'youtube-dl') will look for
+            # the configuration file of any of these three packages
+            for package in ('yt-dlp',):
+                if user:
+                    config, current_path = _readUserConf(package, default=None)
+                else:
+                    current_path = os.path.join(path, '%s.conf' % package)
+                    config = _readOptions(current_path, default=None)
+                if config is not None:
+                    configs[name], paths[name] = config, current_path
+                    return parser.parse_args(config)[0].ignoreconfig
+            return False
 
         def get_configs():
-            if '--config-location' in configs['command-line']:
+            opts, _ = parser.parse_args(configs['command-line'])
+            if opts.config_location is not None:
                 location = compat_expanduser(opts.config_location)
                 if os.path.isdir(location):
                     location = os.path.join(location, 'yt-dlp.conf')
                 if not os.path.exists(location):
                     parser.error('config-location %s does not exist.' % location)
-                configs['custom'] = _readOptions(location, default=None)
-                if configs['custom'] is None:
-                    configs['custom'] = []
-                else:
-                    paths['custom'] = location
-            if '--ignore-config' in configs['command-line']:
+                config = _readOptions(location, default=None)
+                if config:
+                    configs['custom'], paths['config'] = config, location
+
+            if opts.ignoreconfig:
                 return
-            if '--ignore-config' in configs['custom']:
+            if parser.parse_args(configs['custom'])[0].ignoreconfig:
                 return
-
-            def read_options(path, user=False):
-                # Multiple package names can be given here
-                # Eg: ('yt-dlp', 'youtube-dlc', 'youtube-dl') will look for
-                # the configuration file of any of these three packages
-                for package in ('yt-dlp',):
-                    if user:
-                        config, current_path = _readUserConf(package, default=None)
-                    else:
-                        current_path = os.path.join(path, '%s.conf' % package)
-                        config = _readOptions(current_path, default=None)
-                    if config is not None:
-                        return config, current_path
-                return [], None
-
-            configs['portable'], paths['portable'] = read_options(get_executable_path())
-            if '--ignore-config' in configs['portable']:
+            if read_options('portable', get_executable_path()):
                 return
-
-            def get_home_path():
-                opts = parser.parse_args(configs['portable'] + configs['custom'] + configs['command-line'])[0]
-                return expand_path(opts.paths.get('home', '')).strip()
-
-            configs['home'], paths['home'] = read_options(get_home_path())
-            if '--ignore-config' in configs['home']:
+            opts, _ = parser.parse_args(configs['portable'] + configs['custom'] + configs['command-line'])
+            if read_options('home', expand_path(opts.paths.get('home', '')).strip()):
                 return
-
-            configs['system'], paths['system'] = read_options('/etc')
-            if '--ignore-config' in configs['system']:
+            if read_options('system', '/etc'):
                 return
-
-            configs['user'], paths['user'] = read_options('', True)
-            if '--ignore-config' in configs['user']:
+            if read_options('user', None, user=True):
                 configs['system'], paths['system'] = [], None
 
         get_configs()
@@ -1595,10 +1585,9 @@ def get_home_path():
         if opts.verbose:
             for label in ('System', 'User', 'Portable', 'Home', 'Custom', 'Command-line'):
                 key = label.lower()
-                if paths.get(key) is None:
-                    continue
-                if paths[key]:
-                    write_string('[debug] %s config file: %s\n' % (label, paths[key]))
-                write_string('[debug] %s config: %s\n' % (label, repr(_hide_login_info(configs[key]))))
+                if paths.get(key):
+                    write_string(f'[debug] {label} config file: {paths[key]}\n')
+                if paths.get(key) is not None:
+                    write_string(f'[debug] {label} config: {_hide_login_info(configs[key])!r}\n')
 
     return parser, opts, args


Test patch:
diff --git a/test/test_YoutubeDL.py b/test/test_YoutubeDL.py
index e689978fd..e61492ec8 100644
--- a/test/test_YoutubeDL.py
+++ b/test/test_YoutubeDL.py
@@ -649,6 +649,7 @@ def test_add_extra_info(self):
         'title2': '%PATH%',
         'title3': 'foo/bar\\test',
         'title4': 'foo "bar" test',
+        'title5': 'áéí',
         'timestamp': 1618488000,
         'duration': 100000,
         'playlist_index': 1,
@@ -767,6 +768,7 @@ def expect_same_infodict(out):
         test('%(ext)l', 'mp4')
         test('%(formats.:.id) 15l', '  id1, id2, id3')
         test('%(formats)j', (json.dumps(FORMATS), sanitize(json.dumps(FORMATS))))
+        test('%(title5).3B', 'á')
         if compat_os_name == 'nt':
             test('%(title4)q', ('"foo \\"bar\\" test"', "'foo _'bar_' test'"))
         else:
diff --git a/test/test_cookies.py b/test/test_cookies.py
index 6faaaa0c9..6053ebb4e 100644
--- a/test/test_cookies.py
+++ b/test/test_cookies.py
@@ -2,8 +2,8 @@
 from datetime import datetime, timezone
 
 from yt_dlp import cookies
+from yt_dlp.compat import compat_pycrypto_AES
 from yt_dlp.cookies import (
-    CRYPTO_AVAILABLE,
     LinuxChromeCookieDecryptor,
     MacChromeCookieDecryptor,
     WindowsChromeCookieDecryptor,
@@ -53,7 +53,7 @@ def test_chrome_cookie_decryptor_linux_v11(self):
             decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
             self.assertEqual(decryptor.decrypt(encrypted_value), value)
 
-    @unittest.skipIf(not CRYPTO_AVAILABLE, 'cryptography library not available')
+    @unittest.skipIf(not compat_pycrypto_AES, 'cryptography library not available')
     def test_chrome_cookie_decryptor_windows_v10(self):
         with MonkeyPatch(cookies, {
             '_get_windows_v10_key': lambda *args, **kwargs: b'Y\xef\xad\xad\xeerp\xf0Y\xe6\x9b\x12\xc2<z\x16]\n\xbb\xb8\xcb\xd7\x9bA\xc3\x14e\x99{\xd6\xf4&'