Repository: yt-dlp/yt-dlp

Issue:
Unable to download m3u8, unknown type irvine
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a bug unrelated to a specific site
- [X] I've verified that I'm running yt-dlp version **2023.01.06** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Provide a description that is worded well enough to be understood

I'm unable to download new videos from the site wrestle-universe.com. Previous videos are able to downloaded using the regular command of a direct link to the m3u8 file, however the video from today is not able to be downloaded. I've tried a couple other programs, as well as adding the cookies and referer flags, but am getting an error about this "unknown url type: irvine"

I downloaded the m3u8 locally and this line is in the file, hopefully it's just something that urllib or yt-dlp needs to learn how to handle

```#EXT-X-KEY:METHOD=AES-128,URI="irvine://aes",IV=0x5BAC71BEB33197D5600337CE86DE7862```

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', 'https://vod.asset.wrestle-universe.com/9WwMBERet9VdH52yMH3F5/live-a_1080p_2997fps.m3u8', '-o', 's2023e07.mp4']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.01.06 [6becd25] (pip)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-58-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2020.06.20, mutagen-1.46.0, secretstorage-3.3.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Loaded 1760 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.01.06, Current version: 2023.01.06
yt-dlp is up to date (2023.01.06)
[generic] Extracting URL: https://vod.asset.wrestle-universe.com/9WwMBERet9VdH52yMH3F5/live-a_1080p_2997fps.m3u8
[generic] live-a_1080p_2997fps: Downloading webpage
[debug] Identified a direct video link
[generic] live-a_1080p_2997fps: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] live-a_1080p_2997fps: Downloading 1 format(s): 0
[debug] Invoking hlsnative downloader on "https://vod.asset.wrestle-universe.com/9WwMBERet9VdH52yMH3F5/live-a_1080p_2997fps.m3u8"
[hlsnative] Downloading m3u8 manifest
WARNING: Live HLS streams are not supported by the native downloader. If this is a livestream, please add "--downloader ffmpeg --hls-use-mpegts" to your command
[hlsnative] Total fragments: 3113
[download] Destination: s2023e07.mp4
[download] s2023e07.mp4.part-Frag1 has already been downloaded
[download]   0.0% of ~   4.67GiB at   31.19MiB/s ETA Unknown (frag 1/3113)ERROR: unable to download video data: <urlopen error unknown url type: irvine>
Traceback (most recent call last):
  File "/home/mike/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3236, in process_info
    success, real_download = self.dl(temp_filename, info_dict)
  File "/home/mike/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 2959, in dl
    return fd.download(name, new_info, subtitle)
  File "/home/mike/.local/lib/python3.10/site-packages/yt_dlp/downloader/common.py", line 444, in download
    ret = self.real_download(filename, info_dict)
  File "/home/mike/.local/lib/python3.10/site-packages/yt_dlp/downloader/hls.py", line 354, in real_download
    return self.download_and_append_fragments(ctx, fragments, info_dict)
  File "/home/mike/.local/lib/python3.10/site-packages/yt_dlp/downloader/fragment.py", line 521, in download_and_append_fragments
    decrypt_fragment(fragment, self._read_fragment(ctx)), fragment['frag_index'], ctx)
  File "/home/mike/.local/lib/python3.10/site-packages/yt_dlp/downloader/fragment.py", line 363, in decrypt_fragment
    decrypt_info['KEY'] = decrypt_info.get('KEY') or _get_key(info_dict.get('_decryption_key_url') or decrypt_info['URI'])
  File "/home/mike/.local/lib/python3.10/site-packages/yt_dlp/downloader/fragment.py", line 353, in _get_key
    _key_cache[url] = self.ydl.urlopen(self._prepare_url(info_dict, url)).read()
  File "/home/mike/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3730, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/usr/lib/python3.10/urllib/request.py", line 541, in _open
    return self._call_chain(self.handle_open, 'unknown',
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 1419, in unknown_open
    raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: irvine>
```



Discussion:
It's not a standard URL scheme. We'll need to figure out how the website handles this and write an extractor and custom downloader. For that, access to the the actual webpage is needed though
I did a bit more digging myself, it's an encrypted stream. Was able to find the key and download it via hlsdl, but I guess it would need to be an actual proper site support for yt-dlp to be able to pull that info out without having to manually get the key

Is there anything I can do to help with the account part? It's a paid site, so I don't think you can create a free account
Please post the link to the page with any [more info](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-description-of-the-issue-itself-sufficient) you think might be helpful.
You may share the account details (read [this](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed)) with me through [E-Mail](mailto:contact@grub4k.xyz) or Discord (Grub4K#2417).
I'll put a walkthrough of how I got the details on this one page and was able to download

this is the page: https://www.wrestle-universe.com/en/lives/buH9ibbfhdJAY4GKZcEuJX

- go to the ppv page, open the webdev tools
- go to the Souces tab and open to www.wrestle-universe.com->_next/static->chunks->pages->lives->some random bs
- click the curly brackets on the bottom
- search for 'aes128key' and then click the line number on the right to create a breakpoint. this variable exists like 5 times on the page, but any of them should be fine, i just picked the first one
- now click play on the video and it will play a bit then it should stop
- look over in the Scope area on the right. the key field is what you want

Resolving patch:
diff --git a/yt_dlp/extractor/_extractors.py b/yt_dlp/extractor/_extractors.py
--- a/yt_dlp/extractor/_extractors.py
+++ b/yt_dlp/extractor/_extractors.py
@@ -2274,6 +2274,10 @@
     WPPilotIE,
     WPPilotChannelsIE,
 )
+from .wrestleuniverse import (
+    WrestleUniverseVODIE,
+    WrestleUniversePPVIE,
+)
 from .wsj import (
     WSJIE,
     WSJArticleIE,
diff --git a/yt_dlp/extractor/wrestleuniverse.py b/yt_dlp/extractor/wrestleuniverse.py
new file mode 100644
--- /dev/null
+++ b/yt_dlp/extractor/wrestleuniverse.py
@@ -0,0 +1,233 @@
+import base64
+import binascii
+import json
+import time
+
+from .common import InfoExtractor
+from ..dependencies import Cryptodome
+from ..utils import (
+    ExtractorError,
+    int_or_none,
+    jwt_decode_hs256,
+    traverse_obj,
+    try_call,
+    url_or_none,
+)
+
+
+class WrestleUniverseBaseIE(InfoExtractor):
+    _VALID_URL_TMPL = r'https?://(?:www\.)?wrestle-universe\.com/(?:(?P<lang>\w{2})/)?%s/(?P<id>\w+)'
+    _API_PATH = None
+    _TOKEN = None
+    _TOKEN_EXPIRY = None
+
+    def _get_token_cookie(self):
+        if not self._TOKEN or not self._TOKEN_EXPIRY:
+            self._TOKEN = try_call(lambda: self._get_cookies('https://www.wrestle-universe.com/')['token'].value)
+            if not self._TOKEN:
+                self.raise_login_required()
+            expiry = traverse_obj(jwt_decode_hs256(self._TOKEN), ('exp', {int_or_none}))
+            if not expiry:
+                raise ExtractorError('There was a problem with the token cookie')
+            self._TOKEN_EXPIRY = expiry
+
+        if self._TOKEN_EXPIRY <= int(time.time()):
+            raise ExtractorError(
+                'Expired token. Refresh your cookies in browser and try again', expected=True)
+
+        return self._TOKEN
+
+    def _call_api(self, video_id, param='', msg='API', auth=True, data=None, query={}, fatal=True):
+        headers = {'CA-CID': ''}
+        if data:
+            headers['Content-Type'] = 'application/json;charset=utf-8'
+            data = json.dumps(data, separators=(',', ':')).encode()
+        if auth:
+            headers['Authorization'] = f'Bearer {self._get_token_cookie()}'
+        return self._download_json(
+            f'https://api.wrestle-universe.com/v1/{self._API_PATH}/{video_id}{param}', video_id,
+            note=f'Downloading {msg} JSON', errnote=f'Failed to download {msg} JSON',
+            data=data, headers=headers, query=query, fatal=fatal)
+
+    def _call_encrypted_api(self, video_id, param='', msg='API', data={}, query={}, fatal=True):
+        if not Cryptodome:
+            raise ExtractorError('pycryptodomex not found. Please install', expected=True)
+        private_key = Cryptodome.PublicKey.RSA.generate(2048)
+        cipher = Cryptodome.Cipher.PKCS1_OAEP.new(private_key, hashAlgo=Cryptodome.Hash.SHA1)
+
+        def decrypt(data):
+            if not data:
+                return None
+            try:
+                return cipher.decrypt(base64.b64decode(data)).decode()
+            except (ValueError, binascii.Error) as e:
+                raise ExtractorError(f'Could not decrypt data: {e}')
+
+        token = base64.b64encode(private_key.public_key().export_key('DER')).decode()
+        api_json = self._call_api(video_id, param, msg, data={
+            # 'deviceId' (random uuid4 generated at login) is not required yet
+            'token': token,
+            **data,
+        }, query=query, fatal=fatal)
+        return api_json, decrypt
+
+    def _download_metadata(self, url, video_id, lang, props_key):
+        metadata = self._call_api(video_id, msg='metadata', query={'al': lang or 'ja'}, auth=False, fatal=False)
+        if not metadata:
+            webpage = self._download_webpage(url, video_id)
+            nextjs_data = self._search_nextjs_data(webpage, video_id)
+            metadata = traverse_obj(nextjs_data, ('props', 'pageProps', props_key, {dict})) or {}
+        return metadata
+
+    def _get_formats(self, data, path, video_id=None):
+        hls_url = traverse_obj(data, path, get_all=False)
+        if not hls_url and not data.get('canWatch'):
+            self.raise_no_formats(
+                'This account does not have access to the requested content', expected=True)
+        elif not hls_url:
+            self.raise_no_formats('No supported formats found')
+        return self._extract_m3u8_formats(hls_url, video_id, 'mp4', m3u8_id='hls', live=True)
+
+
+class WrestleUniverseVODIE(WrestleUniverseBaseIE):
+    _VALID_URL = WrestleUniverseBaseIE._VALID_URL_TMPL % 'videos'
+    _TESTS = [{
+        'url': 'https://www.wrestle-universe.com/en/videos/dp8mpjmcKfxzUhEHM2uFws',
+        'info_dict': {
+            'id': 'dp8mpjmcKfxzUhEHM2uFws',
+            'ext': 'mp4',
+            'title': 'The 3rd “Futari wa Princess” Max Heart Tournament',
+            'description': 'md5:318d5061e944797fbbb81d5c7dd00bf5',
+            'location': '埼玉・春日部ふれあいキューブ',
+            'channel': 'tjpw',
+            'duration': 7119,
+            'timestamp': 1674979200,
+            'upload_date': '20230129',
+            'thumbnail': 'https://image.asset.wrestle-universe.com/8FjD67P8rZc446RBQs5RBN/8FjD67P8rZc446RBQs5RBN',
+            'chapters': 'count:7',
+            'cast': 'count:18',
+        },
+        'params': {
+            'skip_download': 'm3u8',
+        },
+    }]
+
+    _API_PATH = 'videoEpisodes'
+
+    def _real_extract(self, url):
+        lang, video_id = self._match_valid_url(url).group('lang', 'id')
+        metadata = self._download_metadata(url, video_id, lang, 'videoEpisodeFallbackData')
+        video_data = self._call_api(video_id, ':watch', 'watch', data={
+            # 'deviceId' is required if ignoreDeviceRestriction is False
+            'ignoreDeviceRestriction': True,
+        })
+
+        return {
+            'id': video_id,
+            'formats': self._get_formats(video_data, (
+                (('protocolHls', 'url'), ('chromecastUrls', ...)), {url_or_none}), video_id),
+            **traverse_obj(metadata, {
+                'title': ('displayName', {str}),
+                'description': ('description', {str}),
+                'channel': ('labels', 'group', {str}),
+                'location': ('labels', 'venue', {str}),
+                'timestamp': ('watchStartTime', {int_or_none}),
+                'thumbnail': ('keyVisualUrl', {url_or_none}),
+                'cast': ('casts', ..., 'displayName', {str}),
+                'duration': ('duration', {int}),
+                'chapters': ('videoChapters', lambda _, v: isinstance(v.get('start'), int), {
+                    'title': ('displayName', {str}),
+                    'start_time': ('start', {int}),
+                    'end_time': ('end', {int}),
+                }),
+            }),
+        }
+
+
+class WrestleUniversePPVIE(WrestleUniverseBaseIE):
+    _VALID_URL = WrestleUniverseBaseIE._VALID_URL_TMPL % 'lives'
+    _TESTS = [{
+        'note': 'HLS AES-128 key obtained via API',
+        'url': 'https://www.wrestle-universe.com/en/lives/buH9ibbfhdJAY4GKZcEuJX',
+        'info_dict': {
+            'id': 'buH9ibbfhdJAY4GKZcEuJX',
+            'ext': 'mp4',
+            'title': '【PPV】Beyond the origins, into the future',
+            'description': 'md5:9a872db68cd09be4a1e35a3ee8b0bdfc',
+            'channel': 'tjpw',
+            'location': '東京・Twin Box AKIHABARA',
+            'duration': 10098,
+            'timestamp': 1675076400,
+            'upload_date': '20230130',
+            'thumbnail': 'https://image.asset.wrestle-universe.com/rJs2m7cBaLXrwCcxMdQGRM/rJs2m7cBaLXrwCcxMdQGRM',
+            'thumbnails': 'count:3',
+            'hls_aes': {
+                'key': '5633184acd6e43f1f1ac71c6447a4186',
+                'iv': '5bac71beb33197d5600337ce86de7862',
+            },
+        },
+        'params': {
+            'skip_download': 'm3u8',
+        },
+    }, {
+        'note': 'unencrypted HLS',
+        'url': 'https://www.wrestle-universe.com/en/lives/wUG8hP5iApC63jbtQzhVVx',
+        'info_dict': {
+            'id': 'wUG8hP5iApC63jbtQzhVVx',
+            'ext': 'mp4',
+            'title': 'GRAND PRINCESS \'22',
+            'description': 'md5:e4f43d0d4262de3952ff34831bc99858',
+            'channel': 'tjpw',
+            'location': '東京・両国国技館',
+            'duration': 18044,
+            'timestamp': 1647665400,
+            'upload_date': '20220319',
+            'thumbnail': 'https://image.asset.wrestle-universe.com/i8jxSTCHPfdAKD4zN41Psx/i8jxSTCHPfdAKD4zN41Psx',
+            'thumbnails': 'count:3',
+        },
+        'params': {
+            'skip_download': 'm3u8',
+        },
+    }]
+
+    _API_PATH = 'events'
+
+    def _real_extract(self, url):
+        lang, video_id = self._match_valid_url(url).group('lang', 'id')
+        metadata = self._download_metadata(url, video_id, lang, 'eventFallbackData')
+
+        info = traverse_obj(metadata, {
+            'title': ('displayName', {str}),
+            'description': ('description', {str}),
+            'channel': ('labels', 'group', {str}),
+            'location': ('labels', 'venue', {str}),
+            'timestamp': ('startTime', {int_or_none}),
+            'thumbnails': (('keyVisualUrl', 'alterKeyVisualUrl', 'heroKeyVisualUrl'), {'url': {url_or_none}}),
+        })
+
+        ended_time = traverse_obj(metadata, ('endedTime', {int_or_none}))
+        if info.get('timestamp') and ended_time:
+            info['duration'] = ended_time - info['timestamp']
+
+        video_data, decrypt = self._call_encrypted_api(
+            video_id, ':watchArchive', 'watch archive', data={'method': 1})
+        formats = self._get_formats(video_data, (
+            ('hls', None), ('urls', 'chromecastUrls'), ..., {url_or_none}), video_id)
+        for f in formats:
+            # bitrates are exaggerated in PPV playlists, so avoid wrong/huge filesize_approx values
+            if f.get('tbr'):
+                f['tbr'] = int(f['tbr'] / 2.5)
+
+        hls_aes_key = traverse_obj(video_data, ('hls', 'key', {decrypt}))
+        if not hls_aes_key and traverse_obj(video_data, ('hls', 'encryptType', {int}), default=0) > 0:
+            self.report_warning('HLS AES-128 key was not found in API response')
+
+        return {
+            'id': video_id,
+            'formats': formats,
+            'hls_aes': {
+                'key': hls_aes_key,
+                'iv': traverse_obj(video_data, ('hls', 'iv', {decrypt})),
+            },
+            **info,
+        }


Test patch: