Repository: yt-dlp/yt-dlp

Issue:
AbemaTV-license
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting that a **supported** site is broken
- [X] I've verified that I'm running yt-dlp version **2023.03.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Japan

### Provide a description that is worded well enough to be understood

When I try to download a video from AbemaTV, an error occurs.
I have added the -u and -p parameters as this video must be viewed with a premium account, I have run it using cookies and the situation is the same.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['--verbose', '-u', 'PRIVATE', '-p', 'PRIVATE', 'https://abema.tv/video/episode/19-149_s1_p4']
[debug] Encodings: locale cp932, fs utf-8, pref cp932, out cp932 (No VT), error cp932 (No VT), screen cp932 (No VT)
[debug] yt-dlp version stable@2023.03.04 [392389b7d] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-7-6.1.7601-SP1 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 5.1-full_build-www.gyan.dev (setts), ffprobe 5.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Loading abematv.mymail@outlook.jp from cache
[AbemaTV] Fetching media token
[debug] Loading abematv.mymail@outlook.jp from cache
[debug] [AbemaTV] Skipping logging in
[AbemaTV] Extracting URL: https://abema.tv/video/episode/19-149_s1_p4
[AbemaTV] 19-149_s1_p4: Downloading webpage
[AbemaTV] 19-149_s1_p4: Checking playability
WARNING: [AbemaTV] This is a premium-only stream
[AbemaTV] 19-149_s1_p4: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 19-149_s1_p4: Downloading 1 format(s): 5300
[debug] Invoking hlsnative downloader on "https://vod-abematv.akamaized.net/program/19-149_s1_p4/1080/playlist.m3u8?aver=1"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 240
[download] Destination: #04 まひろとあたらしい友達 [19-149_s1_p4].mp4
[debug] File locking is not supported. Proceeding without locking
[download] #04 まひろとあたらしい友達 [19-149_s1_p4].mp4.part-Frag1 has already been downloaded

[download]   0.4% of ~ 774.84MiB at  807.14MiB/s ETA Unknown (frag 1/240)ERROR: unable to download video data: <urlopen error unknown url type: abematv-license>
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 3247, in process_info
  File "yt_dlp\YoutubeDL.py", line 2970, in dl
  File "yt_dlp\downloader\common.py", line 444, in download
  File "yt_dlp\downloader\hls.py", line 365, in real_download
  File "yt_dlp\downloader\fragment.py", line 523, in download_and_append_fragments
  File "yt_dlp\downloader\fragment.py", line 364, in decrypt_fragment
  File "yt_dlp\downloader\fragment.py", line 353, in _get_key
  File "yt_dlp\YoutubeDL.py", line 3742, in urlopen
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 547, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 1425, in unknown_open
urllib.error.URLError: <urlopen error unknown url type: abematv-license>
```



Discussion:
cc @Lesmiscore 
@pukkandan I suppose it fucked up something in protocol handling? since the error tells that the protocol is not known, instead of decryption or whatever else
```
<urlopen error unknown url type: abematv-license>
```

I'll take a look anyway
PS: At 12:16(UTC), I ran episode 3 (https://abema.tv/video/episode/19-149_s1_p3) of the same title with the exact same command and the download was successful. However, now episode 3 also has an error. It is also a "urlopen error".
And also "urlopen error" occurs for episode 1 (https://abema.tv/video/episode/19-149_s1_p1), which can be viewed without a premium account.
Good luck.
I can't reproduce this on master
```
$ ./yt-dlp.sh https://abema.tv/video/episode/174-27_s4_p501 
[AbemaTV] Extracting URL: https://abema.tv/video/episode/174-27_s4_p501
[AbemaTV] Authorizing
[AbemaTV] 174-27_s4_p501: Downloading webpage
[AbemaTV] 174-27_s4_p501: Checking playability
[AbemaTV] 174-27_s4_p501: Downloading m3u8 information
[info] 174-27_s4_p501: Downloading 1 format(s): 5300
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 9
[download] Destination: PV【2023⧸1⧸5より毎週木曜23_00〜「深章 厄災篇」の地上波先行・独占先行放送が決定！】 [174-27_s4_p501].mp4
[download] 100% of   14.96MiB in 00:00:16 at 956.78KiB/s
[FixupM3u8] Fixing MPEG-TS in MP4 container of "PV【2023⧸1⧸5より毎週木曜23_00〜「深章 厄災篇」の地上波先行・独占先行放送が決定！】 [174-27_s4_p501].mp4"
```
Can you please try with the nightly builds here? https://github.com/yt-dlp/yt-dlp-nightly-builds/releases/latest
Ah, I have found.
Error occurred when using -u and -p

First try
```
[debug] Command-line config: ['-vU', '-u', 'PRIVATE', '-p', 'PRIVATE', 'https://abema.tv/video/episode/174-27_s4_p501']
[debug] Encodings: locale cp932, fs utf-8, pref cp932, out cp932 (No VT), error cp932 (No VT), screen cp932 (No VT)
[debug] yt-dlp version nightly@2023.03.13.014401 [607510b9f] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-7-6.1.7601-SP1 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 5.1-full_build-www.gyan.dev (setts), ffprobe 5.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1788 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version: nightly@2023.03.13.014401, Current version: nightly@2023.03.13.014401
Current Build Hash: 1c92b3a04c54b86ad615ea89a6dd80f3321e10af69cb28c8a914bb12864fe5fa
yt-dlp is up to date (nightly@2023.03.13.014401)
[debug] Loading abematv.me@outlook.jp from cache
[AbemaTV] Fetching media token
[debug] Loading abematv.me@outlook.jp from cache
[debug] [AbemaTV] Skipping logging in
[AbemaTV] Extracting URL: https://abema.tv/video/episode/174-27_s4_p501
[AbemaTV] 174-27_s4_p501: Downloading webpage
[AbemaTV] 174-27_s4_p501: Checking playability
[AbemaTV] 174-27_s4_p501: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 174-27_s4_p501: Downloading 1 format(s): 5300
[debug] Invoking hlsnative downloader on "https://vod-abematv.akamaized.net/program/174-27_s4_p501/1080/playlist.m3u8?aver=1"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 9
[download] Destination: PV【202315より毎週木曜23_00～「深章 厄災篇」の地上波先行・独占先行放送が決定！】 [174-27_s4_p501].mp4
[debug] File locking is not supported. Proceeding without locking

[download]   0.0% of ~  35.08MiB at   24.39KiB/s ETA 24:32 (frag 0/9)
[download]  11.1% of ~  35.08MiB at   19.89MiB/s ETA 00:01 (frag 1/9)
ERROR: unable to download video data: <urlopen error unknown url type: abematv-license>
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 3247, in process_info
  File "yt_dlp\YoutubeDL.py", line 2970, in dl
  File "yt_dlp\downloader\common.py", line 444, in download
  File "yt_dlp\downloader\hls.py", line 365, in real_download
  File "yt_dlp\downloader\fragment.py", line 523, in download_and_append_fragments
  File "yt_dlp\downloader\fragment.py", line 364, in decrypt_fragment
  File "yt_dlp\downloader\fragment.py", line 353, in _get_key
  File "yt_dlp\YoutubeDL.py", line 3743, in urlopen
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 547, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 1425, in unknown_open
urllib.error.URLError: <urlopen error unknown url type: abematv-license>
```
Second try without -u and -p

```
[debug] Command-line config: ['-vU', 'https://abema.tv/video/episode/174-27_s4_p501']
[debug] Encodings: locale cp932, fs utf-8, pref cp932, out cp932 (No VT), error cp932 (No VT), screen cp932 (No VT)
[debug] yt-dlp version nightly@2023.03.13.014401 [607510b9f] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-7-6.1.7601-SP1 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 5.1-full_build-www.gyan.dev (setts), ffprobe 5.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1788 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version: nightly@2023.03.13.014401, Current version: nightly@2023.03.13.014401
Current Build Hash: 1c92b3a04c54b86ad615ea89a6dd80f3321e10af69cb28c8a914bb12864fe5fa
yt-dlp is up to date (nightly@2023.03.13.014401)
[AbemaTV] Extracting URL: https://abema.tv/video/episode/174-27_s4_p501
[AbemaTV] Authorizing
[AbemaTV] 174-27_s4_p501: Downloading webpage
[AbemaTV] 174-27_s4_p501: Checking playability
[AbemaTV] 174-27_s4_p501: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 174-27_s4_p501: Downloading 1 format(s): 5300
[debug] Invoking hlsnative downloader on "https://vod-abematv.akamaized.net/program/174-27_s4_p501/1080/playlist.m3u8?aver=1"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 9
[download] Destination: PV【202315より毎週木曜23_00～「深章 厄災篇」の地上波先行・独占先行放送が決定！】 [174-27_s4_p501].mp4
[debug] File locking is not supported. Proceeding without locking

[download]   0.0% of ~  35.08MiB at   26.32KiB/s ETA 22:45 (frag 0/9)
[AbemaTV] Fetching media token
[AbemaTV] Requesting playback license
[download] 100% of   14.96MiB in 00:00:01 at 13.06MiB/s              
[debug] ffprobe command line: ffprobe -hide_banner -show_format -show_streams -print_format json "file:PV【202315より毎週木曜23_00～「深章 厄災篇」の地上波先行・独占先行放送が決定！】 [174-27_s4_p501].mp4"
[FixupM3u8] Fixing MPEG-TS in MP4 container of "PV【202315より毎週木曜23_00～「深章 厄災篇」の地上波先行・独占先行放送が決定！】 [174-27_s4_p501].mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:PV【202315より毎週木曜23_00～「深章 厄災篇」の地上波先行・独占先行放送が決定！】 [174-27_s4_p501].mp4" -map 0 -dn -ignore_unknown -c copy -f mp4 "-bsf:a" aac_adtstoasc -movflags "+faststart" "file:PV【202315より毎週木曜23_00～「深章 厄災篇」の地上波先行・独占先行放送が決定！】 [174-27_s4_p501].temp.mp4"
```
Unbelievable, but using the 2022.11.11 version I was able to download it without any problems...
I think, maybe the specifications have changed between the newer and older versions.
The latest version has been working to download premium videos, but suddenly it gaves an error even thogh the video used to be downloaded. ( I guess same version of yt-dlp) Now I down grade to 2022.11.11 version, it works again. 

Resolving patch:
diff --git a/yt_dlp/extractor/abematv.py b/yt_dlp/extractor/abematv.py
--- a/yt_dlp/extractor/abematv.py
+++ b/yt_dlp/extractor/abematv.py
@@ -136,11 +136,15 @@ def _get_device_token(self):
         if self._USERTOKEN:
             return self._USERTOKEN
 
+        add_opener(self._downloader, AbemaLicenseHandler(self))
+
         username, _ = self._get_login_info()
-        AbemaTVBaseIE._USERTOKEN = username and self.cache.load(self._NETRC_MACHINE, username)
+        auth_cache = username and self.cache.load(self._NETRC_MACHINE, username, min_ver='2024.01.19')
+        AbemaTVBaseIE._USERTOKEN = auth_cache and auth_cache.get('usertoken')
         if AbemaTVBaseIE._USERTOKEN:
             # try authentication with locally stored token
             try:
+                AbemaTVBaseIE._DEVICE_ID = auth_cache.get('device_id')
                 self._get_media_token(True)
                 return
             except ExtractorError as e:
@@ -159,7 +163,6 @@ def _get_device_token(self):
             })
         AbemaTVBaseIE._USERTOKEN = user_data['token']
 
-        add_opener(self._downloader, AbemaLicenseHandler(self))
         return self._USERTOKEN
 
     def _get_media_token(self, invalidate=False, to_show=True):
@@ -255,7 +258,7 @@ class AbemaTVIE(AbemaTVBaseIE):
 
     def _perform_login(self, username, password):
         self._get_device_token()
-        if self.cache.load(self._NETRC_MACHINE, username) and self._get_media_token():
+        if self.cache.load(self._NETRC_MACHINE, username, min_ver='2024.01.19') and self._get_media_token():
             self.write_debug('Skipping logging in')
             return
 
@@ -278,7 +281,11 @@ def _perform_login(self, username, password):
 
         AbemaTVBaseIE._USERTOKEN = login_response['token']
         self._get_media_token(True)
-        self.cache.store(self._NETRC_MACHINE, username, AbemaTVBaseIE._USERTOKEN)
+        auth_cache = {
+            'device_id': AbemaTVBaseIE._DEVICE_ID,
+            'usertoken': AbemaTVBaseIE._USERTOKEN,
+        }
+        self.cache.store(self._NETRC_MACHINE, username, auth_cache)
 
     def _real_extract(self, url):
         # starting download using infojson from this extractor is undefined behavior,


Test patch: