Repository: yt-dlp/yt-dlp

Issue:
CrunchyRoll Beta does not appears to pay attention to  --dateafter "today- and possibly --playlist-reverse
### Checklist

- [X] I'm reporting a bug unrelated to a specific site
- [X] I've verified that I'm running yt-dlp version **2022.07.18** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Provide a description that is worded well enough to be understood

I have recently had my account forced to use CrunchyRoll Beta and my current scripts have stopped working

I have slowly being trying to diagnose parts of it and slowly re-building the query so it works like it previously did

I have noticed a few commands that do not seem to function as expected when using the -a to pull back some playlists and process them.

It also appears that I can no longer filter on the languages like I did with --extractor-args "crunchyroll:[language=enUS,en-US]" 
 it may be possibly doing the playlist in reverse but is pulling back a language I do not need. 

Does anyone have any have a way I can filter to only ja-JP and en-US as a workaround for now? It does appear now if I use the --no-overwrites it skips the file if already downloaded if the date can not be filtered at this time. I assume this may be the fact its not being exposed from the beta URLs

The good news is it does appear to have fixed some of my naming issues with the seasons coming through as 0 if it's a first season. 

Any help would be much apricated. If I point this at a direct URL for an episode it Downloads as I expect 

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
.\yt-dlp --embed-subs --write-subs --sub-lang en,en_Uncut,en_Uncut_CC,en-US,enUS --parse-metadata "%(series)s - %(title)s S%(season_number)02dE%(episode_number)02d:%(title)s"  -o "%(series)s/Season %(season_number)02d/S%(season_number)02dEP%(episode_number)02d-%(language)s - %(series)s- %(episode)s.%(ext)s" -v --extractor-args "crunchyrollbeta:language=enUS,jaJP" --cookies .\crunchyroll.com_cookies.txt --trim-filenames 110 --dateafter "today-19day" --playlist-reverse --no-overwrites -a .\series.txt
[debug] Command-line config: ['--embed-subs', '--write-subs', '--sub-lang', 'en,en_Uncut,en_Uncut_CC,en-US,enUS', '--parse-metadata', '%(series)s - %(title)s S%(season_number)02dE%(episode_number)02d:%(title)s', '-o', '%(series)s/Season %(season_number)02d/S%(season_number)02dEP%(episode_number)02d-%(language)s - %(series)s- %(episode)s.%(ext)s', '-v', '--extractor-args', 'crunchyrollbeta:language=enUS,jaJP', '--cookies', '.\\crunchyroll.com_cookies.txt', '--trim-filenames', '110', '--dateafter', 'today-19day', '--playlist-reverse', '--no-overwrites', '-a', '.\\series.txt']
[debug] Batch file urls: ['https://www.crunchyroll.com/shikimoris-not-just-a-cutie', 'https://www.crunchyroll.com/rent-a-girlfriend', 'https://www.crunchyroll.com/kaguya-sama-love-is-war']
'\\MyServer\'
CMD.EXE was started with the above path as the current directory.
UNC paths are not supported.  Defaulting to Windows directory.
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.06.29 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 5.0-essentials_build-www.gyan.dev (setts), ffprobe 5.0-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] [crunchyroll:playlist] Extracting URL: https://www.crunchyroll.com/shikimoris-not-just-a-cutie
[crunchyroll:playlist] shikimoris-not-just-a-cutie: Downloading webpage
[crunchyroll:playlist] shikimoris-not-just-a-cutie: Redirected to beta site - https://beta.crunchyroll.com/series/GNVHKN7M4/shikimoris-not-just-a-cutie
[debug] [crunchyroll:playlist:beta] Extracting URL: https://beta.crunchyroll.com/series/GNVHKN7M4/shikimoris-not-just-a-cutie
[crunchyroll:playlist:beta] Retrieving main page
[crunchyroll:playlist:beta] Authenticating with cookie
[crunchyroll:playlist:beta] Retrieving signed policy
[crunchyroll:playlist:beta] shikimoris-not-just-a-cutie: Retrieving series metadata
[crunchyroll:playlist:beta] shikimoris-not-just-a-cutie: Retrieving season list
[download] Downloading playlist: Shikimori's Not Just a Cutie
[crunchyroll:playlist:beta] shikimoris-not-just-a-cutie: Retrieving episode list for shikimoris-not-just-a-cutie
[crunchyroll:playlist:beta] shikimoris-not-just-a-cutie: Retrieving episode list for shikimoris-not-just-a-cutie-german-dub
[crunchyroll:playlist:beta] shikimoris-not-just-a-cutie: Retrieving episode list for shikimoris-not-just-a-cutie-english-dub
[crunchyroll:playlist:beta] shikimoris-not-just-a-cutie: Retrieving episode list for shikimoris-not-just-a-cutie-portuguese-dub
[crunchyroll:playlist:beta] shikimoris-not-just-a-cutie: Retrieving episode list for shikimoris-not-just-a-cutie-spanish-dub
[crunchyroll:playlist:beta] shikimoris-not-just-a-cutie: Retrieving episode list for shikimoris-not-just-a-cutie-russian-dub
[crunchyroll:playlist:beta] Playlist Shikimori's Not Just a Cutie: Downloading 74 videos of 74
[download] Downloading video 1 of 74
[debug] [crunchyroll:beta] Extracting URL: https://beta.crunchyroll.com/watch/G14U403J2/better-than-a-dream
[crunchyroll:beta] better-than-a-dream: Retrieving episode metadata
[crunchyroll:beta] better-than-a-dream: Retrieving stream info
[crunchyroll:beta] better-than-a-dream: Downloading adaptive_hls information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Searching for '(?P<title>.+)' in '%(series)s - %(title)s S%(season_number)02dE%(episode_number)02d'
[MetadataParser] Parsed title from '%(series)s - %(title)s S%(season_number)02dE%(episode_number)02d': "Shikimori's Not Just a Cutie - Shikimori's Not Just a Cutie (Russian Dub) Episode 12 – Better Than a Dream S01E12"
[debug] Default format spec: bestvideo*+bestaudio/best
[info] G14U403J2: Downloading 1 format(s): adaptive_hls-8126
[debug] Invoking hlsnative downloader on "https://pl.crunchyroll.com/evs3/daaae10e5f6c9a639e7bb5ac81b39b32/assets/5258d7d633679f220a4bbcfb860c4b8b_4606448.mp4/index-v1-a1.m3u8?Expires=1659621801&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9wbC5jcnVuY2h5cm9sbC5jb20vZXZzMy9kYWFhZTEwZTVmNmM5YTYzOWU3YmI1YWM4MWIzOWIzMi9hc3NldHMvNTI1OGQ3ZDYzMzY3OWYyMjBhNGJiY2ZiODYwYzRiOGJfNDYwNjQ0OC5tcDQvaW5kZXgtdjEtYTEubTN1OCIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY1OTYyMTgwMX19fV19&Signature=StRr92bfO8K~3BO3fl39gx9iALZBNBFSHxUe8LQM0RqVQLyv7pdRSWqnsJQxw-LlTAc1VcAHbYKcaifJ9EsysT1I5XMFMdLQ4H-3D1u1Dak1HP8sRsOC9U4r0ZNaPbS7tzI-K8SDs0~z8vjH0l5a9OHscTylb6CmxAQn~oCqDf9vcMoeei5J8YITDAqU6odwYGvETHdICrMUXrpB-FyRnF-KaQDkg9A0kGBq8iGa9FeaUAAtgHY306H39pMlHG8rZKytCXVEUWsncBUCQZku8HdW4IaGmeyVS-R526Cxi4DlBK2Df1SmBCtMS0CvNBpIqB71kBSrbb5AQBhfSrdxiQ__&Key-Pair-Id=APKAJMWSQ5S7ZB3MF5VA"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 351
[download] Destination: Shikimori's Not Just a Cutie\Season 01\S01EP12-ru-RU - Shikimori's Not Just a Cutie- Better Than a Dream.mp4
[debug] File locking is not supported. Proceeding without locking
[download]  11.1% of ~1.57GiB at    3.67MiB/s ETA 03:56 (frag 39/351)
ERROR: Interrupted by user
```



Discussion:
To check if dateafter will work for an extractor, check `yt-dlp -O upload_date= "VIDEO_URL"`. If it gives NA, the extractor doesn't fetch the date

As for the rest of the post, there's a ton of random questions all mixed together. Please separate them and ask one by one. And what doesn't work in playlist-reverse?
Hi @pukkandan 

Thank you for the super quick reply. So it does look as if the --playlist-reverse  is working so really sorry about that as it is getting the latest release which just happens to be the RU audio track for that series

It would appear that CrunchyRoll as not tagging either the date or the language onto the URLs or playlists as shown below

.\yt-dlp -O upload_date= --cookies .\crunchyroll.com_cookies.txt https://www.crunchyroll.com/shikimoris-not-just-a-cutie
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'
upload_date = 'NA'

ERROR: Interrupted by user
PS My Drive> .\yt-dlp -O lang= --cookies .\crunchyroll.com_cookies.txt https://www.crunchyroll.com/shikimoris-not-just-a-cutie
lang = 'NA'
lang = 'NA'
lang = 'NA'
lang = 'NA'
lang = 'NA'

ERROR: Interrupted by user
PS My Drive> .\yt-dlp -O lang= --cookies .\crunchyroll.com_cookies.txt https://beta.crunchyroll.com/watch/GVWU0Q1KG/my-girlfriend-is-super-cute
lang = 'NA'
PS My Drive> .\yt-dlp -O upload_date= --cookies .\crunchyroll.com_cookies.txt https://beta.crunchyroll.com/watch/GVWU0Q1KG/my-girlfriend-is-super-cute
upload_date = 'NA'
PS My Drive>


I think if anything all I would want for now is the ability to only download the en-US tracks and then the ja-JP audio tracks, it appears as if they are in another column as shown below without and of the other languages, quite happy for it to taken longer going through the playlist and doing the iterations for now. It does seem we will have to wait for CR too add the other data needed such as date

en-US
![image](https://user-images.githubusercontent.com/19991728/182408606-1e7083a6-4bc4-4e84-84d6-a963e49e3e48.png)


ja-JP

![image](https://user-images.githubusercontent.com/19991728/182408721-c44ce194-1c24-4b3e-9ea5-2deafea097b4.png)

Use `--format`. Eg: `-f "[language=en-US]/[language=ja-JP]"`
For the date, the tests imply that they should be extracted. cc @Burve, @tejing1
For the upload date, that's my bad. The beta api does expose it, but I didn't catch it in the extractor.

For the audio language, unfortunately the beta api doesn't properly report the audio language. There's even a slot for it in the system they're using, but they leave it blank... All you can do is filter by a regexp match on the season title for the `(Something Dub)` ending. It's ugly, but it works. The api *does* supply booleans for `is_dubbed` and `is_subbed`, which seem to always be negations of each other and represent whether the audio language is Japanese or not, but currently the extractor doesn't do anything with them. There was some pushback against putting in too many extractor args when the PR was under review, and I gave up trying to find a way to expose those bools usefully.
It's pretty much a trivial fix for the upload date:
```patch
diff --git a/yt_dlp/extractor/crunchyroll.py b/yt_dlp/extractor/crunchyroll.py
index 6fd74989e..df3af9ae0 100644
--- a/yt_dlp/extractor/crunchyroll.py
+++ b/yt_dlp/extractor/crunchyroll.py
@@ -28,6 +28,7 @@
     join_nonempty,
     lowercase_escape,
     merge_dicts,
+    parse_iso8601,
     qualities,
     remove_end,
     sanitized_Request,
@@ -859,6 +860,7 @@ def _real_extract(self, url):
                 episode_response.get('season_title'), episode_response.get('episode'), episode_response.get('title')),
             'description': try_get(episode_response, lambda x: x['description'].replace(r'\r\n', '\n')),
             'duration': float_or_none(episode_response.get('duration_ms'), 1000),
+            'timestamp': parse_iso8601(episode_response.get('upload_date')),
             'series': episode_response.get('series_title'),
             'series_id': episode_response.get('series_id'),
             'season': episode_response.get('season_title'),
```

It could also provide a `release_timestamp`, but I'm not actually sure what I should set it by. Original air date? When the video became available to premium users? Free users?
Whenever I needed any extra processing, I did that manually from JSON I get from the extractor. There is a parameter "lang" for each episode, that gives out "en-US", but I never used extractor args in such a way.

As for the upload date - technically it is the first time I hear about using that, and it sounds that @tejin1 knows about it way more than I.
Huh, now that I'm looking at it again, I'm not sure if this was true before, but even though `episode_response` has an empty field for it, `stream_reponse` actually contains a proper `audio_locale` field. Not per-stream, but globally. I could pull that out and put it in the episode metadata just fine.
As long as it does not kill the current lang parameter per episode, that gives language in JSON
> As long as it does not kill the current lang parameter per episode, that gives language in JSON

If you mean per-format, no, it won't. Actually, it's exactly the same data as that, which means it was there before. I just didn't pull it out to the episode level, because I wasn't thinking straight.
Scratch that. I didn't extract it at the episode level before because yt-dlp doesn't have a field for that at the episode level. I was misreading docs just now.
Anyway, to reiterate the main answer: `--match-filters 'season !~= "([^()]+ Dub)"' --match-filters 'season ~= "(English Dub)"'` should only look at japanese and english audio episodes, skipping other videos. It's ugly, but it works. Sorry there isn't a better way.

Resolving patch:
diff --git a/yt_dlp/extractor/crunchyroll.py b/yt_dlp/extractor/crunchyroll.py
--- a/yt_dlp/extractor/crunchyroll.py
+++ b/yt_dlp/extractor/crunchyroll.py
@@ -28,6 +28,7 @@
     join_nonempty,
     lowercase_escape,
     merge_dicts,
+    parse_iso8601,
     qualities,
     remove_end,
     sanitized_Request,
@@ -761,43 +762,23 @@ class CrunchyrollBetaIE(CrunchyrollBetaBaseIE):
     _TESTS = [{
         'url': 'https://beta.crunchyroll.com/watch/GY2P1Q98Y/to-the-future',
         'info_dict': {
-            'id': '696363',
+            'id': 'GY2P1Q98Y',
             'ext': 'mp4',
-            'timestamp': 1459610100,
+            'duration': 1380.241,
+            'timestamp': 1459632600,
             'description': 'md5:a022fbec4fbb023d43631032c91ed64b',
-            'uploader': 'Toei Animation',
             'title': 'World Trigger Episode 73 – To the Future',
             'upload_date': '20160402',
-            'episode_number': 73,
             'series': 'World Trigger',
-            'average_rating': 4.9,
-            'episode': 'To the Future',
+            'series_id': 'GR757DMKY',
             'season': 'World Trigger',
-            'thumbnail': 'https://img1.ak.crunchyroll.com/i/spire3-tmb/c870dedca1a83137c2d3d144984155ed1459527119_main.jpg',
+            'season_id': 'GR9P39NJ6',
             'season_number': 1,
+            'episode': 'To the Future',
+            'episode_number': 73,
+            'thumbnail': r're:^https://beta.crunchyroll.com/imgsrv/.*\.jpeg$',
         },
         'params': {'skip_download': 'm3u8'},
-        'expected_warnings': ['Unable to download XML']
-    }, {
-        'url': 'https://beta.crunchyroll.com/watch/GYK53DMPR/wicked-lord-shingan-reborn',
-        'info_dict': {
-            'id': '648781',
-            'ext': 'mp4',
-            'episode_number': 1,
-            'timestamp': 1389173400,
-            'series': 'Love, Chunibyo & Other Delusions - Heart Throb -',
-            'description': 'md5:5579d1a0355cc618558ba23d27067a62',
-            'uploader': 'TBS',
-            'episode': 'Wicked Lord Shingan... Reborn',
-            'average_rating': 4.9,
-            'season': 'Love, Chunibyo & Other Delusions - Heart Throb -',
-            'thumbnail': 'https://img1.ak.crunchyroll.com/i/spire3-tmb/2ba0384e225a5370d5f0ee9496d91ea51389046521_main.jpg',
-            'title': 'Love, Chunibyo & Other Delusions - Heart Throb - Episode 1 – Wicked Lord Shingan... Reborn',
-            'season_number': 2,
-            'upload_date': '20140108',
-        },
-        'params': {'skip_download': 'm3u8'},
-        'expected_warnings': ['Unable to download XML']
     }, {
         'url': 'https://beta.crunchyroll.com/watch/GY2P1Q98Y/',
         'only_matching': True,
@@ -859,6 +840,7 @@ def _real_extract(self, url):
                 episode_response.get('season_title'), episode_response.get('episode'), episode_response.get('title')),
             'description': try_get(episode_response, lambda x: x['description'].replace(r'\r\n', '\n')),
             'duration': float_or_none(episode_response.get('duration_ms'), 1000),
+            'timestamp': parse_iso8601(episode_response.get('upload_date')),
             'series': episode_response.get('series_title'),
             'series_id': episode_response.get('series_id'),
             'season': episode_response.get('season_title'),
@@ -887,17 +869,10 @@ class CrunchyrollBetaShowIE(CrunchyrollBetaBaseIE):
     _TESTS = [{
         'url': 'https://beta.crunchyroll.com/series/GY19NQ2QR/Girl-Friend-BETA',
         'info_dict': {
-            'id': 'girl-friend-beta',
+            'id': 'GY19NQ2QR',
             'title': 'Girl Friend BETA',
         },
         'playlist_mincount': 10,
-    }, {
-        'url': 'https://beta.crunchyroll.com/series/GYJQV73V6/love-chunibyo--other-delusions---heart-throb--',
-        'info_dict': {
-            'id': 'love-chunibyo-other-delusions-heart-throb-',
-            'title': 'Love, Chunibyo & Other Delusions - Heart Throb -',
-        },
-        'playlist_mincount': 10,
     }, {
         'url': 'https://beta.crunchyroll.com/it/series/GY19NQ2QR/Girl-Friend-BETA',
         'only_matching': True,


Test patch: