Repository: yt-dlp/yt-dlp

Issue:
bilibili video related to festivel event cannot be downloaded
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2023.01.06** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Malaysia

### Provide a description that is worded well enough to be understood

Bilibili Videos related to special festival events cannot be downloaded as the page is directed to a different templete. On previous version of yt-dlp (22.09.01) low quality version can still be downloadable but with latest version nothing can be downloaded.

example video
https://www.bilibili.com/video/BV1ay4y1d77f/
https://www.bilibili.com/video/BV1hD4y117je/

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
>yt-dlp.exe  --cookies bilibili_cookies.txt https://www.bilibili.com/video/BV1ay4y1d77f/ -vU
[debug] Command-line config: ['--cookies', 'bilibili_cookies.txt', 'https://www.bilibili.com/video/BV1ay4y1d77f/', '-vU']
[debug] Encodings: locale cp932, fs utf-8, pref cp932, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.01.06 [6becd25] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 2022-01-17-git-dcc9454ab9-full_build-www.gyan.dev (setts), ffprobe 2022-01-17-git-dcc9454ab9-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.16.0, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1760 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.01.06, Current version: 2023.01.06
yt-dlp is up to date (2023.01.06)
[BiliBili] Extracting URL: https://www.bilibili.com/video/BV1ay4y1d77f/
[BiliBili] 1ay4y1d77f: Downloading webpage
ERROR: [BiliBili] 1ay4y1d77f: Unable to extract play info; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 680, in extract
  File "yt_dlp\extractor\bilibili.py", line 288, in _real_extract
  File "yt_dlp\extractor\common.py", line 1242, in _search_json
  File "yt_dlp\extractor\common.py", line 1228, in _search_regex
```



Discussion:
Same as above. Any updates ? 
I don't know any Japanese, but according to a git bisect, prior to ad97487606c87878aa06b736a72ffde15056bdd4 an 18 minute video downloaded, and after that commit, no video downloaded.

That means that an 18 minute video will download on version 2022.10.04, but not on later versions

Here is a verbose log of the first bad commit:
```
[debug] Command-line config: ['https://www.bilibili.com/video/BV1ay4y1d77f/', '--verbose']
[debug] User config "C:\Users\jaybu\AppData\Roaming\yt-dlp\config.txt": ['--ffmpeg-location', 'C:\\Users\\jaybu\\ffmpeg\\bin', '-P', 'C:\\Users\\jaybu\\youtube.dl']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.10.04 [4e0511f27] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: ad9748760
[debug] Python 3.10.8 (CPython 64bit) - Windows-10-10.0.19045-SP0
[debug] Checking exe version: "C:\Users\jaybu\ffmpeg\bin\ffmpeg" -bsfs
[debug] Checking exe version: "C:\Users\jaybu\ffmpeg\bin\ffprobe" -bsfs
[debug] exe versions: ffmpeg N-106498-g854615adf2-20220405 (setts), ffprobe N-106498-g854615adf2-20220405, phantomjs 2.1.1
[debug] Optional libraries: certifi-2022.09.24, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1702 extractors
[debug] [BiliBili] Extracting URL: https://www.bilibili.com/video/BV1ay4y1d77f/
[BiliBili] 1ay4y1d77f: Downloading webpage
ERROR: [BiliBili] 1ay4y1d77f: Unable to extract play info; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "C:\GitHub\yt-dlp-OG\yt_dlp\extractor\common.py", line 674, in extract
    ie_result = self._real_extract(url)
  File "C:\GitHub\yt-dlp-OG\yt_dlp\extractor\bilibili.py", line 289, in _real_extract
    play_info = self._search_json(r'window.__playinfo__\s*=', webpage, 'play info', video_id)['data']
  File "C:\GitHub\yt-dlp-OG\yt_dlp\extractor\common.py", line 1242, in _search_json
    json_string = self._search_regex(
  File "C:\GitHub\yt-dlp-OG\yt_dlp\extractor\common.py", line 1228, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
```

Here's a verbose log of the last good commit:
```
[debug] Command-line config: ['https://www.bilibili.com/video/BV1ay4y1d77f/', '--verbose']
[debug] User config "C:\Users\jaybu\AppData\Roaming\yt-dlp\config.txt": ['--ffmpeg-location', 'C:\\Users\\jaybu\\ffmpeg\\bin', '-P', 'C:\\Users\\jaybu\\youtube.dl']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.10.04 [4e0511f27] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: e091fb92d
[debug] Python 3.10.8 (CPython 64bit) - Windows-10-10.0.19045-SP0
[debug] Checking exe version: "C:\Users\jaybu\ffmpeg\bin\ffmpeg" -bsfs
[debug] Checking exe version: "C:\Users\jaybu\ffmpeg\bin\ffprobe" -bsfs
[debug] exe versions: ffmpeg N-106498-g854615adf2-20220405 (setts), ffprobe N-106498-g854615adf2-20220405, phantomjs 2.1.1
[debug] Optional libraries: certifi-2022.09.24, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1701 extractors
[debug] [BiliBili] Extracting URL: https://www.bilibili.com/video/BV1ay4y1d77f/
[BiliBili] 1ay4y1d77f: Grabbing original ID via API
[BiliBili] 778246196: Downloading webpage
[BiliBili] 778246196: Extracting videos in anthology
[BiliBili] Downloading anthology 778246196 - add --no-playlist to just download video
[download] Downloading playlist: 2023崩坏3新春会
[BiliBili] Playlist 2023崩坏3新春会: Downloading 1 videos of 1
[download] Downloading video 1 of 1
[debug] [BiliBili] Extracting URL: https://www.bilibili.com/video/BV1ay4y1d77f?p=1
[BiliBili] 1ay4y1d77f: Grabbing original ID via API
[BiliBili] 778246196: Downloading webpage
[BiliBili] 778246196: Downloading video info page
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
WARNING: [BiliBili] unable to extract title; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[BiliBili] 778246196: Extracting videos in anthology
WARNING: [BiliBili] unable to extract description; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [BiliBili] unable to extract og:image; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[BiliBili] 778246196: Downloading tags
WARNING: Extractor failed to obtain "title". Creating a generic title instead
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 778246196_part1: Downloading 1 format(s): 0
[debug] Invoking http downloader on "http://upos-hz-mirrorakam.akamaized.net/upgcxcode/64/03/974030364/974030364_nb3-1-16.mp4?e=ig8euxZM2rNcNbuVhwdVtWuVhwdVNEVEuCIv29hEn0lqXg8Y2ENvNCImNEVEUJ1miI7MT96fqj3E9r1qNCNEto8g2ENvN03eN0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8BTrNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&deadline=1677912863&gen=playurl&nbs=1&oi=455093723&os=akam&platform=pc&trid=a78526bf0cb24a788038cf015ee6c13d&uipk=5&upsig=eb8bd70b7bca365ac25866130419ce76&uparams=e,deadline,gen,nbs,oi,os,platform,trid,uipk&hdnts=exp=1677912863~hmac=e6551decf10e16784d008b7670893325c0f4963ee3d3e91dd6f7e16e657d976f&mid=0"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: C:\Users\jaybu\youtube.dl\BiliBili video #778246196_part1 [778246196_part1].mp4
[download] 100% of   29.96MiB in 00:00:31 at 984.04KiB/s
[download] Finished downloading playlist: 2023崩坏3新春会
````
PS: I don't know why @matsumotokenji passed cookies in their verbose log, but this issue definitely doesn't require cookies to reproduce
@lockmatrix 
It is a site-bug, these festival video pages is not same as other normal videos.
Current bilibili extractor can not process it.

I personally have no plans to fix it right now due to the small number of videos involved.

Volunteers are needed
These URL is redirect to `https://www.bilibili.com/festival/`, 
I suggest adding a new extractor for it, and add a check-and-call in BiliBiliIE 
For users: as a temporary workaround, run `yt-dlp --update-to stable@2022.10.04` (or `yt-dlp --update-to 2022.10.04`) before you download the video, and run `yt-dlp --update` after you're done

For developers: https://github.com/yt-dlp/yt-dlp/commit/ad97487606c87878aa06b736a72ffde15056bdd4 is a rather big commit, so I decided to run a bisect to see which individual commit of https://github.com/yt-dlp/yt-dlp/pull/4945 has the regression. Believe it or not, it is the first commit (https://github.com/yt-dlp/yt-dlp/pull/4945/commits/8eee34cf2db684f27ae8986305bcabd9292916a0), although that commit gives a slightly different error message:
```
[debug] Command-line config: ['https://www.bilibili.com/video/BV1ay4y1d77f/', '--verbose']
[debug] User config "C:\Users\jaybu\AppData\Roaming\yt-dlp\config.txt": ['--ffmpeg-location', 'C:\\Users\\jaybu\\ffmpeg\\bin', '-P', 'C:\\Users\\jaybu\\youtube.dl']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.09.01 [5d7c7d656] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: 8eee34cf2
[debug] Python 3.10.8 (CPython 64bit) - Windows-10-10.0.19045-SP0
[debug] Checking exe version: "C:\Users\jaybu\ffmpeg\bin\ffmpeg" -bsfs
[debug] Checking exe version: "C:\Users\jaybu\ffmpeg\bin\ffprobe" -bsfs
[debug] exe versions: ffmpeg N-106498-g854615adf2-20220405 (setts), ffprobe N-106498-g854615adf2-20220405, phantomjs 2.1.1
[debug] Optional libraries: certifi-2022.09.24, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1674 extractors
[debug] [BiliBili] Extracting URL: https://www.bilibili.com/video/BV1ay4y1d77f/
[BiliBili] 1ay4y1d77f: Downloading webpage
ERROR: An extractor error has occurred. (caused by KeyError('bvid')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "C:\GitHub\yt-dlp-OG\yt_dlp\extractor\common.py", line 672, in extract
    ie_result = self._real_extract(url)
  File "C:\GitHub\yt-dlp-OG\yt_dlp\extractor\bilibili.py", line 226, in _real_extract
    bv_id = video_data['bvid']
KeyError: 'bvid'
```
I also observed some other error messages at different commits, as I was running the bisect. Unfortunately I have closed the command prompt window now, and I don't have the patience to run another git bisect.
> For developers: [ad97487](https://github.com/yt-dlp/yt-dlp/commit/ad97487606c87878aa06b736a72ffde15056bdd4) is a rather big commit, so I decided to run a bisect to see which individual commit of #4945 has the regression. Believe it or not, it is the first commit ([8eee34c](https://github.com/yt-dlp/yt-dlp/commit/8eee34cf2db684f27ae8986305bcabd9292916a0)), although that commit gives a slightly different error message:


That commit is a complete refactoring (to be precise, it was rewritten), using a different way of getting video information.

If you can write python, you can try to create a patch for https://www.bilibili.com/festival
Beginnings of a solution at https://github.com/ytdl-org/youtube-dl/issues/31661.
The approach linked above seems not to work in the yt-dlp extractor because there's no `window.__playinfo__`. However punting to the old yt-dl extractor gets a giant FLV, as in the quoted log there.

Not entirely on-topic, but relevant to back-porting new IEs from the yt-dlp module:
* the `title` text isn't quite the same now in `test_BilibiliSpacePlaylist`
* similarly `uploader` for `test_BilibiliAudio`
* similarly `title` for `test_BilibiliAudioAlbum`
* similarly `title` for `test_BiliIntl_3` (ends ` Bilibili`, not ` Bstation`).

Obvs may be others in these tests since only the first disagreement is reported in the download tests. 

Resolving patch:
diff --git a/yt_dlp/extractor/bilibili.py b/yt_dlp/extractor/bilibili.py
--- a/yt_dlp/extractor/bilibili.py
+++ b/yt_dlp/extractor/bilibili.py
@@ -133,7 +133,7 @@ def _get_all_children(self, reply):
 
 
 class BiliBiliIE(BilibiliBaseIE):
-    _VALID_URL = r'https?://www\.bilibili\.com/video/[aAbB][vV](?P<id>[^/?#&]+)'
+    _VALID_URL = r'https?://www\.bilibili\.com/(?:video/|festival/\w+\?(?:[^#]*&)?bvid=)[aAbB][vV](?P<id>[^/?#&]+)'
 
     _TESTS = [{
         'url': 'https://www.bilibili.com/video/BV13x41117TL',
@@ -281,19 +281,60 @@ class BiliBiliIE(BilibiliBaseIE):
             'thumbnail': r're:^https?://.*\.(jpg|jpeg|png)$',
         },
         'params': {'skip_download': True},
+    }, {
+        'note': 'video redirects to festival page',
+        'url': 'https://www.bilibili.com/video/BV1wP4y1P72h',
+        'info_dict': {
+            'id': 'BV1wP4y1P72h',
+            'ext': 'mp4',
+            'title': '牛虎年相交之际，一首传统民族打击乐《牛斗虎》祝大家新春快乐，虎年大吉！【bilibili音乐虎闹新春】',
+            'timestamp': 1643947497,
+            'upload_date': '20220204',
+            'description': 'md5:8681a0d4d2c06b4ae27e59c8080a7fe6',
+            'uploader': '叨叨冯聊音乐',
+            'duration': 246.719,
+            'uploader_id': '528182630',
+            'view_count': int,
+            'like_count': int,
+            'thumbnail': r're:^https?://.*\.(jpg|jpeg|png)$',
+        },
+        'params': {'skip_download': True},
+    }, {
+        'note': 'newer festival video',
+        'url': 'https://www.bilibili.com/festival/2023honkaiimpact3gala?bvid=BV1ay4y1d77f',
+        'info_dict': {
+            'id': 'BV1ay4y1d77f',
+            'ext': 'mp4',
+            'title': '【崩坏3新春剧场】为特别的你送上祝福！',
+            'timestamp': 1674273600,
+            'upload_date': '20230121',
+            'description': 'md5:58af66d15c6a0122dc30c8adfd828dd8',
+            'uploader': '果蝇轰',
+            'duration': 1111.722,
+            'uploader_id': '8469526',
+            'view_count': int,
+            'like_count': int,
+            'thumbnail': r're:^https?://.*\.(jpg|jpeg|png)$',
+        },
+        'params': {'skip_download': True},
     }]
 
     def _real_extract(self, url):
         video_id = self._match_id(url)
         webpage = self._download_webpage(url, video_id)
         initial_state = self._search_json(r'window\.__INITIAL_STATE__\s*=', webpage, 'initial state', video_id)
-        play_info = self._search_json(r'window\.__playinfo__\s*=', webpage, 'play info', video_id)['data']
 
-        video_data = initial_state['videoData']
+        is_festival = 'videoData' not in initial_state
+        if is_festival:
+            video_data = initial_state['videoInfo']
+        else:
+            play_info = self._search_json(r'window\.__playinfo__\s*=', webpage, 'play info', video_id)['data']
+            video_data = initial_state['videoData']
+
         video_id, title = video_data['bvid'], video_data.get('title')
 
         # Bilibili anthologies are similar to playlists but all videos share the same video ID as the anthology itself.
-        page_list_json = traverse_obj(
+        page_list_json = not is_festival and traverse_obj(
             self._download_json(
                 'https://api.bilibili.com/x/player/pagelist', video_id,
                 fatal=False, query={'bvid': video_id, 'jsonp': 'jsonp'},
@@ -316,20 +357,39 @@ def _real_extract(self, url):
 
         cid = traverse_obj(video_data, ('pages', part_id - 1, 'cid')) if part_id else video_data.get('cid')
 
+        festival_info = {}
+        if is_festival:
+            play_info = self._download_json(
+                'https://api.bilibili.com/x/player/playurl', video_id,
+                query={'bvid': video_id, 'cid': cid, 'fnval': 4048},
+                note='Extracting festival video formats')['data']
+
+            festival_info = traverse_obj(initial_state, {
+                'uploader': ('videoInfo', 'upName'),
+                'uploader_id': ('videoInfo', 'upMid', {str_or_none}),
+                'like_count': ('videoStatus', 'like', {int_or_none}),
+                'thumbnail': ('sectionEpisodes', lambda _, v: v['bvid'] == video_id, 'cover'),
+            }, get_all=False)
+
         return {
+            **traverse_obj(initial_state, {
+                'uploader': ('upData', 'name'),
+                'uploader_id': ('upData', 'mid', {str_or_none}),
+                'like_count': ('videoData', 'stat', 'like', {int_or_none}),
+                'tags': ('tags', ..., 'tag_name'),
+                'thumbnail': ('videoData', 'pic', {url_or_none}),
+            }),
+            **festival_info,
+            **traverse_obj(video_data, {
+                'description': 'desc',
+                'timestamp': ('pubdate', {int_or_none}),
+                'view_count': (('viewCount', ('stat', 'view')), {int_or_none}),
+                'comment_count': ('stat', 'reply', {int_or_none}),
+            }, get_all=False),
             'id': f'{video_id}{format_field(part_id, None, "_p%d")}',
             'formats': self.extract_formats(play_info),
             '_old_archive_ids': [make_archive_id(self, old_video_id)] if old_video_id else None,
             'title': title,
-            'description': traverse_obj(initial_state, ('videoData', 'desc')),
-            'view_count': traverse_obj(initial_state, ('videoData', 'stat', 'view')),
-            'uploader': traverse_obj(initial_state, ('upData', 'name')),
-            'uploader_id': traverse_obj(initial_state, ('upData', 'mid')),
-            'like_count': traverse_obj(initial_state, ('videoData', 'stat', 'like')),
-            'comment_count': traverse_obj(initial_state, ('videoData', 'stat', 'reply')),
-            'tags': traverse_obj(initial_state, ('tags', ..., 'tag_name')),
-            'thumbnail': traverse_obj(initial_state, ('videoData', 'pic')),
-            'timestamp': traverse_obj(initial_state, ('videoData', 'pubdate')),
             'duration': float_or_none(play_info.get('timelength'), scale=1000),
             'chapters': self._get_chapters(aid, cid),
             'subtitles': self.extract_subtitles(video_id, aid, cid),


Test patch: