Repository: yt-dlp/yt-dlp

Issue:
bilibili: fix for geo verification error
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting that yt-dlp is broken on a **supported** site
- [X] I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

China

### Provide a description that is worded well enough to be understood

I am using `--geo-verification-proxy` with Unblock Youku's proxy for verification. This proxy doesn't allow downloading, so using `--proxy` is not an option.

I am getting the `GeoRestrictedError` on line 672: https://github.com/yt-dlp/yt-dlp/blob/89f535e2656964b4061c25a7739d4d6ba0a30568/yt_dlp/extractor/bilibili.py#L669-L676

However, if I enable the geo_verification_headers for the first request (`self._download_webpage`), then it works.
Here's a diff of the change:
```diff
@@ -666,14 +666,14 @@ class BiliBiliBangumiIE(BilibiliBaseIE):
 
     def _real_extract(self, url):
         episode_id = self._match_id(url)
-        webpage = self._download_webpage(url, episode_id)
+        headers = {'Referer': url, **self.geo_verification_headers()}
+        webpage = self._download_webpage(url, episode_id, headers=headers)
 
         if '您所在的地区无法观看本片' in webpage:
             raise GeoRestrictedError('This video is restricted')
         elif '正在观看预览，大会员免费看全片' in webpage:
             self.raise_login_required('This video is for premium members only')
 
-        headers = {'Referer': url, **self.geo_verification_headers()}
         play_info = self._download_json(
             'https://api.bilibili.com/pgc/player/web/v2/playurl', episode_id,
             'Extracting episode', query={'fnval': '4048', 'ep_id': episode_id},

```

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', '--geo-verification-proxy', 'http://secure.unblockpro.app:8080/', 'https://www.bilibili.com/bangumi/
play/ep173286']
[debug] Encodings: locale cp936, fs utf-8, pref cp936, out utf-8, error utf-8, screen utf-8 
[debug] yt-dlp version stable@2024.04.09 from yt-dlp/yt-dlp [ff0779267] (win_exe)
[debug] Lazy loading extractors is disabled
[debug] Python 3.12.3 (CPython AMD64 64bit) - Windows-11-10.0.22631-SP0 (OpenSSL 3.0.13 30 Jan 2024) 
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3
-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Plugin directories: ['C:\\Users\\jackl\\AppData\\Roaming\\yt-dlp\\plugins\\yt-dlp-ChromeCookieUnlock\\yt_dlp_plugins']
[debug] Loaded 1810 extractors 
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.04.09 from yt-dlp/yt-dlp 
yt-dlp is up to date (stable@2024.04.09 from yt-dlp/yt-dlp)
[BiliBiliBangumi] Extracting URL: https://www.bilibili.com/bangumi/play/ep173286 
[BiliBiliBangumi] 173286: Downloading webpage
ERROR: [BiliBiliBangumi] 173286: This video is restricted 
You might want to use a VPN or a proxy server (with --proxy) to workaround.
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1606, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "yt_dlp\YoutubeDL.py", line 1741, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "yt_dlp\extractor\common.py", line 735, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "yt_dlp\extractor\bilibili.py", line 672, in _real_extract
    raise GeoRestrictedError('This video is restricted')
yt_dlp.utils.GeoRestrictedError: [BiliBiliBangumi] 173286: This video is restricted
```



Discussion:
After the patch, the download works fine:
```
[debug] Command-line config: ['-vU', '--geo-verification-proxy', 'http://secure.unblockpro.app:8080/', 'https://www.bilibili.com/bangumi/
play/ep173286']
[debug] Encodings: locale cp936, fs utf-8, pref cp936, out utf-8, error utf-8, screen utf-8 
[debug] yt-dlp version stable@2024.04.09 from yt-dlp/yt-dlp [ff0779267] (win_exe)
[debug] Lazy loading extractors is disabled
[debug] Python 3.12.3 (CPython AMD64 64bit) - Windows-11-10.0.22631-SP0 (OpenSSL 3.0.13 30 Jan 2024) 
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3
-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Plugin directories: ['C:\\Users\\jackl\\AppData\\Roaming\\yt-dlp\\plugins\\yt-dlp-ChromeCookieUnlock\\yt_dlp_plugins']
[debug] Loaded 1810 extractors 
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.04.09 from yt-dlp/yt-dlp 
yt-dlp is up to date (stable@2024.04.09 from yt-dlp/yt-dlp)
[BiliBiliBangumi] Extracting URL: https://www.bilibili.com/bangumi/play/ep173286 
[BiliBiliBangumi] 173286: Downloading webpage
[BiliBiliBangumi] 173286: Extracting episode 
[BiliBiliBangumi] Format(s) 1080P 高码率, 1080P 高清, 720P 高清, 4K 超清 are missing; you have to login or become premium member to downl
oad them. Use --cookies-from-browser or --cookies for the authentication. See  https://github.com/yt-dlp/yt-dlp/wiki/FAQ#how-do-i-pass-co
okies-to-yt-dlp  for how to manually pass cookies
[BiliBiliBangumi] 173286: Get episode details
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto,
 vext, aext, hasaud, source, id
[debug] Default format spec: best/bestvideo+bestaudio
[info] 173286: Downloading 1 format(s): 100023+30280
WARNING: You have requested merging of multiple formats but ffmpeg is not installed. The formats won't be merged
[debug] Invoking http downloader on "https://upos-sz-mirrorali.bilivideo.com/upgcxcode/87/20/29662087/29662087_p1-1-100023.m4s?e=ig8euxZM
2rNcNbdlhoNvNC8BqJIzNbfqXBvEqxTEto8BTrNvN0GvT90W5JZMkX_YN0MvXg8gNEV4NC8xNEV4N03eN0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8BT
rNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&uipk=5&nbs=1&deadline=1714176250&gen=playurlv2&os=alibv&oi=2014891429&trid=c6e1e3a82c8a41
c0a5395adf58a2dc28p&mid=0&platform=pc&upsig=6aca74d2d5ed5b9b34446089354335ee&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,mid,platform&bvc=
vod&nettype=0&orderid=0,3&buvid=FCD7BDC5-F4EB-BB19-4C56-CEC0BC22E4AE48957infoc&build=0&f=p_0_0&agrr=1&bw=31891&logo=80000000"
[download] Got error: ('Unable to connect to proxy', OSError('Tunnel connection failed: 403 Forbidden')). Retrying (1/10)... 
[debug] File locking is not supported. Proceeding without locking 
[download] Destination: 1 「爱」与自动手记人偶 [173286].f100023.mp4
[download] 100% of   43.77MiB in 00:01:48 at 412.55KiB/s
[debug] Invoking http downloader on "https://upos-sz-mirrorcoso1.bilivideo.com/upgcxcode/87/20/29662087/29662087_p1-1-30280.m4s?e=ig8euxZ
M2rNcNbdlhoNvNC8BqJIzNbfqXBvEqxTEto8BTrNvN0GvT90W5JZMkX_YN0MvXg8gNEV4NC8xNEV4N03eN0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8B
TrNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&uipk=5&nbs=1&deadline=1714176250&gen=playurlv2&os=coso1bv&oi=2014891429&trid=c6e1e3a82c8
a41c0a5395adf58a2dc28p&mid=0&platform=pc&upsig=f6aa54b3f865e84f5839f5a3b921ac0a&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,mid,platform&b
vc=vod&nettype=0&orderid=0,3&buvid=FCD7BDC5-F4EB-BB19-4C56-CEC0BC22E4AE48957infoc&build=0&f=p_0_0&agrr=1&bw=24215&logo=80000000"
[download] Got error: ('Unable to connect to proxy', OSError('Tunnel connection failed: 403 Forbidden')). Retrying (1/10)... 
[download] Destination: 1 「爱」与自动手记人偶 [173286].f30280.m4a 
[download] 100% of   33.26MiB in 00:00:09 at 3.58MiB/s
```

Resolving patch:
diff --git a/yt_dlp/extractor/bilibili.py b/yt_dlp/extractor/bilibili.py
--- a/yt_dlp/extractor/bilibili.py
+++ b/yt_dlp/extractor/bilibili.py
@@ -93,11 +93,11 @@ def extract_formats(self, play_info):
 
         return formats
 
-    def _download_playinfo(self, video_id, cid):
+    def _download_playinfo(self, video_id, cid, headers=None):
         return self._download_json(
             'https://api.bilibili.com/x/player/playurl', video_id,
             query={'bvid': video_id, 'cid': cid, 'fnval': 4048},
-            note=f'Downloading video formats for cid {cid}')['data']
+            note=f'Downloading video formats for cid {cid}', headers=headers)['data']
 
     def json2srt(self, json_data):
         srt_data = ''
@@ -493,7 +493,8 @@ class BiliBiliIE(BilibiliBaseIE):
 
     def _real_extract(self, url):
         video_id = self._match_id(url)
-        webpage, urlh = self._download_webpage_handle(url, video_id)
+        headers = self.geo_verification_headers()
+        webpage, urlh = self._download_webpage_handle(url, video_id, headers=headers)
         if not self._match_valid_url(urlh.url):
             return self.url_result(urlh.url)
 
@@ -531,7 +532,7 @@ def _real_extract(self, url):
             self._download_json(
                 'https://api.bilibili.com/x/player/pagelist', video_id,
                 fatal=False, query={'bvid': video_id, 'jsonp': 'jsonp'},
-                note='Extracting videos in anthology'),
+                note='Extracting videos in anthology', headers=headers),
             'data', expected_type=list) or []
         is_anthology = len(page_list_json) > 1
 
@@ -552,7 +553,7 @@ def _real_extract(self, url):
 
         festival_info = {}
         if is_festival:
-            play_info = self._download_playinfo(video_id, cid)
+            play_info = self._download_playinfo(video_id, cid, headers=headers)
 
             festival_info = traverse_obj(initial_state, {
                 'uploader': ('videoInfo', 'upName'),
@@ -666,14 +667,15 @@ class BiliBiliBangumiIE(BilibiliBaseIE):
 
     def _real_extract(self, url):
         episode_id = self._match_id(url)
-        webpage = self._download_webpage(url, episode_id)
+        headers = self.geo_verification_headers()
+        webpage = self._download_webpage(url, episode_id, headers=headers)
 
         if '您所在的地区无法观看本片' in webpage:
             raise GeoRestrictedError('This video is restricted')
         elif '正在观看预览，大会员免费看全片' in webpage:
             self.raise_login_required('This video is for premium members only')
 
-        headers = {'Referer': url, **self.geo_verification_headers()}
+        headers['Referer'] = url
         play_info = self._download_json(
             'https://api.bilibili.com/pgc/player/web/v2/playurl', episode_id,
             'Extracting episode', query={'fnval': '4048', 'ep_id': episode_id},
@@ -724,7 +726,7 @@ def _real_extract(self, url):
             'duration': float_or_none(play_info.get('timelength'), scale=1000),
             'subtitles': self.extract_subtitles(episode_id, episode_info.get('cid'), aid=aid),
             '__post_extractor': self.extract_comments(aid),
-            'http_headers': headers,
+            'http_headers': {'Referer': url},
         }
 
 


Test patch: