Repository: yt-dlp/yt-dlp

Issue:
Crunchyroll error 403: Forbidden
### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2021.12.01**. ([update instructions](https://github.com/yt-dlp/yt-dlp#update))
- [X] I've checked that all provided URLs are alive and playable in a browser
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [x] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Australia

### Description

Whenever I try to download from Crunchyroll, it always puts out `403: Forbidden` errors, even when using user agent and cookie file. This has not been happening to me until recently, so I think it may be a change with the site. Before I could just download without using a user agent or cookie file.

Although, I can use `wget` to download the webpages from the site, by just using a user agent, so it could also be an issue within yt-dlp itself.

### Verbose log

```shell
$ yt-dlp -Uv --user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0' --cookies '~/cookies.txt' 'https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083'
[debug] Command-line config: ['-Uv', '--user-agent', 'Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0', '--cookies', '~/cookies.txt', 'https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2021.12.01 [91f071af6]
[debug] Python version 3.9.9 (CPython 64bit) - Linux-5.15.6-arch2-1-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2021.12.01, Current version: 2021.12.01
yt-dlp is up to date (2021.12.01)
[debug] [crunchyroll] Extracting URL: https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083
[crunchyroll] 652083: Downloading webpage
ERROR: [crunchyroll] 652083: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output. (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "/usr/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 715, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 3385, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.9/urllib/request.py", line 555, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 747, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
```



Discussion:
Same issue in NA.
[debug] Encodings: locale cp1252, fs utf-8, out cp1252 (No ANSI), err cp1252 (No ANSI), pref cp1252
[debug] yt-dlp version 2021.12.01 [91f071a] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: none
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[crunchyroll:beta] Downloading login page
ERROR: [crunchyroll:beta] the-events-of-winter: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please 
report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with 
the --verbose flag and include its complete output. (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp 
. Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "yt_dlp\extractor\common.py", line 715, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3385, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 569, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 649, in http_error_default
Try without any options, @epikgamer696969. It works for me:
```
➤ yt-dlp -v https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083
[debug] Command-line config: ['-v', 'https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2021.12.01 [91f071af6] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: dd0228ce1
[debug] Python version 3.10.0 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg N-103892-g71f2a9a2e5-20210927 (setts,fdk), ffprobe N-103892-g71f2a9a2e5-20210927, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [crunchyroll] Extracting URL: https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083
[crunchyroll] 652083: Downloading webpage
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-arME information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-enUS information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-deDE information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-ptBR information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-itIT information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-esLA information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-esES information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-frFR information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[crunchyroll] 652083: Downloading media info
WARNING: [crunchyroll] Unable to download XML: HTTP Error 404: Not Found
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 652083: Downloading 1 format(s): adaptive_hls-audio-jaJP-8115-1
[debug] Invoking downloader on "https://pl.crunchyroll.com/evs1/0e514783c22ff07da34477961c0bd228/assets/59jvz8ht2mxuf8x_1909827.mp4/index-v1-a1.m3u8?Expires=1640107009&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9wbC5jcnVuY2h5cm9sbC5jb20vZXZzMS8wZTUxNDc4M2MyMmZmMDdkYTM0NDc3OTYxYzBiZDIyOC9hc3NldHMvNTlqdno4aHQybXh1Zjh4XyoubTN1OD8qIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNjQwMTA3MDA5fX19XX0_&Signature=pFeb6SD2ABmethINdJdrUDPwIaL3AluNkKXVS2y76rErHCHOmHBhI9wsIF7m3Dg-U4SQXAj-h~lUx8kHiiHSq~ym-48u-nDQP1FXM-xNML7QeM-yMJhhwWTTZZRjhtzseHVV7dLY53S63cW7kRTsYLLmCRP6PwyrWsj8~AwC3678CRa2wvMaqebuEpeZ8Nm2k27rPw2fD6KFox7PfDXLUz66E4mOzx5doSfI1f7TROGoESUeeD2SxQJceJ7kznjz2KcJpsq-507nudLA7RD-HotcuchLUASlfwWK-CVl3agTO5RAddLmPtqOMhHp242uERh2RJbf56p6kj2fMfbAeA__&Key-Pair-Id=APKAJMWSQ5S7ZB3MF5VA&cdn=fy-prod"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 716
[download] Destination: JoJo's Bizarre Adventure Episode 2 – A Letter From the Past [652083].mp4
[download]   3.9% of ~1.35GiB at  3.11MiB/s ETA 09:11 (frag 28/716)
ERROR: Interrupted by user
```

@darnov01 Separate issue with beta crunchyroll https://github.com/yt-dlp/yt-dlp/issues/1911
Sorry but it still doesn't work.
```
$ yt-dlp -v https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083
[debug] Command-line config: ['-v', 'https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2021.12.01 [91f071af6]
[debug] Python version 3.9.9 (CPython 64bit) - Linux-5.15.6-arch2-1-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [crunchyroll] Extracting URL: https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083
[crunchyroll] 652083: Downloading webpage
ERROR: [crunchyroll] 652083: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output. (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "/usr/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 715, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/lib/python3.9/site-packages/yt_dlp/YoutubeDL.py", line 3385, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.9/urllib/request.py", line 555, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 747, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
```

I even tried using a vpn, and tried on a different machine, still the same output.
I'm also having problems when trying to stream from crunchyroll. I noticed the problem today.

Here is what I get when I try to use mpv (I have a symlink youtube-dl > yt-dlp):

```
$ mpv https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083
[ffmpeg] https: HTTP error 403 Forbidden
Failed to open https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083.
[ytdl_hook] 
[ytdl_hook] stack traceback:
[ytdl_hook] 	@ytdl_hook.lua:566: in function 'run_ytdl_hook'
[ytdl_hook] 	@ytdl_hook.lua:725: in function 'fn'
[ytdl_hook] 	mp.defaults:544: in function 'handler'
[ytdl_hook] 	mp.defaults:486: in function 'call_event_handlers'
[ytdl_hook] 	mp.defaults:520: in function 'dispatch_events'
[ytdl_hook] 	mp.defaults:479: in function <mp.defaults:478>
[ytdl_hook] 	[C]: in ?
[ytdl_hook] 	[C]: in ?
[ytdl_hook] Lua error: @ytdl_hook.lua:566: attempt to concatenate local 'err' (a nil value)
client removed during hook handling


Exiting... (Errors when loading file)

```

When trying directly with yt-dlp:

```
yt-dlp -v -F https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083
[debug] Command-line config: ['-v', '-F', 'https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2021.12.01 [91f071af6] (zip)
[debug] Python version 3.9.2 (CPython 64bit) - Linux-5.10.0-10-amd64-x86_64-with-glibc2.31
[debug] exe versions: ffmpeg 4.3.3-0, ffprobe 4.3.3-0, rtmpdump 2.4
[debug] Optional libraries: keyring, sqlite
[debug] Proxy map: {}
[debug] [crunchyroll] Extracting URL: https://www.crunchyroll.com/jojos-bizarre-adventure/episode-2-a-letter-from-the-past-652083
[crunchyroll] 652083: Downloading webpage
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-frFR information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-arME information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-deDE information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-itIT information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-ptBR information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-esLA information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-enUS information
[crunchyroll] 652083: Downloading adaptive_hls-audio-jaJP-hardsub-esES information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[crunchyroll] 652083: Downloading media info
WARNING: [crunchyroll] Unable to download XML: HTTP Error 404: Not Found
[info] Available formats for 652083:
ID                                          EXT RESOLUTION FPS │   TBR PROTO  │ VCODEC        VBR ACODEC    ABR MORE INFO
[...]
```

Btw, I just tried without the -F parameter and it started downloading... I don't know why I can't stream it with mpv like I have been doing. Maybe my problem is unrelated?
@jvschiavo I think your problem is unrelated, because my error is it fails to even connect to the site AT ALL, even to scrape the webpage. It is very odd how I am the only one getting this error, even when using a vpn.
> @jvschiavo I think your problem is unrelated, because my error is it fails to even connect to the site AT ALL, even to scrape the webpage. It is very odd how I am the only one getting this error, even when using a vpn.

its cloudflare thing, i think they banned vpn
@vikkstar12 If they blocked my ip and the vpn's ip, then why can I still connect to the site in a browser?
I need help with this...

https://github.com/yt-dlp/yt-dlp/issues/3678

I exposed loads of useful information but the developer just closed my thread for "Duplicate" (with quotes, because maybe we are not suffering the same problem, in fact I'm not using any kind of VPN and the problem is still not identified for he and for me) without even offering an answer.

I don't think we can expect the developer to try to help us.

But the developer is the only one that can implement a fix at all...

The OP from this thread which was started on December of 2021 he got no solution. 

We are really alone with this issue?.

> @vikkstar12 If they blocked my ip and the vpn's ip, then why can I still connect to the site in a browser?

Hi folk. Same here (although I'm not behind any proxy or VPN). Did you managed to find a workaround for your problem about `HTTP Error 403`?. And in that case could you share the solution with us?. Thanks in advance.
>I don't think we can expect the developer to try to help us.

Well, no. If you look at the [major sites task list](https://github.com/yt-dlp/yt-dlp/projects/3#card-79586288), you'll see that the issue is recorded for future work. Speaking from the viewpoint of yt-dl, it's just that it's not widely reproducible, depends on black magic known only to a third party, and no easy and globally applicable solution has been found.
Thanks for the information, although I don't have much hope anyway because this thread is from 2021 and it doesn't seem like there has been a follow-up to the problem since that date. I understand there are much problems and reports and the devs can't have time to focus on every one of them, but... I consider this is a critical problem that stops the user from be able download anything from a specific site.

I can think of one way to reproduce the problem. Maybe it will reproduce the problem, or maybe it will not. But I think it will, and if i'm not wrong then it would be as simple as doing this:

1. Pick a single url from a **Crunchyroll** episode (eg. `https://www.crunchyroll.com/es-es/86-eighty-six/episode-1-undertaker-811078` )
2. Write a for loop in your favorite programming language to download the video file using **yt-dlp** (don't use special parameters like user agent or cookies) as many times as required until you get blocked and then getting error 403 all the time. This could last for a couple of hours until you start to get error 403 all the tries.

That is essentially what I did until I started to get error 403, but with **youtube-dl**. I mean that I used a for-loop to download one per one all the urls of the episodes from an anime serie from Crunchyroll, and then when I finished to download all the episodes and I attempted to start to download another serie, I noticed that I can't because I started to get error 403 with **youtube-dl**. And if I get the same error using **yt-dlp** on my very first attempt using this program, I'm sure it is because of the same cause that I'm getting that error on **youtube-dl**, and once I can manage to avoid that error 403 with **youtube-dl**, **yt-dlp** will work again too.

If helpful, this is the exact arguments that I used in my for loop in _batch-script_ with **youtube-dl**. I'm not sure if the same syntax applies with **yt-dlp** usage:

`"youtube-dl.exe" -f "bestvideo[height<=1080]+bestaudio/best[height<=1080]" --write-sub --sub-lang "esES,esLA,enUS" --sub-format "ass/srt/best" -o "%CD%\%%(title)s.%%(ext)s" "!URL!"`


> I exposed loads of useful information but the developer just closed my thread for "Duplicate" (with quotes

I closed it as duplicate, because **it is a duplicate**. In fact, it was supposed to be YOUR responsibility to search for duplicates before opening the issue. Be glad I pointed you to the issue of which it is a duplicate of! The "useful information" you presented is still there, it's not like I deleted it... Anyone can reference it anytime.

PS: The `cant-reproduce` label is there because it is not **reliably** reproducible. I cannot check the specific URL you gave because of geo-blocking, but I have never managed to reproduce this with URLs I can access. That ofc doesn't mean the issue doesn't exist. Just that any developers need to go the extra mile to first reproduce this before even attempting to fix it.

> because maybe we are not suffering the same problem, in fact I'm not using any kind of VPN and the problem is still not identified for he and for me) without even offering an answer.

It is the exact same issue. The only difference is that OP of this issue "blocked" due to VPN and you are blocked due to excessive downloading

> I don't think we can expect the developer to try to help us.
> But the developer is the only one that can implement a fix at all...

This is a FOSS project. Anyone can try to figure out a fix and open a PR. If you are expecting me to personally fix every issue in each of the 1500+ websites supported by yt-dlp in addition to maintaining the core project/reviewing PRs, prepare to be disappointed.
Thanks for answer.

> **you are blocked due to excessive downloading**

But why then I can still access Crunchyroll from my web-browser and from [CefSharp](https://cefsharp.github.io/)?. I have the same doubt as the author of this thread had, because we are talking about an alleged blocking that only seems to block **yt-dlp** / **youtube-dl** HTTP requests.

Anyways, and by your experience, could you tell me whether this block will be temporary if you know that?. Or it is forever?.

> If you are expecting me to personally fix every issue in each of the 1500+ websites supported by yt-dlp in addition to maintaining the core project/reviewing PRs, prepare to be disappointed.

Yes, as I said: 

> I understand there are much problems and reports and the devs can't have time to focus on every one of them, but... I consider this is a critical problem that stops the user from be able download anything from a specific site.

Don't worry. But maybe you can read and do something with [this comment ](https://github.com/ytdl-org/youtube-dl/issues/30926#issuecomment-1120476804)where I shared two HAR files in case that this can help to solve this 403 error issue?.

I say something in that comment that I also apply to my thoughts about **yt-dlp**:

> It is possibly that when I attempt to use youtube-dl to access Crunchyroll it throws error 403 because Cloudflare is trying to request to solve a reCaptcha and maybe youtube-dl is not aware and not handling this request in any way and it just exit when receiving the 403 error response?.
> 
> The cookie file with the --cookies parameter is supposed to be enough to avoid this, I'm right?, because I solved the reCaptcha in my browser from which I generated the cookies file, but clearly the cookies file is not enough for me, and I wonder if the reason for this is because, as you or someone else said before, this blocking occurs before even looking at the cookies?. But in my web-browser it does not occur the blocking.

Cheers.

The only solution I found for my specific problem is: wait 24 hours

I shared some suggestions to affected users and a batch-script code example to try to avoid this block in the future, [here](https://github.com/yt-dlp/yt-dlp/issues/3678#issuecomment-1121481852).

Thanks for read.
> The only solution I found for my specific problem is: wait 24 hours
> 
> I shared some suggestions to affected users and a batch-script code example to try to avoid this block in the future, [here](https://github.com/yt-dlp/yt-dlp/issues/3678#issuecomment-1121481852).
> 
> Thanks for read.

I think there has to be more to this than a simple rate limit. I'm pretty sure my ISP provided IP is not shared, or if it is it's a very small pool. Yet I am seeing this 403 problem consistently on first attempt, for example for the Crunhyroll URL provided in the opening message of this issue. I have tried waiting several days between requests. I don't do bulk downloads at all.
The first time got 403, but so far, my experiments have gone this way:
* I tried checking WIth my credentials (using -u -p) with VPN (different IP) with the same 403 error
* I tried on my regular IP without my credentials with success

So far, my conclusion is that either premium/higher quality data access is bugged or some account lock, which I hope is not permanent.

EDIT.

A further investigation showed something interesting - when you try getting JSON without connecting, yt-dlp `Redirecting to non-beta site - https://www.crunchyroll.com/` so, it is also possible, that issue might be related to getting data from beta (with error) or non-beta (fine) site/backend
> EDIT.
> 
> A further investigation showed something interesting - when you try getting JSON without connecting, yt-dlp `Redirecting to non-beta site - https://www.crunchyroll.com/` so, it is also possible, that issue might be related to getting data from beta (with error) or non-beta (fine) site/backend

I've been able to recreate the same issue. Also, attempting to use cookies or credentials against the non-beta URL it attempts to redirect you to the beta site and you get the 403 error.

```bash
[debug] Command-line config: ['--verbose', '-F', '--username', 'PRIVATE', '--password', 'PRIVATE', 'https://www.crunchyroll.com/spy-x-family/operation-strix-842454']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2022.04.08 [7884ade65] (zip)
[debug] Python version 3.8.10 (CPython 64bit) - Linux-5.4.0-109-generic-x86_64-with-glibc2.29
[debug] Checking exe version: ffprobe -bsfs
[debug] Checking exe version: ffmpeg -bsfs
[debug] exe versions: ffmpeg 4.2.4, ffprobe N-103927-g68815d6791
[debug] Optional libraries: certifi-2019.11.28, pycrypto-2.6.1, secretstorage-2.3.1, sqlite3-2.6.0
[debug] Proxy map: {}
[crunchyroll] Getting session id
[crunchyroll] Logging in
[debug] [crunchyroll] Extracting URL: https://www.crunchyroll.com/spy-x-family/operation-strix-842454
[crunchyroll] 842454: Downloading webpage
[crunchyroll] 842454: Redirected to beta site - https://beta.crunchyroll.com/watch/GMKUX838J/operation-strix
[debug] [crunchyroll:beta] Extracting URL: https://beta.crunchyroll.com/watch/GMKUX838J/operation-strix
[crunchyroll:beta] Retrieving main page
[crunchyroll:beta] Authenticating with cookie
[crunchyroll:beta] Retrieving signed policy
[crunchyroll:beta] operation-strix: Retrieving episode metadata
ERROR: [crunchyroll:beta] GMKUX838J: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/bin/youtube-dl/yt_dlp/extractor/common.py", line 642, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/yt_dlp/extractor/crunchyroll.py", line 815, in _real_extract
    episode_response = self._download_json(
  File "/usr/local/bin/youtube-dl/yt_dlp/extractor/common.py", line 1030, in _download_json
    res = self._download_json_handle(
  File "/usr/local/bin/youtube-dl/yt_dlp/extractor/common.py", line 1009, in _download_json_handle
    res = self._download_webpage_handle(
  File "/usr/local/bin/youtube-dl/yt_dlp/extractor/common.py", line 801, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/yt_dlp/extractor/common.py", line 786, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/usr/local/bin/youtube-dl/yt_dlp/extractor/common.py", line 768, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/yt_dlp/YoutubeDL.py", line 3600, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

```
I believe this most recent issue to pop up may not be the same as the initial one. According to another project I follow, Crunchyroll has made changes to their API and added a new subdomain. Hopefully this information is helpful.
I'm getting this _HTTP Error 403: Forbidden_ for the first time ever today. I had no problems yesterday. And, I was only trying to DL a single episode today. Which was definitely more then 24 hours after my last DL yesterday. I hope this is a fixable issue.

**EDIT**

I also get the same error when simply getting _list formats._

F:\YT>yt-dlp.exe --verbose --cookies cookies.txt "https://beta.crunchyroll.com/watch/GJWU25P4E/the-beginning-of-winter" -F
[debug] Command-line config: ['--verbose', '--cookies', 'cookies.txt', 'https://beta.crunchyroll.com/watch/GJWU25P4E/the-beginning-of-winter', '-F']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2022.04.08 [7884ade] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 2022-02-14-git-59c647bcf3-full_build-www.gyan.dev (setts), ffprobe 2022-02-14-git-59c647bcf3-full_build-www.gyan.dev
[debug] Optional libraries: brotli, certifi, Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [crunchyroll:beta] Extracting URL: https://beta.crunchyroll.com/watch/GJWU25P4E/the-beginning-of-winter
[crunchyroll:beta] Retrieving main page
[crunchyroll:beta] Authenticating with cookie
[crunchyroll:beta] Retrieving signed policy
[crunchyroll:beta] the-beginning-of-winter: Retrieving episode metadata
ERROR: [crunchyroll:beta] GJWU25P4E: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 641, in extract
  File "yt_dlp\extractor\crunchyroll.py", line 818, in _real_extract
  File "yt_dlp\extractor\common.py", line 1029, in _download_json
  File "yt_dlp\extractor\common.py", line 1008, in _download_json_handle
  File "yt_dlp\extractor\common.py", line 800, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 785, in _request_webpage

  File "yt_dlp\extractor\common.py", line 767, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3601, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 569, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 403: Forbidden
So, it seems I know where is the problem. Now I only need to figure out how to submit the fix here properly, but before in plain text:

in clurncyroll.py in extractors on lines 731-736 (where we are getting params for the auth)
`            bucket = policy_response['cms']['bucket']
            params = {
                'Policy': policy_response['cms']['policy'],
                'Signature': policy_response['cms']['signature'],
                'Key-Pair-Id': policy_response['cms']['key_pair_id']
            }`
replay json changed (or moved) to beta state (they have 2 versions now there), and if that lines changed to:
`            bucket = policy_response['cms_beta']['bucket']
            params = {
                'Policy': policy_response['cms_beta']['policy'],
                'Signature': policy_response['cms_beta']['signature'],
                'Key-Pair-Id': policy_response['cms_beta']['key_pair_id']
            }`
All works fine, as far as I did a few tests.
OP had reported this issue since December and even ElektroStudios's issue was only temporary. Unless Crunchyroll was A/B testing the change, there may be 2 separate issues at play here.

@Burve For the fix, you'll want to make the extractor look at both keys using `traverse_obj`
Hmm, this look not exactly as original code, but I assume more correct:

            bucket = traverse_obj(policy_response, ('cms_beta', 'bucket'))
            params = {
                'Policy': traverse_obj(policy_response, ('cms_beta', 'policy')),
                'Signature': traverse_obj(policy_response, ('cms_beta', 'signature')),
                'Key-Pair-Id': traverse_obj(policy_response, ('cms_beta', 'key_pair_id'))
            }
> bucket = traverse_obj(policy_response, ('cms_beta', 'bucket'))
>         params = {
>             'Policy': traverse_obj(policy_response, ('cms_beta', 'policy')),
>             'Signature': traverse_obj(policy_response, ('cms_beta', 'signature')),
>             'Key-Pair-Id': traverse_obj(policy_response, ('cms_beta', 'key_pair_id'))
>         }

I applied this manually and it appears to be working. Hopefully the PR will be in soon :)
I can also confirm the fix mentioned above by Burve and krandor solves this issue for me.

Edit: see comments below & disregard the rest of this comment.

<details>
<summary>
<strike>For convenience, here's a git diff against release tag `2022.05.18` for the change:</strike>

</summary>

```diff
diff --git a/yt_dlp/extractor/crunchyroll.py b/yt_dlp/extractor/crunchyroll.py
index bb1dbbaad..42c72fd29 100644
--- a/yt_dlp/extractor/crunchyroll.py
+++ b/yt_dlp/extractor/crunchyroll.py
@@ -728,11 +728,11 @@ def _get_params(self, lang):
                 headers={
                     'Authorization': auth_response['token_type'] + ' ' + auth_response['access_token']
                 })
-            bucket = policy_response['cms']['bucket']
+            bucket = traverse_obj(policy_response, ('cms_beta', 'bucket'))
             params = {
-                'Policy': policy_response['cms']['policy'],
-                'Signature': policy_response['cms']['signature'],
-                'Key-Pair-Id': policy_response['cms']['key_pair_id']
+                'Policy': traverse_obj(policy_response, ('cms_beta', 'policy')),
+                'Signature': traverse_obj(policy_response, ('cms_beta', 'signature')),
+                'Key-Pair-Id': traverse_obj(policy_response, ('cms_beta', 'key_pair_id'))
             }
             locale = traverse_obj(initial_state, ('localization', 'locale'))
             if locale:
```
</details>
```py
        cms = traverse_obj(policy_response, 'cms_beta', 'cms')
        bucket = cms['bucket']
        params = {
            'Policy': cms['policy'],
            'Signature': cms['signature'],
            'Key-Pair-Id': cms['key_pair_id'],
        }
```

This is what I meant. This way, if `cms_beta` is not found, `cms` will be used
> ```python
>         cms = traverse_obj(policy_response, 'cms_beta', 'cms')
>         bucket = cms['bucket']
>         params = {
>             'Policy': cms['policy'],
>             'Signature': cms['signature'],
>             'Key-Pair-Id': cms['key_pair_id'],
>         }
> ```
> 
> This is what I meant. This way, if `cms_beta` is not found, `cms` will be used

This was my original assumption, but when I tried it I got None as a result for the cms. And then checking how `locale = traverse_obj(initial_state, ('localization', 'locale'))` works and what was inside `initial_state` I understood, that `traverse_obj` might be a fancy way to get nested elements even if they deep inside
`traverse_obj(O, (a, b), (c, d))` ~= `O[a][b] or O[c][d]`
and 
`traverse_obj(O, a, b)` is a shortened form of `traverse_obj(O, (a,), (b,))` ~= `O[a] or O[b]`

See `help(yt_dlp.utils.traverse_obj)` for details of the syntax
oh, I see my issue from yesterday, I was looking at
`locale = traverse_obj(initial_state, ('localization', 'locale'))`
and then I assumed to do this
`cms = traverse_obj(policy_response, ('cms_beta', 'cms'))`
and then the rest as you said here to go from cms, but that had no sense, hense None, and correct was
`cms = traverse_obj(policy_response, 'cms_beta', 'cms')`


Resolving patch:
diff --git a/yt_dlp/extractor/crunchyroll.py b/yt_dlp/extractor/crunchyroll.py
--- a/yt_dlp/extractor/crunchyroll.py
+++ b/yt_dlp/extractor/crunchyroll.py
@@ -728,11 +728,12 @@ def _get_params(self, lang):
                 headers={
                     'Authorization': auth_response['token_type'] + ' ' + auth_response['access_token']
                 })
-            bucket = policy_response['cms']['bucket']
+            cms = traverse_obj(policy_response, 'cms_beta', 'cms')
+            bucket = cms['bucket']
             params = {
-                'Policy': policy_response['cms']['policy'],
-                'Signature': policy_response['cms']['signature'],
-                'Key-Pair-Id': policy_response['cms']['key_pair_id']
+                'Policy': cms['policy'],
+                'Signature': cms['signature'],
+                'Key-Pair-Id': cms['key_pair_id']
             }
             locale = traverse_obj(initial_state, ('localization', 'locale'))
             if locale:


Test patch: