Repository: yt-dlp/yt-dlp

Issue:
[ARD:mediathek] individual videos and series download not working (Unable to download JSON metadata: HTTP Error 410: Gone)
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting that yt-dlp is broken on a **supported** site
- [X] I've verified that I'm running yt-dlp version **2023.07.06** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Germany

### Provide a description that is worded well enough to be understood

Downloading other videos and shows from the ARD Mediathek works as intended. [This](https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ?isChildContent) is the only series I found that causes this error for both downloading the whole playlist and individual videos. There was a similar [issue](https://github.com/yt-dlp/yt-dlp/issues/6784), but the provided solution doesn't work here, so I opened a new one.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', 'https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ?isChildContent']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.07.06 [b532a3481] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19042-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 6.0-full_build-www.gyan.dev (setts), ffprobe 6.0-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.07.06, Current version: stable@2023.07.06
Current Build Hash: 5ff3e702171a50175c34397494e2d18ce35d771c2110b1e59bd173ec2fb352aa
yt-dlp is up to date (stable@2023.07.06)
[debug] Using fake IP 53.150.139.24 (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ?isChildContent
[ARD:mediathek] Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ: Downloading webpage
[ARD:mediathek] Downloading media JSON
ERROR: [ARD:mediathek] None: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 710, in extract
  File "yt_dlp\extractor\ard.py", line 275, in _real_extract
  File "yt_dlp\extractor\ard.py", line 28, in _extract_media_info
  File "yt_dlp\extractor\common.py", line 1064, in download_content
  File "yt_dlp\extractor\common.py", line 1028, in download_handle
  File "yt_dlp\extractor\common.py", line 898, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 855, in _request_webpage

  File "yt_dlp\extractor\common.py", line 837, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3863, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 569, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone
```

[ARD:mediathek] individual videos and series download not working (Unable to download JSON metadata: HTTP Error 410: Gone)
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting that yt-dlp is broken on a **supported** site
- [X] I've verified that I'm running yt-dlp version **2023.07.06** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Germany

### Provide a description that is worded well enough to be understood

Downloading other videos and shows from the ARD Mediathek works as intended. [This](https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ?isChildContent) is the only series I found that causes this error for both downloading the whole playlist and individual videos. There was a similar [issue](https://github.com/yt-dlp/yt-dlp/issues/6784), but the provided solution doesn't work here, so I opened a new one.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', 'https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ?isChildContent']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.07.06 [b532a3481] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19042-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 6.0-full_build-www.gyan.dev (setts), ffprobe 6.0-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.07.06, Current version: stable@2023.07.06
Current Build Hash: 5ff3e702171a50175c34397494e2d18ce35d771c2110b1e59bd173ec2fb352aa
yt-dlp is up to date (stable@2023.07.06)
[debug] Using fake IP 53.150.139.24 (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ?isChildContent
[ARD:mediathek] Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ: Downloading webpage
[ARD:mediathek] Downloading media JSON
ERROR: [ARD:mediathek] None: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 710, in extract
  File "yt_dlp\extractor\ard.py", line 275, in _real_extract
  File "yt_dlp\extractor\ard.py", line 28, in _extract_media_info
  File "yt_dlp\extractor\common.py", line 1064, in download_content
  File "yt_dlp\extractor\common.py", line 1028, in download_handle
  File "yt_dlp\extractor\common.py", line 898, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 855, in _request_webpage

  File "yt_dlp\extractor\common.py", line 837, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3863, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 569, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone
```



Discussion:
I've just spotted some more links with the same error:
* https://www.ardmediathek.de/video/serien/wapo-berlin/hr-fernsehen/YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm
* https://www.ardmediathek.de/video/serien/in-aller-freundschaft-die-jungen-aerzte/hr-fernsehen/ZTdiOTY2NjktYWRlOS00NjFiLTgwYmEtNDIzZjNmYzdjNWNl
* https://www.ardmediathek.de/video/serien/herr-hesselbach-und-das-juwel/hr-fernsehen/YWQ5Zjk4MzAtOWMxYy00ODAxLWFhZjktOGJhNmU1YWZjZTQ4
* https://www.ardmediathek.de/video/serien/das-fest-6-oder-die-laubenpieper/hr-fernsehen/Y2RiMDJlZmYtOTkwNS00ZjAxLWI1MzYtNzI2ZTg1NjY4OGY2
* ...

They are all from https://www.ardmediathek.de/sendung/serien/Y3JpZDovL2hyLW9ubGluZS8zODIyMDEwOQ.

The easy visible difference:
* ```[ARD:mediathek]``` extractor for those failing links.
* ```[ARDBetaMediathek]``` extractor for all normally working links.
Die Ratgeber vom 24.07.2023
https://www.ardmediathek.de/video/die-ratgeber/die-ratgeber-vom-24-07-2023/hr-fernsehen/Y3JpZDovL2hyLW9ubGluZS8yMDc5NTM
```
$ curl 'https://api.ardmediathek.de/public-gateway' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.ardmediathek.de' --data-binary '{"query":"query { playerPage(client: \"ard\", clipId: \"Y3JpZDovL2hyLW9ubGluZS8yMDc5NTM\" ) { title broadcastedOn } }"}'
{"data":{"playerPage":{"title":"Die Ratgeber vom 24.07.2023","broadcastedOn":"2023-07-24T16:45:00Z"}}}
```
Good!

Broadcasted one day later:
Die Ratgeber vom 25.07.2023
https://www.ardmediathek.de/video/die-ratgeber/die-ratgeber-vom-25-07-2023/hr-fernsehen/NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl
```
$ curl 'https://api.ardmediathek.de/public-gateway' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.ardmediathek.de' --data-binary '{"query":"query { playerPage(client: \"ard\", clipId: \"NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl\" ) { title broadcastedOn } }"}'
{"errors":[{"message":"invalid clipId: NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl","locations":[{"line":1,"column":9}],"path":["playerPage"],"extensions":{"code":404}}],"data":{"playerPage":null}}
```
Bad.

So, ARD mediathek refuses to serve recently published videos through ```public-gateway```.
Hi,

here the simular issue

https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fernsehen/OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw

```
$ ./yt-dlp_linux --version
2023.07.06

```

`curl 'https://api.ardmediathek.de/public-gateway' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.ardmediathek.de' --data-binary '{"query":"query { playerPage(client: \"ard\", clipId: \"OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw\" ) { title broadcastedOn } }"}'
`

`{"errors":[{"message":"invalid clipId: OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw","locations":[{"line":1,"column":9}],"path":["playerPage"],"extensions":{"code":404}}],"data":{"playerPage":null}}
`


```
~$ ./yt-dlp_linux  https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fernsehen/OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fer...MzUtMGY1NjNjMGM2ZTMw
[ARD:mediathek] OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw: Downloading webpage
[ARD:mediathek] 94636404: Downloading media JSON
ERROR: [ARD:mediathek] None: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

```
The original problem URL was a playlist page that should have been handled by the ARDBetaMediathek extractor but wasn't because the extractor expected a final `/` (problem 1). But then the playlist items aren't `suitable()` for the extractor (problem 2) -- they match ARDMediathek:
```
$ yt-dlp -v -F 'https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent'
[debug] Command-line config: ['-v', '-F', 'https://www.ardmediathek.de/sendung/g
arfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), erro
r utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version stable@2023.06.22 [812cdfa06] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: de4cf77ec
[debug] Python 3.9.16 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-g
libc2.23 (OpenSSL 1.1.1u  30 May 2023, glibc 2.23)
[debug] exe versions: ffmpeg 4.3, ffprobe 4.3
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2019.11.28, secretstorage
-3.2.0, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[debug] Using fake IP 53.66.212.115 (DE) as X-Forwarded-For
[ARDBetaMediathek] Extracting URL: https://www.ardmediathek.de/sendung/garfield/
Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent
[ARDBetaMediathek] [Playlist] garfield: Downloading JSON metadata
[ARDBetaMediathek] [Playlist] garfield: Downloading JSON metadata
[download] Downloading playlist: garfield
[ARDBetaMediathek] Playlist garfield: Downloading 96 items of 96
[download] Downloading item 1 of 96
ERROR: No suitable extractor (ARDBetaMediathek) found for URL https://www.ardmed
iathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk
...
```
Forcing ARDMediathek gives 404 (problem 3). Checking the playlist page, the URLs should have a penultimate `/kika/`:
```diff
-https://www.ardmediathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk
+https://www.ardmediathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/kika/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk
```
The corrected URL now gives 410 (problem 4).

Looking at the wapo-berlin page, the hydration JSON contains this URL https://api.ardmediathek.de/page-gateway/pages/ard/item/YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm?embedded=false&mcV6=true in this structure:
```json
[
  [URL, {
    "url": URL,
    ...
    "data": {
      /* this  is the JSON object is that the URL returns */
    },
    ...
  }]
]
```
This is a new page and API structure that isn't handled by any existing ARD extractor module (problem 5).

If it's the case that pages handled by ARDMediathek expire regularly (there are no permanent test pages), that extractor can probably be removed or replaced.

Similarly, no test video pages work for ARDBetaMediathek. If no-one can find a valid video page for ARDBetaMediathek, both Mediathek extractors can be replaced with a new one.

Parsing the new page structure (trial yt-dl code) gives this (and similarly for the Garfield kika link):
```console
[info] Available formats for YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm:
format code  extension  resolution note
SD           mp4        640x360    [de] 1200k
SD_480p      mp4        960x540    [de] 1600k
HD_Ready     mp4        1280x720   [de] 3200k
Full_HD      mp4        1920x1080  [de] 5000k
hls-543      mp4        480x270    [de]  543k , avc1.4d401e, 50.0fps, mp4a.40.2
hls-841      mp4        640x360    [de]  841k , avc1.4d401f, 50.0fps, mp4a.40.2
hls-1260     mp4        960x540    [de] 1260k , avc1.4d401f, 50.0fps, mp4a.40.2
hls-2154     mp4        1280x720   [de] 2154k , avc1.640020, 50.0fps, mp4a.40.2
hls-3456     mp4        1920x1080  [de] 3456k , avc1.64002a, 50.0fps, mp4a.40.2 (best)
```
The sort is dubious: probably `tbr` from the mp4 format URLs is a peak bit-rate while the HLS formats are showing `AVERAGE-BANDWIDTH` from the manifest.
I’ve just experienced the same issue with KIKA URL‘s.

https://www.kika.de/emmas-chatroom/videos/zwei-netzreisende-1

Maybe mediathekview can be helpful ?
https://github.com/mediathekview/MediathekView

It's not the same page structure. The quoted page is 404 but a valid page (https://www.kika.de/emmas-chatroom/videos/zwoelf-das-tagebuch-100, say) is still handled by the outstanding PR #5788 for Kika.
Can not download, which is only available for another 3 hours until midnight:
https://www.ardmediathek.de/video/filme/die-drei-tage-des-condor-oder-thriller/kika/MGUzZTE3ZGItYzQ4Yi00NzgxLWFjMDktODA0ZjNjNTM0ZDM4

(But https://www.zdf.de/filme/spielfilm-highlights/salt-102.html is possible, even if it is not late enough for the age restricted movie, I know ZDF is not ARD.)
another log if needed:

<details>
<summary>log</summary>

```
yt-dlp https://www.ardmediathek.de/video/NmY5ZGM4YzQtNDcxMy00ZGYxLThmMDgtZTAxZTcwNzE2MTEw --no-config -v -f-
[debug] Command-line config: ['https://www.ardmediathek.de/video/NmY5ZGM4YzQtNDcxMy00ZGYxLThmMDgtZTAxZTcwNzE2MTEw', '--no-config', '-v', '-f-']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2023.10.22.230540 [a40e0b37d] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, requests-2.31.0, sqlite3-3.35.5, urllib3-2.0.7, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests
[debug] Loaded 1890 extractors
[debug] Using fake IP 53.190.50.12 (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/NmY5ZGM4YzQtNDcxMy00ZGYxLThmMDgtZTAxZTcwNzE2MTEw
[ARD:mediathek] NmY5ZGM4YzQtNDcxMy00ZGYxLThmMDgtZTAxZTcwNzE2MTEw: Downloading webpage
[ARD:mediathek] 12850285: Downloading media JSON
ERROR: [ARD:mediathek] Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: Gone>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 715, in extract
  File "yt_dlp\extractor\ard.py", line 275, in _real_extract
  File "yt_dlp\extractor\ard.py", line 28, in _extract_media_info
  File "yt_dlp\extractor\common.py", line 1069, in download_content
  File "yt_dlp\extractor\common.py", line 1033, in download_handle
  File "yt_dlp\extractor\common.py", line 903, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 860, in _request_webpage

  File "yt_dlp\YoutubeDL.py", line 4051, in urlopen
  File "yt_dlp\networking\common.py", line 114, in send
  File "yt_dlp\networking\_helper.py", line 204, in wrapper
  File "yt_dlp\networking\common.py", line 325, in send
  File "yt_dlp\networking\_requests.py", line 340, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 847, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 4073, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 410: Gone
```


</details>
arm Version

yt-dlp_linux_armv7l https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5

[debug] Command-line config: ['-v', 'https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.10.13 (linux_armv7l_exe)
[debug] Python 3.8.0 (CPython armv7l 32bit) - Linux-4.4.35-armv7l-with-glibc2.7 (OpenSSL 1.1.1  11 Sep 2018, glibc 2.37)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.22.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Using fake IP 53.123.100.252 (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[ARD:mediathek] ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5: Downloading webpage
[ARD:mediathek] 12822193: Downloading media JSON
ERROR: [ARD:mediathek] Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: Gone>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp/extractor/common.py", line 715, in extract
  File "yt_dlp/extractor/ard.py", line 275, in _real_extract
  File "yt_dlp/extractor/ard.py", line 28, in _extract_media_info
  File "yt_dlp/extractor/common.py", line 1069, in download_content
  File "yt_dlp/extractor/common.py", line 1033, in download_handle
  File "yt_dlp/extractor/common.py", line 903, in _download_webpage_handle
  File "yt_dlp/extractor/common.py", line 860, in _request_webpage

  File "yt_dlp/networking/_urllib.py", line 410, in _send
  File "urllib/request.py", line 531, in open
  File "urllib/request.py", line 640, in http_response
  File "urllib/request.py", line 569, in error
  File "urllib/request.py", line 502, in _call_chain
  File "urllib/request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/YoutubeDL.py", line 4051, in urlopen
  File "yt_dlp/networking/common.py", line 114, in send
  File "yt_dlp/networking/_helper.py", line 204, in wrapper
  File "yt_dlp/networking/common.py", line 325, in send
  File "yt_dlp/networking/_urllib.py", line 415, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/extractor/common.py", line 847, in _request_webpage
  File "yt_dlp/YoutubeDL.py", line 4070, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 410: Gone

-----

Intel Version:

./yt-dlp_linux -v https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[debug] Command-line config: ['-v', 'https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.10.13 [b634ba742] (linux_exe)
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-6.1.0-13-amd64-x86_64-with-glibc2.36 (OpenSSL 3.1.3 19 Sep 2023, glibc 2.36)
[debug] exe versions: ffmpeg 5.1.3-1 (setts), ffprobe 5.1.3-1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.43.2, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Using fake IP 53.63.86.236 (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[ARD:mediathek] ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5: Downloading webpage
[ARD:mediathek] 12822193: Downloading media JSON
ERROR: [ARD:mediathek] Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: Gone>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp/extractor/common.py", line 715, in extract
  File "yt_dlp/extractor/ard.py", line 275, in _real_extract
  File "yt_dlp/extractor/ard.py", line 28, in _extract_media_info
  File "yt_dlp/extractor/common.py", line 1069, in download_content
  File "yt_dlp/extractor/common.py", line 1033, in download_handle
  File "yt_dlp/extractor/common.py", line 903, in _download_webpage_handle
  File "yt_dlp/extractor/common.py", line 860, in _request_webpage

  File "yt_dlp/networking/_urllib.py", line 410, in _send
  File "urllib/request.py", line 525, in open
  File "urllib/request.py", line 634, in http_response
  File "urllib/request.py", line 563, in error
  File "urllib/request.py", line 496, in _call_chain
  File "urllib/request.py", line 643, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/YoutubeDL.py", line 4051, in urlopen
  File "yt_dlp/networking/common.py", line 114, in send
  File "yt_dlp/networking/_helper.py", line 204, in wrapper
  File "yt_dlp/networking/common.py", line 325, in send
  File "yt_dlp/networking/_urllib.py", line 415, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/extractor/common.py", line 847, in _request_webpage
  File "yt_dlp/YoutubeDL.py", line 4070, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 410: Gone


I've just spotted some more links with the same error:
* https://www.ardmediathek.de/video/serien/wapo-berlin/hr-fernsehen/YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm
* https://www.ardmediathek.de/video/serien/in-aller-freundschaft-die-jungen-aerzte/hr-fernsehen/ZTdiOTY2NjktYWRlOS00NjFiLTgwYmEtNDIzZjNmYzdjNWNl
* https://www.ardmediathek.de/video/serien/herr-hesselbach-und-das-juwel/hr-fernsehen/YWQ5Zjk4MzAtOWMxYy00ODAxLWFhZjktOGJhNmU1YWZjZTQ4
* https://www.ardmediathek.de/video/serien/das-fest-6-oder-die-laubenpieper/hr-fernsehen/Y2RiMDJlZmYtOTkwNS00ZjAxLWI1MzYtNzI2ZTg1NjY4OGY2
* ...

They are all from https://www.ardmediathek.de/sendung/serien/Y3JpZDovL2hyLW9ubGluZS8zODIyMDEwOQ.

The easy visible difference:
* ```[ARD:mediathek]``` extractor for those failing links.
* ```[ARDBetaMediathek]``` extractor for all normally working links.
Die Ratgeber vom 24.07.2023
https://www.ardmediathek.de/video/die-ratgeber/die-ratgeber-vom-24-07-2023/hr-fernsehen/Y3JpZDovL2hyLW9ubGluZS8yMDc5NTM
```
$ curl 'https://api.ardmediathek.de/public-gateway' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.ardmediathek.de' --data-binary '{"query":"query { playerPage(client: \"ard\", clipId: \"Y3JpZDovL2hyLW9ubGluZS8yMDc5NTM\" ) { title broadcastedOn } }"}'
{"data":{"playerPage":{"title":"Die Ratgeber vom 24.07.2023","broadcastedOn":"2023-07-24T16:45:00Z"}}}
```
Good!

Broadcasted one day later:
Die Ratgeber vom 25.07.2023
https://www.ardmediathek.de/video/die-ratgeber/die-ratgeber-vom-25-07-2023/hr-fernsehen/NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl
```
$ curl 'https://api.ardmediathek.de/public-gateway' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.ardmediathek.de' --data-binary '{"query":"query { playerPage(client: \"ard\", clipId: \"NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl\" ) { title broadcastedOn } }"}'
{"errors":[{"message":"invalid clipId: NTNjNTRkYWMtZWMxMC00N2MyLWJmNzItZDg0NmFmZDJhMjVl","locations":[{"line":1,"column":9}],"path":["playerPage"],"extensions":{"code":404}}],"data":{"playerPage":null}}
```
Bad.

So, ARD mediathek refuses to serve recently published videos through ```public-gateway```.
Hi,

here the simular issue

https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fernsehen/OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw

```
$ ./yt-dlp_linux --version
2023.07.06

```

`curl 'https://api.ardmediathek.de/public-gateway' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.ardmediathek.de' --data-binary '{"query":"query { playerPage(client: \"ard\", clipId: \"OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw\" ) { title broadcastedOn } }"}'
`

`{"errors":[{"message":"invalid clipId: OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw","locations":[{"line":1,"column":9}],"path":["playerPage"],"extensions":{"code":404}}],"data":{"playerPage":null}}
`


```
~$ ./yt-dlp_linux  https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fernsehen/OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-14-abenteuer-allrad-und-oldtimer/hr-fer...MzUtMGY1NjNjMGM2ZTMw
[ARD:mediathek] OGU3OTFkZmUtNGFjNy00ZWFkLWEzMzUtMGY1NjNjMGM2ZTMw: Downloading webpage
[ARD:mediathek] 94636404: Downloading media JSON
ERROR: [ARD:mediathek] None: Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: 'Gone'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

```
The original problem URL was a playlist page that should have been handled by the ARDBetaMediathek extractor but wasn't because the extractor expected a final `/` (problem 1). But then the playlist items aren't `suitable()` for the extractor (problem 2) -- they match ARDMediathek:
```
$ yt-dlp -v -F 'https://www.ardmediathek.de/sendung/garfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent'
[debug] Command-line config: ['-v', '-F', 'https://www.ardmediathek.de/sendung/g
arfield/Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), erro
r utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version stable@2023.06.22 [812cdfa06] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: de4cf77ec
[debug] Python 3.9.16 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-g
libc2.23 (OpenSSL 1.1.1u  30 May 2023, glibc 2.23)
[debug] exe versions: ffmpeg 4.3, ffprobe 4.3
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2019.11.28, secretstorage
-3.2.0, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[debug] Using fake IP 53.66.212.115 (DE) as X-Forwarded-For
[ARDBetaMediathek] Extracting URL: https://www.ardmediathek.de/sendung/garfield/
Y3JpZDovL2hyLW9ubGluZS8zODIyMDA2MQ/?isChildContent
[ARDBetaMediathek] [Playlist] garfield: Downloading JSON metadata
[ARDBetaMediathek] [Playlist] garfield: Downloading JSON metadata
[download] Downloading playlist: garfield
[ARDBetaMediathek] Playlist garfield: Downloading 96 items of 96
[download] Downloading item 1 of 96
ERROR: No suitable extractor (ARDBetaMediathek) found for URL https://www.ardmed
iathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk
...
```
Forcing ARDMediathek gives 404 (problem 3). Checking the playlist page, the URLs should have a penultimate `/kika/`:
```diff
-https://www.ardmediathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk
+https://www.ardmediathek.de/ard/video/garfield/178-garfield-der-pirat-teil-2/kika/NzMxZGMyMmUtMjAxMi00ZDM1LWFiOWQtYzY1MzJhNDEzYzlk
```
The corrected URL now gives 410 (problem 4).

Looking at the wapo-berlin page, the hydration JSON contains this URL https://api.ardmediathek.de/page-gateway/pages/ard/item/YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm?embedded=false&mcV6=true in this structure:
```json
[
  [URL, {
    "url": URL,
    ...
    "data": {
      /* this  is the JSON object is that the URL returns */
    },
    ...
  }]
]
```
This is a new page and API structure that isn't handled by any existing ARD extractor module (problem 5).

If it's the case that pages handled by ARDMediathek expire regularly (there are no permanent test pages), that extractor can probably be removed or replaced.

Similarly, no test video pages work for ARDBetaMediathek. If no-one can find a valid video page for ARDBetaMediathek, both Mediathek extractors can be replaced with a new one.

Parsing the new page structure (trial yt-dl code) gives this (and similarly for the Garfield kika link):
```console
[info] Available formats for YmY1YTQ4NzItNzE0Yi00MzRmLTlhNTUtZDY3OTliZjA0MDBm:
format code  extension  resolution note
SD           mp4        640x360    [de] 1200k
SD_480p      mp4        960x540    [de] 1600k
HD_Ready     mp4        1280x720   [de] 3200k
Full_HD      mp4        1920x1080  [de] 5000k
hls-543      mp4        480x270    [de]  543k , avc1.4d401e, 50.0fps, mp4a.40.2
hls-841      mp4        640x360    [de]  841k , avc1.4d401f, 50.0fps, mp4a.40.2
hls-1260     mp4        960x540    [de] 1260k , avc1.4d401f, 50.0fps, mp4a.40.2
hls-2154     mp4        1280x720   [de] 2154k , avc1.640020, 50.0fps, mp4a.40.2
hls-3456     mp4        1920x1080  [de] 3456k , avc1.64002a, 50.0fps, mp4a.40.2 (best)
```
The sort is dubious: probably `tbr` from the mp4 format URLs is a peak bit-rate while the HLS formats are showing `AVERAGE-BANDWIDTH` from the manifest.
I’ve just experienced the same issue with KIKA URL‘s.

https://www.kika.de/emmas-chatroom/videos/zwei-netzreisende-1

Maybe mediathekview can be helpful ?
https://github.com/mediathekview/MediathekView

It's not the same page structure. The quoted page is 404 but a valid page (https://www.kika.de/emmas-chatroom/videos/zwoelf-das-tagebuch-100, say) is still handled by the outstanding PR #5788 for Kika.
Can not download, which is only available for another 3 hours until midnight:
https://www.ardmediathek.de/video/filme/die-drei-tage-des-condor-oder-thriller/kika/MGUzZTE3ZGItYzQ4Yi00NzgxLWFjMDktODA0ZjNjNTM0ZDM4

(But https://www.zdf.de/filme/spielfilm-highlights/salt-102.html is possible, even if it is not late enough for the age restricted movie, I know ZDF is not ARD.)
another log if needed:

<details>
<summary>log</summary>

```
yt-dlp https://www.ardmediathek.de/video/NmY5ZGM4YzQtNDcxMy00ZGYxLThmMDgtZTAxZTcwNzE2MTEw --no-config -v -f-
[debug] Command-line config: ['https://www.ardmediathek.de/video/NmY5ZGM4YzQtNDcxMy00ZGYxLThmMDgtZTAxZTcwNzE2MTEw', '--no-config', '-v', '-f-']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2023.10.22.230540 [a40e0b37d] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, requests-2.31.0, sqlite3-3.35.5, urllib3-2.0.7, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests
[debug] Loaded 1890 extractors
[debug] Using fake IP 53.190.50.12 (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/NmY5ZGM4YzQtNDcxMy00ZGYxLThmMDgtZTAxZTcwNzE2MTEw
[ARD:mediathek] NmY5ZGM4YzQtNDcxMy00ZGYxLThmMDgtZTAxZTcwNzE2MTEw: Downloading webpage
[ARD:mediathek] 12850285: Downloading media JSON
ERROR: [ARD:mediathek] Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: Gone>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 715, in extract
  File "yt_dlp\extractor\ard.py", line 275, in _real_extract
  File "yt_dlp\extractor\ard.py", line 28, in _extract_media_info
  File "yt_dlp\extractor\common.py", line 1069, in download_content
  File "yt_dlp\extractor\common.py", line 1033, in download_handle
  File "yt_dlp\extractor\common.py", line 903, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 860, in _request_webpage

  File "yt_dlp\YoutubeDL.py", line 4051, in urlopen
  File "yt_dlp\networking\common.py", line 114, in send
  File "yt_dlp\networking\_helper.py", line 204, in wrapper
  File "yt_dlp\networking\common.py", line 325, in send
  File "yt_dlp\networking\_requests.py", line 340, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 847, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 4073, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 410: Gone
```


</details>
arm Version

yt-dlp_linux_armv7l https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5

[debug] Command-line config: ['-v', 'https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.10.13 (linux_armv7l_exe)
[debug] Python 3.8.0 (CPython armv7l 32bit) - Linux-4.4.35-armv7l-with-glibc2.7 (OpenSSL 1.1.1  11 Sep 2018, glibc 2.37)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.22.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Using fake IP 53.123.100.252 (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[ARD:mediathek] ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5: Downloading webpage
[ARD:mediathek] 12822193: Downloading media JSON
ERROR: [ARD:mediathek] Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: Gone>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp/extractor/common.py", line 715, in extract
  File "yt_dlp/extractor/ard.py", line 275, in _real_extract
  File "yt_dlp/extractor/ard.py", line 28, in _extract_media_info
  File "yt_dlp/extractor/common.py", line 1069, in download_content
  File "yt_dlp/extractor/common.py", line 1033, in download_handle
  File "yt_dlp/extractor/common.py", line 903, in _download_webpage_handle
  File "yt_dlp/extractor/common.py", line 860, in _request_webpage

  File "yt_dlp/networking/_urllib.py", line 410, in _send
  File "urllib/request.py", line 531, in open
  File "urllib/request.py", line 640, in http_response
  File "urllib/request.py", line 569, in error
  File "urllib/request.py", line 502, in _call_chain
  File "urllib/request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/YoutubeDL.py", line 4051, in urlopen
  File "yt_dlp/networking/common.py", line 114, in send
  File "yt_dlp/networking/_helper.py", line 204, in wrapper
  File "yt_dlp/networking/common.py", line 325, in send
  File "yt_dlp/networking/_urllib.py", line 415, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/extractor/common.py", line 847, in _request_webpage
  File "yt_dlp/YoutubeDL.py", line 4070, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 410: Gone

-----

Intel Version:

./yt-dlp_linux -v https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[debug] Command-line config: ['-v', 'https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.10.13 [b634ba742] (linux_exe)
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-6.1.0-13-amd64-x86_64-with-glibc2.36 (OpenSSL 3.1.3 19 Sep 2023, glibc 2.36)
[debug] exe versions: ffmpeg 5.1.3-1 (setts), ffprobe 5.1.3-1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.43.2, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Using fake IP 53.63.86.236 (DE) as X-Forwarded-For
[ARD:mediathek] Extracting URL: https://www.ardmediathek.de/video/der-camping-check/folge-16-camper-selbstgebaut/hr-fernsehen/ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5
[ARD:mediathek] ZjFiOWEyNTQtYmVlZi00NGVhLTgwZjYtN2I0MGM4ODgzMDM5: Downloading webpage
[ARD:mediathek] 12822193: Downloading media JSON
ERROR: [ARD:mediathek] Unable to download JSON metadata: HTTP Error 410: Gone (caused by <HTTPError 410: Gone>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp/extractor/common.py", line 715, in extract
  File "yt_dlp/extractor/ard.py", line 275, in _real_extract
  File "yt_dlp/extractor/ard.py", line 28, in _extract_media_info
  File "yt_dlp/extractor/common.py", line 1069, in download_content
  File "yt_dlp/extractor/common.py", line 1033, in download_handle
  File "yt_dlp/extractor/common.py", line 903, in _download_webpage_handle
  File "yt_dlp/extractor/common.py", line 860, in _request_webpage

  File "yt_dlp/networking/_urllib.py", line 410, in _send
  File "urllib/request.py", line 525, in open
  File "urllib/request.py", line 634, in http_response
  File "urllib/request.py", line 563, in error
  File "urllib/request.py", line 496, in _call_chain
  File "urllib/request.py", line 643, in http_error_default
urllib.error.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/YoutubeDL.py", line 4051, in urlopen
  File "yt_dlp/networking/common.py", line 114, in send
  File "yt_dlp/networking/_helper.py", line 204, in wrapper
  File "yt_dlp/networking/common.py", line 325, in send
  File "yt_dlp/networking/_urllib.py", line 415, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 410: Gone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/extractor/common.py", line 847, in _request_webpage
  File "yt_dlp/YoutubeDL.py", line 4070, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 410: Gone



Resolving patch:
diff --git a/yt_dlp/extractor/ard.py b/yt_dlp/extractor/ard.py
--- a/yt_dlp/extractor/ard.py
+++ b/yt_dlp/extractor/ard.py
@@ -292,7 +292,7 @@ class ARDIE(InfoExtractor):
     _TESTS = [{
         # available till 7.12.2023
         'url': 'https://www.daserste.de/information/talk/maischberger/videos/maischberger-video-424.html',
-        'md5': 'a438f671e87a7eba04000336a119ccc4',
+        'md5': '94812e6438488fb923c361a44469614b',
         'info_dict': {
             'id': 'maischberger-video-424',
             'display_id': 'maischberger-video-424',
@@ -403,26 +403,25 @@ class ARDBetaMediathekIE(ARDMediathekBaseIE):
     _VALID_URL = r'''(?x)https://
         (?:(?:beta|www)\.)?ardmediathek\.de/
         (?:(?P<client>[^/]+)/)?
-        (?:player|live|video|(?P<playlist>sendung|sammlung))/
+        (?:player|live|video|(?P<playlist>sendung|serie|sammlung))/
         (?:(?P<display_id>(?(playlist)[^?#]+?|[^?#]+))/)?
         (?P<id>(?(playlist)|Y3JpZDovL)[a-zA-Z0-9]+)
         (?(playlist)/(?P<season>\d+)?/?(?:[?#]|$))'''
 
     _TESTS = [{
-        'url': 'https://www.ardmediathek.de/video/filme-im-mdr/wolfsland-die-traurigen-schwestern/mdr-fernsehen/Y3JpZDovL21kci5kZS9iZWl0cmFnL2Ntcy8xZGY0ZGJmZS00ZWQwLTRmMGItYjhhYy0wOGQ4ZmYxNjVhZDI',
-        'md5': '3fd5fead7a370a819341129c8d713136',
+        'url': 'https://www.ardmediathek.de/video/filme-im-mdr/liebe-auf-vier-pfoten/mdr-fernsehen/Y3JpZDovL21kci5kZS9zZW5kdW5nLzI4MjA0MC80MjIwOTEtNDAyNTM0',
+        'md5': 'b6e8ab03f2bcc6e1f9e6cef25fcc03c4',
         'info_dict': {
-            'display_id': 'filme-im-mdr/wolfsland-die-traurigen-schwestern/mdr-fernsehen',
-            'id': '12172961',
-            'title': 'Wolfsland - Die traurigen Schwestern',
-            'description': r're:^Als der Polizeiobermeister Raaben',
-            'duration': 5241,
-            'thumbnail': 'https://api.ardmediathek.de/image-service/images/urn:ard:image:efa186f7b0054957',
-            'timestamp': 1670710500,
-            'upload_date': '20221210',
+            'display_id': 'filme-im-mdr/liebe-auf-vier-pfoten/mdr-fernsehen',
+            'id': '12939099',
+            'title': 'Liebe auf vier Pfoten',
+            'description': r're:^Claudia Schmitt, Anwältin in Salzburg',
+            'duration': 5222,
+            'thumbnail': 'https://api.ardmediathek.de/image-service/images/urn:ard:image:aee7cbf8f06de976?w=960&ch=ae4d0f2ee47d8b9b',
+            'timestamp': 1701343800,
+            'upload_date': '20231130',
             'ext': 'mp4',
-            'age_limit': 12,
-            'episode': 'Wolfsland - Die traurigen Schwestern',
+            'episode': 'Liebe auf vier Pfoten',
             'series': 'Filme im MDR'
         },
     }, {
@@ -454,7 +453,7 @@ class ARDBetaMediathekIE(ARDMediathekBaseIE):
             'duration': 915,
             'episode': 'tagesschau, 20:00 Uhr',
             'series': 'tagesschau',
-            'thumbnail': 'https://api.ardmediathek.de/image-service/images/urn:ard:image:fbb21142783b0a49',
+            'thumbnail': 'https://api.ardmediathek.de/image-service/images/urn:ard:image:fbb21142783b0a49?w=960&ch=ee69108ae344f678',
         },
     }, {
         'url': 'https://beta.ardmediathek.de/ard/video/Y3JpZDovL2Rhc2Vyc3RlLmRlL3RhdG9ydC9mYmM4NGM1NC0xNzU4LTRmZGYtYWFhZS0wYzcyZTIxNGEyMDE',
@@ -475,6 +474,10 @@ class ARDBetaMediathekIE(ARDMediathekBaseIE):
         # playlist of type 'sendung'
         'url': 'https://www.ardmediathek.de/ard/sendung/doctor-who/Y3JpZDovL3dkci5kZS9vbmUvZG9jdG9yIHdobw/',
         'only_matching': True,
+    }, {
+        # playlist of type 'serie'
+        'url': 'https://www.ardmediathek.de/serie/nachtstreife/staffel-1/Y3JpZDovL3N3ci5kZS9zZGIvc3RJZC8xMjQy/1',
+        'only_matching': True,
     }, {
         # playlist of type 'sammlung'
         'url': 'https://www.ardmediathek.de/ard/sammlung/team-muenster/5JpTzLSbWUAK8184IOvEir/',
@@ -487,10 +490,11 @@ class ARDBetaMediathekIE(ARDMediathekBaseIE):
         'only_matching': True,
     }]
 
-    def _ARD_load_playlist_snipped(self, playlist_id, display_id, client, mode, pageNumber):
+    def _ARD_load_playlist_snippet(self, playlist_id, display_id, client, mode, page_number):
         """ Query the ARD server for playlist information
         and returns the data in "raw" format """
-        if mode == 'sendung':
+        assert mode in ('sendung', 'serie', 'sammlung')
+        if mode in ('sendung', 'serie'):
             graphQL = json.dumps({
                 'query': '''{
                     showPage(
@@ -507,7 +511,7 @@ def _ARD_load_playlist_snipped(self, playlist_id, display_id, client, mode, page
                             links { target { id href title } }
                             type
                         }
-                    }}''' % (client, playlist_id, pageNumber),
+                    }}''' % (client, playlist_id, page_number),
             }).encode()
         else:  # mode == 'sammlung'
             graphQL = json.dumps({
@@ -528,7 +532,7 @@ def _ARD_load_playlist_snipped(self, playlist_id, display_id, client, mode, page
                                 type
                             }
                         }
-                    }}''' % (client, playlist_id, pageNumber),
+                    }}''' % (client, playlist_id, page_number),
             }).encode()
         # Ressources for ARD graphQL debugging:
         # https://api-test.ardmediathek.de/public-gateway
@@ -538,7 +542,7 @@ def _ARD_load_playlist_snipped(self, playlist_id, display_id, client, mode, page
             data=graphQL,
             headers={'Content-Type': 'application/json'})['data']
         # align the structure of the returned data:
-        if mode == 'sendung':
+        if mode in ('sendung', 'serie'):
             show_page = show_page['showPage']
         else:  # mode == 'sammlung'
             show_page = show_page['morePage']['widget']
@@ -546,12 +550,12 @@ def _ARD_load_playlist_snipped(self, playlist_id, display_id, client, mode, page
 
     def _ARD_extract_playlist(self, url, playlist_id, display_id, client, mode):
         """ Collects all playlist entries and returns them as info dict.
-        Supports playlists of mode 'sendung' and 'sammlung', and also nested
-        playlists. """
+        Supports playlists of mode 'sendung', 'serie', and 'sammlung',
+        as well as nested playlists. """
         entries = []
         pageNumber = 0
         while True:  # iterate by pageNumber
-            show_page = self._ARD_load_playlist_snipped(
+            show_page = self._ARD_load_playlist_snippet(
                 playlist_id, display_id, client, mode, pageNumber)
             for teaser in show_page['teasers']:  # process playlist items
                 if '/compilation/' in teaser['links']['target']['href']:


Test patch: