Repository: yt-dlp/yt-dlp

Issue:
ARD age verification
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm requesting a site-specific feature
- [X] I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Germany

### Example URLs

https://www.ardmediathek.de/video/oderbruch/folge-3-blindgaenger-s01-e03/das-erste/Y3JpZDovL2Rhc2Vyc3RlLmRlL29kZXJicnVjaC8yMDI0LTAxLTIwXzAwLTAwLU1FWg

### Provide a description that is worded well enough to be understood

To play that video in the browser, you need to sign in and share your passport to verify your age. Afterwards you can play any video with FSK at any time. There is also a PIN verification, it might be a client side verification only from what I see.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', 'https://www.ardmediathek.de/video/oderbruch/folge-3-blindgaenger-s01-e03/das-erste/Y3JpZDovL2Rhc2Vyc3RlLmRlL29kZXJicnVjaC8yMDI0LTAxLTIwXzAwLTAwLU1FWg', '--cookies-from-browser', 'brave']
Extracting cookies from brave
[debug] Extracting cookies from: "/Users/myusername/Library/Application Support/BraveSoftware/Brave-Browser/Default/Cookies"
[debug] using find-generic-password to obtain password from OSX keychain
Extracted 838 cookies from brave
[debug] cookie version breakdown: {'v10': 827, 'other': 0, 'unencrypted': 11}
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8 (No ANSI), error utf-8 (No ANSI), screen utf-8 (No ANSI)
[debug] yt-dlp version stable@2023.12.30 from yt-dlp/yt-dlp [f10589e34] (source)
[debug] Git HEAD: f24e44e8c
[debug] Python 3.9.6 (CPython arm64 64bit) - macOS-14.2.1-arm64-arm-64bit (LibreSSL 2.8.3)
[debug] exe versions: ffmpeg 6.1.1 (setts), ffprobe 6.1.1
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, requests-2.31.0, sqlite3-3.43.2, urllib3-2.1.0, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1886 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2023.12.30 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2023.12.30 from yt-dlp/yt-dlp)
[ARDMediathek] Extracting URL: https://www.ardmediathek.de/video/oderbruch/folge-3-blindgaenger-s01-e03/das-erste/Y3JpZDovL2Rhc2Vyc3RlLmRlL29kZXJicnVjaC8yMDI0LTAxLTIwXzAwLTAwLU1FWg
[ARDMediathek] Y3JpZDovL2Rhc2Vyc3RlLmRlL29kZXJicnVjaC8yMDI0LTAxLTIwXzAwLTAwLU1FWg: Downloading JSON metadata
[debug] Using fake IP 53.146.162.25 (DE) as X-Forwarded-For
ERROR: [ARDMediathek] Y3JpZDovL2Rhc2Vyc3RlLmRlL29kZXJicnVjaC8yMDI0LTAxLTIwXzAwLTAwLU1FWg: This video is only available after 22:00
  File "/projects/yt-dlp/yt_dlp/extractor/common.py", line 718, in extract
    ie_result = self._real_extract(url)
  File "/projects/yt-dlp/yt_dlp/extractor/ard.py", line 386, in _real_extract
    self.raise_no_formats('This video is only available after 22:00', expected=True)
  File "/projects/yt-dlp/yt_dlp/extractor/common.py", line 1196, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)
```



Discussion:

Resolving patch:
diff --git a/yt_dlp/extractor/ard.py b/yt_dlp/extractor/ard.py
--- a/yt_dlp/extractor/ard.py
+++ b/yt_dlp/extractor/ard.py
@@ -8,6 +8,7 @@
     determine_ext,
     int_or_none,
     join_nonempty,
+    jwt_decode_hs256,
     make_archive_id,
     parse_duration,
     parse_iso8601,
@@ -238,6 +239,7 @@ class ARDBetaMediathekIE(InfoExtractor):
         (?P<id>[a-zA-Z0-9]+)
         /?(?:[?#]|$)'''
     _GEO_COUNTRIES = ['DE']
+    _TOKEN_URL = 'https://sso.ardmediathek.de/sso/token'
 
     _TESTS = [{
         'url': 'https://www.ardmediathek.de/video/filme-im-mdr/liebe-auf-vier-pfoten/mdr-fernsehen/Y3JpZDovL21kci5kZS9zZW5kdW5nLzI4MjA0MC80MjIwOTEtNDAyNTM0',
@@ -359,12 +361,27 @@ def _extract_episode_info(self, title):
 
     def _real_extract(self, url):
         display_id = self._match_id(url)
+        query = {'embedded': 'false', 'mcV6': 'true'}
+        headers = {}
+
+        if self._get_cookies(self._TOKEN_URL).get('ams'):
+            token = self._download_json(
+                self._TOKEN_URL, display_id, 'Fetching token for age verification',
+                'Unable to fetch age verification token', fatal=False)
+            id_token = traverse_obj(token, ('idToken', {str}))
+            decoded_token = traverse_obj(id_token, ({jwt_decode_hs256}, {dict}))
+            user_id = traverse_obj(decoded_token, (('user_id', 'sub'), {str}), get_all=False)
+            if not user_id:
+                self.report_warning('Unable to extract token, continuing without authentication')
+            else:
+                headers['x-authorization'] = f'Bearer {id_token}'
+                query['userId'] = user_id
+                if decoded_token.get('age_rating') != 18:
+                    self.report_warning('Account is not verified as 18+; video may be unavailable')
 
         page_data = self._download_json(
-            f'https://api.ardmediathek.de/page-gateway/pages/ard/item/{display_id}', display_id, query={
-                'embedded': 'false',
-                'mcV6': 'true',
-            })
+            f'https://api.ardmediathek.de/page-gateway/pages/ard/item/{display_id}',
+            display_id, query=query, headers=headers)
 
         # For user convenience we use the old contentId instead of the longer crid
         # Ref: https://github.com/yt-dlp/yt-dlp/issues/8731#issuecomment-1874398283
@@ -383,7 +400,7 @@ def _real_extract(self, url):
         media_data = traverse_obj(player_data, ('mediaCollection', 'embedded', {dict}))
 
         if player_data.get('blockedByFsk'):
-            self.raise_no_formats('This video is only available after 22:00', expected=True)
+            self.raise_login_required('This video is only available for age verified users or after 22:00')
 
         formats = []
         subtitles = {}


Test patch: