Repository: yt-dlp/yt-dlp

Issue:
[RTBF.be] Extration problem for the "Live" streams
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.08.08** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Belgique

### Provide a description that is worded well enough to be understood

Extration problem for the "Live" streams found here: https://www.rtbf.be/auvio/direct#/

It seems to be blocked because of Subtitle...

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
$ yt-dlp -vU -g --cookies-from-browser firefox https://www.rtbf.be/auvio/direct_le-beau-velo-de-ravel?lid=408218#/
[debug] Command-line config: ['-vU', '-g', '--cookies-from-browser', 'firefox', 'https://www.rtbf.be/auvio/direct_le-beau-velo-de-ravel?lid=408218#/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.08.08 [3157158] (pip)
[debug] Python 3.9.13 (CPython 64bit) - macOS-10.15.7-x86_64-i386-64bit 
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 5.1 (setts), ffprobe 5.1
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[Cookies] Extracting cookies from firefox
[debug] Extracting cookies from: "/Users/xxx/Library/Application Support/Firefox/Profiles/xc8x3893.default-release/cookies.sqlite"
[Cookies] Extracted 888 cookies from firefox
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
[debug] [RTBF] Extracting URL: https://www.rtbf.be/auvio/direct_le-beau-velo-de-ravel?lid=408218#/
[RTBF] 408218: Downloading webpage
ERROR: 408218: An extractor error has occurred. (caused by KeyError('subtitle')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/Cellar/yt-dlp/2022.8.8/libexec/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 666, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/Cellar/yt-dlp/2022.8.8/libexec/lib/python3.9/site-packages/yt_dlp/extractor/redbee.py", line 283, in _real_extract
    title = data['subtitle']
KeyError: 'subtitle'
```



Discussion:

Resolving patch:
diff --git a/yt_dlp/extractor/redbee.py b/yt_dlp/extractor/redbee.py
--- a/yt_dlp/extractor/redbee.py
+++ b/yt_dlp/extractor/redbee.py
@@ -69,6 +69,10 @@ def _get_formats_and_subtitles(self, asset_id, **kwargs):
                 fmts, subs = self._extract_m3u8_formats_and_subtitles(
                     format['mediaLocator'], asset_id, fatal=False)
 
+            if format.get('drm'):
+                for f in fmts:
+                    f['has_drm'] = True
+
             formats.extend(fmts)
             self._merge_subtitles(subs, target=subtitles)
 
@@ -269,8 +273,17 @@ def _real_extract(self, url):
         embed_page = self._download_webpage(
             'https://www.rtbf.be/auvio/embed/' + ('direct' if live else 'media'),
             media_id, query={'id': media_id})
-        data = self._parse_json(self._html_search_regex(
-            r'data-media="([^"]+)"', embed_page, 'media data'), media_id)
+
+        media_data = self._html_search_regex(r'data-media="([^"]+)"', embed_page, 'media data', fatal=False)
+        if not media_data:
+            if re.search(r'<div[^>]+id="js-error-expired"[^>]+class="(?![^"]*hidden)', embed_page):
+                raise ExtractorError('Livestream has ended.', expected=True)
+            if re.search(r'<div[^>]+id="js-sso-connect"[^>]+class="(?![^"]*hidden)', embed_page):
+                self.raise_login_required()
+
+            raise ExtractorError('Could not find media data')
+
+        data = self._parse_json(media_data, media_id)
 
         error = data.get('error')
         if error:
@@ -280,15 +293,20 @@ def _real_extract(self, url):
         if provider in self._PROVIDERS:
             return self.url_result(data['url'], self._PROVIDERS[provider])
 
-        title = data['subtitle']
+        title = traverse_obj(data, 'subtitle', 'title')
         is_live = data.get('isLive')
         height_re = r'-(\d+)p\.'
-        formats = []
+        formats, subtitles = [], {}
 
-        m3u8_url = data.get('urlHlsAes128') or data.get('urlHls')
+        # The old api still returns m3u8 and mpd manifest for livestreams, but these are 'fake'
+        # since all they contain is a 20s video that is completely unrelated.
+        # https://github.com/yt-dlp/yt-dlp/issues/4656#issuecomment-1214461092
+        m3u8_url = None if data.get('isLive') else traverse_obj(data, 'urlHlsAes128', 'urlHls')
         if m3u8_url:
-            formats.extend(self._extract_m3u8_formats(
-                m3u8_url, media_id, 'mp4', m3u8_id='hls', fatal=False))
+            fmts, subs = self._extract_m3u8_formats_and_subtitles(
+                m3u8_url, media_id, 'mp4', m3u8_id='hls', fatal=False)
+            formats.extend(fmts)
+            self._merge_subtitles(subs, target=subtitles)
 
         fix_url = lambda x: x.replace('//rtbf-vod.', '//rtbf.') if '/geo/drm/' in x else x
         http_url = data.get('url')
@@ -319,10 +337,12 @@ def _real_extract(self, url):
                     'height': height,
                 })
 
-        mpd_url = data.get('urlDash')
+        mpd_url = None if data.get('isLive') else data.get('urlDash')
         if mpd_url and (self.get_param('allow_unplayable_formats') or not data.get('drm')):
-            formats.extend(self._extract_mpd_formats(
-                mpd_url, media_id, mpd_id='dash', fatal=False))
+            fmts, subs = self._extract_mpd_formats_and_subtitles(
+                mpd_url, media_id, mpd_id='dash', fatal=False)
+            formats.extend(fmts)
+            self._merge_subtitles(subs, target=subtitles)
 
         audio_url = data.get('urlAudio')
         if audio_url:
@@ -332,7 +352,6 @@ def _real_extract(self, url):
                 'vcodec': 'none',
             })
 
-        subtitles = {}
         for track in (data.get('tracks') or {}).values():
             sub_url = track.get('url')
             if not sub_url:
@@ -342,7 +361,7 @@ def _real_extract(self, url):
             })
 
         if not formats:
-            fmts, subs = self._get_formats_and_subtitles(url, media_id)
+            fmts, subs = self._get_formats_and_subtitles(url, f'live_{media_id}' if is_live else media_id)
             formats.extend(fmts)
             self._merge_subtitles(subs, target=subtitles)
 


Test patch: