Repository: yt-dlp/yt-dlp

Issue:
[Broken] ParliamentLive.TV has changed streaming platforms
## Checklist

- [X] I'm reporting a broken site support
- [X] I've verified that I'm running yt-dlp version **2021.09.25**
- [X] I've checked that all provided URLs are alive and playable in a browser
- [X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [X] I've searched the bugtracker for similar issues including closed ones


## Verbose log

<!--
Provide the complete verbose output of yt-dlp that clearly demonstrates the problem.
Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
 [debug] System config: []
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
 [debug] yt-dlp version 2021.09.25
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
 <more lines>
-->

```
$ yt-dlp --verbose  https://www.parliamentlive.tv/Event/Index/34be5e69-5133-4bcd-ae29-dd96908a4457
[debug] Command-line config: ['--verbose', 'https://www.parliamentlive.tv/Event/Index/34be5e69-5133-4bcd-ae29-dd96908a4457']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] yt-dlp version 2021.09.25 (zip)
[debug] Python version 3.7.3 (CPython 64bit) - Darwin-20.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.1.1, ffprobe 3.1.1
[debug] Optional libraries: sqlite
[debug] Proxy map: {}
[debug] [parliamentlive.tv] Extracting URL: https://www.parliamentlive.tv/Event/Index/34be5e69-5133-4bcd-ae29-dd96908a4457
[parliamentlive.tv] 34be5e69-5133-4bcd-ae29-dd96908a4457: Downloading webpage
ERROR: [parliamentlive.tv] 34be5e69-5133-4bcd-ae29-dd96908a4457: Unable to extract kaltura widget config; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/Users/sams/bin/yt-dlp/yt_dlp/extractor/common.py", line 585, in extract
    ie_result = self._real_extract(url)
  File "/Users/sams/bin/yt-dlp/yt_dlp/extractor/parliamentliveuk.py", line 32, in _real_extract
    webpage, 'kaltura widget config'), video_id)
  File "/Users/sams/bin/yt-dlp/yt_dlp/extractor/common.py", line 1149, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
yt_dlp.utils.RegexNotFoundError: Unable to extract kaltura widget config; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/Users/sams/bin/yt-dlp/yt_dlp/extractor/common.py", line 585, in extract
    ie_result = self._real_extract(url)
  File "/Users/sams/bin/yt-dlp/yt_dlp/extractor/parliamentliveuk.py", line 32, in _real_extract
    webpage, 'kaltura widget config'), video_id)
  File "/Users/sams/bin/yt-dlp/yt_dlp/extractor/common.py", line 1149, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
yt_dlp.utils.RegexNotFoundError: Unable to extract kaltura widget config; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/sams/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1227, in wrapper
    return func(self, *args, **kwargs)
  File "/Users/sams/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1252, in __extract_info
    ie_result = ie.extract(url)
  File "/Users/sams/bin/yt-dlp/yt_dlp/extractor/common.py", line 602, in extract
    e.msg, video_id=video_id, ie=self.IE_NAME, tb=e.traceback, expected=e.expected, cause=e.cause)
yt_dlp.utils.ExtractorError: [parliamentlive.tv] 34be5e69-5133-4bcd-ae29-dd96908a4457: Unable to extract kaltura widget config; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; see  https://github.com/yt-dlp/yt-dlp  on how to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.

```
<!--
Do not remove the above ```
-->


## Description

The ParliamentLive.TV site (video from the UK Parliament) has recently changed streaming platforms, and the existing extractor is broken as all the streaming code has been changed. This extractor should likely be removed/archived:
https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/parliamentliveuk.py

Separately, the new platform is USP which seems to have several extractors built in, and I'm not sure which is easiest to use with the new site. Examples:
https://www.parliamentlive.tv/Event/Index/34be5e69-5133-4bcd-ae29-dd96908a4457
https://www.parliamentlive.tv/Event/Index/39877885-7c59-4a9a-9f03-8549a083f03c


Discussion:

Resolving patch:
diff --git a/yt_dlp/extractor/parliamentliveuk.py b/yt_dlp/extractor/parliamentliveuk.py
--- a/yt_dlp/extractor/parliamentliveuk.py
+++ b/yt_dlp/extractor/parliamentliveuk.py
@@ -1,6 +1,14 @@
+# coding: utf-8
 from __future__ import unicode_literals
 
+import json
+import uuid
+
 from .common import InfoExtractor
+from ..utils import (
+    unified_timestamp,
+    try_get,
+)
 
 
 class ParliamentLiveUKIE(InfoExtractor):
@@ -11,12 +19,14 @@ class ParliamentLiveUKIE(InfoExtractor):
     _TESTS = [{
         'url': 'http://parliamentlive.tv/Event/Index/c1e9d44d-fd6c-4263-b50f-97ed26cc998b',
         'info_dict': {
-            'id': '1_af9nv9ym',
+            'id': 'c1e9d44d-fd6c-4263-b50f-97ed26cc998b',
             'ext': 'mp4',
             'title': 'Home Affairs Committee',
-            'uploader_id': 'FFMPEG-01',
-            'timestamp': 1422696664,
-            'upload_date': '20150131',
+            'timestamp': 1395153872,
+            'upload_date': '20140318',
+        },
+        'params': {
+            'format': 'bestvideo',
         },
     }, {
         'url': 'http://parliamentlive.tv/event/index/3f24936f-130f-40bf-9a5d-b3d6479da6a4',
@@ -25,19 +35,49 @@ class ParliamentLiveUKIE(InfoExtractor):
 
     def _real_extract(self, url):
         video_id = self._match_id(url)
-        webpage = self._download_webpage(
-            'http://vodplayer.parliamentlive.tv/?mid=' + video_id, video_id)
-        widget_config = self._parse_json(self._search_regex(
-            r'(?s)kWidgetConfig\s*=\s*({.+});',
-            webpage, 'kaltura widget config'), video_id)
-        kaltura_url = 'kaltura:%s:%s' % (
-            widget_config['wid'][1:], widget_config['entry_id'])
-        event_title = self._download_json(
-            'http://parliamentlive.tv/Event/GetShareVideo/' + video_id, video_id)['event']['title']
+        video_info = self._download_json(f'https://www.parliamentlive.tv/Event/GetShareVideo/{video_id}', video_id)
+        _DEVICE_ID = str(uuid.uuid4())
+        auth = 'Bearer ' + self._download_json(
+            'https://exposure.api.redbee.live/v2/customer/UKParliament/businessunit/ParliamentLive/auth/anonymous',
+            video_id, headers={
+                'Origin': 'https://videoplayback.parliamentlive.tv',
+                'Accept': 'application/json, text/plain, */*',
+                'Content-Type': 'application/json;charset=utf-8'
+            }, data=json.dumps({
+                'deviceId': _DEVICE_ID,
+                'device': {
+                    'deviceId': _DEVICE_ID,
+                    'width': 653,
+                    'height': 368,
+                    'type': 'WEB',
+                    'name': ' Mozilla Firefox 91'
+                }
+            }).encode('utf-8'))['sessionToken']
+
+        video_urls = self._download_json(
+            f'https://exposure.api.redbee.live/v2/customer/UKParliament/businessunit/ParliamentLive/entitlement/{video_id}/play',
+            video_id, headers={'Authorization': auth, 'Accept': 'application/json, text/plain, */*'})['formats']
+
+        formats = []
+        for format in video_urls:
+            if not format.get('mediaLocator'):
+                continue
+            if format.get('format') == 'DASH':
+                formats.extend(self._extract_mpd_formats(
+                    format['mediaLocator'], video_id, mpd_id='dash', fatal=False))
+            elif format.get('format') == 'SMOOTHSTREAMING':
+                formats.extend(self._extract_ism_formats(
+                    format['mediaLocator'], video_id, ism_id='ism', fatal=False))
+            elif format.get('format') == 'HLS':
+                formats.extend(self._extract_m3u8_formats(
+                    format['mediaLocator'], video_id, m3u8_id='hls', fatal=False))
+
+        self._sort_formats(formats)
+
         return {
-            '_type': 'url_transparent',
-            'title': event_title,
-            'description': '',
-            'url': kaltura_url,
-            'ie_key': 'Kaltura',
+            'id': video_id,
+            'formats': formats,
+            'title': video_info['event']['title'],
+            'timestamp': unified_timestamp(try_get(video_info, lambda x: x['event']['publishedStartTime'])),
+            'thumbnail': video_info.get('thumbnailUrl'),
         }


Test patch: