Repository: yt-dlp/yt-dlp

Issue:
vod.tvp.pl broken after redesign (new url schemes)
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.10.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Poland

### Provide a description that is worded well enough to be understood

after the redesign there are new URLs for both the series and episodes. the generic extractor doesn't handle them (on the example it downloads some 43 seconds trailer for the series).

series:
old: https://vod.tvp.pl/website/ranczo,1667874
new: https://vod.tvp.pl/seriale,18/ranczo-odcinki,316445

series episodes:
old: https://vod.tvp.pl/video/czas-honoru,i-seria-odc-13,194536
new: https://vod.tvp.pl/seriale,18/czas-honoru-odcinki,292065/odcinek-13,S01E13,313867

standalone movies:
old: https://vod.tvp.pl/website/ukrainski-sluga-narodu,51640041
new: https://vod.tvp.pl/filmy-dokumentalne,163/ukrainski-sluga-narodu,339667

the website seems to be more restrictive to region lock than the APIs we use, as I can download the episodes and standalones from NL. the series extractor is broken for everything, even from PL IPs, though it is freely available in NL as it's just metadata.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['--no-check-certificate', '-v', 'https://vod.tvp.pl/seriale,18/ranczo-odcinki,316445']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.10.04 [4e0511f27] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: 2530b68d4
[debug] Python 3.10.8 (CPython 64bit) - Linux-5.15.73-0-lts-x86_64-with 
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 5.1.2 (setts), ffprobe 5.1.2
[debug] Optional libraries: Crypto-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {'https': '127.0.0.1:8080'}
[debug] Loaded 1700 extractors
[debug] [generic] Extracting URL: https://vod.tvp.pl/seriale,18/ranczo-odcinki,316445
[generic] ranczo-odcinki,316445: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] ranczo-odcinki,316445: Extracting information
[debug] Looking for embeds
[debug] Identified a JSON LD
[debug] [generic] Extracting URL: https://r.dcs.redcdn.pl/file/o2/TVP/video_trailer/c33e61d40cae4025b43f9dae6e62631f.mp4#__youtubedl_smuggle=%7B%22force_videoid%22%3A+%22ranczo-odcinki%2C316445%22%2C+%22to_generic%22%3A+true%2C+%22http_headers%22%3A+%7B%22Referer%22%3A+%22https%3A%2F%2Fvod.tvp.pl%2Fseriale%2C18%2Franczo-odcinki%2C316445%22%7D%7D
[generic] ranczo-odcinki,316445: Downloading webpage
[redirect] Following redirect to https://n-6-5.dcs.redcdn.pl/file/o2/TVP/video_trailer/c33e61d40cae4025b43f9dae6e62631f.mp4
[debug] [generic] Extracting URL: https://n-6-5.dcs.redcdn.pl/file/o2/TVP/video_trailer/c33e61d40cae4025b43f9dae6e62631f.mp4#__youtubedl_smuggle=%7B%22force_videoid%22%3A+%22ranczo-odcinki%2C316445%22%7D
[generic] ranczo-odcinki,316445: Downloading webpage
[debug] Identified a direct video link
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] ranczo-odcinki,316445: Downloading 1 format(s): mp4
[debug] Invoking http downloader on "https://n-6-5.dcs.redcdn.pl/file/o2/TVP/video_trailer/c33e61d40cae4025b43f9dae6e62631f.mp4"
[download] Ranczo [ranczo-odcinki,316445].mp4 has already been downloaded
[download] 100% of   16.62MiB
```



Discussion:

Resolving patch:
diff --git a/yt_dlp/extractor/_extractors.py b/yt_dlp/extractor/_extractors.py
--- a/yt_dlp/extractor/_extractors.py
+++ b/yt_dlp/extractor/_extractors.py
@@ -1937,7 +1937,8 @@
     TVPEmbedIE,
     TVPIE,
     TVPStreamIE,
-    TVPWebsiteIE,
+    TVPVODSeriesIE,
+    TVPVODVideoIE,
 )
 from .tvplay import (
     TVPlayIE,
diff --git a/yt_dlp/extractor/tvp.py b/yt_dlp/extractor/tvp.py
--- a/yt_dlp/extractor/tvp.py
+++ b/yt_dlp/extractor/tvp.py
@@ -4,40 +4,51 @@
 
 from .common import InfoExtractor
 from ..utils import (
+    clean_html,
     determine_ext,
     dict_get,
     ExtractorError,
     int_or_none,
     js_to_json,
-    orderedSet,
     str_or_none,
+    strip_or_none,
+    traverse_obj,
     try_get,
+    url_or_none,
 )
 
 
 class TVPIE(InfoExtractor):
     IE_NAME = 'tvp'
     IE_DESC = 'Telewizja Polska'
-    _VALID_URL = r'https?://(?:[^/]+\.)?(?:tvp(?:parlament)?\.(?:pl|info)|polandin\.com)/(?:video/(?:[^,\s]*,)*|(?:(?!\d+/)[^/]+/)*)(?P<id>\d+)'
+    _VALID_URL = r'https?://(?:[^/]+\.)?(?:tvp(?:parlament)?\.(?:pl|info)|tvpworld\.com|swipeto\.pl)/(?:(?!\d+/)[^/]+/)*(?P<id>\d+)'
 
     _TESTS = [{
         # TVPlayer 2 in js wrapper
-        'url': 'https://vod.tvp.pl/video/czas-honoru,i-seria-odc-13,194536',
+        'url': 'https://swipeto.pl/64095316/uliczny-foxtrot-wypozyczalnia-kaset-kto-pamieta-dvdvideo',
         'info_dict': {
-            'id': '194536',
+            'id': '64095316',
             'ext': 'mp4',
-            'title': 'Czas honoru, odc. 13 – Władek',
-            'description': 'md5:437f48b93558370b031740546b696e24',
-            'age_limit': 12,
+            'title': 'Uliczny Foxtrot — Wypożyczalnia kaset. Kto pamięta DVD-Video?',
+            'age_limit': 0,
+            'duration': 374,
+            'thumbnail': r're:https://.+',
         },
+        'expected_warnings': [
+            'Failed to download ISM manifest: HTTP Error 404: Not Found',
+            'Failed to download m3u8 information: HTTP Error 404: Not Found',
+        ],
     }, {
         # TVPlayer legacy
-        'url': 'http://www.tvp.pl/there-can-be-anything-so-i-shortened-it/17916176',
+        'url': 'https://www.tvp.pl/polska-press-video-uploader/wideo/62042351',
         'info_dict': {
-            'id': '17916176',
+            'id': '62042351',
             'ext': 'mp4',
-            'title': 'TVP Gorzów pokaże filmy studentów z podroży dookoła świata',
-            'description': 'TVP Gorzów pokaże filmy studentów z podroży dookoła świata',
+            'title': 'Wideo',
+            'description': 'Wideo Kamera',
+            'duration': 24,
+            'age_limit': 0,
+            'thumbnail': r're:https://.+',
         },
     }, {
         # TVPlayer 2 in iframe
@@ -48,6 +59,8 @@ class TVPIE(InfoExtractor):
             'title': 'Dzieci na sprzedaż dla homoseksualistów',
             'description': 'md5:7d318eef04e55ddd9f87a8488ac7d590',
             'age_limit': 12,
+            'duration': 259,
+            'thumbnail': r're:https://.+',
         },
     }, {
         # TVPlayer 2 in client-side rendered website (regional; window.__newsData)
@@ -58,7 +71,11 @@ class TVPIE(InfoExtractor):
             'title': 'Studio Yayo',
             'upload_date': '20160616',
             'timestamp': 1466075700,
-        }
+            'age_limit': 0,
+            'duration': 20,
+            'thumbnail': r're:https://.+',
+        },
+        'skip': 'Geo-blocked outside PL',
     }, {
         # TVPlayer 2 in client-side rendered website (tvp.info; window.__videoData)
         'url': 'https://www.tvp.info/52880236/09042021-0800',
@@ -66,7 +83,10 @@ class TVPIE(InfoExtractor):
             'id': '52880236',
             'ext': 'mp4',
             'title': '09.04.2021, 08:00',
+            'age_limit': 0,
+            'thumbnail': r're:https://.+',
         },
+        'skip': 'Geo-blocked outside PL',
     }, {
         # client-side rendered (regional) program (playlist) page
         'url': 'https://opole.tvp.pl/9660819/rozmowa-dnia',
@@ -122,7 +142,7 @@ class TVPIE(InfoExtractor):
         'url': 'https://www.tvpparlament.pl/retransmisje-vod/inne/wizyta-premiera-mateusza-morawieckiego-w-firmie-berotu-sp-z-oo/48857277',
         'only_matching': True,
     }, {
-        'url': 'https://polandin.com/47942651/pln-10-billion-in-subsidies-transferred-to-companies-pm',
+        'url': 'https://tvpworld.com/48583640/tescos-polish-business-bought-by-danish-chain-netto',
         'only_matching': True,
     }]
 
@@ -151,16 +171,13 @@ def _extract_vue_video(self, video_data, page_id=None):
         is_website = video_data.get('type') == 'website'
         if is_website:
             url = video_data['url']
-            fucked_up_url_parts = re.match(r'https?://vod\.tvp\.pl/(\d+)/([^/?#]+)', url)
-            if fucked_up_url_parts:
-                url = f'https://vod.tvp.pl/website/{fucked_up_url_parts.group(2)},{fucked_up_url_parts.group(1)}'
         else:
             url = 'tvp:' + str_or_none(video_data.get('_id') or page_id)
         return {
             '_type': 'url_transparent',
             'id': str_or_none(video_data.get('_id') or page_id),
             'url': url,
-            'ie_key': 'TVPEmbed' if not is_website else 'TVPWebsite',
+            'ie_key': (TVPIE if is_website else TVPEmbedIE).ie_key(),
             'title': str_or_none(video_data.get('title')),
             'description': str_or_none(video_data.get('lead')),
             'timestamp': int_or_none(video_data.get('release_date_long')),
@@ -217,8 +234,9 @@ def _real_extract(self, url):
 
         # The URL may redirect to a VOD
         # example: https://vod.tvp.pl/48463890/wadowickie-spotkania-z-janem-pawlem-ii
-        if TVPWebsiteIE.suitable(urlh.url):
-            return self.url_result(urlh.url, ie=TVPWebsiteIE.ie_key(), video_id=page_id)
+        for ie_cls in (TVPVODSeriesIE, TVPVODVideoIE):
+            if ie_cls.suitable(urlh.url):
+                return self.url_result(urlh.url, ie=ie_cls.ie_key(), video_id=page_id)
 
         if re.search(
                 r'window\.__(?:video|news|website|directory)Data\s*=',
@@ -297,12 +315,13 @@ def _real_extract(self, url):
 class TVPEmbedIE(InfoExtractor):
     IE_NAME = 'tvp:embed'
     IE_DESC = 'Telewizja Polska'
+    _GEO_BYPASS = False
     _VALID_URL = r'''(?x)
         (?:
             tvp:
             |https?://
                 (?:[^/]+\.)?
-                (?:tvp(?:parlament)?\.pl|tvp\.info|polandin\.com)/
+                (?:tvp(?:parlament)?\.pl|tvp\.info|tvpworld\.com|swipeto\.pl)/
                 (?:sess/
                         (?:tvplayer\.php\?.*?object_id
                         |TVPlayer2/(?:embed|api)\.php\?.*[Ii][Dd])
@@ -320,6 +339,12 @@ class TVPEmbedIE(InfoExtractor):
             'title': 'Czas honoru, odc. 13 – Władek',
             'description': 'md5:76649d2014f65c99477be17f23a4dead',
             'age_limit': 12,
+            'duration': 2652,
+            'series': 'Czas honoru',
+            'episode': 'Episode 13',
+            'episode_number': 13,
+            'season': 'sezon 1',
+            'thumbnail': r're:https://.+',
         },
     }, {
         'url': 'https://www.tvp.pl/sess/tvplayer.php?object_id=51247504&amp;autoplay=false',
@@ -327,6 +352,9 @@ class TVPEmbedIE(InfoExtractor):
             'id': '51247504',
             'ext': 'mp4',
             'title': 'Razmova 091220',
+            'duration': 876,
+            'age_limit': 0,
+            'thumbnail': r're:https://.+',
         },
     }, {
         # TVPlayer2 embed URL
@@ -361,40 +389,48 @@ def _real_extract(self, url):
         # stripping JSONP padding
         datastr = webpage[15 + len(callback):-3]
         if datastr.startswith('null,'):
-            error = self._parse_json(datastr[5:], video_id)
-            raise ExtractorError(error[0]['desc'])
+            error = self._parse_json(datastr[5:], video_id, fatal=False)
+            error_desc = traverse_obj(error, (0, 'desc'))
+
+            if error_desc == 'Obiekt wymaga płatności':
+                raise ExtractorError('Video requires payment and log-in, but log-in is not implemented')
+
+            raise ExtractorError(error_desc or 'unexpected JSON error')
 
         content = self._parse_json(datastr, video_id)['content']
         info = content['info']
         is_live = try_get(info, lambda x: x['isLive'], bool)
 
+        if info.get('isGeoBlocked'):
+            # actual country list is not provided, we just assume it's always available in PL
+            self.raise_geo_restricted(countries=['PL'])
+
         formats = []
         for file in content['files']:
-            video_url = file.get('url')
+            video_url = url_or_none(file.get('url'))
             if not video_url:
                 continue
-            if video_url.endswith('.m3u8'):
+            ext = determine_ext(video_url, None)
+            if ext == 'm3u8':
                 formats.extend(self._extract_m3u8_formats(video_url, video_id, m3u8_id='hls', fatal=False, live=is_live))
-            elif video_url.endswith('.mpd'):
+            elif ext == 'mpd':
                 if is_live:
                     # doesn't work with either ffmpeg or native downloader
                     continue
                 formats.extend(self._extract_mpd_formats(video_url, video_id, mpd_id='dash', fatal=False))
-            elif video_url.endswith('.f4m'):
+            elif ext == 'f4m':
                 formats.extend(self._extract_f4m_formats(video_url, video_id, f4m_id='hds', fatal=False))
             elif video_url.endswith('.ism/manifest'):
                 formats.extend(self._extract_ism_formats(video_url, video_id, ism_id='mss', fatal=False))
             else:
-                # mp4, wmv or something
-                quality = file.get('quality', {})
                 formats.append({
                     'format_id': 'direct',
                     'url': video_url,
-                    'ext': determine_ext(video_url, file['type']),
-                    'fps': int_or_none(quality.get('fps')),
-                    'tbr': int_or_none(quality.get('bitrate')),
-                    'width': int_or_none(quality.get('width')),
-                    'height': int_or_none(quality.get('height')),
+                    'ext': ext or file.get('type'),
+                    'fps': int_or_none(traverse_obj(file, ('quality', 'fps'))),
+                    'tbr': int_or_none(traverse_obj(file, ('quality', 'bitrate')), scale=1000),
+                    'width': int_or_none(traverse_obj(file, ('quality', 'width'))),
+                    'height': int_or_none(traverse_obj(file, ('quality', 'height'))),
                 })
 
         self._sort_formats(formats)
@@ -449,57 +485,105 @@ def _real_extract(self, url):
         return info_dict
 
 
-class TVPWebsiteIE(InfoExtractor):
-    IE_NAME = 'tvp:series'
-    _VALID_URL = r'https?://vod\.tvp\.pl/website/(?P<display_id>[^,]+),(?P<id>\d+)'
+class TVPVODBaseIE(InfoExtractor):
+    _API_BASE_URL = 'https://vod.tvp.pl/api/products'
+
+    def _call_api(self, resource, video_id, **kwargs):
+        return self._download_json(
+            f'{self._API_BASE_URL}/{resource}', video_id,
+            query={'lang': 'pl', 'platform': 'BROWSER'}, **kwargs)
+
+    def _parse_video(self, video):
+        return {
+            '_type': 'url',
+            'url': 'tvp:' + video['externalUid'],
+            'ie_key': TVPEmbedIE.ie_key(),
+            'title': video.get('title'),
+            'description': traverse_obj(video, ('lead', 'description')),
+            'age_limit': int_or_none(video.get('rating')),
+            'duration': int_or_none(video.get('duration')),
+        }
+
+
+class TVPVODVideoIE(TVPVODBaseIE):
+    IE_NAME = 'tvp:vod'
+    _VALID_URL = r'https?://vod\.tvp\.pl/[a-z\d-]+,\d+/[a-z\d-]+(?<!-odcinki)(?:-odcinki,\d+/odcinek-\d+,S\d+E\d+)?,(?P<id>\d+)(?:\?[^#]+)?(?:#.+)?$'
 
     _TESTS = [{
-        # series
-        'url': 'https://vod.tvp.pl/website/wspaniale-stulecie,17069012/video',
+        'url': 'https://vod.tvp.pl/dla-dzieci,24/laboratorium-alchemika-odcinki,309338/odcinek-24,S01E24,311357',
         'info_dict': {
-            'id': '17069012',
+            'id': '60468609',
+            'ext': 'mp4',
+            'title': 'Laboratorium alchemika, Tusze termiczne. Jak zobaczyć niewidoczne. Odcinek 24',
+            'description': 'md5:1d4098d3e537092ccbac1abf49b7cd4c',
+            'duration': 300,
+            'episode_number': 24,
+            'episode': 'Episode 24',
+            'age_limit': 0,
+            'series': 'Laboratorium alchemika',
+            'thumbnail': 're:https://.+',
         },
-        'playlist_count': 312,
     }, {
-        # film
-        'url': 'https://vod.tvp.pl/website/krzysztof-krawczyk-cale-moje-zycie,51374466',
+        'url': 'https://vod.tvp.pl/filmy-dokumentalne,163/ukrainski-sluga-narodu,339667',
         'info_dict': {
-            'id': '51374509',
+            'id': '51640077',
             'ext': 'mp4',
-            'title': 'Krzysztof Krawczyk – całe moje życie, Krzysztof Krawczyk – całe moje życie',
-            'description': 'md5:2e80823f00f5fc263555482f76f8fa42',
+            'title': 'Ukraiński sługa narodu, Ukraiński sługa narodu',
+            'series': 'Ukraiński sługa narodu',
+            'description': 'md5:b7940c0a8e439b0c81653a986f544ef3',
             'age_limit': 12,
+            'episode': 'Episode 0',
+            'episode_number': 0,
+            'duration': 3051,
+            'thumbnail': 're:https://.+',
         },
-        'params': {
-            'skip_download': True,
-        },
-        'add_ie': ['TVPEmbed'],
-    }, {
-        'url': 'https://vod.tvp.pl/website/lzy-cennet,38678312',
-        'only_matching': True,
     }]
 
-    def _entries(self, display_id, playlist_id):
-        url = 'https://vod.tvp.pl/website/%s,%s/video' % (display_id, playlist_id)
-        for page_num in itertools.count(1):
-            page = self._download_webpage(
-                url, display_id, 'Downloading page %d' % page_num,
-                query={'page': page_num})
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+
+        return self._parse_video(self._call_api(f'vods/{video_id}', video_id))
 
-            video_ids = orderedSet(re.findall(
-                r'<a[^>]+\bhref=["\']/video/%s,[^,]+,(\d+)' % display_id,
-                page))
 
-            if not video_ids:
-                break
+class TVPVODSeriesIE(TVPVODBaseIE):
+    IE_NAME = 'tvp:vod:series'
+    _VALID_URL = r'https?://vod\.tvp\.pl/[a-z\d-]+,\d+/[a-z\d-]+-odcinki,(?P<id>\d+)(?:\?[^#]+)?(?:#.+)?$'
+
+    _TESTS = [{
+        'url': 'https://vod.tvp.pl/seriale,18/ranczo-odcinki,316445',
+        'info_dict': {
+            'id': '316445',
+            'title': 'Ranczo',
+            'age_limit': 12,
+            'categories': ['seriale'],
+        },
+        'playlist_count': 129,
+    }, {
+        'url': 'https://vod.tvp.pl/programy,88/rolnik-szuka-zony-odcinki,284514',
+        'only_matching': True,
+    }, {
+        'url': 'https://vod.tvp.pl/dla-dzieci,24/laboratorium-alchemika-odcinki,309338',
+        'only_matching': True,
+    }]
 
-            for video_id in video_ids:
-                yield self.url_result(
-                    'tvp:%s' % video_id, ie=TVPEmbedIE.ie_key(),
-                    video_id=video_id)
+    def _entries(self, seasons, playlist_id):
+        for season in seasons:
+            episodes = self._call_api(
+                f'vods/serials/{playlist_id}/seasons/{season["id"]}/episodes', playlist_id,
+                note=f'Downloading episode list for {season["title"]}')
+            yield from map(self._parse_video, episodes)
 
     def _real_extract(self, url):
-        mobj = self._match_valid_url(url)
-        display_id, playlist_id = mobj.group('display_id', 'id')
+        playlist_id = self._match_id(url)
+        metadata = self._call_api(
+            f'vods/serials/{playlist_id}', playlist_id,
+            note='Downloading serial metadata')
+        seasons = self._call_api(
+            f'vods/serials/{playlist_id}/seasons', playlist_id,
+            note='Downloading season list')
         return self.playlist_result(
-            self._entries(display_id, playlist_id), playlist_id)
+            self._entries(seasons, playlist_id), playlist_id, strip_or_none(metadata.get('title')),
+            clean_html(traverse_obj(metadata, ('description', 'lead'), expected_type=strip_or_none)),
+            categories=[traverse_obj(metadata, ('mainCategory', 'name'))],
+            age_limit=int_or_none(metadata.get('rating')),
+        )


Test patch: