Repository: yt-dlp/yt-dlp

Issue:
Dowloading file from svtplay.se sometimes results in the wrong file being downloaded
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Sweden

### Provide a description that is worded well enough to be understood

Trying to download a video with URLs of format https://www.svtplay.se/video/${id}/${name} sometimes results in the wrong file being downloaded.

Examples of breaking URLs: 
- https://www.svtplay.se/video/K5wLrGb/deadline
- https://www.svtplay.se/video/j4zJ9Nn/bilranaren

For videos that don't require this verification the downloads seem to work fine.

Exampels of working URLs:
- https://www.svtplay.se/video/jLdDpwA/den-basta-sommaren
- https://www.svtplay.se/video/8YyBQXp/handerna

I believe the extractor for svtplay.se will need to be updated, but I don't have enough insight into the inner workings of this project to provide a solution.

 In the section below, attempting to download https://www.svtplay.se/video/K5wLrGb/deadline results in the file https://www.svtplay.se/video/KBM7ZXx/mid90s from being downloaded.

**Edit**: Edited this section to remove parts that I no longer believe to be relevant to this problem.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', 'https://www.svtplay.se/video/K5wLrGb/deadline']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (linux_exe)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-6.0.9-arch1-1-x86_64-with-glibc2.36 (OpenSSL 3.0.7 1 Nov 2022, glibc 2.36)
[debug] exe versions: ffmpeg N-62475-g130d19bf20-static (setts), ffprobe N-62475-g130d19bf20-static, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] Using fake IP 78.76.216.231 (SE) as X-Forwarded-For
[debug] [SVTPlay] Extracting URL: https://www.svtplay.se/video/K5wLrGb/deadline
[SVTPlay] K5wLrGb: Downloading webpage
[SVTPlay] KM3WEXY: Downloading JSON metadata
[SVTPlay] KM3WEXY: Downloading MPD manifest
[SVTPlay] KM3WEXY: Downloading MPD manifest
[SVTPlay] KM3WEXY: Downloading MPD manifest
[SVTPlay] KM3WEXY: Downloading MPD manifest
[SVTPlay] KM3WEXY: Downloading m3u8 information
[SVTPlay] KM3WEXY: Downloading MPD manifest
[SVTPlay] KM3WEXY: Downloading m3u8 information
[SVTPlay] KM3WEXY: Downloading m3u8 information
[SVTPlay] KM3WEXY: Downloading MPD manifest
[SVTPlay] KM3WEXY: Downloading MPD manifest
[SVTPlay] KM3WEXY: Downloading m3u8 information
[SVTPlay] KM3WEXY: Downloading m3u8 information
[SVTPlay] KM3WEXY: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] KM3WEXY: Downloading 1 format(s): dash-hbbtv-hevc-0+dash-full-15
[debug] Invoking dashsegments downloader on "https://svt-vod-9g.akamaized.net/d0/se/20221104/68d3e83a-6bf3-4c68-baa1-01e2516f46ce/dash-hbbtv-hevc.mpd"
[dashsegments] Total fragments: 1281
[download] Destination: Mid90s [KM3WEXY].fdash-hbbtv-hevc-0.mp4
[download] 100% of    1.77GiB in 00:01:51 at 16.28MiB/s
[debug] Invoking dashsegments downloader on "https://svt-vod-9g.akamaized.net/d0/se/20221104/68d3e83a-6bf3-4c68-baa1-01e2516f46ce/dash-full.mpd"
[dashsegments] Total fragments: 1281
[download] Destination: Mid90s [KM3WEXY].fdash-full-15.m4a
[download] 100% of  113.54MiB in 00:00:58 at 1.95MiB/s
[Merger] Merging formats into "Mid90s [KM3WEXY].mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:Mid90s [KM3WEXY].fdash-hbbtv-hevc-0.mp4' -i 'file:Mid90s [KM3WEXY].fdash-full-15.m4a' -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart 'file:Mid90s [KM3WEXY].temp.mp4'
Deleting original file Mid90s [KM3WEXY].fdash-full-15.m4a (pass -k to keep)
Deleting original file Mid90s [KM3WEXY].fdash-hbbtv-hevc-0.mp4 (pass -k to keep)
```



Discussion:
https://www.svtplay.se/video/K5wLrGb/deadline
->
`ERROR: [SVTPlay] K5wLrGb: This video is only available in Sweden`
but with metadata `e3ppQZE` instead of `KM3WEXY`.
The videos are only available in Sweden, but should be accessible using a Swedish VPN server. AFAIK there's no VPN DRM.

A workaround for now is to click on the play button (labeled "Spela"), which redirects to a URL with format https://www.svtplay.se/video/${number_id}/${name}?id=${id}. For example, https://www.svtplay.se/video/K5wLrGb/deadline redirects to https://www.svtplay.se/video/17540346/deadline?id=8YwEpmA. Using this URL downloads the correct video.

```
[debug] Command-line config: ['-vU', 'https://www.svtplay.se/video/17540346/deadline?id=8YwEpmA']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (linux_exe)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-6.0.9-arch1-1-x86_64-with-glibc2.36 (OpenSSL 3.0.7 1 Nov 2022, glibc 2.36)
[debug] exe versions: ffmpeg N-62475-g130d19bf20-static (setts), ffprobe N-62475-g130d19bf20-static, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] Using fake IP 78.75.51.182 (SE) as X-Forwarded-For
[debug] [SVTPlay] Extracting URL: https://www.svtplay.se/video/17540346/deadline?id=8YwEpmA
[SVTPlay] 8YwEpmA: Downloading JSON metadata
[SVTPlay] 8YwEpmA: Downloading m3u8 information
[SVTPlay] 8YwEpmA: Downloading m3u8 information
[SVTPlay] 8YwEpmA: Downloading m3u8 information
[SVTPlay] 8YwEpmA: Downloading MPD manifest
[SVTPlay] 8YwEpmA: Downloading MPD manifest
[SVTPlay] 8YwEpmA: Downloading m3u8 information
[SVTPlay] 8YwEpmA: Downloading MPD manifest
[SVTPlay] 8YwEpmA: Downloading m3u8 information
[SVTPlay] 8YwEpmA: Downloading MPD manifest
[SVTPlay] 8YwEpmA: Downloading MPD manifest
[SVTPlay] 8YwEpmA: Downloading m3u8 information
[SVTPlay] 8YwEpmA: Downloading MPD manifest
[SVTPlay] 8YwEpmA: Downloading MPD manifest
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 8YwEpmA: Downloading 1 format(s): dash-hbbtv-hevc-0+dashhbbtv-5
[download] Deadline [8YwEpmA].mp4 has already been downloaded
```

When the "wrong" file is downloaded, is it a completely different video, or the same basic show with (eg) audio description as in #5164?
I have seen this issue too.

> When the "wrong" file is downloaded, is it a completely different video, or the same basic show with (eg) audio description as in #5164?

It downloads a completely different video and is not related to #5164. For example, https://www.svtplay.se/video/jz2rYz7/anders-hansen-moter/james-fallon?info=visa downloads https://www.svtplay.se/video/jNnWaXy/anders-hansen-moter/stuart-russell?info=visa (in this case two different episodes of the same series).
The extractor tries to match random IDs in the `__NEXT_DATA__` without analysing it.

This patch against 2022.11.11 works for the examples above:
```diff
--- old/yt_dlp/extractor/svt.py
+++ new/yt_dlp/extractor/svt.py
@@ -250,12 +250,10 @@
         if not svt_id:
             svt_id = self._search_regex(
                 (r'<video[^>]+data-video-id=["\']([\da-zA-Z-]+)',
-                 r'<[^>]+\bdata-rt=["\']top-area-play-button["\'][^>]+\bhref=["\'][^"\']*video/%s/[^"\']*\b(?:modalId|id)=([\da-zA-Z-]+)' % re.escape(video_id),
-                 r'["\']videoSvtId["\']\s*:\s*["\']([\da-zA-Z-]+)',
-                 r'["\']videoSvtId\\?["\']\s*:\s*\\?["\']([\da-zA-Z-]+)',
+                 r'<[^>]+\bdata-rt=["\']top-area-play-button["\'][^>]+\bhref=["\'][^"\']*video/[\da-zA-Z-]+/[^"\']*\b(?:modalId|id)=([\da-zA-Z-]+)',
                  r'"content"\s*:\s*{.*?"id"\s*:\s*"([\da-zA-Z-]+)"',
-                 r'["\']svtId["\']\s*:\s*["\']([\da-zA-Z-]+)',
-                 r'["\']svtId\\?["\']\s*:\s*\\?["\']([\da-zA-Z-]+)'),
+                 # removed patterns for videoSvtId, svtId that would randomly match __NEXT_DATA__
+                ),
                 webpage, 'video id')
 
         info_dict = self._extract_by_video_id(svt_id, webpage)
```
Then (working from the UK):
```shellsession
$ python3.9 -m yt_dlp -v --get-title 'https://www.svtplay.se/video/jz2rYz7/anders-hansen-moter/james-fallon?info=visa'
[debug] Command-line config: ['-v', '--get-title', 'https://www.svtplay.se/video/jz2rYz7/anders-hansen-moter/james-fallon?info=visa']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b644025b] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: c73355510
[debug] Python 3.9.16 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-glibc2.23 (OpenSSL 1.1.1s  1 Nov 2022, glibc 2.23)
[debug] exe versions: ffmpeg 4.3, ffprobe 4.3
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2019.11.28, secretstorage-3.2.0, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1735 extractors
[debug] Using fake IP 78.74.29.81 (SE) as X-Forwarded-For
[SVTPlay] Extracting URL: https://www.svtplay.se/video/jz2rYz7/anders-hansen-moter/james-fallon?info=visa
[SVTPlay] jz2rYz7: Downloading webpage
[SVTPlay] jvXAGVb: Downloading JSON metadata
[SVTPlay] jvXAGVb: Downloading m3u8 information
[SVTPlay] jvXAGVb: Downloading MPD manifest
[SVTPlay] jvXAGVb: Downloading MPD manifest
[SVTPlay] jvXAGVb: Downloading MPD manifest
[SVTPlay] jvXAGVb: Downloading MPD manifest
[SVTPlay] jvXAGVb: Downloading m3u8 information
[SVTPlay] jvXAGVb: Downloading m3u8 information
[SVTPlay] jvXAGVb: Downloading m3u8 information
[SVTPlay] jvXAGVb: Downloading MPD manifest
[SVTPlay] jvXAGVb: Downloading MPD manifest
[SVTPlay] jvXAGVb: Downloading m3u8 information
[SVTPlay] jvXAGVb: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] jvXAGVb: Downloading 1 format(s): dash-hbbtv-hevc-0+dash-hbbtv-hevc-2
James Fallon
$
```
Hey, i ran into this also for https://www.svtplay.se/video/jMdpzp2/leave-no-trace and more or less ended up with @dirkf conclusion and fix. Can i help out with something to get it fixed?
> Can i help out with something to get it fixed?

I use the patch from @dirkf's [comment](https://github.com/yt-dlp/yt-dlp/issues/5595#issuecomment-1427036023), and it fixes your link as well. So a fix is basically available. Perhaps a Pull Request would put it up for review?

If you cannot patch your own copy of yt-dlp, it _might_ be possible to fix it with a plugin (which you could just place into some folder) - but I have no experience with that yet.
@barsnick i would mostly be worried about breaking something else by removing the regexps
The whole point of the change is that the deleted REs were breaking the extraction, by finding the wrong video. Making the change won't affect any other sites. So you shouldn't worry on that score.

You can run the extractor test before (it fails) and after making a change, as well as keeping a backup, etc.

If you should find a page at SVT that appeared to work before and doesn't after the change, obviously that would be something to report.


Ok! i know more or less nothing about how that code works, my worry was that those REs are there to handle some special cases etc as it look looks like the extract method is used for many variants of svtplay web pages.
Sometimes it's simpler to add a new pattern for the latest page structure and not worry whether the previously handled structures are still being sent. Over time you get code like that above with a whole swathe of REs that may no longer be relevant. When they also break the extraction it's time to start again ...
Yeap is very much a moving target :( had some time to poke around a bit and run the tests. Seems like half of them are broken, either the URL is not valid anymore or the test asserts on a difference. The patch above seems to work for some URLs but does not work with the (valid) URL https://www.svtplay.se/video/30479064 that is used by `test/test_download.py TestDownload.test_SVTPlay` 
> Yeap is very much a moving target :(

To be honest, a guess-timate of 50 % of supported sites are moving targets. ;) That's just the way it is.

> Seems like half of them are broken, either the URL is not valid anymore or the test asserts on a difference.

This phenomenon affects many many extractors!

> The patch above seems to work for some URLs but does not work with the (valid) URL https://www.svtplay.se/video/30479064 that is used by `test/test_download.py TestDownload.test_SVTPlay`

Do _not_ forget to validate in a browser. That particular video says:

> Avsnittet är inte tillgängligt längre. Senast tillgänglig tor 20 jul

(For those of you who don't comprehend: This episode is no longer available. It was last accessible July 20.)
In other words: It currently cannot work with yt-dlp.

Feel free to update the extractor with new tests (as a [PR](https://github.com/yt-dlp/yt-dlp/pulls)), but I'm not sure such PRs are approved.

Resolving patch:
diff --git a/yt_dlp/extractor/svt.py b/yt_dlp/extractor/svt.py
--- a/yt_dlp/extractor/svt.py
+++ b/yt_dlp/extractor/svt.py
@@ -1,3 +1,4 @@
+import json
 import re
 
 from .common import InfoExtractor
@@ -6,10 +7,11 @@
     determine_ext,
     dict_get,
     int_or_none,
-    unified_timestamp,
     str_or_none,
     strip_or_none,
+    traverse_obj,
     try_get,
+    unified_timestamp,
 )
 
 
@@ -163,10 +165,46 @@ class SVTPlayIE(SVTPlayBaseIE):
             },
         },
         'params': {
-            # skip for now due to download test asserts that segment is > 10000 bytes and svt uses
-            # init segments that are smaller
-            # AssertionError: Expected test_SVTPlay_jNwpV9P.mp4 to be at least 9.77KiB, but it's only 864.00B
-            'skip_download': True,
+            'skip_download': 'm3u8',
+        },
+        'skip': 'Episode is no longer available',
+    }, {
+        'url': 'https://www.svtplay.se/video/emBxBQj',
+        'md5': '2382036fd6f8c994856c323fe51c426e',
+        'info_dict': {
+            'id': 'eyBd9aj',
+            'ext': 'mp4',
+            'title': '1. Farlig kryssning',
+            'timestamp': 1491019200,
+            'upload_date': '20170401',
+            'duration': 2566,
+            'thumbnail': r're:^https?://(?:.*[\.-]jpg|www.svtstatic.se/image/.*)$',
+            'age_limit': 0,
+            'episode': '1. Farlig kryssning',
+            'series': 'Rederiet',
+            'subtitles': {
+                'sv': 'count:3'
+            },
+        },
+        'params': {
+            'skip_download': 'm3u8',
+        },
+    }, {
+        'url': 'https://www.svtplay.se/video/jz2rYz7/anders-hansen-moter/james-fallon?info=visa',
+        'info_dict': {
+            'id': 'jvXAGVb',
+            'ext': 'mp4',
+            'title': 'James Fallon',
+            'timestamp': 1673917200,
+            'upload_date': '20230117',
+            'duration': 1081,
+            'thumbnail': r're:^https?://(?:.*[\.-]jpg|www.svtstatic.se/image/.*)$',
+            'age_limit': 0,
+            'episode': 'James Fallon',
+            'series': 'Anders Hansen möter...',
+        },
+        'params': {
+            'skip_download': 'dash',
         },
     }, {
         'url': 'https://www.svtplay.se/video/30479064/husdrommar/husdrommar-sasong-8-designdrommar-i-stenungsund?modalId=8zVbDPA',
@@ -247,15 +285,16 @@ def _real_extract(self, url):
                 data, lambda x: x['statistics']['dataLake']['content']['id'],
                 compat_str)
 
+        if not svt_id:
+            nextjs_data = self._search_nextjs_data(webpage, video_id, fatal=False)
+            svt_id = traverse_obj(nextjs_data, (
+                'props', 'urqlState', ..., 'data', {json.loads}, 'detailsPageByPath',
+                'video', 'svtId', {str}), get_all=False)
+
         if not svt_id:
             svt_id = self._search_regex(
                 (r'<video[^>]+data-video-id=["\']([\da-zA-Z-]+)',
-                 r'<[^>]+\bdata-rt=["\']top-area-play-button["\'][^>]+\bhref=["\'][^"\']*video/%s/[^"\']*\b(?:modalId|id)=([\da-zA-Z-]+)' % re.escape(video_id),
-                 r'["\']videoSvtId["\']\s*:\s*["\']([\da-zA-Z-]+)',
-                 r'["\']videoSvtId\\?["\']\s*:\s*\\?["\']([\da-zA-Z-]+)',
-                 r'"content"\s*:\s*{.*?"id"\s*:\s*"([\da-zA-Z-]+)"',
-                 r'["\']svtId["\']\s*:\s*["\']([\da-zA-Z-]+)',
-                 r'["\']svtId\\?["\']\s*:\s*\\?["\']([\da-zA-Z-]+)'),
+                 r'<[^>]+\bdata-rt=["\']top-area-play-button["\'][^>]+\bhref=["\'][^"\']*video/[\w-]+/[^"\']*\b(?:modalId|id)=([\w-]+)'),
                 webpage, 'video id')
 
         info_dict = self._extract_by_video_id(svt_id, webpage)


Test patch: