Repository: yt-dlp/yt-dlp

Issue:
[VK Video] unknown_video extension while downloading some VK video playlists
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a broken site
- [X] I've verified that I'm running yt-dlp version **2022.11.11** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Ukraine, but not limited to.

### Provide a description that is worded well enough to be understood

Reason to uncheck sharing account credentials: No account needed. Reproduces without cookies.

URL [https://vk.com/video/playlist/-24440848_54694294](https://vk.com/video/playlist/-24440848_54694294)
yt-dlp creates a file called VIDEONAME.unknown_video, instead of a file called VIDEONAME.mp4


### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', 'https://vk.com/video/playlist/-24440848_54694294']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk:uservideos] Extracting URL: https://vk.com/video/playlist/-24440848_54694294
[vk:uservideos] -24440848_54694294: Downloading webpage
[download] Downloading playlist: Расколотая битвой синева небес: Целый год
[vk:uservideos] -24440848: Downloading JSON metadata
[vk:uservideos] Playlist Расколотая битвой синева небес: Целый год: Downloading 20 videos of 20
[download] Downloading video 1 of 20
[debug] [vk] Extracting URL: http://vk.com/video-24440848_456240567
[vk] -24440848_456240567: Downloading JSON metadata
[vk] -24440848_456240567: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] -24440848_456240567: Downloading 1 format(s): url1080
[debug] Invoking http downloader on "https://vkvd17.mycdn.me/?expires=1670023931822&srcIp=104.28.192.97&pr=40&srcAg=CHROME&ms=185.226.52.148&type=5&sig=MUJTO_j6BWs&ct=0&urls=45.136.22.143&clientType=13&appId=512000384397&id=3432280689393"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: Расколотая битвой синева небес： Целый год (серия 20) [-24440848_456240567].unknown_video
[download]  42.0% of  683.68MiB at    1.19MiB/s ETA 05:32


[debug] Command-line config: ['-UvF', 'https://vk.com/video/playlist/-24440848_54694294']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk:uservideos] Extracting URL: https://vk.com/video/playlist/-24440848_54694294
[vk:uservideos] -24440848_54694294: Downloading webpage
[download] Downloading playlist: Расколотая битвой синева небес: Целый год
[vk:uservideos] -24440848: Downloading JSON metadata
[vk:uservideos] Playlist Расколотая битвой синева небес: Целый год: Downloading 20 videos of 20
[download] Downloading video 1 of 20
[debug] [vk] Extracting URL: http://vk.com/video-24440848_456240567
[vk] -24440848_456240567: Downloading JSON metadata
[vk] -24440848_456240567: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for -24440848_456240567:
ID       EXT           RESOLUTION FPS │   FILESIZE   TBR PROTO │ VCODEC    VBR ACODEC  ABR
──────────────────────────────────────────────────────────────────────────────────────────
hls-200  mp4           256x106     25 │ ~ 31.01MiB  201k m3u8  │ unknown  201k unknown  0k
hls-390  mp4           426x178     25 │ ~ 60.38MiB  390k m3u8  │ unknown  390k unknown  0k
url240   unknown_video 240p           │                  https │ unknown       unknown
hls-790  mp4           640x268     25 │ ~122.32MiB  791k m3u8  │ unknown  791k unknown  0k
hls-1313 mp4           852x356     25 │ ~203.18MiB 1314k m3u8  │ unknown 1314k unknown  0k
url360   unknown_video 360p           │                  https │ unknown       unknown
url480   unknown_video 480p           │                  https │ unknown       unknown
hls-2690 mp4           1280x536    25 │ ~416.19MiB 2691k m3u8  │ unknown 2691k unknown  0k
url720   unknown_video 720p           │                  https │ unknown       unknown
hls-4684 mp4           1920x804    25 │ ~724.46MiB 4684k m3u8  │ unknown 4684k unknown  0k
url1080  unknown_video 1080p          │                  https │ unknown       unknown
```



Discussion:
In the extractor, `ext` is not explicity set for the direct http formats, and the video URLs look like this:

`https://vkvd17.mycdn.me/?srcIp=1.2.3.4&pr=40&expires=1670327657930&srcAg=CHROME&fromCache=1&ms=1.2.3.4&type=5&sig=ctT8xnSEW5U&ct=0&urls=1.2.3.4&clientType=13&appId=512000384397&zs=43&id=3432280689393`

so yt-dlp doesn't have any idea which file extension to use.

Does anyone know if VK hosts any direct http videos that are *not* mp4?
If they're all mp4, we could add `'ext': 'mp4'` in here:
https://github.com/yt-dlp/yt-dlp/blob/0e96b408b994678764a89cabbb3879b2c383624a/yt_dlp/extractor/vk.py#L491-L499
@bashonly 
> Does anyone know if VK hosts any direct http videos that are _not_ mp4?

All videos I came across in VK were mp4 but I can't say that VK has only mp4.

IMHO, this issue appears due to the fact `yt-dlp` selects the last format in the format-list and this in turn occurs since it tries to follow `bestvideo*+bestaudio/best`.
For instance:
**1. video with resulted `unknow_video` extension.**
```
[debug] Command-line config: ['-vU', 'http://vk.com/video-24440848_456240567']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk] Extracting URL: http://vk.com/video-24440848_456240567
[vk] -24440848_456240567: Downloading JSON metadata
[vk] -24440848_456240567: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] -24440848_456240567: Downloading 1 format(s): url1080
[debug] Invoking http downloader on "https://vkvd17.mycdn.me/?expires=1670251065151&srcIp=104.28.224.99&pr=40&srcAg=CHROME&ms=185.226.52.148&type=5&sig=RK7RNlZNwyc&ct=0&urls=45.136.22.143&clientType=13&appId=512000384397&id=3432280689393"
[download] Resuming download at byte 29359104
[debug] File locking is not supported. Proceeding without locking
[download] Destination: Расколотая битвой синева небес： Целый год (серия 20) [-24440848_456240567].unknown_video
[download]   5.9% of  683.68MiB at    6.74MiB/s ETA 01:35
ERROR: Interrupted by user

[debug] Command-line config: ['-vUF', 'http://vk.com/video-24440848_456240567']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk] Extracting URL: http://vk.com/video-24440848_456240567
[vk] -24440848_456240567: Downloading JSON metadata
[vk] -24440848_456240567: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for -24440848_456240567:
ID       EXT           RESOLUTION FPS │   FILESIZE   TBR PROTO │ VCODEC    VBR ACODEC  ABR
──────────────────────────────────────────────────────────────────────────────────────────
hls-200  mp4           256x106     25 │ ~ 31.01MiB  201k m3u8  │ unknown  201k unknown  0k
hls-390  mp4           426x178     25 │ ~ 60.38MiB  390k m3u8  │ unknown  390k unknown  0k
url240   unknown_video 240p           │                  https │ unknown       unknown
hls-790  mp4           640x268     25 │ ~122.32MiB  791k m3u8  │ unknown  791k unknown  0k
hls-1313 mp4           852x356     25 │ ~203.18MiB 1314k m3u8  │ unknown 1314k unknown  0k
url360   unknown_video 360p           │                  https │ unknown       unknown
url480   unknown_video 480p           │                  https │ unknown       unknown
hls-2690 mp4           1280x536    25 │ ~416.19MiB 2691k m3u8  │ unknown 2691k unknown  0k
url720   unknown_video 720p           │                  https │ unknown       unknown
hls-4684 mp4           1920x804    25 │ ~724.46MiB 4684k m3u8  │ unknown 4684k unknown  0k
url1080  unknown_video 1080p          │                  https │ unknown       unknown
```

**2. video with correct extension (`mp4`).**
```
[debug] Command-line config: ['-vU', 'http://vk.com/video-213978958_456240301']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk] Extracting URL: http://vk.com/video-213978958_456240301
[vk] -213978958_456240301: Downloading JSON metadata
[vk] -213978958_456240301: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] -213978958_456240301: Downloading 1 format(s): hls-5472
[debug] Invoking hlsnative downloader on "https://vkvd240.mycdn.me/expires/1670251108698/srcIp/104.28.224.99/pr/40/srcAg/CHROME/ms/45.136.20.153/type/5/sig/F834_6bUHfM/ct/8/urls/45.136.22.146/clientType/13/id/3177770453570/video/"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 99
[download] Destination: Звёздный домен： Сороковое тысячелетие ⧸ Сорок тысяч лет культивации 11 серия [TV-1] [FHD 1080p] ⧸ Xing Yu Siwan Nian [-213978958_456240301].mp4
[debug] File locking is not supported. Proceeding without locking
[download]   3.0% of ~ 714.38MiB at    4.64MiB/s ETA 02:29 (frag 3/99)
ERROR: Interrupted by user

[debug] Command-line config: ['-vUF', 'http://vk.com/video-213978958_456240301']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk] Extracting URL: http://vk.com/video-213978958_456240301
[vk] -213978958_456240301: Downloading JSON metadata
[vk] -213978958_456240301: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for -213978958_456240301:
ID       EXT           RESOLUTION FPS │   FILESIZE   TBR PROTO │ VCODEC    VBR ACODEC  ABR
──────────────────────────────────────────────────────────────────────────────────────────
hls-195  mp4           256x144     24 │ ~ 23.38MiB  195k m3u8  │ unknown  195k unknown  0k
url240   unknown_video 240p           │                  https │ unknown       unknown
hls-376  mp4           426x240     24 │ ~ 45.16MiB  377k m3u8  │ unknown  377k unknown  0k
url360   unknown_video 360p           │                  https │ unknown       unknown
hls-756  mp4           640x360     24 │ ~ 90.69MiB  757k m3u8  │ unknown  757k unknown  0k
url480   unknown_video 480p           │                  https │ unknown       unknown
hls-1244 mp4           852x480     24 │ ~149.20MiB 1245k m3u8  │ unknown 1245k unknown  0k
url720   unknown_video 720p           │                  https │ unknown       unknown
hls-2613 mp4           1280x720    24 │ ~313.26MiB 2613k m3u8  │ unknown 2613k unknown  0k
url1080  unknown_video 1080p          │                  https │ unknown       unknown
hls-5472 mp4           1920x1080   24 │ ~656.06MiB 5473k m3u8  │ unknown 5473k unknown  0k
```

Is it possible to check previous format(s) in format-list and use extension from it if the best(last) one is unknown?
Vimeo extractor had the same problem #1402.
Would it make sense for yt-dlp to `ffprobe` the downloaded video and try to deduce the format from there?
Too late by then: the extension is needed for format selection **before** downloading.

Maybe a HEAD, or other, request for the video would reveal a MIME type, as [mentioned above](https://github.com/yt-dlp/yt-dlp/pull/1477/commits/1c5f2503dc38a4d8bd549b4974dfe81c550ce789).
I thought, the problem is that the downloaded file is `video.unknown_video` instead of `video.mp4`. That could be solved after the download by doing `ffprobe` on the file to determine the correct format, I think.
> Too late by then: the extension is needed for format selection **before** downloading.
> 
> Maybe a HEAD, or other, request for the video would reveal a MIME type, as [mentioned above](https://github.com/yt-dlp/yt-dlp/pull/1477/commits/1c5f2503dc38a4d8bd549b4974dfe81c550ce789).

making a head request to every single video url just for format selection wouldn't make sense, imo, unless the user passes `--check-all-formats`

I like the ffprobe idea, although it's not my decision

Or why not just write a proposal for `.unknown_video` to be standardised as a video format? :trollface:

now that I think about it, a similar file extension likely already exists (UPDATE: apparently .vid exists, but nothing seems to support it)
>making a head request to every single video url just for format selection wouldn't make sense, imo, unless the user passes `--check-all-formats`

Only when a video/media URL is found that doesn't convey any format information, via `determine_ext()` or an accompanying MIME type, or similar.

Resolving patch:
diff --git a/yt_dlp/extractor/vk.py b/yt_dlp/extractor/vk.py
--- a/yt_dlp/extractor/vk.py
+++ b/yt_dlp/extractor/vk.py
@@ -451,6 +451,7 @@ def _real_extract(self, url):
             info_page, 'view count', default=None))
 
         formats = []
+        subtitles = {}
         for format_id, format_url in data.items():
             format_url = url_or_none(format_url)
             if not format_url or not format_url.startswith(('http', '//', 'rtmp')):
@@ -462,12 +463,21 @@ def _real_extract(self, url):
                 formats.append({
                     'format_id': format_id,
                     'url': format_url,
+                    'ext': 'mp4',
+                    'source_preference': 1,
                     'height': height,
                 })
             elif format_id == 'hls':
-                formats.extend(self._extract_m3u8_formats(
+                fmts, subs = self._extract_m3u8_formats_and_subtitles(
                     format_url, video_id, 'mp4', 'm3u8_native',
-                    m3u8_id=format_id, fatal=False, live=is_live))
+                    m3u8_id=format_id, fatal=False, live=is_live)
+                formats.extend(fmts)
+                self._merge_subtitles(subs, target=subtitles)
+            elif format_id.startswith('dash_'):
+                fmts, subs = self._extract_mpd_formats_and_subtitles(
+                    format_url, video_id, mpd_id=format_id, fatal=False)
+                formats.extend(fmts)
+                self._merge_subtitles(subs, target=subtitles)
             elif format_id == 'rtmp':
                 formats.append({
                     'format_id': format_id,
@@ -475,7 +485,6 @@ def _real_extract(self, url):
                     'ext': 'flv',
                 })
 
-        subtitles = {}
         for sub in data.get('subs') or {}:
             subtitles.setdefault(sub.get('lang', 'en'), []).append({
                 'ext': sub.get('title', '.srt').split('.')[-1],
@@ -496,6 +505,7 @@ def _real_extract(self, url):
             'comment_count': int_or_none(mv_data.get('commcount')),
             'is_live': is_live,
             'subtitles': subtitles,
+            '_format_sort_fields': ('res', 'source'),
         }
 
 


Test patch: