Repository: ytdl-org/youtube-dl

Issue:
Allow merging formats
Allow individual formats in `-f` to consist of multiple format IDs, for example

```
-f 137+141/137+160/37/mp4+mp3
```

A format is picked only if all the IDs it consists of are present (and ffmpeg is available?).  If this options is set, the format name is automatically appended to the video filename while downloading, i.e. we'll get `title-id.ext.f137` and `title-id.ext.f1141` by default. If a format with a plus is picked, the extracted information contains a `_postprocessing: {type:"ffmpeg-merge", "files": ["title-id.ext.f137", "title-id.ext.f1141"]}` hint, and ffmpeg is automatically invoked to create a single file.



Discussion:
Great, this will take care of missing 1080p. Pls. do this with priority.

We also need  -f 135 as 480p is now only available as dash video.

This need to be put as default if other 1080p is not available.

Perfect!

I just came here to file an issue asking for much less (saving audio from dash streams with an extension different from those of the videos---namely, saving AAC audio with extension `m4a`).

Otherwise, to download 480p videos (which I do for lectures and so on with other projects of mine, like `edx-dl`) I have to call youtube-dl twice: once for format 135 and another for format 140, since the old (?) format 35 files are much smaller than the lower resolution 360p files (due to the former being encoded in High profile vs. the latter being encoded in Constrained Baseline profile).

The way we are right now, calling youtube-dl once for format 135 results in a file ending with extension mp4 and doing a successive download call to format 140 gives me the message that the file was already downloaded.

I can obviously workaround this, renaming the files, taking care of the muxing of both the audio and the video etc., but that would be a pain.

Great that other people also felt the same way that I did. :)

Regards,

Rogério.

Another thing: the templates for file naming don't seem to have (after a quick look) a string for the format of the video. This would be a very welcome addition, IMVHO.

It would also fix a bug that I received in Debian quite some time ago: bugs.debian.org/cgi-bin/bugreport.cgi?bug=643831

Should I open a separate issue for the bug above?

Thanks.

That is entirely a separate bug. I think you should open a separate ticket.

On Sat, Oct 19, 2013 at 1:27 AM, Rogério Brito notifications@github.comwrote:

> Another thing: the templates for file naming don't seem to have (after a
> quick look) a string for the format of the video. This would be a very
> welcome addition, IMVHO.
> 
> It would also fix a bug that I received in Debian quite some time ago:
> bugs.debian.org/cgi-bin/bugreport.cgi?bug=643831
> 
> Should I open a separate issue for the bug above?
> 
> Thanks.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/rg3/youtube-dl/issues/1612#issuecomment-26627161
> .

@rbrito 

<pre>
$ youtube-dl b6vSf0cA9qY -o "%(format)s.%(ext)s"
[youtube] Setting language
[youtube] b6vSf0cA9qY: Downloading video webpage
[youtube] b6vSf0cA9qY: Downloading video info webpage
[youtube] b6vSf0cA9qY: Extracting video information
[download] Destination: 22 - 720x1280.mp4
[download] 100% of 46.49MiB in 00:14
</pre>

@rbrito I have documented the format name in the filename template in fdefe96.

@phihag, thanks. I missed that, but I also sent a micro-pull request with another way of working around the output naming.

---

@hdevelop Thank you, we are well aware of basic ffmpeg muxing. If you post, please don't include the names of malicious software. Thanks!

Just for the record, when muxing we may want to partially get rid of
youtube-only metadata via `-map_metadata -1`.

With this option turned on, two muxes of the same audio and video are
(at least in my tests) bit-identical (provided you use the same
versions of ffmpeg/avconv, since they include a tag along the lines
"encoded by libavfoo <version>").

Nice way to help deduplicating filesystems if you happen to download
the same file twice (which I frequently do, from playlists of
lectures, like those from edX).

Regards,

## 

Rogério Brito : rbrito@{ime.usp.br,gmail.com} : GPG key 4096R/BCFCAAAA
http://cynic.cc/blog/ : github.com/rbrito : profiles.google.com/rbrito
DebianQA: http://qa.debian.org/developer.php?login=rbrito%40ime.usp.br

---

With Windows Media Player Classic when you play a DASH-video (mp4), it automatically detects the DASH-audio (m4a). To make it work, I changed the following setting :
enable menu view+options+player+formats+MPEG-4 Audio+choose DirectShow instead of QuickTime

I don't know if the other video players also automatically detects m4a-files when playing mp4-files. I would prefer that in youtube-dl, there is an option to enable/disable the automatically merge of DASH-video+DASH-audio to a single file with ffmpeg.

ok thanx to info by jaimeFM   managed a 4k here which plays on my ubuntu perfectly with mpv  https://launchpad.net/~mc3man/+archive/mpv-tests

 youtube-dl  -t -f 138 k_okcNVZqqI
[youtube] Setting language
[youtube] k_okcNVZqqI: Downloading video webpage
[youtube] k_okcNVZqqI: Downloading video info webpage
[youtube] k_okcNVZqqI: Extracting video information
[download] Destination: 00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.mp4
[download] 100% of 376.18MiB in 12:50

 mediainfo '00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.mp4'
General
Complete name                            : 00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.mp4
Format                                   : dash
Codec ID                                 : dash
File size                                : 376 MiB
Duration                                 : 3mn 4s
Overall bit rate                         : 17.1 Mbps
Encoded date                             : UTC 2013-09-16 07:46:45
Tagged date                              : UTC 2013-09-16 07:46:45

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings, CABAC                   : No
Format settings, ReFrames                : 2 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 3mn 4s
Bit rate mode                            : Constant
Bit rate                                 : 17.1 Mbps
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 23.976 fps
Minimum frame rate                       : 23.974 fps
Maximum frame rate                       : 2 195.122 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.086
Stream size                              : 376 MiB (100%)
Encoded date                             : UTC 2013-09-16 07:46:45
Tagged date                              : UTC 2013-09-16 07:46:45

4kdrops.mp4
General
Complete name                            : 4kdrops.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom
File size                                : 382 MiB
Duration                                 : 3mn 4s
Overall bit rate                         : 17.4 Mbps
Writing application                      : Lavf55.19.103

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings, CABAC                   : No
Format settings, ReFrames                : 2 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 3mn 4s
Bit rate                                 : 17.1 Mbps
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.086
Stream size                              : 376 MiB (99%)

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 3mn 4s
Bit rate mode                            : Constant
Bit rate                                 : 254 Kbps
Channel count                            : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 44.1 KHz
Compression mode                         : Lossy
Delay relative to video                  : 42ms
Stream size                              : 5.58 MiB (1%)

then audio

youtube-dl   -t --extract-audio -f 141 k_okcNVZqqI
[youtube] Setting language
[youtube] k_okcNVZqqI: Downloading video webpage
[youtube] k_okcNVZqqI: Downloading video info webpage4kdrops.mp4
General4kdrops.mp4
General
Complete name                            : 4kdrops.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom
File size                                : 382 MiB
Duration                                 : 3mn 4s
Overall bit rate                         : 17.4 Mbps
Writing application                      : Lavf55.19.103

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings, CABAC                   : No
Format settings, ReFrames                : 2 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 3mn 4s
Bit rate                                 : 17.1 Mbps
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.086
Stream size                              : 376 MiB (99%)

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 3mn 4s
Bit rate mode                            : Constant
Bit rate                                 : 254 Kbps
Channel count                            : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 44.1 KHz
Compression mode                         : Lossy
Delay relative to video                  : 42ms
Stream size                              : 5.58 MiB (1%)

Complete name                            : 4kdrops.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom
File size                                : 382 MiB
Duration                                 : 3mn 4s
Overall bit rate                         : 17.4 Mbps
Writing application                      : Lavf55.19.103

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings, CABAC                   : No
Format settings, ReFrames                : 2 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 3mn 4s
Bit rate                                 : 17.1 Mbps
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.086
Stream size                              : 376 MiB (99%)

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 3mn 4s
Bit rate mode                            : Constant
Bit rate                                 : 254 Kbps
Channel count                            : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 44.1 KHz
Compression mode                         : Lossy
Delay relative to video                  : 42ms
Stream size                              : 5.58 MiB (1%)

[youtube] k_okcNVZqqI: Extracting video information
[download] Destination: 00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.m4a
[download] 100% of 5.61MiB in 00:11
[youtube] Post-process file 00001-INK DROPS 4K (ULTRA HD)-k_okcNVZqqI.m4a exists, skipping

to bind mute mp4 and sound m4a 256k

ffmpeg -i _INK_.mp4 -i _INK_.m4a  -c copy 4kdrops.mp4  

which gives us

4kdrops.mp4
General
Complete name                            : 4kdrops.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom
File size                                : 382 MiB
Duration                                 : 3mn 4s
Overall bit rate                         : 17.4 Mbps
Writing application                      : Lavf55.19.103

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L5.1
Format settings, CABAC                   : No
Format settings, ReFrames                : 2 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 3mn 4s
Bit rate                                 : 17.1 Mbps
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.086
Stream size                              : 376 MiB (99%)

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 3mn 4s
Bit rate mode                            : Constant
Bit rate                                 : 254 Kbps
Channel count                            : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 44.1 KHz
Compression mode                         : Lossy
Delay relative to video                  : 42ms
Stream size                              : 5.58 MiB (1%)


Resolving patch:
diff --git a/youtube_dl/PostProcessor.py b/youtube_dl/PostProcessor.py
--- a/youtube_dl/PostProcessor.py
+++ b/youtube_dl/PostProcessor.py
@@ -508,3 +508,11 @@ def run(self, info):
         os.remove(encodeFilename(filename))
         os.rename(encodeFilename(temp_filename), encodeFilename(filename))
         return True, info
+
+
+class FFmpegMergerPP(FFmpegPostProcessor):
+    def run(self, info):
+        filename = info['filepath']
+        args = ['-c', 'copy']
+        self.run_ffmpeg_multiple_files(info['__files_to_merge'], filename, args)
+        return True, info
diff --git a/youtube_dl/YoutubeDL.py b/youtube_dl/YoutubeDL.py
--- a/youtube_dl/YoutubeDL.py
+++ b/youtube_dl/YoutubeDL.py
@@ -51,9 +51,11 @@
     write_json_file,
     write_string,
     YoutubeDLHandler,
+    prepend_extension,
 )
 from .extractor import get_info_extractor, gen_extractors
 from .downloader import get_suitable_downloader
+from .PostProcessor import FFmpegMergerPP
 from .version import __version__
 
 
@@ -704,7 +706,17 @@ def process_video_result(self, info_dict, download=True):
             # the first that is available, starting from left
             req_formats = req_format.split('/')
             for rf in req_formats:
-                selected_format = self.select_format(rf, formats)
+                if re.match(r'.+?\+.+?', rf) is not None:
+                    # Two formats have been requested like '137+139'
+                    format_1, format_2 = rf.split('+')
+                    formats_info = (self.select_format(format_1, formats),
+                        self.select_format(format_2, formats))
+                    if all(formats_info):
+                        selected_format = {'requested_formats': formats_info}
+                    else:
+                        selected_format = None
+                else:
+                    selected_format = self.select_format(rf, formats)
                 if selected_format is not None:
                     formats_to_download = [selected_format]
                     break
@@ -880,10 +892,27 @@ def process_info(self, info_dict):
                 success = True
             else:
                 try:
-                    fd = get_suitable_downloader(info_dict)(self, self.params)
-                    for ph in self._progress_hooks:
-                        fd.add_progress_hook(ph)
-                    success = fd.download(filename, info_dict)
+                    def dl(name, info):
+                        fd = get_suitable_downloader(info)(self, self.params)
+                        for ph in self._progress_hooks:
+                            fd.add_progress_hook(ph)
+                        return fd.download(name, info)
+                    if info_dict.get('requested_formats') is not None:
+                        downloaded = []
+                        success = True
+                        for f in info_dict['requested_formats']:
+                            new_info = dict(info_dict)
+                            new_info.update(f)
+                            fname = self.prepare_filename(new_info)
+                            fname = prepend_extension(fname, 'f%s' % f['format_id'])
+                            downloaded.append(fname)
+                            partial_success = dl(fname, new_info)
+                            success = success and partial_success
+                        info_dict['__postprocessors'] = [FFmpegMergerPP(self)]
+                        info_dict['__files_to_merge'] = downloaded
+                    else:
+                        # Just a single file
+                        success = dl(filename, info_dict)
                 except (compat_urllib_error.URLError, compat_http_client.HTTPException, socket.error) as err:
                     self.report_error(u'unable to download video data: %s' % str(err))
                     return
@@ -940,7 +969,11 @@ def post_process(self, filename, ie_info):
         info = dict(ie_info)
         info['filepath'] = filename
         keep_video = None
-        for pp in self._pps:
+        pps_chain = []
+        if ie_info.get('__postprocessors') is not None:
+            pps_chain.extend(ie_info['__postprocessors'])
+        pps_chain.extend(self._pps)
+        for pp in pps_chain:
             try:
                 keep_video_wish, new_info = pp.run(info)
                 if keep_video_wish is not None:


Test patch: