Repository: yt-dlp
Issue: [FFmpegFixupDuplicateMoov] Improve performance
Labels: enhancement,PR-needed,core:post-processor

### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a bug unrelated to a specific site
- [X] I've verified that I'm running yt-dlp version **2023.06.22** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Provide a description that is worded well enough to be understood

When downloading livestreams or DVRs, yt-dlp sometimes downloads them as segmented. This causes ffmpeg to put out a bunch of "Found duplicated MOOV Atom" warnings and slows down the merge/copy by a lot. The slowdown depends on the specific IO setup - on my SSD it isn't as noticeable, but on my server with an HDD array it causes some long livestreams to stall at the ffmpeg command for hours (and this is before ffmpeg actually creates the output file, at the probing stage).

yt-dlp seems to have features such as "FixupM4a" and "FixupDuplicateMoov" to remedy this issue, but they don't seem to run for me when merging formats. There was a commit (26010b5cec50193b98ad7845d1d77450f9f14c2b) recently that affected the latter - I've made sure that my yt-dlp is built from a source that includes this commit. While I've never looked into this issue in detail before, I've experienced similar extreme slowdowns from downloading streams a lot in the past, so I assume that this has been an issue for a while and isn't caused by that commit.

As an example, we can try downloading a few minutes of an active livestream from the beginning:  
`yt-dlp -vU --ignore-config --live-from-start -f 137+140 -k https://www.youtube.com/watch?v=jfKfPfyJRdk`
(output is in *Complete Verbose Output*, I pressed ctrl+c after 5 minutes was downloaded)

Depending on how long we download for, the final merge will be slow. In this example the final merge was fast because I only downloaded 5 minutes of the stream, but we can still look at the downloaded files kept by `-k` to see the ffmpeg behaviour.

Here I ran `ffmpeg -i` on just the video (this will not copy/encode anything):

```
$ ffmpeg -i 'lofi hip hop radio 📚 - beats to relax⧸study to [jfKfPfyJRdk].f137.mp4'
ffmpeg version N-66244-g468615f204-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      58. 13.101 / 58. 13.101
  libavcodec     60. 21.100 / 60. 21.100
  libavformat    60.  9.100 / 60.  9.100
  libavdevice    60.  2.100 / 60.  2.100
  libavfilter     9.  8.102 /  9.  8.102
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5e73a00] Found duplicated MOOV Atom. Skipped it
    Last message repeated 412 times
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'lofi hip hop radio 📚 - beats to relax⧸study to [jfKfPfyJRdk].f137.mp4':
  Metadata:
    minor_version   : 0
    major_brand     : dash
    compatible_brands: iso6avc1mp41
  Duration: 6619:48:03.61, start: 23829213.605233, bitrate: 0 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 0 kb/s, 30 fps, 30 tbr, 90k tbn (default)
    Metadata:
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
At least one output file must be specified
```

You can see all of the repeated "Found duplicated MOOV Atom" messages, which seem to cause the slowdown. Running this command with the f137 audio also results in the same behaviour. I've used strace to check the slow ffmpeg run by yt-dlp directly and the behaviour matches that of this.

For a more practical example that has really bad slowdown, I took the m4a from a yt-dlp download of a 7 hour livestream and ran it through `ffmpeg -i`:

1. On my HDD (over 9p)
2. On my SSD
3. On my SSD, but fixed (run through `ffmpeg -c copy` beforehand)

[Here](https://www.youtube.com/watch?v=Oh1IuithsFE) are the results as a screen recording, but in summary:

1. The HDD was very slow, at the rate it was going it would've taken 1.5 hours to finish.
2. The SSD took about 20 seconds
3. The fixed file took less than a second

I hope this issue doesn't sound too rambly, feel free to ask for more info if necessary.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', '--ignore-config', '--live-from-start', '-f', '137+140', '-k', 'https://www.youtube.com/watch?v=jfKfPfyJRdk']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.06.22 [812cdfa06]
[debug] Lazy loading extractors is disabled
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-75-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, secretstorage-3.3.1, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.06.22, Current version: stable@2023.06.22
yt-dlp is up to date (stable@2023.06.22)
[youtube] Extracting URL: https://www.youtube.com/watch?v=jfKfPfyJRdk
[youtube] jfKfPfyJRdk: Downloading webpage
[youtube] jfKfPfyJRdk: Downloading ios player API JSON
[youtube] jfKfPfyJRdk: Downloading android player API JSON
[youtube] jfKfPfyJRdk: Downloading MPD manifest
[youtube] jfKfPfyJRdk: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[info] jfKfPfyJRdk: Downloading 1 format(s): 137+140
[debug] Invoking dashsegments downloader on "https://manifest.googlevideo.com/api/manifest/dash/expire/1688081263/ei/D7-dZIGZLMmUv_IP6v65sAo/ip/165.231.178.20/id/jfKfPfyJRdk.2/source/yt_live_broadcast/requiressl/yes/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/Ul2SqzlbJFKYc5o4u4JTOP_AaKgMtYI/vprv/1/pacing/0/itag_bl/376%2C377%2C384%2C385%2C612%2C613%2C617%2C619%2C623%2C628%2C655%2C656%2C660%2C662%2C666%2C671/keepalive/yes/fexp/24007246%2C51000013/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AOq0QJ8wRQIgL08qIWlRb38CjsOmMh0g7G7c3-a2vSrAZayaHQep3ccCIQDf4cfU63QSoXWhwmEAXlQaxblYRItb_g2rBEgm6GDLAw%3D%3D", "https://manifest.googlevideo.com/api/manifest/dash/expire/1688081263/ei/D7-dZIGZLMmUv_IP6v65sAo/ip/165.231.178.20/id/jfKfPfyJRdk.2/source/yt_live_broadcast/requiressl/yes/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/spc/Ul2SqzlbJFKYc5o4u4JTOP_AaKgMtYI/vprv/1/pacing/0/itag_bl/376%2C377%2C384%2C385%2C612%2C613%2C617%2C619%2C623%2C628%2C655%2C656%2C660%2C662%2C666%2C671/keepalive/yes/fexp/24007246%2C51000013/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Cas%2Cspc%2Cvprv%2Citag%2Cplaylist_type/sig/AOq0QJ8wRQIgL08qIWlRb38CjsOmMh0g7G7c3-a2vSrAZayaHQep3ccCIQDf4cfU63QSoXWhwmEAXlQaxblYRItb_g2rBEgm6GDLAw%3D%3D"
[dashsegments] Total fragments: unknown (live)
[download] Destination: lofi hip hop radio 📚 - beats to relax⧸study to [jfKfPfyJRdk].f137.mp4
[dashsegments] Total fragments: unknown (live)
[download] Destination: lofi hip hop radio 📚 - beats to relax⧸study to [jfKfPfyJRdk].f140.mp4
WARNING: [youtube] Starting download from the last 120 hours of the live stream since YouTube does not have data before that. If you think this is wrong, please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[debug] [youtube] [jfKfPfyJRdk] Generating fragments for format 137
[debug] [youtube] [jfKfPfyJRdk] Generating fragments for format 140
[download] 100% of  191.56MiB in 00:05:23 at 605.59KiB/s
[download] 100% of   49.17MiB in 00:05:23 at 155.46KiB/s
[Merger] Merging formats into "lofi hip hop radio 📚 - beats to relax⧸study to [jfKfPfyJRdk].mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:lofi hip hop radio 📚 - beats to relax⧸study to [jfKfPfyJRdk].f137.mp4' -i 'file:lofi hip hop radio 📚 - beats to relax⧸study to [jfKfPfyJRdk].f140.mp4' -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart 'file:lofi hip hop radio 📚 - beats to relax⧸study to [jfKfPfyJRdk].temp.mp4'
```