Repository: yt-dlp
Issue: YouTube --live-from-start keep track of correct fragment number that the downloaded file started with.
Labels: site-enhancement,triage

### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm requesting a site-specific feature
- [X] I've verified that I'm running yt-dlp version **2023.03.04** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

United States of America

### Example URLs

https://www.youtube.com/watch?v=EmHxMxeakBg

### Provide a description that is worded well enough to be understood

When using the experimental feature "--live-from-start" in yt-dlp on a long running videocam livestream in YouTube, the available fragments on the YouTube side tend to fall off the soft cliff of available fragments, at least according to yt-dlp.  yt-dlp handles this by only going back so far.  The problem is that during bugs in the download process such as race conditions in yt-dlp or other network failures that yt-dlp does not know how to handle, sometimes the yt-dlp has to be (manually, or by a script) restarted.  At first, this seems nice, since it can continue the yt-dlp session from the last known fragment downloaded for each file (audio and video separately) using the data in the .ytdl files, but in practice from a live-from-start stream, the fragment resumed is not the correct fragment, since the amount of time that passed between invocations of the yt-dlp commands the first and the second (continued) run time is different and therefore the seek-back in the livestream is different, so the 0 basis for what fragment # is being downloaded is wrong.

In a current example, I am downloading a video in which I seeked back around 8 hours.  The total number of fragments listed is currently 1065517 (and growing every second).  However, it started the download at pseudo-fragment number "0".  It even says it is downloading "0 of 1065517"!  (Then 1, then 2, and so on.)  That's impossible, since the pseudo-fragment number can never get to fragment number 1065517.  I think using absolute fragment numbers would be better; the download would thus start at fragment number, e.g. (I don't actually know), 1054517/1065517 or whatever, and count up from there.  Then, when yt-dlp fails during this download process for whatever reason, and I have to restart, it is able to recalculate the last absolute fragment number downloaded, and continue from there.

I propose an enhancement to the --live-from-start feature that fixes this.  My proposed enhancement is to use absolute fragment numbers when downloading rather than relative pseudo-fragment numbers, and then to use a fragment number offset for beginning of file, rather than a pseudo-fragment number that shifts through each invocation of yt-dlp and incorrectly assumes a start of file at psuedo-fragment number 0 which almost never stays the same across multiple continued invocations of yt-dlp of the same stream.  (As an alternative, if yt-dlp architects are wed to the idea of using pseudo-fragment numbers (yuck), then they should add a way to track what the pseudo-fragment number offset is, so that it can be recreated upon continuance; obviously, I say "yuck" because this just seems like a messy error prone way to do things.)  I suspect this enhancement (and bug fix) requires someone with a strong sense of absolute vs shifting relative references, and shifting done vs not done ranges of such, and does not get confused by such things.

I also propose a second enhancement, that we be able to set begin and end points of the video download, in the past.  I don't even know if this is available as an option for non-livestreams, so this enhancement request also implies such a feature request for non-livestream downloads as well, if it does not already exist, but to be perfectly honest, I don't need that, so no need for me to demand that implication.  I don't know how to specify this time point; it could be so many seconds ago to so many seconds ago.  If this feature is implemented, it would imply that when using absolute fragment numbers as above, that it figure out which absolute fragment range is to be downloaded, and then if continuing from an existing download, to go to the next absolute fragment number to be downloaded, once again to have contiguous continued files without fragments and time skipped.   I suspect this feature request requires a fair bit of coding with someone who has a solid understanding in absolute reference points and relative shifts and shifting done vs not done ranges, and does not get confused by this sort of thing.

I also propose a third enhancement, that we be able to set the maximum allowed amount of time backward we are allowed to download a YouTube livestream.  Currently in the release yt-dlp program, it is set to 5 days (expressed as seconds in the code), but I know for a fact that it goes several days more back, and have successfully downloaded such files, except for the race conditions errors that caused the above bugs.  In my feature request, this option would allow me to set the maximum number of seconds back in a livestream from the start that yt-dlp will attempt to download from, rather than have it hardcoded in the program.  This is a trivial feature request and could be implemented in a few minutes for anyone familiar with Python.  Also, this could be implemented before the above two feature requests, as a poor man's way of limiting downloads to the last # of seconds if we don't want to go back the hardcoded "max" amount, but just want, say, the last 3 days, rather than the last 5 days.  (Perhaps we already have the data from 5 and 4 days ago, and only need days 3, 2, 1, and 0 ago now.  In fact, this is a rather common occurrence in real life for archivists, and is the main usage for --live-from-start usage.  Specifying this in seconds and not days is obviously the correct way to do this, but I used days as a way to describe this conceptually.)

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
These files are so huge, that I don't yet have time to disturb one of my huge long running invocations, and don't have -vU verbose output.  I can return with that the next few times I try this if it makes sense.  For now, I will show a redacted version of a current run:  In this, I manually coded the maximum number of seconds to seek back to 19366 seconds (and recompiled yt-dlp myself):

$ ./yt-dlp/yt-dlp.19366 --live-from-start `cat ../../../yt`
[youtube] Extracting URL: sdjksdjksdk
[youtube] sdjksdjksdk: Downloading webpage
[youtube] sdjksdjksdk: Downloading android player API JSON
[youtube] sdjksdjksdk: Downloading MPD manifest
[youtube] sdjksdjksdk: Downloading MPD manifest
[info] sdjksdjksdk: Downloading 1 format(s): 248+140
[dashsegments] Total fragments: unknown (live)
[download] Destination: Video File Name [sdjksdjksdk].f248.mkv
[dashsegments] Total fragments: unknown (live)
[download] Destination: Video File Name [sdjksdjksdk].f140.mkv
WARNING: [youtube] Starting download from the last xxx hours of the live stream since YouTube does not have data before that. If you think this is wrong, please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[download]   68.65MiB at  474.10KiB/s (00:01:39) (frag 238/1065518)[download] Got error: The read operation timed out. Retrying (1/10)...
[download]  138.74MiB at  490.22KiB/s (00:03:47) (frag 477/1065518)
[download]  139.85MiB at  453.31KiB/s (00:04:08) (frag 481/1065518)
[download]  167.93MiB at  516.70KiB/s (00:05:18) (frag 580/1065518)
[download]  196.97MiB at    2.78KiB/s (00:07:18) (frag 678/1065518)
[download]  217.06MiB at  377.87KiB/s (00:08:02) (frag 751/1065518)[download] Got error: The read operation timed out. Retrying (1/10)...
[download]   42.44MiB at   87.75KiB/s (00:07:57) (frag 1043/1065517)
[download]  311.98MiB at  561.83KiB/s (00:11:19) (frag 1100/1065518)[download] Got error: The read operation timed out. Retrying (1/10)...
[download]  341.95MiB at  445.26KiB/s (00:12:12) (frag 1201/1065518)
[download]  372.42MiB at  504.62KiB/s (00:13:28) (frag 1311/1065518)[download] Got error: The read operation timed out. Retrying (1/10)...
[download]  459.33MiB at  539.46KiB/s (00:16:22) (frag 1663/1065518)
[download]  476.96MiB at  432.65KiB/s (00:17:23) (frag 1737/1065518)[download] Got error: The read operation timed out. Retrying (1/10)...
[download]  612.61MiB at  602.70KiB/s (00:22:32) (frag 2358/1065518)
[download]  631.08MiB at  427.42KiB/s (00:23:34) (frag 2454/1065518)
[download]  641.69MiB at  535.15KiB/s (00:24:18) (frag 2502/1065518)[download] Got error: The read operation timed out. Retrying (1/10)...
[download]  118.61MiB at   82.28KiB/s (00:24:06) (frag 2915/1065517)
[download]  648.51MiB at  321.98KiB/s (00:24:52) (frag 2534/1065518)
[download]  733.53MiB at  592.55KiB/s (00:28:16) (frag 2893/1065518)[download] Got error: <urlopen error _ssl.c:1114: The handshake operation timed out>. Retrying (1/10)...
[download]  797.02MiB at  613.04KiB/s (00:30:29) (frag 3126/1065518)
```