Repository: yt-dlp/yt-dlp

Issue:
Some Youtube m3u8 formats fail to download with a 404
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting that yt-dlp is broken on a **supported** site
- [X] I've verified that I'm running yt-dlp version **2023.07.06** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [ ] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

Netherlands

### Provide a description that is worded well enough to be understood

This video (https://www.youtube.com/watch?v=V5yN4xSKxRM) cannot be downloaded, exiting with the error 
`ERROR: unable to download video data: HTTP Error 404: Not Found`. 
I can play the video in a browser just fine, even without being logged in on Youtube. Other videos from the channel can be downloaded without issues. I remember encountering the same problem on another video some time ago (unfortunately I am not sure whether it was the same channel), but thought then it was just a temporary glitch.
 

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
~ yt-dlp -vU https://www.youtube.com/watch\?v\=V5yN4xSKxRM
[debug] Command-line config: ['--config-location', '/Users/alekoe/yt-dlp/config', '-vU', 'https://www.youtube.com/watch?v=V5yN4xSKxRM']
[debug] | Config "/Users/alekoe/yt-dlp/config": ['-o', '/Users/alekoe/Movies/Youtube/%(uploader)s/%(title)s.%(ext)s', '-f', 'bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4] / bv*+ba/b', '--download-archive', '/Users/alekoe/yt-dlp/archive.txt']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.07.06 [b532a3481] (zip)
[debug] Python 3.11.4 (CPython arm64 64bit) - macOS-13.4.1-arm64-arm-64bit (OpenSSL 3.1.1 30 May 2023)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0
[debug] Optional libraries: sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Loading archive file '/Users/alekoe/yt-dlp/archive.txt'
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.07.06, Current version: stable@2023.07.06
Current Build Hash: cedf44a9cf3c241520a692c1006d276e6f434d4d945727c47cb1a70d32953f4a
yt-dlp is up to date (stable@2023.07.06)
[youtube] Extracting URL: https://www.youtube.com/watch?v=V5yN4xSKxRM
[youtube] V5yN4xSKxRM: Downloading webpage
[youtube] V5yN4xSKxRM: Downloading ios player API JSON
[youtube] V5yN4xSKxRM: Downloading android player API JSON
[debug] Loading youtube-nsig.0e6aaa83 from cache
[debug] [youtube] Decrypted nsig 3b4yG4cOBVYUqfLKTB => xREnxB6z0FDm4Q
[youtube] V5yN4xSKxRM: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[info] V5yN4xSKxRM: Downloading 1 format(s): 617+140
[debug] Invoking hlsnative downloader on "https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1690951820/ei/LIzJZOPlJMLQ1gLf5YHYDg/ip/2a02:a212:2241:3b80:8c17:9ec9:1d0:a29d/id/579c8de3148ac513/itag/617/source/youtube/requiressl/yes/ratebypass/yes/pfa/1/wft/1/sgovp/clen%3D441607860%3Bdur%3D1517.016%3Bgir%3Dyes%3Bitag%3D303%3Blmt%3D1690855624695842/hls_chunk_host/rr2---sn-5hneknee.googlevideo.com/mh/Un/mm/31,29/mn/sn-5hneknee,sn-5hne6nsy/ms/au,rdu/mv/m/mvi/2/pl/28/initcwndbps/2168750/bui/AYlvQAubp_5xZFxoJPwO7ClBPOCe_5Ur5ozlzjVc_eok-DdjFybKUS0ezqzuH9TPNWtm26AidxufaWGJioOKZMvZpvHXuF0Y/vprv/1/playlist_type/DVR/dover/13/txp/3309224/mt/1690929897/fvip/4/short_key/1/keepalive/yes/fexp/24007246,51000014/beids/24350018/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,pfa,wft,sgovp,bui,vprv,playlist_type/sig/AOq0QJ8wRQIga-bVEuOKllOaG83nAEVb4_nto79dDK0EPCMhwGijWowCIQC4Vds1rER_wDnsZ3z2SIh9VTnoUM4CTUNJpDifNn44qg%3D%3D/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl,initcwndbps/lsig/AG3C_xAwRQIgdZAKONrPWnHIAbaHkCP1PlquuGJIX9R2K4d-vbJ0zmwCIQDrH0ScbgVeyaxze8_DV5TQ7O7l65mAnQQtw1t0XjqEmA%3D%3D/playlist/index.m3u8"
[hlsnative] Downloading m3u8 manifest
ERROR: unable to download video data: HTTP Error 404: Not Found
Traceback (most recent call last):
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3345, in process_info
    partial_success, real_download = self.dl(fname, new_info)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3084, in dl
    return fd.download(name, new_info, subtitle)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/yt-dlp/yt_dlp/downloader/common.py", line 461, in download
    ret = self.real_download(filename, info_dict)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/yt-dlp/yt_dlp/downloader/hls.py", line 77, in real_download
    urlh = self.ydl.urlopen(self._prepare_url(info_dict, man_url))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3863, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
```



Discussion:

Resolving patch:
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -2338,7 +2338,7 @@ def _check_formats(formats):
                 return
 
             for f in formats:
-                if f.get('has_drm'):
+                if f.get('has_drm') or f.get('__needs_testing'):
                     yield from self._check_formats([f])
                 else:
                     yield f
diff --git a/yt_dlp/extractor/youtube.py b/yt_dlp/extractor/youtube.py
--- a/yt_dlp/extractor/youtube.py
+++ b/yt_dlp/extractor/youtube.py
@@ -4560,6 +4560,14 @@ def process_language(container, base_url, lang_code, sub_name, query):
                 self._parse_time_text(self._get_text(vpir, 'dateText'))) or upload_date
         info['upload_date'] = upload_date
 
+        if upload_date and live_status not in ('is_live', 'post_live', 'is_upcoming'):
+            # Newly uploaded videos' HLS formats are potentially problematic and need to be checked
+            upload_datetime = datetime_from_str(upload_date).replace(tzinfo=datetime.timezone.utc)
+            if upload_datetime >= datetime_from_str('today-1day'):
+                for fmt in info['formats']:
+                    if fmt.get('protocol') == 'm3u8_native':
+                        fmt['__needs_testing'] = True
+
         for s_k, d_k in [('artist', 'creator'), ('track', 'alt_title')]:
             v = info.get(s_k)
             if v:


Test patch: