Repository: ytdl-org/youtube-dl

Issue:
[Vimeo] ERROR: Unable to download JSON metadata: HTTP Error 404
since the tool asked me to report im putting this here and hope its as intended 
i just put in the output with the error 
i used the latest version and also updated ffmpeg to check if it has something to do with it

-------------------

youtube-dl -f bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best https://vimeo.com/581039021/9603038895 -o B:\yt/%(title)s.%(ext)s--verbose --merge-output-format mp4
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best', 'https://vimeo.com/581039021/9603038895', '-o', 'B:\\yt/%(title)s.%(ext)s', '--verbose', '--merge-output-format', 'mp4']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 4.2.2
[debug] Proxy map: {}
[vimeo] 9603038895: Downloading webpage
[vimeo] 9603038895: Downloading JSON metadata
[vimeo] 9603038895: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpkqxnwl31\build\youtube_dl\extractor\common.py", line 634, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpkqxnwl31\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

-------------------

ive checked with a older video to see if there is some vimeo related issue but the older url works perfectly -- gonna post the "**working**" example now

------------------


youtube-dl -f bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best https://vimeo.com/580888053/a127b5ccd0 -o B:\yt/%(title)s.%(ext)s--verbose --merge-output-format mp4
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best', 'https://vimeo.com/580888053/a127b5ccd0', '-o', 'B:\\yt/%(title)s.%(ext)s', '--verbose', '--merge-output-format', 'mp4']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 2021-07-27-git-0068b3d0f0-full_build-www.gyan.dev
[debug] Proxy map: {}
[vimeo] 580888053: Downloading JSON metadata
[vimeo] 580888053: Downloading JSON metadata
[vimeo] 580888053: Downloading JSON metadata
[vimeo] 580888053: Downloading fastly_skyfire m3u8 information
[vimeo] 580888053: Downloading fastly_skyfire m3u8 information
[vimeo] 580888053: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 580888053: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 580888053: Downloading fastly_skyfire MPD information
[vimeo] 580888053: Downloading fastly_skyfire MPD information
[vimeo] 580888053: Downloading akfire_interconnect_quic MPD information
[vimeo] 580888053: Downloading akfire_interconnect_quic MPD information
[debug] Invoking downloader on 'https://skyfire.vimeocdn.com/1627634832-0x16a4d5563ea72c9611c90b1709e96e32fa1843c6/7062d513-389f-4cc6-adb9-a9c2cf1a35bf/sep/video/b72d696c,360af58c,c69af52d,fa60c372,ebaf0f3a/master.mpd?base64_init=1'
[dashsegments] Total fragments: 156
[download] Destination: B:\yt\Thursday, July 29, 2021 BMA Mid-Day Video Update.fdash-fastly_skyfire_sep-video-fa60c372.mp4
[download] 100% of 118.54MiB in 00:11
[debug] Invoking downloader on 'https://skyfire.vimeocdn.com/1627634832-0x16a4d5563ea72c9611c90b1709e96e32fa1843c6/7062d513-389f-4cc6-adb9-a9c2cf1a35bf/sep/video/b72d696c,360af58c,c69af52d,fa60c372,ebaf0f3a/master.mpd?base64_init=1'
[dashsegments] Total fragments: 156
[download] Destination: B:\yt\Thursday, July 29, 2021 BMA Mid-Day Video Update.fdash-fastly_skyfire_sep-audio-b72d696c.m4a
[download] 100% of 28.62MiB in 00:07
[ffmpeg] Merging formats into "B:\yt\Thursday, July 29, 2021 BMA Mid-Day Video Update.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:B:\yt\Thursday, July 29, 2021 BMA Mid-Day Video Update.fdash-fastly_skyfire_sep-video-fa60c372.mp4" -i "file:B:\yt\Thursday, July 29, 2021 BMA Mid-Day Video Update.fdash-fastly_skyfire_sep-audio-b72d696c.m4a" -c copy -map "0:v:0" -map "1:a:0" "file:B:\yt\Thursday, July 29, 2021 BMA Mid-Day Video Update.temp.mp4"
Deleting original file B:\yt\Thursday, July 29, 2021 BMA Mid-Day Video Update.fdash-fastly_skyfire_sep-video-fa60c372.mp4 (pass -k to keep)
Deleting original file B:\yt\Thursday, July 29, 2021 BMA Mid-Day Video Update.fdash-fastly_skyfire_sep-audio-b72d696c.m4a (pass -k to keep)

------------------

the working video was uploaded a few hours befor the "not working" one
working: https://vimeo.com/580888053/a127b5ccd0 (upload around 12 hours agoaround 3 PM CST Thursday July 29 2021)
not working: https://vimeo.com/581039021/9603038895 (upload around 3 hours ago around 11 PM CST Thursday July 29 2021)

in case more information are required let me know im hoping ive done everything as wanted 
dayta

side note: im using win7 thus using the latest version of phthon that was available and working tryed different ones without any change in result.. the newest version of python does not work on win 7 but that shouldnt be part of the problem itself since the error only occours with this one url out of alot of others which are still working perfectly


Discussion:
The problem comes when fetching the video details.

The working video is correctly identified as having the 'ID/hash' (`580888053/a127b5ccd0`) pattern and the extractor goes on to fetch the video details using `api.vimeo.com` without loading the original page itself. From the debug log we can see that it is being handled with ID 580888053. The similar URL from the extractor's tests also works.

The failing video is identified with ID 9603038895 instead of 581039021 and unsurprisingly this means that the extractor fails to find the video details.

The problem occurs when a Vimeo URL ends with '/ID/hash' and both ID and hash are numeric. The pattern match skips over the ID part and assigns the ID from the hash part. One component of the URL pattern needs to be constrained (non-greedy `(?:.*?/)??` instead of `(?:.*?/)?`), and then we get:
```
# youtube-dl -F -v 'https://vimeo.com/581039021/9603038895'
[debug] System config: [u'--restrict-filenames', u'--prefer-ffmpeg', u'-f', u'best[height<=?1080][fps<=?60]', u'-o', u'/media/drive1/Video/%(title)s.%(ext)s']
[debug] User config: [u'-f', u'(best/bestvideo+bestaudio)[height<=?1080][fps<=?60][tbr<=?1900]']
[debug] Custom config: []
[debug] Command-line args: [u'-F', u'-v', u'https://vimeo.com/581039021/9603038895']
[debug] Encodings: locale ASCII, fs ASCII, out ASCII, pref ASCII
[debug] youtube-dl version 2021.06.06.1
[debug] Python version 2.7.1 (CPython) - Linux-2.6.18-7.1-7405b0-smp-with-libc0
[debug] exe versions: ffmpeg 4.1, ffprobe 4.1
[debug] Proxy map: {}
[vimeo] 581039021: Downloading JSON metadata
[vimeo] 581039021: Downloading JSON metadata
[vimeo] 581039021: Downloading JSON metadata
[vimeo] 581039021: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 581039021: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 581039021: Downloading fastly_skyfire m3u8 information
[vimeo] 581039021: Downloading fastly_skyfire m3u8 information
[vimeo] 581039021: Downloading akfire_interconnect_quic MPD information
[vimeo] 581039021: Downloading akfire_interconnect_quic MPD information
[vimeo] 581039021: Downloading fastly_skyfire MPD information
[vimeo] 581039021: Downloading fastly_skyfire MPD information
[info] Available formats for 581039021:
format code                                                                                        extension  resolution note
hls-fastly_skyfire-1501                                                                            mp4        1920x1080  1501k , avc1.640028, 30.0fps, mp4a.40.2
hls-akfire_interconnect_quic-1501                                                                  mp4        1920x1080  1501k , avc1.640028, 30.0fps, mp4a.40.2
http-1080p                                                                                         mp4        1920x1080  30fps
hls-fastly_skyfire-916                                                                             mp4        1280x720    916k , avc1.640020, 30.0fps, mp4a.40.2
hls-akfire_interconnect_quic-916                                                                   mp4        1280x720    916k , avc1.640020, 30.0fps, mp4a.40.2
http-720p                                                                                          mp4        1280x720   30fps
dash-fastly_skyfire-video-7248713b                                                                 mp4        960x540    DASH video 1115k , mp4_dash container, avc1.64001F, 30fps, mp4a.40.2 (48000Hz)
dash-akfire_interconnect_quic-video-7248713b                                                       mp4        960x540    DASH video 1115k , mp4_dash container, avc1.64001F, 30fps, mp4a.40.2 (48000Hz)
hls-fastly_skyfire-661                                                                             mp4        960x540     661k , avc1.64001F, 30.0fps, mp4a.40.2
hls-akfire_interconnect_quic-661                                                                   mp4        960x540     661k , avc1.64001F, 30.0fps, mp4a.40.2
http-540p                                                                                          mp4        960x540    30fps
dash-fastly_skyfire-video-ded7a2c1                                                                 mp4        640x360    DASH video  440k , mp4_dash container, avc1.64001E, 30fps, mp4a.40.2 (48000Hz)
dash-akfire_interconnect_quic-video-ded7a2c1                                                       mp4        640x360    DASH video  440k , mp4_dash container, avc1.64001E, 30fps, mp4a.40.2 (48000Hz)
hls-fastly_skyfire-337                                                                             mp4        640x360     337k , avc1.64001E, 30.0fps, mp4a.40.2
hls-akfire_interconnect_quic-337                                                                   mp4        640x360     337k , avc1.64001E, 30.0fps, mp4a.40.2
http-360p                                                                                          mp4        640x360    30fps
http-240p                                                                                          mp4        426x240    30fps
hls-fastly_skyfire_sep-1501+dash-fastly_skyfire_sep-audio-7248713b                                 mp4        1920x1080  avc1.640028, 30.0fps, mp4a.40.2
hls-akfire_interconnect_quic_sep-1501+dash-akfire_interconnect_quic_sep-audio-7248713b             mp4        1920x1080  avc1.640028, 30.0fps, mp4a.40.2
dash-fastly_skyfire_sep-video-36b397cb+dash-fastly_skyfire_sep-audio-ded7a2c1                      mp4        1280x720   avc1.640020, 30fps, mp4a.40.2
dash-akfire_interconnect_quic_sep-video-36b397cb+dash-akfire_interconnect_quic_sep-audio-ded7a2c1  mp4        1280x720   avc1.640020, 30fps, mp4a.40.2
hls-fastly_skyfire_sep-916+dash-fastly_skyfire_sep-audio-06e865e1                                  mp4        1280x720   avc1.640020, 30.0fps, opus 
hls-akfire_interconnect_quic_sep-916+dash-akfire_interconnect_quic_sep-audio-06e865e1              mp4        1280x720   avc1.640020, 30.0fps, opus 
dash-fastly_skyfire_sep-video-7248713b+dash-fastly_skyfire_sep-audio-8f1b4276                      mp4        960x540    avc1.64001F, 30fps, opus 
dash-akfire_interconnect_quic_sep-video-7248713b+dash-akfire_interconnect_quic_sep-audio-8f1b4276  mp4        960x540    avc1.64001F, 30fps, opus 
hls-fastly_skyfire_sep-533+hls-fastly_skyfire_sep-audio-medium-audio                               mp4        960x540    avc1.64001F, 30.0fps
hls-akfire_interconnect_quic_sep-533+hls-fastly_skyfire_sep-audio-high-audio                       mp4        960x540    avc1.64001F, 30.0fps
hls-fastly_skyfire_sep-336+hls-akfire_interconnect_quic_sep-audio-medium-audio                     mp4        640x360    avc1.64001E, 30.0fps
hls-akfire_interconnect_quic_sep-336+hls-akfire_interconnect_quic_sep-audio-high-audio             mp4        640x360    avc1.64001E, 30.0fps (best)
#
``` 


Resolving patch:
diff --git a/youtube_dl/YoutubeDL.py b/youtube_dl/YoutubeDL.py
--- a/youtube_dl/YoutubeDL.py
+++ b/youtube_dl/YoutubeDL.py
@@ -1529,7 +1529,7 @@ def sanitize_numeric_fields(info):
                 # see http://bugs.python.org/issue1646728)
                 try:
                     upload_date = datetime.datetime.utcfromtimestamp(info_dict[ts_key])
-                    info_dict[date_key] = upload_date.strftime('%Y%m%d')
+                    info_dict[date_key] = compat_str(upload_date.strftime('%Y%m%d'))
                 except (ValueError, OverflowError, OSError):
                     pass
 
diff --git a/youtube_dl/extractor/vimeo.py b/youtube_dl/extractor/vimeo.py
--- a/youtube_dl/extractor/vimeo.py
+++ b/youtube_dl/extractor/vimeo.py
@@ -271,7 +271,7 @@ class VimeoIE(VimeoBaseInfoExtractor):
                         )?
                         vimeo(?:pro)?\.com/
                         (?!(?:channels|album|showcase)/[^/?#]+/?(?:$|[?#])|[^/]+/review/|ondemand/)
-                        (?:.*?/)?
+                        (?:.*?/)??
                         (?:
                             (?:
                                 play_redirect_hls|
@@ -517,14 +517,28 @@ class VimeoIE(VimeoBaseInfoExtractor):
             'url': 'https://vimeo.com/7809605',
             'only_matching': True,
         },
-        {
-            'url': 'https://vimeo.com/160743502/abd0e13fb4',
-            'only_matching': True,
-        },
         {
             # requires passing unlisted_hash(a52724358e) to load_download_config request
             'url': 'https://vimeo.com/392479337/a52724358e',
             'only_matching': True,
+        },
+        {
+            # similar, but all numeric: ID must be 581039021, not 9603038895
+            # issue #29690
+            'url': 'https://vimeo.com/581039021/9603038895',
+            'info_dict': {
+                'id': '581039021',
+                # these have to be provided but we don't care
+                'ext': 'mp4',
+                'timestamp': 1627621014,
+                'title': 're:.+',
+                'uploader_id': 're:.+',
+                'uploader': 're:.+',
+                'upload_date': r're:\d+',
+            },
+            'params': {
+                'skip_download': True,
+            },
         }
         # https://gettingthingsdone.com/workflowmap/
         # vimeo embed with check-password page protected by Referer header


Test patch:
diff --git a/test/test_YoutubeDL.py b/test/test_YoutubeDL.py
--- a/test/test_YoutubeDL.py
+++ b/test/test_YoutubeDL.py
@@ -997,6 +997,25 @@ def _real_extract(self, url):
         self.assertEqual(downloaded['extractor'], 'Video')
         self.assertEqual(downloaded['extractor_key'], 'Video')
 
+    def test_default_times(self):
+        """Test addition of missing upload/release/_date from /release_/timestamp"""
+        info = {
+            'id': '1234',
+            'url': TEST_URL,
+            'title': 'Title',
+            'ext': 'mp4',
+            'timestamp': 1631352900,
+            'release_timestamp': 1632995931,
+        }
+
+        params = {'simulate': True, }
+        ydl = FakeYDL(params)
+        out_info = ydl.process_ie_result(info)
+        self.assertTrue(isinstance(out_info['upload_date'], compat_str))
+        self.assertEqual(out_info['upload_date'], '20210911')
+        self.assertTrue(isinstance(out_info['release_date'], compat_str))
+        self.assertEqual(out_info['release_date'], '20210930')
+
 
 if __name__ == '__main__':
     unittest.main()