Repository: yt-dlp/yt-dlp

Issue:
SHA2-256SUMS cannot be verified using standard Linux tools
<!--

######################################################################
  WARNING!
  IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################

-->


## Checklist

<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.01. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
- Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
- Read bugs section in FAQ: https://github.com/yt-dlp/yt-dlp
- Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
-->

- [ ] I'm reporting a broken site support issue
- [x] I've verified that I'm running yt-dlp version **2021.06.01**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [x] I've searched the bugtracker for similar bug reports including closed ones
- [x] I've read bugs section in FAQ


## Verbose log

<!--
Provide the complete verbose output of yt-dlp that clearly demonstrates the problem.
Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
 [debug] System config: []
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
 [debug] yt-dlp version 2021.06.01
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
 <more lines>
-->

```bash
$ sha256sum -c SHA2-256SUMS
sha256sum: SHA2-256SUMS: no properly formatted SHA256 checksum lines found
```
<!--
Do not remove the above ```
-->


## Description

<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->

SHA256 provided with the release cannot be verified using standard Linux tools. The reason is two-fold:
1. `file:hash` is not a standard checksum syntax, `coreutils` understands only GNU `hash file` or BSD `SHA256(file) = hash` styles.
2. For some reason, `SHA2-256SUMS` has Windows line endings, not Unix ones.

The current format was inherited from `youtube-dlc`. Autoupdater depends on it, but will only issue a warning when the format is changed. Support for the new format was added in https://github.com/yt-dlp/yt-dlp/pull/383.

When a substantial number of people have updated their `yt-dlp`s, checksum format should change to comply with UNIX standards.
possible to remove temp folder extractions for binaries?
## Checklist

- [ X] I'm asking a question
- [ X I've looked through the README and FAQ for similar questions
- [ X] I've searched the bugtracker for similar questions including closed ones


## Question

It is possible to run  yt-dlp without extracting binaries inside temp folder on Windows (in my case it was _MEI101362 folder and files inside) like it was on original yt-dl?
The problem is each time I'm trying to download something, it extracts some binaries inside temp folder and those binaries needs network access so my firewall always asking if I want to allow them network access or not and I cannot setup rules for temp files auto deleted after the program is closed.

SHA2-256SUMS cannot be verified using standard Linux tools
<!--

######################################################################
  WARNING!
  IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################

-->


## Checklist

<!--
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.01. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
- Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
- Read bugs section in FAQ: https://github.com/yt-dlp/yt-dlp
- Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
-->

- [ ] I'm reporting a broken site support issue
- [x] I've verified that I'm running yt-dlp version **2021.06.01**
- [ ] I've checked that all provided URLs are alive and playable in a browser
- [x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
- [x] I've searched the bugtracker for similar bug reports including closed ones
- [x] I've read bugs section in FAQ


## Verbose log

<!--
Provide the complete verbose output of yt-dlp that clearly demonstrates the problem.
Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your command line>`), copy the WHOLE output and insert it below. It should look similar to this:
 [debug] System config: []
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
 [debug] yt-dlp version 2021.06.01
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
 <more lines>
-->

```bash
$ sha256sum -c SHA2-256SUMS
sha256sum: SHA2-256SUMS: no properly formatted SHA256 checksum lines found
```
<!--
Do not remove the above ```
-->


## Description

<!--
Provide an explanation of your issue in an arbitrary form. Please make sure the description is worded well enough to be understood, see https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient. Provide any additional information, suggested solution and as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
-->

SHA256 provided with the release cannot be verified using standard Linux tools. The reason is two-fold:
1. `file:hash` is not a standard checksum syntax, `coreutils` understands only GNU `hash file` or BSD `SHA256(file) = hash` styles.
2. For some reason, `SHA2-256SUMS` has Windows line endings, not Unix ones.

The current format was inherited from `youtube-dlc`. Autoupdater depends on it, but will only issue a warning when the format is changed. Support for the new format was added in https://github.com/yt-dlp/yt-dlp/pull/383.

When a substantial number of people have updated their `yt-dlp`s, checksum format should change to comply with UNIX standards.


Discussion:
I was about to report the same essentially.

I had to use the SHA2-*512*SUMS file for my tool in question to be able to extract the hash. Hopefully the *256* SUMS file generation can be updated to output the same format as what is in the 512 file.

I was about to report the same essentially.

I had to use the SHA2-*512*SUMS file for my tool in question to be able to extract the hash. Hopefully the *256* SUMS file generation can be updated to output the same format as what is in the 512 file.

Resolving patch:
diff --git a/pyinst.py b/pyinst.py
--- a/pyinst.py
+++ b/pyinst.py
@@ -15,9 +15,11 @@
 
 arch = sys.argv[1] if len(sys.argv) > 1 else platform.architecture()[0][:2]
 assert arch in ('32', '64')
-print('Building %sbit version' % arch)
 _x86 = '_x86' if arch == '32' else ''
 
+opts = sys.argv[2:] or ['--onefile']
+print(f'Building {arch}bit version with options {opts}')
+
 FILE_DESCRIPTION = 'yt-dlp%s' % (' (32 Bit)' if _x86 else '')
 
 # root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
@@ -72,11 +74,12 @@
 
 PyInstaller.__main__.run([
     '--name=yt-dlp%s' % _x86,
-    '--onefile',
     '--icon=devscripts/logo.ico',
     *[f'--exclude-module={module}' for module in excluded_modules],
     *[f'--hidden-import={module}' for module in dependancies],
     '--upx-exclude=vcruntime140.dll',
+    '--noconfirm',
+    *opts,
     'yt_dlp/__main__.py',
 ])
 SetVersion('dist/yt-dlp%s.exe' % _x86, VERSION_FILE)
diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py
--- a/yt_dlp/YoutubeDL.py
+++ b/yt_dlp/YoutubeDL.py
@@ -27,7 +27,6 @@
 import random
 
 from string import ascii_letters
-from zipimport import zipimporter
 
 from .compat import (
     compat_basestring,
@@ -142,6 +141,7 @@
     FFmpegPostProcessor,
     MoveFilesAfterDownloadPP,
 )
+from .update import detect_variant
 from .version import __version__
 
 if compat_os_name == 'nt':
@@ -3231,12 +3231,8 @@ def print_debug_header(self):
                 self.get_encoding()))
         write_string(encoding_str, encoding=None)
 
-        source = (
-            '(exe)' if hasattr(sys, 'frozen')
-            else '(zip)' if isinstance(globals().get('__loader__'), zipimporter)
-            else '(source)' if os.path.basename(sys.argv[0]) == '__main__.py'
-            else '')
-        self._write_string('[debug] yt-dlp version %s %s\n' % (__version__, source))
+        source = detect_variant()
+        self._write_string('[debug] yt-dlp version %s%s\n' % (__version__, '' if source == 'unknown' else f' ({source})'))
         if _LAZY_LOADER:
             self._write_string('[debug] Lazy loading extractors enabled\n')
         if _PLUGIN_CLASSES:
diff --git a/yt_dlp/update.py b/yt_dlp/update.py
--- a/yt_dlp/update.py
+++ b/yt_dlp/update.py
@@ -31,6 +31,18 @@ def rsa_verify(message, signature, key):
 '''
 
 
+def detect_variant():
+    if hasattr(sys, 'frozen') and getattr(sys, '_MEIPASS', None):
+        if sys._MEIPASS == os.path.dirname(sys.executable):
+            return 'dir'
+        return 'exe'
+    elif isinstance(globals().get('__loader__'), zipimporter):
+        return 'zip'
+    elif os.path.basename(sys.argv[0]) == '__main__.py':
+        return 'source'
+    return 'unknown'
+
+
 def update_self(to_screen, verbose, opener):
     ''' Exists for backward compatibility. Use run_update(ydl) instead '''
 
@@ -87,13 +99,14 @@ def calc_sha256sum(path):
                 h.update(mv[:n])
         return h.hexdigest()
 
-    err = None
-    if isinstance(globals().get('__loader__'), zipimporter):
-        pass
-    elif hasattr(sys, 'frozen'):
-        pass
-    else:
-        err = 'It looks like you installed yt-dlp with a package manager, pip, setup.py or a tarball. Please use that to update'
+    ERRORS = {
+        'exe': None,
+        'zip': None,
+        'dir': 'Auto-update is not supported for unpackaged windows executable. Re-download the latest release',
+        'source': 'You cannot update when running from source code',
+        'unknown': 'It looks like you installed yt-dlp with a package manager, pip, setup.py or a tarball. Use that to update',
+    }
+    err = ERRORS.get(detect_variant(), ERRORS['unknown'])
     if err:
         return report_error(err, expected=True)
 
@@ -138,12 +151,7 @@ def get_sha256sum(bin_or_exe, version):
         if not urlh:
             return None
         hash_data = ydl._opener.open(urlh).read().decode('utf-8')
-        if hash_data.startswith('version:'):
-            # Old colon-separated hash file
-            return dict(ln.split(':') for ln in hash_data.splitlines()).get(filename)
-        else:
-            # GNU-style hash file
-            return dict(ln.split()[::-1] for ln in hash_data.splitlines()).get(filename)
+        return dict(ln.split()[::-1] for ln in hash_data.splitlines()).get(filename)
 
     if not os.access(filename, os.W_OK):
         return report_error('no write permissions on %s' % filename, expected=True)


Test patch: