Repository: yt-dlp/yt-dlp
Pull request #4355 (2022-07-14 19:54:32)

make pyinst.py build name more PEP425 compatible
<details open><summary>Template</summary> <!-- OPEN is intentional -->

<!--

# PLEASE FOLLOW THE GUIDE BELOW

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like

-->

### Before submitting a *pull request* make sure you have:
- [ ] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [ ] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [ ] Fix or improvement to an extractor (Make sure to add/update tests)
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [x] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))

### Description of your *pull request* and other information

</details>

<!--

Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible

-->

DESCRIPTION

Pyinst.py generates build name "yt-dlp_linux_x86" on ARM(32bit). Also "x86" is not compatible with PEP 425.

Fixes #

Introduces MACHINE variable filled by platform.machine() and replaces "AMD64" with "x86_64" according to PEP 425.

pukkandan: The PEP is for wheels... The binary names created by pyinst will not be changed
pukkandan: Actually, let me clarify. The names for the x86/64 will not be changed. Name change for just the ARM variants are welcome


Patch:
diff --git a/README.md b/README.md
index af5fb46ae..1f756ca31 100644
--- a/README.md
+++ b/README.md
@@ -1161,6 +1161,12 @@ # Save all videos under YouTube directory in your home directory
 
 You can use `--ignore-config` if you want to disable all configuration files for a particular yt-dlp run. If `--ignore-config` is found inside any configuration file, no further configuration will be loaded. For example, having the option in the portable configuration file prevents loading of home, user, and system configurations. Additionally, (for backward compatibility) if `--ignore-config` is found inside the system configuration file, the user configuration is not loaded.
 
+### Config file encoding
+
+The config files are decoded according to the UTF BOM if present, and in the encoding from system locale otherwise.
+
+If you want your file to be decoded differently, add `# coding: ENCODING` to the beginning of the file (e.g. `# coding: shift-jis`). There must be no characters before that, even spaces or BOM.
+
 ### Authentication with `.netrc` file
 
 You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every yt-dlp execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in `--netrc-location` and restrict permissions to read/write by only you:
diff --git a/pyinst.py b/pyinst.py
index a8c8dd7b7..55df1a78f 100644
--- a/pyinst.py
+++ b/pyinst.py
@@ -6,7 +6,11 @@
 
 from PyInstaller.__main__ import run as run_pyinstaller
 
-OS_NAME, ARCH = sys.platform, platform.architecture()[0][:2]
+OS_NAME, MACHINE = sys.platform, platform.machine()
+if MACHINE in ('x86_64', 'amd64'):
+    MACHINE = ''
+elif 'i' in MACHINE and '86' in MACHINE:
+    MACHINE = 'x86'
 
 
 def main():
@@ -18,7 +22,7 @@ def main():
         opts.append('--onefile')
 
     name, final_file = exe(onedir)
-    print(f'Building yt-dlp v{version} {ARCH}bit for {OS_NAME} with options {opts}')
+    print(f'Building yt-dlp v{version} for {OS_NAME} {platform.machine()} with options {opts}')
     print('Remember to update the version using  "devscripts/update-version.py"')
     if not os.path.isfile('yt_dlp/extractor/lazy_extractors.py'):
         print('WARNING: Building without lazy_extractors. Run  '
@@ -47,6 +51,7 @@ def parse_options():
     # Compatibility with older arguments
     opts = sys.argv[1:]
     if opts[0:1] in (['32'], ['64']):
+        ARCH = platform.architecture()[0][:2]
         if ARCH != opts[0]:
             raise Exception(f'{opts[0]}bit executable cannot be built on a {ARCH}bit system')
         opts = opts[1:]
@@ -65,7 +70,7 @@ def exe(onedir):
     name = '_'.join(filter(None, (
         'yt-dlp',
         {'win32': '', 'darwin': 'macos'}.get(OS_NAME, OS_NAME),
-        ARCH == '32' and 'x86'
+        MACHINE
     )))
     return name, ''.join(filter(None, (
         'dist/',
@@ -122,7 +127,7 @@ def windows_set_version(exe, version):
     )
 
     version_list = version_to_list(version)
-    suffix = '_x86' if ARCH == '32' else ''
+    suffix = MACHINE and f'_{MACHINE}'
     SetVersion(exe, VSVersionInfo(
         ffi=FixedFileInfo(
             filevers=version_list,
@@ -136,9 +141,9 @@ def windows_set_version(exe, version):
         ),
         kids=[
             StringFileInfo([StringTable('040904B0', [
-                StringStruct('Comments', 'yt-dlp%s Command Line Interface.' % suffix),
+                StringStruct('Comments', 'yt-dlp%s Command Line Interface' % suffix),
                 StringStruct('CompanyName', 'https://github.com/yt-dlp'),
-                StringStruct('FileDescription', 'yt-dlp%s' % (' (32 Bit)' if ARCH == '32' else '')),
+                StringStruct('FileDescription', 'yt-dlp%s' % (MACHINE and f' ({MACHINE})')),
                 StringStruct('FileVersion', version),
                 StringStruct('InternalName', f'yt-dlp{suffix}'),
                 StringStruct('LegalCopyright', 'pukkandan.ytdlp@gmail.com | UNLICENSE'),
diff --git a/yt_dlp/extractor/_extractors.py b/yt_dlp/extractor/_extractors.py
index 070729ce5..6cf4677d2 100644
--- a/yt_dlp/extractor/_extractors.py
+++ b/yt_dlp/extractor/_extractors.py
@@ -1794,6 +1794,7 @@
 from .trunews import TruNewsIE
 from .trutv import TruTVIE
 from .tube8 import Tube8IE
+from .tubetugraz import TubeTuGrazIE, TubeTuGrazSeriesIE
 from .tubitv import (
     TubiTvIE,
     TubiTvShowIE,
diff --git a/yt_dlp/extractor/common.py b/yt_dlp/extractor/common.py
index 68f08dfea..47c829857 100644
--- a/yt_dlp/extractor/common.py
+++ b/yt_dlp/extractor/common.py
@@ -929,39 +929,37 @@ def _webpage_read_content(self, urlh, url_or_request, video_id, note=None, errno
 
         return content
 
-    def _parse_xml(self, xml_string, video_id, transform_source=None, fatal=True):
+    def __print_error(self, errnote, fatal, video_id, err):
+        if fatal:
+            raise ExtractorError(f'{video_id}: {errnote} ', cause=err)
+        elif errnote:
+            self.report_warning(f'{video_id}: {errnote} {err}')
+
+    def _parse_xml(self, xml_string, video_id, transform_source=None, fatal=True, errnote=None):
         if transform_source:
             xml_string = transform_source(xml_string)
         try:
             return compat_etree_fromstring(xml_string.encode('utf-8'))
         except xml.etree.ElementTree.ParseError as ve:
-            errmsg = '%s: Failed to parse XML ' % video_id
-            if fatal:
-                raise ExtractorError(errmsg, cause=ve)
-            else:
-                self.report_warning(errmsg + str(ve))
+            self.__print_error('Failed to parse XML' if errnote is None else errnote, fatal, video_id, ve)
 
-    def _parse_json(self, json_string, video_id, transform_source=None, fatal=True, **parser_kwargs):
+    def _parse_json(self, json_string, video_id, transform_source=None, fatal=True, errnote=None, **parser_kwargs):
         try:
             return json.loads(
                 json_string, cls=LenientJSONDecoder, strict=False, transform_source=transform_source, **parser_kwargs)
         except ValueError as ve:
-            errmsg = f'{video_id}: Failed to parse JSON'
-            if fatal:
-                raise ExtractorError(errmsg, cause=ve)
-            else:
-                self.report_warning(f'{errmsg}: {ve}')
+            self.__print_error('Failed to parse JSON' if errnote is None else errnote, fatal, video_id, ve)
 
-    def _parse_socket_response_as_json(self, data, video_id, transform_source=None, fatal=True):
-        return self._parse_json(
-            data[data.find('{'):data.rfind('}') + 1],
-            video_id, transform_source, fatal)
+    def _parse_socket_response_as_json(self, data, *args, **kwargs):
+        return self._parse_json(data[data.find('{'):data.rfind('}') + 1], *args, **kwargs)
 
     def __create_download_methods(name, parser, note, errnote, return_value):
 
-        def parse(ie, content, *args, **kwargs):
+        def parse(ie, content, *args, errnote=errnote, **kwargs):
             if parser is None:
                 return content
+            if errnote is False:
+                kwargs['errnote'] = errnote
             # parser is fetched by name so subclasses can override it
             return getattr(ie, parser)(content, *args, **kwargs)
 
@@ -973,7 +971,7 @@ def download_handle(self, url_or_request, video_id, note=note, errnote=errnote,
             if res is False:
                 return res
             content, urlh = res
-            return parse(self, content, video_id, transform_source=transform_source, fatal=fatal), urlh
+            return parse(self, content, video_id, transform_source=transform_source, fatal=fatal, errnote=errnote), urlh
 
         def download_content(self, url_or_request, video_id, note=note, errnote=errnote, transform_source=None,
                              fatal=True, encoding=None, data=None, headers={}, query={}, expected_status=None):
@@ -988,7 +986,7 @@ def download_content(self, url_or_request, video_id, note=note, errnote=errnote,
                     self.report_warning(f'Unable to load request from disk: {e}')
                 else:
                     content = self.__decode_webpage(webpage_bytes, encoding, url_or_request.headers)
-                    return parse(self, content, video_id, transform_source, fatal)
+                    return parse(self, content, video_id, transform_source=transform_source, fatal=fatal, errnote=errnote)
             kwargs = {
                 'note': note,
                 'errnote': errnote,
diff --git a/yt_dlp/extractor/instagram.py b/yt_dlp/extractor/instagram.py
index 5a824b500..04afacb90 100644
--- a/yt_dlp/extractor/instagram.py
+++ b/yt_dlp/extractor/instagram.py
@@ -1,17 +1,17 @@
-import itertools
 import hashlib
+import itertools
 import json
 import re
 import time
+import urllib.error
 
 from .common import InfoExtractor
-from ..compat import (
-    compat_HTTPError,
-)
 from ..utils import (
     ExtractorError,
-    format_field,
+    decode_base_n,
+    encode_base_n,
     float_or_none,
+    format_field,
     get_element_by_attribute,
     int_or_none,
     lowercase_escape,
@@ -22,6 +22,18 @@
     urlencode_postdata,
 )
 
+_ENCODING_CHARS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'
+
+
+def _pk_to_id(id):
+    """Source: https://stackoverflow.com/questions/24437823/getting-instagram-post-url-from-media-id"""
+    return encode_base_n(int(id.split('_')[0]), table=_ENCODING_CHARS)
+
+
+def _id_to_pk(shortcode):
+    """Covert a shortcode to a numeric value"""
+    return decode_base_n(shortcode[:11], table=_ENCODING_CHARS)
+
 
 class InstagramBaseIE(InfoExtractor):
     _NETRC_MACHINE = 'instagram'
@@ -156,6 +168,15 @@ def _extract_product(self, product_info):
         if isinstance(product_info, list):
             product_info = product_info[0]
 
+        comment_data = traverse_obj(product_info, ('edge_media_to_parent_comment', 'edges'))
+        comments = [{
+            'author': traverse_obj(comment_dict, ('node', 'owner', 'username')),
+            'author_id': traverse_obj(comment_dict, ('node', 'owner', 'id')),
+            'id': traverse_obj(comment_dict, ('node', 'id')),
+            'text': traverse_obj(comment_dict, ('node', 'text')),
+            'timestamp': traverse_obj(comment_dict, ('node', 'created_at'), expected_type=int_or_none),
+        } for comment_dict in comment_data] if comment_data else None
+
         user_info = product_info.get('user') or {}
         info_dict = {
             'id': product_info.get('code') or product_info.get('id'),
@@ -168,6 +189,7 @@ def _extract_product(self, product_info):
             'view_count': int_or_none(product_info.get('view_count')),
             'like_count': int_or_none(product_info.get('like_count')),
             'comment_count': int_or_none(product_info.get('comment_count')),
+            'comments': comments,
             'http_headers': {
                 'Referer': 'https://www.instagram.com/',
             }
@@ -214,23 +236,9 @@ class InstagramIOSIE(InfoExtractor):
         'add_ie': ['Instagram']
     }]
 
-    def _get_id(self, id):
-        """Source: https://stackoverflow.com/questions/24437823/getting-instagram-post-url-from-media-id"""
-        chrs = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'
-        media_id = int(id.split('_')[0])
-        shortened_id = ''
-        while media_id > 0:
-            r = media_id % 64
-            media_id = (media_id - r) // 64
-            shortened_id = chrs[r] + shortened_id
-        return shortened_id
-
     def _real_extract(self, url):
-        return {
-            '_type': 'url_transparent',
-            'url': f'http://instagram.com/tv/{self._get_id(self._match_id(url))}/',
-            'ie_key': 'Instagram',
-        }
+        video_id = _pk_to_id(self._match_id(url))
+        return self.url_result(f'http://instagram.com/tv/{video_id}', InstagramIE, video_id)
 
 
 class InstagramIE(InstagramBaseIE):
@@ -358,39 +366,49 @@ def _extract_embed_url(webpage):
 
     def _real_extract(self, url):
         video_id, url = self._match_valid_url(url).group('id', 'url')
-        webpage, urlh = self._download_webpage_handle(url, video_id)
-        if 'www.instagram.com/accounts/login' in urlh.geturl():
-            self.report_warning('Main webpage is locked behind the login page. '
-                                'Retrying with embed webpage (Note that some metadata might be missing)')
-            webpage = self._download_webpage(
-                'https://www.instagram.com/p/%s/embed/' % video_id, video_id, note='Downloading embed webpage')
-
-        shared_data = self._parse_json(
-            self._search_regex(
-                r'window\._sharedData\s*=\s*({.+?});',
-                webpage, 'shared data', default='{}'),
-            video_id, fatal=False)
-        media = traverse_obj(
-            shared_data,
-            ('entry_data', 'PostPage', 0, 'graphql', 'shortcode_media'),
-            ('entry_data', 'PostPage', 0, 'media'),
-            expected_type=dict)
-
-        # _sharedData.entry_data.PostPage is empty when authenticated (see
-        # https://github.com/ytdl-org/youtube-dl/pull/22880)
+        general_info = self._download_json(
+            f'https://www.instagram.com/graphql/query/?query_hash=9f8827793ef34641b2fb195d4d41151c'
+            f'&variables=%7B"shortcode":"{video_id}",'
+            '"parent_comment_count":10,"has_threaded_comments":true}', video_id, fatal=False, errnote=False,
+            headers={
+                'Accept': '*',
+                'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
+                'Authority': 'www.instagram.com',
+                'Referer': 'https://www.instagram.com',
+                'x-ig-app-id': '936619743392459',
+            })
+        media = traverse_obj(general_info, ('data', 'shortcode_media')) or {}
         if not media:
-            additional_data = self._parse_json(
-                self._search_regex(
-                    r'window\.__additionalDataLoaded\s*\(\s*[^,]+,\s*({.+?})\s*\);',
-                    webpage, 'additional data', default='{}'),
-                video_id, fatal=False)
-            product_item = traverse_obj(additional_data, ('items', 0), expected_type=dict)
-            if product_item:
-                return self._extract_product(product_item)
-            media = traverse_obj(additional_data, ('graphql', 'shortcode_media'), 'shortcode_media', expected_type=dict) or {}
-
-        if not media and 'www.instagram.com/accounts/login' in urlh.geturl():
-            self.raise_login_required('You need to log in to access this content')
+            self.report_warning('General metadata extraction failed', video_id)
+
+        info = self._download_json(
+            f'https://i.instagram.com/api/v1/media/{_id_to_pk(video_id)}/info/', video_id,
+            fatal=False, note='Downloading video info', errnote=False, headers={
+                'Accept': '*',
+                'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
+                'Authority': 'www.instagram.com',
+                'Referer': 'https://www.instagram.com',
+                'x-ig-app-id': '936619743392459',
+            })
+        if info:
+            media.update(info['items'][0])
+            return self._extract_product(media)
+
+        webpage = self._download_webpage(
+            f'https://www.instagram.com/p/{video_id}/embed/', video_id,
+            note='Downloading embed webpage', fatal=False)
+        if not webpage:
+            self.raise_login_required('Requested content was not found, the content might be private')
+
+        additional_data = self._search_json(
+            r'window\.__additionalDataLoaded\s*\(\s*[^,]+,\s*', webpage, 'additional data', video_id, fatal=False)
+        product_item = traverse_obj(additional_data, ('items', 0), expected_type=dict)
+        if product_item:
+            media.update(product_item)
+            return self._extract_product(media)
+
+        media.update(traverse_obj(
+            additional_data, ('graphql', 'shortcode_media'), 'shortcode_media', expected_type=dict) or {})
 
         username = traverse_obj(media, ('owner', 'username')) or self._search_regex(
             r'"owner"\s*:\s*{\s*"username"\s*:\s*"(.+?)"', webpage, 'username', fatal=False)
@@ -519,7 +537,7 @@ def _extract_graphql(self, data, url):
                 except ExtractorError as e:
                     # if it's an error caused by a bad query, and there are
                     # more GIS templates to try, ignore it and keep trying
-                    if isinstance(e.cause, compat_HTTPError) and e.cause.code == 403:
+                    if isinstance(e.cause, urllib.error.HTTPError) and e.cause.code == 403:
                         if gis_tmpl != gis_tmpls[-1]:
                             continue
                     raise
@@ -629,41 +647,36 @@ class InstagramStoryIE(InstagramBaseIE):
 
     def _real_extract(self, url):
         username, story_id = self._match_valid_url(url).groups()
-
-        story_info_url = f'{username}/{story_id}/?__a=1' if username == 'highlights' else f'{username}/?__a=1'
-        story_info = self._download_json(f'https://www.instagram.com/stories/{story_info_url}', story_id, headers={
-            'X-IG-App-ID': 936619743392459,
-            'X-ASBD-ID': 198387,
-            'X-IG-WWW-Claim': 0,
-            'X-Requested-With': 'XMLHttpRequest',
-            'Referer': url,
-        })
-        user_id = story_info['user']['id']
-        highlight_title = traverse_obj(story_info, ('highlight', 'title'))
+        story_info = self._download_webpage(url, story_id)
+        user_info = self._search_json(r'"user":', story_info, 'user info', story_id, fatal=False)
+        if not user_info:
+            self.raise_login_required('This content is unreachable')
+        user_id = user_info.get('id')
 
         story_info_url = user_id if username != 'highlights' else f'highlight:{story_id}'
-        videos = self._download_json(f'https://i.instagram.com/api/v1/feed/reels_media/?reel_ids={story_info_url}', story_id, headers={
-            'X-IG-App-ID': 936619743392459,
-            'X-ASBD-ID': 198387,
-            'X-IG-WWW-Claim': 0,
-        })['reels']
-
-        full_name = traverse_obj(videos, ('user', 'full_name'))
-
-        user_info = {}
-        if not (username and username != 'highlights' and full_name):
-            user_info = self._download_json(
-                f'https://i.instagram.com/api/v1/users/{user_id}/info/', story_id, headers={
-                    'User-Agent': 'Mozilla/5.0 (Linux; Android 11; SM-A505F Build/RP1A.200720.012; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/96.0.4664.45 Mobile Safari/537.36 Instagram 214.1.0.29.120 Android (30/11; 450dpi; 1080x2122; samsung; SM-A505F; a50; exynos9610; en_US; 333717274)',
-                }, note='Downloading user info')
+        videos = traverse_obj(self._download_json(
+            f'https://i.instagram.com/api/v1/feed/reels_media/?reel_ids={story_info_url}',
+            story_id, errnote=False, fatal=False, headers={
+                'X-IG-App-ID': 936619743392459,
+                'X-ASBD-ID': 198387,
+                'X-IG-WWW-Claim': 0,
+            }), 'reels')
+        if not videos:
+            self.raise_login_required('You need to log in to access this content')
 
-        username = traverse_obj(user_info, ('user', 'username')) or username
-        full_name = traverse_obj(user_info, ('user', 'full_name')) or full_name
+        full_name = traverse_obj(videos, (f'highlight:{story_id}', 'user', 'full_name'), (str(user_id), 'user', 'full_name'))
+        story_title = traverse_obj(videos, (f'highlight:{story_id}', 'title'))
+        if not story_title:
+            story_title = f'Story by {username}'
 
         highlights = traverse_obj(videos, (f'highlight:{story_id}', 'items'), (str(user_id), 'items'))
-        return self.playlist_result([{
-            **self._extract_product(highlight),
-            'title': f'Story by {username}',
-            'uploader': full_name,
-            'uploader_id': user_id,
-        } for highlight in highlights], playlist_id=story_id, playlist_title=highlight_title)
+        info_data = []
+        for highlight in highlights:
+            highlight_data = self._extract_product(highlight)
+            if highlight_data.get('formats'):
+                info_data.append({
+                    **highlight_data,
+                    'uploader': full_name,
+                    'uploader_id': user_id,
+                })
+        return self.playlist_result(info_data, playlist_id=story_id, playlist_title=story_title)
diff --git a/yt_dlp/extractor/tubetugraz.py b/yt_dlp/extractor/tubetugraz.py
new file mode 100644
index 000000000..89371b6eb
--- /dev/null
+++ b/yt_dlp/extractor/tubetugraz.py
@@ -0,0 +1,234 @@
+from .common import InfoExtractor
+from ..utils import (
+    float_or_none,
+    parse_resolution,
+    traverse_obj,
+    urlencode_postdata,
+    variadic,
+)
+
+
+class TubeTuGrazBaseIE(InfoExtractor):
+    _NETRC_MACHINE = 'tubetugraz'
+
+    _API_EPISODE = 'https://tube.tugraz.at/search/episode.json'
+    _FORMAT_TYPES = ('presentation', 'presenter')
+
+    def _perform_login(self, username, password):
+        urlh = self._request_webpage(
+            'https://tube.tugraz.at/Shibboleth.sso/Login?target=/paella/ui/index.html',
+            None, fatal=False, note='downloading login page', errnote='unable to fetch login page')
+        if not urlh:
+            return
+
+        urlh = self._request_webpage(
+            urlh.geturl(), None, fatal=False, headers={'referer': urlh.geturl()},
+            note='logging in', errnote='unable to log in', data=urlencode_postdata({
+                'lang': 'de',
+                '_eventId_proceed': '',
+                'j_username': username,
+                'j_password': password
+            }))
+
+        if urlh and urlh.geturl() != 'https://tube.tugraz.at/paella/ui/index.html':
+            self.report_warning('unable to login: incorrect password')
+
+    def _extract_episode(self, episode_info):
+        id = episode_info.get('id')
+        formats = list(self._extract_formats(
+            traverse_obj(episode_info, ('mediapackage', 'media', 'track')), id))
+        self._sort_formats(formats)
+
+        title = traverse_obj(episode_info, ('mediapackage', 'title'), 'dcTitle')
+        series_title = traverse_obj(episode_info, ('mediapackage', 'seriestitle'))
+        creator = ', '.join(variadic(traverse_obj(
+            episode_info, ('mediapackage', 'creators', 'creator'), 'dcCreator', default='')))
+        return {
+            'id': id,
+            'title': title,
+            'creator': creator or None,
+            'duration': traverse_obj(episode_info, ('mediapackage', 'duration'), 'dcExtent'),
+            'series': series_title,
+            'series_id': traverse_obj(episode_info, ('mediapackage', 'series'), 'dcIsPartOf'),
+            'episode': series_title and title,
+            'formats': formats
+        }
+
+    def _set_format_type(self, formats, type):
+        for f in formats:
+            f['format_note'] = type
+            if not type.startswith(self._FORMAT_TYPES[0]):
+                f['preference'] = -2
+        return formats
+
+    def _extract_formats(self, format_list, id):
+        has_hls, has_dash = False, False
+
+        for format_info in format_list or []:
+            url = traverse_obj(format_info, ('tags', 'url'), 'url')
+            if url is None:
+                continue
+
+            type = format_info.get('type') or 'unknown'
+            transport = (format_info.get('transport') or 'https').lower()
+
+            if transport == 'https':
+                formats = [{
+                    'url': url,
+                    'abr': float_or_none(traverse_obj(format_info, ('audio', 'bitrate')), 1000),
+                    'vbr': float_or_none(traverse_obj(format_info, ('video', 'bitrate')), 1000),
+                    'fps': traverse_obj(format_info, ('video', 'framerate')),
+                    **parse_resolution(traverse_obj(format_info, ('video', 'resolution'))),
+                }]
+            elif transport == 'hls':
+                has_hls, formats = True, self._extract_m3u8_formats(
+                    url, id, 'mp4', fatal=False, note=f'downloading {type} HLS manifest')
+            elif transport == 'dash':
+                has_dash, formats = True, self._extract_mpd_formats(
+                    url, id, fatal=False, note=f'downloading {type} DASH manifest')
+            else:
+                # RTMP, HDS, SMOOTH, and unknown formats
+                # - RTMP url fails on every tested entry until now
+                # - HDS url 404's on every tested entry until now
+                # - SMOOTH url 404's on every tested entry until now
+                continue
+
+            yield from self._set_format_type(formats, type)
+
+        # TODO: Add test for these
+        for type in self._FORMAT_TYPES:
+            if not has_hls:
+                hls_formats = self._extract_m3u8_formats(
+                    f'https://wowza.tugraz.at/matterhorn_engage/smil:engage-player_{id}_{type}.smil/playlist.m3u8',
+                    id, 'mp4', fatal=False, note=f'Downloading {type} HLS manifest', errnote=False) or []
+                yield from self._set_format_type(hls_formats, type)
+
+            if not has_dash:
+                dash_formats = self._extract_mpd_formats(
+                    f'https://wowza.tugraz.at/matterhorn_engage/smil:engage-player_{id}_{type}.smil/manifest_mpm4sav_mvlist.mpd',
+                    id, fatal=False, note=f'Downloading {type} DASH manifest', errnote=False)
+                yield from self._set_format_type(dash_formats, type)
+
+
+class TubeTuGrazIE(TubeTuGrazBaseIE):
+    IE_DESC = 'tube.tugraz.at'
+
+    _VALID_URL = r'''(?x)
+        https?://tube\.tugraz\.at/paella/ui/watch.html\?id=
+        (?P<id>[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12})
+    '''
+    _TESTS = [
+        {
+            'url': 'https://tube.tugraz.at/paella/ui/watch.html?id=f2634392-e40e-4ac7-9ddc-47764aa23d40',
+            'md5': 'a23a3d5c9aaca2b84932fdba66e17145',
+            'info_dict': {
+                'id': 'f2634392-e40e-4ac7-9ddc-47764aa23d40',
+                'ext': 'mp4',
+                'title': '#6 (23.11.2017)',
+                'episode': '#6 (23.11.2017)',
+                'series': '[INB03001UF] Einführung in die strukturierte Programmierung',
+                'creator': 'Safran C',
+                'duration': 3295818,
+                'series_id': 'b1192fff-2aa7-4bf0-a5cf-7b15c3bd3b34',
+            }
+        }, {
+            'url': 'https://tube.tugraz.at/paella/ui/watch.html?id=2df6d787-e56a-428d-8ef4-d57f07eef238',
+            'md5': 'de0d854a56bf7318d2b693fe1adb89a5',
+            'info_dict': {
+                'id': '2df6d787-e56a-428d-8ef4-d57f07eef238',
+                'title': 'TubeTuGraz video #2df6d787-e56a-428d-8ef4-d57f07eef238',
+                'ext': 'mp4',
+            },
+            'expected_warnings': ['Extractor failed to obtain "title"'],
+        }
+    ]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        episode_data = self._download_json(
+            self._API_EPISODE, video_id, query={'id': video_id, 'limit': 1}, note='Downloading episode metadata')
+
+        episode_info = traverse_obj(episode_data, ('search-results', 'result'), default={'id': video_id})
+        return self._extract_episode(episode_info)
+
+
+class TubeTuGrazSeriesIE(TubeTuGrazBaseIE):
+    _VALID_URL = r'''(?x)
+        https?://tube\.tugraz\.at/paella/ui/browse\.html\?series=
+        (?P<id>[0-9a-fA-F]{8}-(?:[0-9a-fA-F]{4}-){3}[0-9a-fA-F]{12})
+    '''
+    _TESTS = [{
+        'url': 'https://tube.tugraz.at/paella/ui/browse.html?series=0e6351b7-c372-491e-8a49-2c9b7e21c5a6',
+        'id': '0e6351b7-c372-491e-8a49-2c9b7e21c5a6',
+        'info_dict': {
+            'id': '0e6351b7-c372-491e-8a49-2c9b7e21c5a6',
+            'title': '[209351] Strassenwesen',
+        },
+        'playlist': [
+            {
+                'info_dict': {
+                    'id': 'ee17ce5d-34e2-48b7-a76a-fed148614e11',
+                    'series_id': '0e6351b7-c372-491e-8a49-2c9b7e21c5a6',
+                    'ext': 'mp4',
+                    'title': '#4 Detailprojekt',
+                    'episode': '#4 Detailprojekt',
+                    'series': '[209351] Strassenwesen',
+                    'creator': 'Neuhold R',
+                    'duration': 6127024,
+                }
+            },
+            {
+                'info_dict': {
+                    'id': '87350498-799a-44d3-863f-d1518a98b114',
+                    'series_id': '0e6351b7-c372-491e-8a49-2c9b7e21c5a6',
+                    'ext': 'mp4',
+                    'title': '#3 Generelles Projekt',
+                    'episode': '#3 Generelles Projekt',
+                    'series': '[209351] Strassenwesen',
+                    'creator': 'Neuhold R',
+                    'duration': 5374422,
+                }
+            },
+            {
+                'info_dict': {
+                    'id': '778599ea-489e-4189-9e05-3b4888e19bcd',
+                    'series_id': '0e6351b7-c372-491e-8a49-2c9b7e21c5a6',
+                    'ext': 'mp4',
+                    'title': '#2 Vorprojekt',
+                    'episode': '#2 Vorprojekt',
+                    'series': '[209351] Strassenwesen',
+                    'creator': 'Neuhold R',
+                    'duration': 5566404,
+                }
+            },
+            {
+                'info_dict': {
+                    'id': '75e4c71c-d99d-4e56-b0e6-4f2bcdf11f29',
+                    'series_id': '0e6351b7-c372-491e-8a49-2c9b7e21c5a6',
+                    'ext': 'mp4',
+                    'title': '#1 Variantenstudium',
+                    'episode': '#1 Variantenstudium',
+                    'series': '[209351] Strassenwesen',
+                    'creator': 'Neuhold R',
+                    'duration': 5420200,
+                }
+            }
+        ],
+        'min_playlist_count': 4
+    }]
+
+    def _real_extract(self, url):
+        id = self._match_id(url)
+        episodes_data = self._download_json(self._API_EPISODE, id, query={'sid': id}, note='Downloading episode list')
+        series_data = self._download_json(
+            'https://tube.tugraz.at/series/series.json', id, fatal=False,
+            note='downloading series metadata', errnote='failed to download series metadata',
+            query={
+                'seriesId': id,
+                'count': 1,
+                'sort': 'TITLE'
+            })
+
+        return self.playlist_result(
+            map(self._extract_episode, episodes_data['search-results']['result']), id,
+            traverse_obj(series_data, ('catalogs', 0, 'http://purl.org/dc/terms/', 'title', 0, 'value')))
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
index 6e0c31c01..7648b6fce 100644
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -3485,17 +3485,19 @@ def age_restricted(content_limit, age_limit):
     return age_limit < content_limit
 
 
+# List of known byte-order-marks (BOM)
+BOMS = [
+    (b'\xef\xbb\xbf', 'utf-8'),
+    (b'\x00\x00\xfe\xff', 'utf-32-be'),
+    (b'\xff\xfe\x00\x00', 'utf-32-le'),
+    (b'\xff\xfe', 'utf-16-le'),
+    (b'\xfe\xff', 'utf-16-be'),
+]
+
+
 def is_html(first_bytes):
     """ Detect whether a file contains HTML by examining its first bytes. """
 
-    BOMS = [
-        (b'\xef\xbb\xbf', 'utf-8'),
-        (b'\x00\x00\xfe\xff', 'utf-32-be'),
-        (b'\xff\xfe\x00\x00', 'utf-32-le'),
-        (b'\xff\xfe', 'utf-16-le'),
-        (b'\xfe\xff', 'utf-16-be'),
-    ]
-
     encoding = 'utf-8'
     for bom, enc in BOMS:
         while first_bytes.startswith(bom):
@@ -5394,6 +5396,24 @@ def read_stdin(what):
     return sys.stdin
 
 
+def determine_file_encoding(data):
+    """
+    Detect the text encoding used
+    @returns (encoding, bytes to skip)
+    """
+
+    # BOM marks are given priority over declarations
+    for bom, enc in BOMS:
+        if data.startswith(bom):
+            return enc, len(bom)
+
+    # Strip off all null bytes to match even when UTF-16 or UTF-32 is used.
+    # We ignore the endianness to get a good enough match
+    data = data.replace(b'\0', b'')
+    mobj = re.match(rb'(?m)^#\s*coding\s*:\s*(\S+)\s*$', data)
+    return mobj.group(1).decode() if mobj else None, 0
+
+
 class Config:
     own_args = None
     parsed_args = None
@@ -5445,12 +5465,17 @@ def __str__(self):
     @staticmethod
     def read_file(filename, default=[]):
         try:
-            optionf = open(filename)
+            optionf = open(filename, 'rb')
         except OSError:
             return default  # silently skip if file is not present
+        try:
+            enc, skip = determine_file_encoding(optionf.read(512))
+            optionf.seek(skip, io.SEEK_SET)
+        except OSError:
+            enc = None  # silently skip read errors
         try:
             # FIXME: https://github.com/ytdl-org/youtube-dl/commit/dfe5fa49aed02cf36ba9f743b11b0903554b5e56
-            contents = optionf.read()
+            contents = optionf.read().decode(enc or preferredencoding())
             res = shlex.split(contents, comments=True)
         except Exception as err:
             raise ValueError(f'Unable to parse "{filename}": {err}')


Test patch:
diff --git a/test/test_utils.py b/test/test_utils.py
index 948d5d059..bf46bdc61 100644
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -39,6 +39,7 @@
     datetime_from_str,
     detect_exe_version,
     determine_ext,
+    determine_file_encoding,
     dfxp2srt,
     dict_get,
     encode_base_n,
@@ -1822,6 +1823,25 @@ def test_locked_file(self):
             with contextlib.suppress(OSError):
                 os.remove(FILE)
 
+    def test_determine_file_encoding(self):
+        self.assertEqual(determine_file_encoding(b''), (None, 0))
+        self.assertEqual(determine_file_encoding(b'--verbose -x --audio-format mkv\n'), (None, 0))
+
+        self.assertEqual(determine_file_encoding(b'\xef\xbb\xbf'), ('utf-8', 3))
+        self.assertEqual(determine_file_encoding(b'\x00\x00\xfe\xff'), ('utf-32-be', 4))
+        self.assertEqual(determine_file_encoding(b'\xff\xfe'), ('utf-16-le', 2))
+
+        self.assertEqual(determine_file_encoding(b'\xff\xfe# coding: utf-8\n--verbose'), ('utf-16-le', 2))
+
+        self.assertEqual(determine_file_encoding(b'# coding: utf-8\n--verbose'), ('utf-8', 0))
+        self.assertEqual(determine_file_encoding(b'# coding: someencodinghere-12345\n--verbose'), ('someencodinghere-12345', 0))
+
+        self.assertEqual(determine_file_encoding(b'#coding:utf-8\n--verbose'), ('utf-8', 0))
+        self.assertEqual(determine_file_encoding(b'#  coding:   utf-8   \r\n--verbose'), ('utf-8', 0))
+
+        self.assertEqual(determine_file_encoding('# coding: utf-32-be'.encode('utf-32-be')), ('utf-32-be', 0))
+        self.assertEqual(determine_file_encoding('# coding: utf-16-le'.encode('utf-16-le')), ('utf-16-le', 0))
+
 
 if __name__ == '__main__':
     unittest.main()