Repository: yt-dlp/yt-dlp

Issue:
[Version 2023.01.02] /etc/yt-dlp.conf is not loaded
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting a bug unrelated to a specific site
- [X] I've verified that I'm running yt-dlp version **2023.01.02** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit)
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)

### Provide a description that is worded well enough to be understood

Hi,
it seems that since version 2023.01.02 the upload date from YouTube-Videos can't (?) be extracted by the following output template:

-o %(title)s_[%(upload_date>%Y-%m-%d)s]_[%(id)s].%(ext)s

Title and ID are extracted correectly.
Template configuration is stored in stored in /etc/yt-dlp.conf and worked until New Years Eve.

Can anybody confirm?
Best Regards /M.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
~/!_temp$ yt-dlp -vU aqz-KE-bpKQ
[debug] Command-line config: ['-vU', 'aqz-KE-bpKQ']
[debug] User config: []
[debug] System config: []
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.01.02 [d83b0ad] (zip)
[debug] Python 3.8.6 (CPython x86_64 64bit) - Linux-3.10.105-x86_64-with-glibc2.2.5 (OpenSSL 1.0.2u-fips  20 Dec 2019, glibc 2.20-2014.11)
[debug] exe versions: ffmpeg 2.7.7 (needs_adtstoasc)
[debug] Optional libraries: sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1754 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.01.02, Current version: 2023.01.02
yt-dlp is up to date (2023.01.02)
[youtube] Extracting URL: aqz-KE-bpKQ
[youtube] aqz-KE-bpKQ: Downloading webpage
[youtube] aqz-KE-bpKQ: Downloading android player API JSON
[youtube] aqz-KE-bpKQ: Downloading player e5f6cbd5
[debug] Saving youtube-nsig.e5f6cbd5 to cache
[debug] [youtube] Decrypted nsig KM0AnFlHKvzynxTEb => M-TXZDH19wD2Gw
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] aqz-KE-bpKQ: Downloading 1 format(s): 315+258
[debug] Invoking http downloader on "https://rr5---sn-4g5edn6r.googlevideo.com/videoplayback?expire=1672887224&ei=WOe1Y-yXAYi-1wKk6IXQAg&ip=2003%3Aea%3Aef05%3Aeb96%3A211%3A32ff%3Afe6c%3A2425&id=o-AGvLJndvkTeT6li5AUwg5mnE6UUjuUVETaKwyvERggfH&itag=315&source=youtube&requiressl=yes&mh=aP&mm=31%2C26&mn=sn-4g5edn6r%2Csn-5hnekn7k&ms=au%2Conr&mv=m&mvi=5&pl=35&initcwndbps=1205000&spc=zIddbFRRa6UKdjxzwGyjfRYDNLe4VyE&vprv=1&svpuc=1&mime=video%2Fwebm&gir=yes&clen=1536155487&dur=634.566&lmt=1662347928284893&mt=1672865118&fvip=5&keepalive=yes&fexp=24007246&c=ANDROID&txp=553C434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhANdZpV1XvXGH7Wmns5qLfBZUvdbSk3G7y9ssW_O9g6q7AiEAw4ybzvEiuBk5zrgiz286CiYAJe-IYqa0Jexz9Ulp7jc%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgNqrEiAh7LhPh0amLC0Ogq90mTTFBi-YcGLcUUE0IOHMCID_TozeBlYc0f2LfvwLf03VbnL4U7iaMYL9DFKg-u81K"
[download] Destination: Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f315.webm
[download] 100% of    1.43GiB in 00:03:55 at 6.23MiB/s
[debug] Invoking http downloader on "https://rr5---sn-4g5edn6r.googlevideo.com/videoplayback?expire=1672887224&ei=WOe1Y-yXAYi-1wKk6IXQAg&ip=2003%3Aea%3Aef05%3Aeb96%3A211%3A32ff%3Afe6c%3A2425&id=o-AGvLJndvkTeT6li5AUwg5mnE6UUjuUVETaKwyvERggfH&itag=258&source=youtube&requiressl=yes&mh=aP&mm=31%2C26&mn=sn-4g5edn6r%2Csn-5hnekn7k&ms=au%2Conr&mv=m&mvi=5&pl=35&initcwndbps=1205000&spc=zIddbFRRa6UKdjxzwGyjfRYDNLe4VyE&vprv=1&svpuc=1&mime=audio%2Fmp4&gir=yes&clen=30767520&dur=634.624&lmt=1662204997981909&mt=1672865118&fvip=5&keepalive=yes&fexp=24007246&c=ANDROID&txp=5532434&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgRDJv5TpU6rOr20YsqgG-4CrdYdBVYX9KBaR_WpbyXMgCIQDkaCtNYYB3xn2XdiwW0Ur5G6EBfyOQ2s5y-vX9VUvSjQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgNqrEiAh7LhPh0amLC0Ogq90mTTFBi-YcGLcUUE0IOHMCID_TozeBlYc0f2LfvwLf03VbnL4U7iaMYL9DFKg-u81K"
[download] Destination: Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f258.m4a
[download] 100% of   29.34MiB in 00:00:04 at 6.16MiB/s
[Merger] Merging formats into "Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].mkv"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f315.webm' -i 'file:Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f258.m4a' -c copy -map 0:v:0 -map 1:a:0 -movflags +faststart 'file:Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].temp.mkv'
Deleting original file Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f315.webm (pass -k to keep)
Deleting original file Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film [aqz-KE-bpKQ].f258.m4a (pass -k to keep)
```



Discussion:

Resolving patch:
diff --git a/yt_dlp/options.py b/yt_dlp/options.py
--- a/yt_dlp/options.py
+++ b/yt_dlp/options.py
@@ -40,49 +40,28 @@
 
 
 def parseOpts(overrideArguments=None, ignore_config_files='if_override'):
+    PACKAGE_NAME = 'yt-dlp'
+
     root = Config(create_parser())
     if ignore_config_files == 'if_override':
         ignore_config_files = overrideArguments is not None
 
+    def read_config(*paths):
+        path = os.path.join(*paths)
+        conf = Config.read_file(path, default=None)
+        if conf is not None:
+            return conf, path
+
     def _load_from_config_dirs(config_dirs):
         for config_dir in config_dirs:
-            conf_file_path = os.path.join(config_dir, 'config')
-            conf = Config.read_file(conf_file_path, default=None)
-            if conf is None:
-                conf_file_path += '.txt'
-                conf = Config.read_file(conf_file_path, default=None)
-            if conf is not None:
-                return conf, conf_file_path
-        return None, None
-
-    def _read_user_conf(package_name, default=None):
-        # .config/package_name.conf
-        xdg_config_home = os.getenv('XDG_CONFIG_HOME') or compat_expanduser('~/.config')
-        user_conf_file = os.path.join(xdg_config_home, '%s.conf' % package_name)
-        user_conf = Config.read_file(user_conf_file, default=None)
-        if user_conf is not None:
-            return user_conf, user_conf_file
-
-        # home (~/package_name.conf or ~/package_name.conf.txt)
-        user_conf_file = os.path.join(compat_expanduser('~'), '%s.conf' % package_name)
-        user_conf = Config.read_file(user_conf_file, default=None)
-        if user_conf is None:
-            user_conf_file += '.txt'
-            user_conf = Config.read_file(user_conf_file, default=None)
-        if user_conf is not None:
-            return user_conf, user_conf_file
-
-        # Package config directories (e.g. ~/.config/package_name/package_name.txt)
-        user_conf, user_conf_file = _load_from_config_dirs(get_user_config_dirs(package_name))
-        if user_conf is not None:
-            return user_conf, user_conf_file
-        return default if default is not None else [], None
+            head, tail = os.path.split(config_dir)
+            assert tail == PACKAGE_NAME or config_dir == os.path.join(compat_expanduser('~'), f'.{PACKAGE_NAME}')
 
-    def _read_system_conf(package_name, default=None):
-        system_conf, system_conf_file = _load_from_config_dirs(get_system_config_dirs(package_name))
-        if system_conf is not None:
-            return system_conf, system_conf_file
-        return default if default is not None else [], None
+            yield read_config(head, f'{PACKAGE_NAME}.conf')
+            if tail.startswith('.'):  # ~/.PACKAGE_NAME
+                yield read_config(head, f'{PACKAGE_NAME}.conf.txt')
+            yield read_config(config_dir, 'config')
+            yield read_config(config_dir, 'config.txt')
 
     def add_config(label, path=None, func=None):
         """ Adds config and returns whether to continue """
@@ -90,21 +69,21 @@ def add_config(label, path=None, func=None):
             return False
         elif func:
             assert path is None
-            args, current_path = func('yt-dlp')
+            args, current_path = next(
+                filter(None, _load_from_config_dirs(func(PACKAGE_NAME))), (None, None))
         else:
             current_path = os.path.join(path, 'yt-dlp.conf')
             args = Config.read_file(current_path, default=None)
         if args is not None:
             root.append_config(args, current_path, label=label)
-            return True
         return True
 
     def load_configs():
         yield not ignore_config_files
         yield add_config('Portable', get_executable_path())
         yield add_config('Home', expand_path(root.parse_known_args()[0].paths.get('home', '')).strip())
-        yield add_config('User', func=_read_user_conf)
-        yield add_config('System', func=_read_system_conf)
+        yield add_config('User', func=get_user_config_dirs)
+        yield add_config('System', func=get_system_config_dirs)
 
     opts = optparse.Values({'verbose': True, 'print_help': False})
     try:
diff --git a/yt_dlp/plugins.py b/yt_dlp/plugins.py
--- a/yt_dlp/plugins.py
+++ b/yt_dlp/plugins.py
@@ -5,7 +5,6 @@
 import importlib.util
 import inspect
 import itertools
-import os
 import pkgutil
 import sys
 import traceback
@@ -14,11 +13,11 @@
 from zipfile import ZipFile
 
 from .compat import functools  # isort: split
-from .compat import compat_expanduser
 from .utils import (
     get_executable_path,
     get_system_config_dirs,
     get_user_config_dirs,
+    orderedSet,
     write_string,
 )
 
@@ -57,7 +56,7 @@ def search_locations(self, fullname):
         candidate_locations = []
 
         def _get_package_paths(*root_paths, containing_folder='plugins'):
-            for config_dir in map(Path, root_paths):
+            for config_dir in orderedSet(map(Path, root_paths), lazy=True):
                 plugin_dir = config_dir / containing_folder
                 if not plugin_dir.is_dir():
                     continue
@@ -65,15 +64,15 @@ def _get_package_paths(*root_paths, containing_folder='plugins'):
 
         # Load from yt-dlp config folders
         candidate_locations.extend(_get_package_paths(
-            *get_user_config_dirs('yt-dlp'), *get_system_config_dirs('yt-dlp'),
+            *get_user_config_dirs('yt-dlp'),
+            *get_system_config_dirs('yt-dlp'),
             containing_folder='plugins'))
 
         # Load from yt-dlp-plugins folders
         candidate_locations.extend(_get_package_paths(
             get_executable_path(),
-            compat_expanduser('~'),
-            '/etc',
-            os.getenv('XDG_CONFIG_HOME') or compat_expanduser('~/.config'),
+            *get_user_config_dirs(''),
+            *get_system_config_dirs(''),
             containing_folder='yt-dlp-plugins'))
 
         candidate_locations.extend(map(Path, sys.path))  # PYTHONPATH
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -5387,36 +5387,22 @@ def get_executable_path():
 
 
 def get_user_config_dirs(package_name):
-    locations = set()
-
     # .config (e.g. ~/.config/package_name)
     xdg_config_home = os.getenv('XDG_CONFIG_HOME') or compat_expanduser('~/.config')
-    config_dir = os.path.join(xdg_config_home, package_name)
-    if os.path.isdir(config_dir):
-        locations.add(config_dir)
+    yield os.path.join(xdg_config_home, package_name)
 
     # appdata (%APPDATA%/package_name)
     appdata_dir = os.getenv('appdata')
     if appdata_dir:
-        config_dir = os.path.join(appdata_dir, package_name)
-        if os.path.isdir(config_dir):
-            locations.add(config_dir)
+        yield os.path.join(appdata_dir, package_name)
 
     # home (~/.package_name)
-    user_config_directory = os.path.join(compat_expanduser('~'), '.%s' % package_name)
-    if os.path.isdir(user_config_directory):
-        locations.add(user_config_directory)
-
-    return locations
+    yield os.path.join(compat_expanduser('~'), f'.{package_name}')
 
 
 def get_system_config_dirs(package_name):
-    locations = set()
     # /etc/package_name
-    system_config_directory = os.path.join('/etc', package_name)
-    if os.path.isdir(system_config_directory):
-        locations.add(system_config_directory)
-    return locations
+    yield os.path.join('/etc', package_name)
 
 
 def traverse_obj(


Test patch:
diff --git a/test/test_config.py b/test/test_config.py
new file mode 100644
--- /dev/null
+++ b/test/test_config.py
@@ -0,0 +1,227 @@
+#!/usr/bin/env python3
+
+# Allow direct execution
+import os
+import sys
+import unittest
+import unittest.mock
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import contextlib
+import itertools
+from pathlib import Path
+
+from yt_dlp.compat import compat_expanduser
+from yt_dlp.options import create_parser, parseOpts
+from yt_dlp.utils import Config, get_executable_path
+
+ENVIRON_DEFAULTS = {
+    'HOME': None,
+    'XDG_CONFIG_HOME': '/_xdg_config_home/',
+    'USERPROFILE': 'C:/Users/testing/',
+    'APPDATA': 'C:/Users/testing/AppData/Roaming/',
+    'HOMEDRIVE': 'C:/',
+    'HOMEPATH': 'Users/testing/',
+}
+
+
+@contextlib.contextmanager
+def set_environ(**kwargs):
+    saved_environ = os.environ.copy()
+
+    for name, value in {**ENVIRON_DEFAULTS, **kwargs}.items():
+        if value is None:
+            os.environ.pop(name, None)
+        else:
+            os.environ[name] = value
+
+    yield
+
+    os.environ.clear()
+    os.environ.update(saved_environ)
+
+
+def _generate_expected_groups():
+    xdg_config_home = os.getenv('XDG_CONFIG_HOME') or compat_expanduser('~/.config')
+    appdata_dir = os.getenv('appdata')
+    home_dir = compat_expanduser('~')
+    return {
+        'Portable': [
+            Path(get_executable_path(), 'yt-dlp.conf'),
+        ],
+        'Home': [
+            Path('yt-dlp.conf'),
+        ],
+        'User': [
+            Path(xdg_config_home, 'yt-dlp.conf'),
+            Path(xdg_config_home, 'yt-dlp', 'config'),
+            Path(xdg_config_home, 'yt-dlp', 'config.txt'),
+            *((
+                Path(appdata_dir, 'yt-dlp.conf'),
+                Path(appdata_dir, 'yt-dlp', 'config'),
+                Path(appdata_dir, 'yt-dlp', 'config.txt'),
+            ) if appdata_dir else ()),
+            Path(home_dir, 'yt-dlp.conf'),
+            Path(home_dir, 'yt-dlp.conf.txt'),
+            Path(home_dir, '.yt-dlp', 'config'),
+            Path(home_dir, '.yt-dlp', 'config.txt'),
+        ],
+        'System': [
+            Path('/etc/yt-dlp.conf'),
+            Path('/etc/yt-dlp/config'),
+            Path('/etc/yt-dlp/config.txt'),
+        ]
+    }
+
+
+class TestConfig(unittest.TestCase):
+    maxDiff = None
+
+    @set_environ()
+    def test_config__ENVIRON_DEFAULTS_sanity(self):
+        expected = make_expected()
+        self.assertCountEqual(
+            set(expected), expected,
+            'ENVIRON_DEFAULTS produces non unique names')
+
+    def test_config_all_environ_values(self):
+        for name, value in ENVIRON_DEFAULTS.items():
+            for new_value in (None, '', '.', value or '/some/dir'):
+                with set_environ(**{name: new_value}):
+                    self._simple_grouping_test()
+
+    def test_config_default_expected_locations(self):
+        files, _ = self._simple_config_test()
+        self.assertEqual(
+            files, make_expected(),
+            'Not all expected locations have been checked')
+
+    def test_config_default_grouping(self):
+        self._simple_grouping_test()
+
+    def _simple_grouping_test(self):
+        expected_groups = make_expected_groups()
+        for name, group in expected_groups.items():
+            for index, existing_path in enumerate(group):
+                result, opts = self._simple_config_test(existing_path)
+                expected = expected_from_expected_groups(expected_groups, existing_path)
+                self.assertEqual(
+                    result, expected,
+                    f'The checked locations do not match the expected ({name}, {index})')
+                self.assertEqual(
+                    opts.outtmpl['default'], '1',
+                    f'The used result value was incorrect ({name}, {index})')
+
+    def _simple_config_test(self, *stop_paths):
+        encountered = 0
+        paths = []
+
+        def read_file(filename, default=[]):
+            nonlocal encountered
+            path = Path(filename)
+            paths.append(path)
+            if path in stop_paths:
+                encountered += 1
+                return ['-o', f'{encountered}']
+
+        with ConfigMock(read_file):
+            _, opts, _ = parseOpts([], False)
+
+        return paths, opts
+
+    @set_environ()
+    def test_config_early_exit_commandline(self):
+        self._early_exit_test(0, '--ignore-config')
+
+    @set_environ()
+    def test_config_early_exit_files(self):
+        for index, _ in enumerate(make_expected(), 1):
+            self._early_exit_test(index)
+
+    def _early_exit_test(self, allowed_reads, *args):
+        reads = 0
+
+        def read_file(filename, default=[]):
+            nonlocal reads
+            reads += 1
+
+            if reads > allowed_reads:
+                self.fail('The remaining config was not ignored')
+            elif reads == allowed_reads:
+                return ['--ignore-config']
+
+        with ConfigMock(read_file):
+            parseOpts(args, False)
+
+    @set_environ()
+    def test_config_override_commandline(self):
+        self._override_test(0, '-o', 'pass')
+
+    @set_environ()
+    def test_config_override_files(self):
+        for index, _ in enumerate(make_expected(), 1):
+            self._override_test(index)
+
+    def _override_test(self, start_index, *args):
+        index = 0
+
+        def read_file(filename, default=[]):
+            nonlocal index
+            index += 1
+
+            if index > start_index:
+                return ['-o', 'fail']
+            elif index == start_index:
+                return ['-o', 'pass']
+
+        with ConfigMock(read_file):
+            _, opts, _ = parseOpts(args, False)
+
+        self.assertEqual(
+            opts.outtmpl['default'], 'pass',
+            'The earlier group did not override the later ones')
+
+
+@contextlib.contextmanager
+def ConfigMock(read_file=None):
+    with unittest.mock.patch('yt_dlp.options.Config') as mock:
+        mock.return_value = Config(create_parser())
+        if read_file is not None:
+            mock.read_file = read_file
+
+        yield mock
+
+
+def make_expected(*filepaths):
+    return expected_from_expected_groups(_generate_expected_groups(), *filepaths)
+
+
+def make_expected_groups(*filepaths):
+    return _filter_expected_groups(_generate_expected_groups(), filepaths)
+
+
+def expected_from_expected_groups(expected_groups, *filepaths):
+    return list(itertools.chain.from_iterable(
+        _filter_expected_groups(expected_groups, filepaths).values()))
+
+
+def _filter_expected_groups(expected, filepaths):
+    if not filepaths:
+        return expected
+
+    result = {}
+    for group, paths in expected.items():
+        new_paths = []
+        for path in paths:
+            new_paths.append(path)
+            if path in filepaths:
+                break
+
+        result[group] = new_paths
+
+    return result
+
+
+if __name__ == '__main__':
+    unittest.main()