Repository: yt-dlp/yt-dlp
Pull request #6180 (2023-02-06 22:28:49)

[utils] `traverse_obj`: Fix several behavioral problems
**IMPORTANT**: PRs without the template will be CLOSED

### Description of your *pull request* and other information

<!--

Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible

-->

This PR adapts the expected results from the previous version to closer mimic what is desirable in an extractor context.
Branch prediction is removed and instead replaced by the `apply_key` result of `branching, result` (either `False, <value>` or `True, <values iterable>`).

### Changes
- Values now get pruned if they are decided unhelpful (that is they are falsy and not `bool`, `int` or `float`).
- A trailing `dict` key now always returns a dictionary, making `**traverse_obj(None, {})` safe.
- `traverse_string` is now correctly handled when `is_user_input` is `False` (though not branching).
- `is_user_input` no longer supports string keys containing `:`.
- `default` now always applies to the values in the last `dict` key (`traverse_obj(None, {}, default=...)` gives `{}`)
- `default` now applies to nested dict keys. Try to avoid this scenario.

<details open><summary>Template</summary> <!-- OPEN is intentional -->

<!--

# PLEASE FOLLOW THE GUIDE BELOW

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like

-->

### Before submitting a *pull request* make sure you have:
- [x] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [ ] Fix or improvement to an extractor (Make sure to add/update tests)
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [x] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))

</details>



Patch:
diff --git a/Makefile b/Makefile
index ca7d641ab..d5d47629b 100644
--- a/Makefile
+++ b/Makefile
@@ -74,7 +74,7 @@ offlinetest: codetest
 	$(PYTHON) -m pytest -k "not download"
 
 # XXX: This is hard to maintain
-CODE_FOLDERS = yt_dlp yt_dlp/downloader yt_dlp/extractor yt_dlp/postprocessor yt_dlp/compat
+CODE_FOLDERS = yt_dlp yt_dlp/downloader yt_dlp/extractor yt_dlp/postprocessor yt_dlp/compat yt_dlp/dependencies
 yt-dlp: yt_dlp/*.py yt_dlp/*/*.py
 	mkdir -p zip
 	for d in $(CODE_FOLDERS) ; do \
diff --git a/yt_dlp/aes.py b/yt_dlp/aes.py
index 60ce99cb1..deff0a2b3 100644
--- a/yt_dlp/aes.py
+++ b/yt_dlp/aes.py
@@ -2,17 +2,17 @@
 from math import ceil
 
 from .compat import compat_ord
-from .dependencies import Cryptodome_AES
+from .dependencies import Cryptodome
 from .utils import bytes_to_intlist, intlist_to_bytes
 
-if Cryptodome_AES:
+if Cryptodome:
     def aes_cbc_decrypt_bytes(data, key, iv):
         """ Decrypt bytes with AES-CBC using pycryptodome """
-        return Cryptodome_AES.new(key, Cryptodome_AES.MODE_CBC, iv).decrypt(data)
+        return Cryptodome.Cipher.AES.new(key, Cryptodome.Cipher.AES.MODE_CBC, iv).decrypt(data)
 
     def aes_gcm_decrypt_and_verify_bytes(data, key, tag, nonce):
         """ Decrypt bytes with AES-GCM using pycryptodome """
-        return Cryptodome_AES.new(key, Cryptodome_AES.MODE_GCM, nonce).decrypt_and_verify(data, tag)
+        return Cryptodome.Cipher.AES.new(key, Cryptodome.Cipher.AES.MODE_GCM, nonce).decrypt_and_verify(data, tag)
 
 else:
     def aes_cbc_decrypt_bytes(data, key, iv):
diff --git a/yt_dlp/compat/__init__.py b/yt_dlp/compat/__init__.py
index 5d3db4b4c..5cc78ebc2 100644
--- a/yt_dlp/compat/__init__.py
+++ b/yt_dlp/compat/__init__.py
@@ -8,7 +8,7 @@
 
 # XXX: Implement this the same way as other DeprecationWarnings without circular import
 passthrough_module(__name__, '._legacy', callback=lambda attr: warnings.warn(
-    DeprecationWarning(f'{__name__}.{attr} is deprecated'), stacklevel=3))
+    DeprecationWarning(f'{__name__}.{attr} is deprecated'), stacklevel=5))
 
 
 # HTMLParseError has been deprecated in Python 3.3 and removed in
diff --git a/yt_dlp/compat/compat_utils.py b/yt_dlp/compat/compat_utils.py
index 82e176281..f8679c98e 100644
--- a/yt_dlp/compat/compat_utils.py
+++ b/yt_dlp/compat/compat_utils.py
@@ -1,5 +1,6 @@
 import collections
 import contextlib
+import functools
 import importlib
 import sys
 import types
@@ -10,61 +11,87 @@
 
 
 def get_package_info(module):
-    parent = module.__name__.split('.')[0]
-    parent_module = None
-    with contextlib.suppress(ImportError):
-        parent_module = importlib.import_module(parent)
-
-    for attr in ('__version__', 'version_string', 'version'):
-        version = getattr(parent_module, attr, None)
-        if version is not None:
-            break
-    return _Package(getattr(module, '_yt_dlp__identifier', parent), str(version))
+    return _Package(
+        name=getattr(module, '_yt_dlp__identifier', module.__name__),
+        version=str(next(filter(None, (
+            getattr(module, attr, None)
+            for attr in ('__version__', 'version_string', 'version')
+        )), None)))
 
 
 def _is_package(module):
-    try:
-        module.__getattribute__('__path__')
-    except AttributeError:
-        return False
-    return True
-
-
-def passthrough_module(parent, child, allowed_attributes=None, *, callback=lambda _: None):
-    parent_module = importlib.import_module(parent)
-    child_module = None  # Import child module only as needed
-
-    class PassthroughModule(types.ModuleType):
-        def __getattr__(self, attr):
-            if _is_package(parent_module):
-                with contextlib.suppress(ImportError):
-                    return importlib.import_module(f'.{attr}', parent)
-
-            ret = self.__from_child(attr)
-            if ret is _NO_ATTRIBUTE:
-                raise AttributeError(f'module {parent} has no attribute {attr}')
-            callback(attr)
-            return ret
-
-        def __from_child(self, attr):
-            if allowed_attributes is None:
-                if attr.startswith('__') and attr.endswith('__'):
-                    return _NO_ATTRIBUTE
-            elif attr not in allowed_attributes:
+    return '__path__' in vars(module)
+
+
+def _is_dunder(name):
+    return name.startswith('__') and name.endswith('__')
+
+
+class EnhancedModule(types.ModuleType):
+    def __new__(cls, name, *args, **kwargs):
+        if name not in sys.modules:
+            return super().__new__(cls, name, *args, **kwargs)
+
+        assert not args and not kwargs, 'Cannot pass additional arguments to an existing module'
+        module = sys.modules[name]
+        module.__class__ = cls
+        return module
+
+    def __init__(self, name, *args, **kwargs):
+        # Prevent __new__ from trigerring __init__ again
+        if name not in sys.modules:
+            super().__init__(name, *args, **kwargs)
+
+    def __bool__(self):
+        return vars(self).get('__bool__', lambda: True)()
+
+    def __getattribute__(self, attr):
+        try:
+            ret = super().__getattribute__(attr)
+        except AttributeError:
+            if _is_dunder(attr):
+                raise
+            getter = getattr(self, '__getattr__', None)
+            if not getter:
+                raise
+            ret = getter(attr)
+        return ret.fget() if isinstance(ret, property) else ret
+
+
+def passthrough_module(parent, child, allowed_attributes=(..., ), *, callback=lambda _: None):
+    """Passthrough parent module into a child module, creating the parent if necessary"""
+    parent = EnhancedModule(parent)
+
+    def __getattr__(attr):
+        if _is_package(parent):
+            with contextlib.suppress(ImportError):
+                return importlib.import_module(f'.{attr}', parent.__name__)
+
+        ret = from_child(attr)
+        if ret is _NO_ATTRIBUTE:
+            raise AttributeError(f'module {parent.__name__} has no attribute {attr}')
+        callback(attr)
+        return ret
+
+    @functools.lru_cache(maxsize=None)
+    def from_child(attr):
+        nonlocal child
+        if attr not in allowed_attributes:
+            if ... not in allowed_attributes or _is_dunder(attr):
                 return _NO_ATTRIBUTE
 
-            nonlocal child_module
-            child_module = child_module or importlib.import_module(child, parent)
+        if isinstance(child, str):
+            child = importlib.import_module(child, parent.__name__)
 
-            with contextlib.suppress(AttributeError):
-                return getattr(child_module, attr)
+        if _is_package(child):
+            with contextlib.suppress(ImportError):
+                return passthrough_module(f'{parent.__name__}.{attr}',
+                                          importlib.import_module(f'.{attr}', child.__name__))
 
-            if _is_package(child_module):
-                with contextlib.suppress(ImportError):
-                    return importlib.import_module(f'.{attr}', child)
+        with contextlib.suppress(AttributeError):
+            return getattr(child, attr)
 
-            return _NO_ATTRIBUTE
+        return _NO_ATTRIBUTE
 
-    # Python 3.6 does not have module level __getattr__
-    # https://peps.python.org/pep-0562/
-    sys.modules[parent].__class__ = PassthroughModule
+    parent.__getattr__ = __getattr__
+    return parent
diff --git a/yt_dlp/dependencies/Cryptodome.py b/yt_dlp/dependencies/Cryptodome.py
new file mode 100644
index 000000000..580ce0753
--- /dev/null
+++ b/yt_dlp/dependencies/Cryptodome.py
@@ -0,0 +1,28 @@
+from ..compat import functools
+from ..compat.compat_utils import EnhancedModule, passthrough_module
+
+try:
+    import Cryptodome as _parent
+except ImportError:
+    try:
+        import Crypto as _parent
+    except (ImportError, SyntaxError):  # Old Crypto gives SyntaxError in newer Python
+        _parent = EnhancedModule('Cryptodome')
+        __bool__ = lambda: False
+
+passthrough_module(__name__, _parent, (..., '__version__'))
+del passthrough_module, EnhancedModule
+
+
+@property
+@functools.cache
+def _yt_dlp__identifier():
+    if _parent.__name__ == 'Crypto':
+        from Crypto.Cipher import AES
+        try:
+            # In pycrypto, mode defaults to ECB. See:
+            # https://www.pycryptodome.org/en/latest/src/vs_pycrypto.html#:~:text=not%20have%20ECB%20as%20default%20mode
+            AES.new(b'abcdefghijklmnop')
+        except TypeError:
+            return 'pycrypto'
+    return _parent.__name__
diff --git a/yt_dlp/dependencies.py b/yt_dlp/dependencies/__init__.py
similarity index 75%
rename from yt_dlp/dependencies.py
rename to yt_dlp/dependencies/__init__.py
index 5a5363adb..c2214e6db 100644
--- a/yt_dlp/dependencies.py
+++ b/yt_dlp/dependencies/__init__.py
@@ -23,24 +23,6 @@
         certifi = None
 
 
-try:
-    from Cryptodome.Cipher import AES as Cryptodome_AES
-except ImportError:
-    try:
-        from Crypto.Cipher import AES as Cryptodome_AES
-    except (ImportError, SyntaxError):  # Old Crypto gives SyntaxError in newer Python
-        Cryptodome_AES = None
-    else:
-        try:
-            # In pycrypto, mode defaults to ECB. See:
-            # https://www.pycryptodome.org/en/latest/src/vs_pycrypto.html#:~:text=not%20have%20ECB%20as%20default%20mode
-            Cryptodome_AES.new(b'abcdefghijklmnop')
-        except TypeError:
-            pass
-        else:
-            Cryptodome_AES._yt_dlp__identifier = 'pycrypto'
-
-
 try:
     import mutagen
 except ImportError:
@@ -84,10 +66,14 @@
         xattr._yt_dlp__identifier = 'pyxattr'
 
 
+from . import Cryptodome
+
 all_dependencies = {k: v for k, v in globals().items() if not k.startswith('_')}
+available_dependencies = {k: v for k, v in all_dependencies.items() if v}
 
 
-available_dependencies = {k: v for k, v in all_dependencies.items() if v}
+# Deprecated
+Cryptodome_AES = Cryptodome.Cipher.AES if Cryptodome else None
 
 
 __all__ = [
diff --git a/yt_dlp/downloader/hls.py b/yt_dlp/downloader/hls.py
index 2010f3dc9..ae18ac419 100644
--- a/yt_dlp/downloader/hls.py
+++ b/yt_dlp/downloader/hls.py
@@ -7,7 +7,7 @@
 from .external import FFmpegFD
 from .fragment import FragmentFD
 from .. import webvtt
-from ..dependencies import Cryptodome_AES
+from ..dependencies import Cryptodome
 from ..utils import bug_reports_message, parse_m3u8_attributes, update_url_query
 
 
@@ -63,7 +63,7 @@ def real_download(self, filename, info_dict):
         can_download, message = self.can_download(s, info_dict, self.params.get('allow_unplayable_formats')), None
         if can_download:
             has_ffmpeg = FFmpegFD.available()
-            no_crypto = not Cryptodome_AES and '#EXT-X-KEY:METHOD=AES-128' in s
+            no_crypto = not Cryptodome and '#EXT-X-KEY:METHOD=AES-128' in s
             if no_crypto and has_ffmpeg:
                 can_download, message = False, 'The stream has AES-128 encryption and pycryptodomex is not available'
             elif no_crypto:
diff --git a/yt_dlp/extractor/bilibili.py b/yt_dlp/extractor/bilibili.py
index d4b05248f..266d57871 100644
--- a/yt_dlp/extractor/bilibili.py
+++ b/yt_dlp/extractor/bilibili.py
@@ -6,6 +6,7 @@
 import urllib.parse
 
 from .common import InfoExtractor, SearchInfoExtractor
+from ..dependencies import Cryptodome
 from ..utils import (
     ExtractorError,
     GeoRestrictedError,
@@ -893,22 +894,15 @@ def _parse_video_metadata(self, video_data):
         }
 
     def _perform_login(self, username, password):
-        try:
-            from Cryptodome.PublicKey import RSA
-            from Cryptodome.Cipher import PKCS1_v1_5
-        except ImportError:
-            try:
-                from Crypto.PublicKey import RSA
-                from Crypto.Cipher import PKCS1_v1_5
-            except ImportError:
-                raise ExtractorError('pycryptodomex not found. Please install', expected=True)
+        if not Cryptodome:
+            raise ExtractorError('pycryptodomex not found. Please install', expected=True)
 
         key_data = self._download_json(
             'https://passport.bilibili.tv/x/intl/passport-login/web/key?lang=en-US', None,
             note='Downloading login key', errnote='Unable to download login key')['data']
 
-        public_key = RSA.importKey(key_data['key'])
-        password_hash = PKCS1_v1_5.new(public_key).encrypt((key_data['hash'] + password).encode('utf-8'))
+        public_key = Cryptodome.PublicKey.RSA.importKey(key_data['key'])
+        password_hash = Cryptodome.Cipher.PKCS1_v1_5.new(public_key).encrypt((key_data['hash'] + password).encode('utf-8'))
         login_post = self._download_json(
             'https://passport.bilibili.tv/x/intl/passport-login/web/login/password?lang=en-US', None, data=urlencode_postdata({
                 'username': username,
diff --git a/yt_dlp/extractor/ivi.py b/yt_dlp/extractor/ivi.py
index dc6a48196..96220bea9 100644
--- a/yt_dlp/extractor/ivi.py
+++ b/yt_dlp/extractor/ivi.py
@@ -2,11 +2,8 @@
 import re
 
 from .common import InfoExtractor
-from ..utils import (
-    ExtractorError,
-    int_or_none,
-    qualities,
-)
+from ..dependencies import Cryptodome
+from ..utils import ExtractorError, int_or_none, qualities
 
 
 class IviIE(InfoExtractor):
@@ -94,18 +91,8 @@ def _real_extract(self, url):
         for site in (353, 183):
             content_data = (data % site).encode()
             if site == 353:
-                try:
-                    from Cryptodome.Cipher import Blowfish
-                    from Cryptodome.Hash import CMAC
-                    pycryptodome_found = True
-                except ImportError:
-                    try:
-                        from Crypto.Cipher import Blowfish
-                        from Crypto.Hash import CMAC
-                        pycryptodome_found = True
-                    except ImportError:
-                        pycryptodome_found = False
-                        continue
+                if not Cryptodome:
+                    continue
 
                 timestamp = (self._download_json(
                     self._LIGHT_URL, video_id,
@@ -118,7 +105,8 @@ def _real_extract(self, url):
 
                 query = {
                     'ts': timestamp,
-                    'sign': CMAC.new(self._LIGHT_KEY, timestamp.encode() + content_data, Blowfish).hexdigest(),
+                    'sign': Cryptodome.Hash.CMAC.new(self._LIGHT_KEY, timestamp.encode() + content_data,
+                                                     Cryptodome.Cipher.Blowfish).hexdigest(),
                 }
             else:
                 query = {}
@@ -138,7 +126,7 @@ def _real_extract(self, url):
                     extractor_msg = 'Video %s does not exist'
                 elif site == 353:
                     continue
-                elif not pycryptodome_found:
+                elif not Cryptodome:
                     raise ExtractorError('pycryptodomex not found. Please install', expected=True)
                 elif message:
                     extractor_msg += ': ' + message
diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py
index e1e0f7b25..878b2b6a8 100644
--- a/yt_dlp/utils.py
+++ b/yt_dlp/utils.py
@@ -5420,7 +5420,7 @@ def traverse_obj(
     Each of the provided `paths` is tested and the first producing a valid result will be returned.
     The next path will also be tested if the path branched but no results could be found.
     Supported values for traversal are `Mapping`, `Sequence` and `re.Match`.
-    A value of None is treated as the absence of a value.
+    Unhelpful values (`[]`, `{}`, `None`) are treated as the absence of a value and discarded.
 
     The paths will be wrapped in `variadic`, so that `'key'` is conveniently the same as `('key', )`.
 
@@ -5446,6 +5446,8 @@ def traverse_obj(
 
     @params paths           Paths which to traverse by.
     @param default          Value to return if the paths do not match.
+                            If the last key in the path is a `dict`, it will apply to each value inside
+                            the dict instead, depth first. Try to avoid if using nested `dict` keys.
     @param expected_type    If a `type`, only accept final values of this type.
                             If any other callable, try to call the function on each result.
                             If the last key in the path is a `dict`, it will apply to each value inside
@@ -5460,12 +5462,15 @@ def traverse_obj(
     @param traverse_string  Whether to traverse into objects as strings.
                             If `True`, any non-compatible object will first be
                             converted into a string and then traversed into.
+                            The return value of that path will be a string instead,
+                            not respecting any further branching.
 
 
     @returns                The result of the object traversal.
                             If successful, `get_all=True`, and the path branches at least once,
                             then a list of results is returned instead.
-                            A list is always returned if the last path branches and no `default` is given.
+                            If no `default` is given and the last path branches, a `list` of results
+                            is always returned. If a path ends on a `dict` that result will always be a `dict`.
     """
     is_sequence = lambda x: isinstance(x, collections.abc.Sequence) and not isinstance(x, (str, bytes))
     casefold = lambda k: k.casefold() if isinstance(k, str) else k
@@ -5475,87 +5480,94 @@ def traverse_obj(
     else:
         type_test = lambda val: try_call(expected_type or IDENTITY, args=(val,))
 
-    def apply_key(key, test_type, obj):
+    def apply_key(key, obj, is_last):
+        branching = False
+        result = None
+
         if obj is None:
-            return
+            pass
 
         elif key is None:
-            yield obj
+            result = obj
 
         elif isinstance(key, set):
             assert len(key) == 1, 'Set should only be used to wrap a single item'
             item = next(iter(key))
             if isinstance(item, type):
                 if isinstance(obj, item):
-                    yield obj
+                    result = obj
             else:
-                yield try_call(item, args=(obj,))
+                result = try_call(item, args=(obj,))
 
         elif isinstance(key, (list, tuple)):
-            for branch in key:
-                _, result = apply_path(obj, branch, test_type)
-                yield from result
+            branching = True
+            result = itertools.chain.from_iterable(
+                apply_path(obj, branch, is_last)[0] for branch in key)
 
         elif key is ...:
+            branching = True
             if isinstance(obj, collections.abc.Mapping):
-                yield from obj.values()
+                result = obj.values()
             elif is_sequence(obj):
-                yield from obj
+                result = obj
             elif isinstance(obj, re.Match):
-                yield from obj.groups()
+                result = obj.groups()
             elif traverse_string:
-                yield from str(obj)
+                branching = False
+                result = str(obj)
+            else:
+                result = ()
 
         elif callable(key):
-            if is_sequence(obj):
-                iter_obj = enumerate(obj)
-            elif isinstance(obj, collections.abc.Mapping):
+            branching = True
+            if isinstance(obj, collections.abc.Mapping):
                 iter_obj = obj.items()
+            elif is_sequence(obj):
+                iter_obj = enumerate(obj)
             elif isinstance(obj, re.Match):
                 iter_obj = itertools.chain(
                     enumerate((obj.group(), *obj.groups())),
                     obj.groupdict().items())
             elif traverse_string:
+                branching = False
                 iter_obj = enumerate(str(obj))
             else:
-                return
-            yield from (v for k, v in iter_obj if try_call(key, args=(k, v)))
+                iter_obj = ()
+
+            result = (v for k, v in iter_obj if try_call(key, args=(k, v)))
+            if not branching:  # string traversal
+                result = ''.join(result)
 
         elif isinstance(key, dict):
-            iter_obj = ((k, _traverse_obj(obj, v, test_type=test_type)) for k, v in key.items())
-            yield {k: v if v is not None else default for k, v in iter_obj
-                   if v is not None or default is not NO_DEFAULT}
+            iter_obj = ((k, _traverse_obj(obj, v, False, is_last)) for k, v in key.items())
+            result = {
+                k: v if v is not None else default for k, v in iter_obj
+                if v is not None or default is not NO_DEFAULT
+            } or None
 
         elif isinstance(obj, collections.abc.Mapping):
-            yield (obj.get(key) if casesense or (key in obj)
-                   else next((v for k, v in obj.items() if casefold(k) == key), None))
+            result = (obj.get(key) if casesense or (key in obj) else
+                      next((v for k, v in obj.items() if casefold(k) == key), None))
 
         elif isinstance(obj, re.Match):
             if isinstance(key, int) or casesense:
                 with contextlib.suppress(IndexError):
-                    yield obj.group(key)
-                    return
+                    result = obj.group(key)
 
-            if not isinstance(key, str):
-                return
-
-            yield next((v for k, v in obj.groupdict().items() if casefold(k) == key), None)
-
-        else:
-            if is_user_input:
-                key = (int_or_none(key) if ':' not in key
-                       else slice(*map(int_or_none, key.split(':'))))
-
-            if not isinstance(key, (int, slice)):
-                return
+            elif isinstance(key, str):
+                result = next((v for k, v in obj.groupdict().items() if casefold(k) == key), None)
 
+        elif isinstance(key, (int, slice)):
             if not is_sequence(obj):
-                if not traverse_string:
-                    return
-                obj = str(obj)
+                if traverse_string:
+                    with contextlib.suppress(IndexError):
+                        result = str(obj)[key]
+            else:
+                branching = isinstance(key, slice)
+                with contextlib.suppress(IndexError):
+                    result = obj[key]
 
-            with contextlib.suppress(IndexError):
-                yield obj[key]
+        return branching, result if branching else (result,)
 
     def lazy_last(iterable):
         iterator = iter(iterable)
@@ -5569,45 +5581,54 @@ def lazy_last(iterable):
 
         yield True, prev
 
-    def apply_path(start_obj, path, test_type=False):
+    def apply_path(start_obj, path, test_type):
         objs = (start_obj,)
         has_branched = False
 
         key = None
         for last, key in lazy_last(variadic(path, (str, bytes, dict, set))):
-            if is_user_input and key == ':':
-                key = ...
+            if is_user_input and isinstance(key, str):
+                if key == ':':
+                    key = ...
+                elif ':' in key:
+                    key = slice(*map(int_or_none, key.split(':')))
+                elif int_or_none(key) is not None:
+                    key = int(key)
 
             if not casesense and isinstance(key, str):
                 key = key.casefold()
 
-            if key is ... or isinstance(key, (list, tuple)) or callable(key):
-                has_branched = True
-
             if __debug__ and callable(key):
                 # Verify function signature
                 inspect.signature(key).bind(None, None)
 
-            key_func = functools.partial(apply_key, key, last)
-            objs = itertools.chain.from_iterable(map(key_func, objs))
+            new_objs = []
+            for obj in objs:
+                branching, results = apply_key(key, obj, last)
+                has_branched |= branching
+                new_objs.append(results)
+
+            objs = itertools.chain.from_iterable(new_objs)
 
         if test_type and not isinstance(key, (dict, list, tuple)):
             objs = map(type_test, objs)
 
-        return has_branched, objs
-
-    def _traverse_obj(obj, path, use_list=True, test_type=True):
-        has_branched, results = apply_path(obj, path, test_type)
-        results = LazyList(x for x in results if x is not None)
+        return objs, has_branched, isinstance(key, dict)
 
+    def _traverse_obj(obj, path, allow_empty, test_type):
+        results, has_branched, is_dict = apply_path(obj, path, test_type)
+        results = LazyList(item for item in results if item not in (None, [], {}))
         if get_all and has_branched:
-            return results.exhaust() if results or use_list else None
+            if results:
+                return results.exhaust()
+            if allow_empty:
+                return [] if default is NO_DEFAULT else default
+            return None
 
-        return results[0] if results else None
+        return results[0] if results else {} if allow_empty and is_dict else None
 
     for index, path in enumerate(paths, 1):
-        use_list = default is NO_DEFAULT and index == len(paths)
-        result = _traverse_obj(obj, path, use_list)
+        result = _traverse_obj(obj, path, index == len(paths), True)
         if result is not None:
             return result
 


Test patch:
diff --git a/test/test_aes.py b/test/test_aes.py
index 8e8fc0b3e..18f15fecb 100644
--- a/test/test_aes.py
+++ b/test/test_aes.py
@@ -26,7 +26,7 @@
     key_expansion,
     pad_block,
 )
-from yt_dlp.dependencies import Cryptodome_AES
+from yt_dlp.dependencies import Cryptodome
 from yt_dlp.utils import bytes_to_intlist, intlist_to_bytes
 
 # the encrypted data can be generate with 'devscripts/generate_aes_testdata.py'
@@ -48,7 +48,7 @@ def test_cbc_decrypt(self):
         data = b'\x97\x92+\xe5\x0b\xc3\x18\x91ky9m&\xb3\xb5@\xe6\x27\xc2\x96.\xc8u\x88\xab9-[\x9e|\xf1\xcd'
         decrypted = intlist_to_bytes(aes_cbc_decrypt(bytes_to_intlist(data), self.key, self.iv))
         self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
-        if Cryptodome_AES:
+        if Cryptodome:
             decrypted = aes_cbc_decrypt_bytes(data, intlist_to_bytes(self.key), intlist_to_bytes(self.iv))
             self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
 
@@ -78,7 +78,7 @@ def test_gcm_decrypt(self):
         decrypted = intlist_to_bytes(aes_gcm_decrypt_and_verify(
             bytes_to_intlist(data), self.key, bytes_to_intlist(authentication_tag), self.iv[:12]))
         self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
-        if Cryptodome_AES:
+        if Cryptodome:
             decrypted = aes_gcm_decrypt_and_verify_bytes(
                 data, intlist_to_bytes(self.key), authentication_tag, intlist_to_bytes(self.iv[:12]))
             self.assertEqual(decrypted.rstrip(b'\x08'), self.secret_msg)
diff --git a/test/test_compat.py b/test/test_compat.py
index e3d775bc1..003a97abf 100644
--- a/test/test_compat.py
+++ b/test/test_compat.py
@@ -31,6 +31,9 @@ def test_compat_passthrough(self):
         # TODO: Test submodule
         # compat.asyncio.events  # Must not raise error
 
+        with self.assertWarns(DeprecationWarning):
+            compat.compat_pycrypto_AES  # Must not raise error
+
     def test_compat_expanduser(self):
         old_home = os.environ.get('HOME')
         test_str = R'C:\Documents and Settings\тест\Application Data'
diff --git a/test/test_utils.py b/test/test_utils.py
index ffe1b729f..190e4ef9b 100644
--- a/test/test_utils.py
+++ b/test/test_utils.py
@@ -2000,8 +2000,8 @@ def test_traverse_obj(self):
 
         # Test Ellipsis behavior
         self.assertCountEqual(traverse_obj(_TEST_DATA, ...),
-                              (item for item in _TEST_DATA.values() if item is not None),
-                              msg='`...` should give all values except `None`')
+                              (item for item in _TEST_DATA.values() if item not in (None, [], {})),
+                              msg='`...` should give all non discarded values')
         self.assertCountEqual(traverse_obj(_TEST_DATA, ('urls', 0, ...)), _TEST_DATA['urls'][0].values(),
                               msg='`...` selection for dicts should select all values')
         self.assertEqual(traverse_obj(_TEST_DATA, (..., ..., 'url')),
@@ -2084,15 +2084,23 @@ def test_traverse_obj(self):
                          {0: ['https://www.example.com/1', 'https://www.example.com/0']},
                          msg='tripple nesting in dict path should be treated as branches')
         self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}), {},
-                         msg='remove `None` values when dict key')
+                         msg='remove `None` values when top level dict key fails')
         self.assertEqual(traverse_obj(_TEST_DATA, {0: 'fail'}, default=...), {0: ...},
-                         msg='do not remove `None` values if `default`')
-        self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}), {0: {}},
-                         msg='do not remove empty values when dict key')
-        self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}, default=...), {0: {}},
-                         msg='do not remove empty values when dict key and a default')
-        self.assertEqual(traverse_obj(_TEST_DATA, {0: ('dict', ...)}), {0: []},
-                         msg='if branch in dict key not successful, return `[]`')
+                         msg='use `default` if key fails and `default`')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}), {},
+                         msg='remove empty values when dict key')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: 'dict'}, default=...), {0: ...},
+                         msg='use `default` when dict key and `default`')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 'fail'}}), {},
+                         msg='remove empty values when nested dict key fails')
+        self.assertEqual(traverse_obj(None, {0: 'fail'}), {},
+                         msg='default to dict if pruned')
+        self.assertEqual(traverse_obj(None, {0: 'fail'}, default=...), {},
+                         msg='default to dict if pruned and default is given')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: {0: 'fail'}}, default=...), {0: {0: ...}},
+                         msg='use nested `default` when nested dict key fails and `default`')
+        self.assertEqual(traverse_obj(_TEST_DATA, {0: ('dict', ...)}), {},
+                         msg='remove key if branch in dict key not successful')
 
         # Testing default parameter behavior
         _DEFAULT_DATA = {'None': None, 'int': 0, 'list': []}
@@ -2183,14 +2191,17 @@ def test_traverse_obj(self):
                                       traverse_string=True), '.',
                          msg='traverse into converted data if `traverse_string`')
         self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', ...),
-                                      traverse_string=True), list('str'),
-                         msg='`...` branching into string should result in list')
+                                      traverse_string=True), 'str',
+                         msg='`...` should result in string (same value) if `traverse_string`')
+        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', slice(0, None, 2)),
+                                      traverse_string=True), 'sr',
+                         msg='`slice` should result in string if `traverse_string`')
+        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', lambda i, v: i or v == "s"),
+                                      traverse_string=True), 'str',
+                         msg='function should result in string if `traverse_string`')
         self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', (0, 2)),
                                       traverse_string=True), ['s', 'r'],
-                         msg='branching into string should result in list')
-        self.assertEqual(traverse_obj(_TRAVERSE_STRING_DATA, ('str', lambda _, x: x),
-                                      traverse_string=True), list('str'),
-                         msg='function branching into string should result in list')
+                         msg='branching should result in list if `traverse_string`')
 
         # Test is_user_input behavior
         _IS_USER_INPUT_DATA = {'range8': list(range(8))}