Repository: pyinstaller/pyinstaller-hooks-contrib (language: python)

Issue:
cryptography with OpenSSL 3.x
I'm using the `cryptography` module and trying to use the latest OpenSSL 3.3.0, then I encountered this error when trying to use cryptography:
```
RuntimeError: OpenSSL 3.0's legacy provider failed to load. This is a fatal error by default, but cryptography supports running without legacy algorithms by setting the environment variable CRYPTOGRAPHY_OPENSSL_NO_LEGACY. If you did not expect this error, you have likely made a mistake with your OpenSSL configuration. 
```

After some debugging, I found out that the legacy modules are built in a separate shared object called `legacy.so`, in my case it is located in `/usr/lib/ossl-modules/legacy.so`, and PyInstaller is not collecting it.

Can someone point me out what will be the best way to implement it? should I change the `hook-cryptography.py` or create a new one? 



Discussion / hints:
Do you actually need those legacy algorithms? If you don't, it might be better to just set that `CRYPTOGRAPHY_OPENSSL_NO_LEGACY` environment variable before importing `cryptography`...

Otherwise, `cryptography` hook would need to explicitly look for these additional OpenSSL shared libraries (and account for differences between distributions).
@rokm Unfortunately, I still need them 😥 
Do you have any suggestions on how to locate them without hardcoding the full path? 

I tried using `collect_dynamic_libs` but I can't figure it out as it is not able to locate it
`collect_dynamic_libs` is for shared libs that are bundled within python packages, so it won't work with this.

Either you will need to look into bunch of standard hard-coded locations, or try to determine the location of `libssl.so.3` (either using `ctypes` helpers, or using `PyInstaller.depend.bindepend.resolve_library_path`), and then look for `ossl-modules` directory next to it.

On my Fedora system, this would be `/lib64/libssl.so.3`  and `/lib64/ossl-modules/legacy.so`; on debian-based systems, it seems to be `/usr/lib/x86_64-linux-gnu/libssl.so.3` and `/usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so`, etc. 

I need to check how this is handled on Windows and macOS, though. And actually, how does OpenSSL determine the search path (is it also using relative search path, or does it have hardcoded absolute search location, that might be overridden via environment variable)...

The fastest way for your application build right now is to hard-code the path for your build system (and you can do that in the .spec file instead of the hook), though, until more generic solution is added to the hook.
Thanks, I think that this is what I will do for now, would appreciate any help getting it into the official hooks 😃 
@rokm I added the file in my spec file and I do see it collected inside the application directory but this still not working, looking at `strace` output I see that it tries to load it from the system path and not from the application directory
```
openat(AT_FDCWD, "/usr/lib64/ossl-modules/legacy.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
```
Is there something I need to do to fix the path? 
I think the search location can be manipulated via `OPENSSL_MODULES` environment variable; can you first try to set it manually outside of the executable? And if that works, then dynamically/programmatically at the very top of your program?
Thanks, that seems to do the trick, I will go with adding a "runtime hook" that will set this environment variable for now until we will be able to work around this issue

Fix patch:
diff --git a/news/724.update.rst b/news/724.update.rst
new file mode 100644
index 000000000..f81423a4a
--- /dev/null
+++ b/news/724.update.rst
@@ -0,0 +1,8 @@
+Extend ``cryptography`` hook to collect OpenSSL modules (the
+``ossl-modules`` directory) when available. Add a run-time hook that
+overrides OpenSSL module search path by setting the ``OPENSSL_MODULES``
+environment variable to the bundled ``ossl-modules`` directory. This
+fixes ``RuntimeError: OpenSSL 3.0's legacy provider failed to load.``
+error when using ``cryptography`` with OpenSSL >= 3.0 builds that have
+modules enabled (e.g., most Linux distributions, msys/MinGW on Windows,
+and Homebrew on macOS).
diff --git a/src/_pyinstaller_hooks_contrib/hooks/rthooks.dat b/src/_pyinstaller_hooks_contrib/hooks/rthooks.dat
index 25fd8edc6..538e94909 100644
--- a/src/_pyinstaller_hooks_contrib/hooks/rthooks.dat
+++ b/src/_pyinstaller_hooks_contrib/hooks/rthooks.dat
@@ -1,4 +1,5 @@
 {
+    'cryptography': ['pyi_rth_cryptography_openssl.py'],
     'enchant': ['pyi_rth_enchant.py'],
     'ffpyplayer': ['pyi_rth_ffpyplayer.py'],
     'osgeo': ['pyi_rth_osgeo.py'],
diff --git a/src/_pyinstaller_hooks_contrib/hooks/rthooks/pyi_rth_cryptography_openssl.py b/src/_pyinstaller_hooks_contrib/hooks/rthooks/pyi_rth_cryptography_openssl.py
new file mode 100644
index 000000000..24b81590e
--- /dev/null
+++ b/src/_pyinstaller_hooks_contrib/hooks/rthooks/pyi_rth_cryptography_openssl.py
@@ -0,0 +1,20 @@
+#-----------------------------------------------------------------------------
+# Copyright (c) 2024, PyInstaller Development Team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+#
+# The full license is in the file COPYING.txt, distributed with this software.
+#
+# SPDX-License-Identifier: Apache-2.0
+#-----------------------------------------------------------------------------
+
+import os
+import sys
+
+# If we collected OpenSSL modules into `ossl-modules` directory, override the OpenSSL search path by setting the
+# `OPENSSL_MODULES` environment variable.
+_ossl_modules_dir = os.path.join(sys._MEIPASS, 'ossl-modules')
+if os.path.isdir(_ossl_modules_dir):
+    os.environ['OPENSSL_MODULES'] = _ossl_modules_dir
+del _ossl_modules_dir
diff --git a/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-cryptography.py b/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-cryptography.py
index 7edc4ed5b..50f16593f 100644
--- a/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-cryptography.py
+++ b/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-cryptography.py
@@ -13,12 +13,19 @@
 Hook for cryptography module from the Python Cryptography Authority.
 """
 
-import os.path
+import os
 import glob
+import pathlib
 
-from PyInstaller.compat import EXTENSION_SUFFIXES
-from PyInstaller.utils.hooks import collect_submodules, get_module_file_attribute
-from PyInstaller.utils.hooks import copy_metadata
+from PyInstaller import compat
+from PyInstaller import isolated
+from PyInstaller.utils.hooks import (
+    collect_submodules,
+    copy_metadata,
+    get_module_file_attribute,
+    is_module_satisfies,
+    logger,
+)
 
 # get the package data so we can load the backends
 datas = copy_metadata('cryptography')
@@ -34,9 +41,74 @@
 # The cffi verifier expects to find them inside the package directory for
 # the main module. We cannot use hiddenimports because that would add the modules
 # outside the package.
+# NOTE: this is not true anymore with PyInstaller >= 6.0, but we keep it like this for compatibility with 5.x series.
 binaries = []
 cryptography_dir = os.path.dirname(get_module_file_attribute('cryptography'))
-for ext in EXTENSION_SUFFIXES:
+for ext in compat.EXTENSION_SUFFIXES:
     ffimods = glob.glob(os.path.join(cryptography_dir, '*_cffi_*%s*' % ext))
     for f in ffimods:
         binaries.append((f, 'cryptography'))
+
+
+# Check if cryptography was built with support for OpenSSL >= 3.0.0. In that case, we might need to collect external
+# OpenSSL modules, if OpenSSL was built with modules support. It seems the best indication of this is the presence
+# of ossl-modules directory next to the OpenSSL shared library.
+try:
+    @isolated.decorate
+    def _check_cryptography_openssl3():
+        from cryptography.hazmat.backends.openssl.backend import backend
+        return bool(backend._lib.CRYPTOGRAPHY_OPENSSL_300_OR_GREATER)
+
+    uses_openssl3 = _check_cryptography_openssl3()
+except Exception:
+    logger.warning(
+        "hook-cryptography: failed to determine whether cryptography is using OpenSSL >= 3.0.0", exc_info=True
+    )
+    uses_openssl3 = False
+
+if uses_openssl3:
+    # Determine location of OpenSSL shared library.
+    # This requires the new PyInstaller.bindepend API from PyInstaller >= 6.0.
+    openssl_lib = None
+    if is_module_satisfies("PyInstaller >= 6.0"):
+        from PyInstaller.depend import bindepend
+
+        if compat.is_win:
+            # Resolve the given library name; first, search the python library directory for python-provided OpenSSL.
+            lib_name = 'libssl-3-x64.dll' if compat.is_64bits else 'libssl-3.dll'
+            openssl_lib = bindepend.resolve_library_path(lib_name, search_paths=[compat.base_prefix])
+        elif compat.is_darwin:
+            # First, attempt to resolve using only {sys.base_prefix}/lib - `bindepend.resolve_library_path` uses
+            # standard dyld search semantics and uses the given search paths as fallback (and would therefore
+            # favor Homebrew-provided version of the library).
+            lib_name = 'libssl.3.dylib'
+            base_prefix_lib_dir = os.path.join(compat.base_prefix, 'lib')
+            openssl_lib = bindepend._resolve_library_path_in_search_paths(lib_name, search_paths=[base_prefix_lib_dir])
+            if openssl_lib is None:
+                openssl_lib = bindepend.resolve_library_path(lib_name, search_paths=[base_prefix_lib_dir])
+        else:
+            # Linux and other POSIX systems
+            lib_name = 'libssl.so.3'
+            openssl_lib = bindepend.resolve_library_path(lib_name)
+    else:
+        logger.warning(
+            "hook-cryptography: full support for cryptography + OpenSSL >= 3.0.0 requires PyInstaller >= 6.0"
+        )
+
+    # Check for presence of ossl-modules directory next to the OpenSSL shared library.
+    if openssl_lib:
+        logger.debug("hook-cryptography: OpenSSL shared library location: %r", openssl_lib)
+
+        openssl_lib_dir = pathlib.Path(openssl_lib).parent
+
+        # Collect whole ossl-modules directory, if it exists.
+        ossl_modules_dir = openssl_lib_dir / 'ossl-modules'
+
+        # Msys2/MinGW installations on Windows put the shared library into `bin` directory, but the modules are
+        # located in `lib` directory. Account for that possibility.
+        if not ossl_modules_dir.is_dir() and openssl_lib_dir.name == 'bin':
+            ossl_modules_dir = openssl_lib_dir.parent / 'lib' / 'ossl-modules'
+
+        if ossl_modules_dir.is_dir():
+            logger.debug("hook-cryptography: collecting OpenSSL modules directory: %r", str(ossl_modules_dir))
+            binaries.append((str(ossl_modules_dir), 'ossl-modules'))


Test patch:
diff --git a/requirements-test-libraries.txt b/requirements-test-libraries.txt
index 11b30f86d..29b0a4df3 100644
--- a/requirements-test-libraries.txt
+++ b/requirements-test-libraries.txt
@@ -22,6 +22,7 @@ cloudpickle==3.0.0
 cloudscraper==1.2.71
 # compliance-checker requires cf-units, so same constraints apply.
 compliance-checker==5.1.0; sys_platform != 'win32'
+cryptography==42.0.5
 dash==2.16.1
 dash-bootstrap-components==1.6.0
 dash-uploader==0.6.0
diff --git a/src/_pyinstaller_hooks_contrib/tests/test_libraries.py b/src/_pyinstaller_hooks_contrib/tests/test_libraries.py
index b0b12015a..0f1dc988a 100644
--- a/src/_pyinstaller_hooks_contrib/tests/test_libraries.py
+++ b/src/_pyinstaller_hooks_contrib/tests/test_libraries.py
@@ -1979,3 +1979,19 @@ def test_opentelemetry(pyi_builder):
         with tracer.start_as_current_span("foo"):
             print("Hello world!")
     """)
+
+
+# Basic test for cryptography package
+@importorskip('cryptography')
+def test_cryptography(pyi_builder):
+    pyi_builder.test_source("""
+        from cryptography.fernet import Fernet
+
+        key = Fernet.generate_key()
+        f = Fernet(key)
+
+        token = f.encrypt(b"This is test.")
+        print(f"Encrypted message: {token}")
+
+        print(f"Decrypted message: {f.decrypt(token)}")
+    """)