Repository: easybuilders/easybuild-easyblocks

Issue:
OpenSSL wrapper does not include corresponding pkg-config files
It appears that installing OpenSSL with OpenSSL-1.1.eb does not include the corresponding pkg-config files:
```
$ ml
Currently Loaded Modules:
  1) GCCcore/10.3.0   2) OpenSSL/1.1   3) pkg-config/0.29.2-GCCcore-10.3.0

$ pkg-config --libs --cflags openssl
Package openssl was not found in the pkg-config search path.
Perhaps you should add the directory containing `openssl.pc'
to the PKG_CONFIG_PATH environment variable
No package 'openssl' found

$ echo $PKG_CONFIG_PATH
(empty return)
```
This is not the expected behaviour and crops up when I tried to install `Rust-1.52.1-GCCcore-10.3.0.eb` 




Discussion:
Note: We should generate new ones, not symlink, as we are doing a bit of a rename trick with the symlinks (for openssl11)

> > (maybe we should symlink in the pkgconfig files as well?)
> 
> Those files cannot be symlinked, we would have to re-create them with the appropriate contents.
> _Originally posted by @lexming in https://github.com/easybuilders/easybuild-easyblocks/issues/2429#issuecomment-842340663_
Required files:

- libssl.pc
- openssl.pc
- libcrypto.pc

@lexming In what sense? Probably replacing the prefix might be enough already?
For e.g. Centos7 using `openssl11-devel` we'd need to change the Libs and Cflags parts as well; `Libs: -L${libdir} -L${libdir}/openssl11 -lcrypto` and `Cflags: -I${includedir} -I${includedir}/openssl11` won't be correct. Would be cleaner just to generate a file that simply fits the paths we use in the wrapper rather than modifying an existing PC file.
It looks like that Debian/Ubuntu have the files here:
* /usr/lib/x86_64-linux-gnu/pkgconfig/openssl.pc
* /usr/lib/x86_64-linux-gnu/pkgconfig/libssl.pc
* /usr/lib/x86_64-linux-gnu/pkgconfig/libcrypto.pc
Just so you know. 
@Micket Yeah, looks like it. For reference:

```
$ cat /usr/lib64/pkgconfig/openssl.pc
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib64
includedir=${prefix}/include

Name: OpenSSL
Description: Secure Sockets Layer and cryptography libraries and tools
Version: 1.0.2k
Requires: libssl libcrypto

$ cat /usr/lib64/pkgconfig/libssl.pc 
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib64
includedir=${prefix}/include

Name: OpenSSL-libssl
Description: Secure Sockets Layer and cryptography libraries
Version: 1.0.2k
Requires: libcrypto
Libs: -L${libdir} -lssl
Libs.private: -ldl -lz -lgssapi_krb5 -lkrb5 -lcom_err -lk5crypto
Cflags: -I${includedir} -I/usr/include

$ cat /usr/lib64/pkgconfig/libcrypto.pc 
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib64
includedir=${prefix}/include
enginesdir=/usr/lib64/openssl/engines

Name: OpenSSL-libcrypto
Description: OpenSSL cryptography library
Version: 1.0.2k
Requires: 
Libs: -L${libdir} -lcrypto
Libs.private: -ldl -lz
Cflags: -I${includedir}
```

Resolving patch:
diff --git a/easybuild/easyblocks/o/openssl_wrapper.py b/easybuild/easyblocks/o/openssl_wrapper.py
--- a/easybuild/easyblocks/o/openssl_wrapper.py
+++ b/easybuild/easyblocks/o/openssl_wrapper.py
@@ -35,7 +35,7 @@
 from easybuild.easyblocks.generic.bundle import Bundle
 from easybuild.framework.easyconfig import CUSTOM
 from easybuild.tools.build_log import EasyBuildError, print_warning
-from easybuild.tools.filetools import change_dir, expand_glob_paths, mkdir, read_file, symlink, which
+from easybuild.tools.filetools import change_dir, expand_glob_paths, mkdir, read_file, symlink, which, write_file
 from easybuild.tools.py2vs3 import string_type
 from easybuild.tools.run import run_cmd
 from easybuild.tools.systemtools import DARWIN, LINUX, get_os_type, get_shared_lib_ext, find_library_path
@@ -95,7 +95,8 @@ def __init__(self, *args, **kwargs):
             raise EasyBuildError(err_msg, min_openssl_version, self.version)
 
         # Regex pattern to find version strings in OpenSSL libraries and headers
-        openssl_version_regex = re.compile(r'OpenSSL\s+([0-9]+\.[0-9]+(\.[0-9]+.)*)', re.M)
+        full_version_regex = re.compile(r'[0-9]+\.[0-9]+\.[0-9]+[a-z]')
+        openssl_version_regex = re.compile(r'OpenSSL\s+([0-9]+\.[0-9]+(\.[0-9]+[a-z])*)', re.M)
 
         # Libraries packaged in OpenSSL
         openssl_libs = ['libssl', 'libcrypto']
@@ -142,7 +143,7 @@ def __init__(self, *args, **kwargs):
             'bin': None,
             'engines': None,
             'include': None,
-            'lib': None,
+            'libs': [],
         }
 
         # early return when we're not wrapping the system OpenSSL installation
@@ -156,41 +157,46 @@ def __init__(self, *args, **kwargs):
             for solib in solibs:
                 system_solib = find_library_path(solib)
                 if system_solib:
-                    # check version string of system library
-                    solib_strings = read_file(system_solib, mode="rb").decode('utf-8', 'replace')
+                    openssl_version = 0
+                    # get version of system library filename
                     try:
-                        openssl_version = openssl_version_regex.search(solib_strings).group(1)
+                        openssl_version = full_version_regex.search(os.path.realpath(system_solib)).group(0)
                     except AttributeError:
-                        # some distributions ship libraries without version strings, ignore such libraries
-                        dbg_msg = "System OpenSSL library '%s' does not contain any recognizable version string"
-                        self.log.debug(dbg_msg, system_solib)
+                        # filename lacks the full version, fallback to version strings within the library
+                        solib_strings = read_file(system_solib, mode="rb").decode('utf-8', 'replace')
+                        try:
+                            openssl_version = openssl_version_regex.search(solib_strings).group(1)
+                        except AttributeError:
+                            dbg_msg = "Could not detect the full version of system OpenSSL library: %s"
+                            self.log.debug(dbg_msg, system_solib)
+                    # check that system version fulfills requirements
+                    if LooseVersion(openssl_version) >= LooseVersion(min_openssl_version):
+                        dbg_msg = "System OpenSSL library '%s' with version %s fulfills requested version %s"
+                        self.log.debug(dbg_msg, system_solib, openssl_version, min_openssl_version)
+                        self.system_ssl['libs'].append(system_solib)
                     else:
-                        if LooseVersion(openssl_version) >= LooseVersion(min_openssl_version):
-                            dbg_msg = "System OpenSSL library '%s' version %s fulfills requested version %s"
-                            self.log.debug(dbg_msg, system_solib, openssl_version, min_openssl_version)
-                            self.system_ssl['lib'] = system_solib
-                            break
-                        else:
-                            dbg_msg = "System OpenSSL library '%s' version %s is older than requested version %s"
-                            self.log.debug(dbg_msg, system_solib, openssl_version, min_openssl_version)
+                        dbg_msg = "System OpenSSL library '%s' with version %s is older than requested version %s"
+                        self.log.debug(dbg_msg, system_solib, openssl_version, min_openssl_version)
                 else:
                     # one of the OpenSSL libraries is missing, switch to next group of versioned libs
+                    self.system_ssl['libs'] = []
                     break
 
-            if self.system_ssl['lib']:
-                # keep the libraries found as possible targets for this installation
+            if len(self.system_ssl['libs']) == len(openssl_libs):
+                # keep these libraries as possible targets for this installation
                 target_system_ssl_libs = system_versioned_libs[idx]
                 break
 
-        if self.system_ssl['lib']:
+        if len(self.system_ssl['libs']) == len(openssl_libs):
+            self.system_ssl['version'] = openssl_version
             info_msg = "Found OpenSSL library version %s in host system: %s"
-            self.log.info(info_msg, openssl_version, os.path.dirname(self.system_ssl['lib']))
+            self.log.info(info_msg, self.system_ssl['version'], os.path.dirname(self.system_ssl['libs'][0]))
         else:
             self.log.info("OpenSSL library not found in host system, falling back to OpenSSL in EasyBuild")
             return
 
         # Directory with engine libraries
-        lib_dir = os.path.dirname(self.system_ssl['lib'])
+        lib_dir = os.path.dirname(self.system_ssl['libs'][0])
         lib_engines_dir = [
             os.path.join(lib_dir, 'openssl', self.target_ssl_engine),
             os.path.join(lib_dir, self.target_ssl_engine),
@@ -203,7 +209,6 @@ def __init__(self, *args, **kwargs):
                 break
 
         if not self.system_ssl['engines']:
-            self.system_ssl['lib'] = None
             self.log.info("OpenSSL engines not found in host system, falling back to OpenSSL in EasyBuild")
             return
 
@@ -239,14 +244,14 @@ def __init__(self, *args, **kwargs):
                     err_msg = "System OpenSSL header '%s' does not contain any recognizable version string"
                     raise EasyBuildError(err_msg, opensslv_path)
 
-                if header_version == openssl_version:
+                if header_version == self.system_ssl['version']:
                     self.system_ssl['include'] = include_dir
                     info_msg = "Found OpenSSL headers v%s in host system: %s"
                     self.log.info(info_msg, header_version, self.system_ssl['include'])
                     break
                 else:
                     dbg_msg = "System OpenSSL header version '%s' doesn not match library version '%s'"
-                    self.log.debug(dbg_msg, header_version, openssl_version)
+                    self.log.debug(dbg_msg, header_version, self.system_ssl['version'])
             else:
                 self.log.info("System OpenSSL header file %s not found", opensslv_path)
 
@@ -274,18 +279,17 @@ def __init__(self, *args, **kwargs):
 
     def fetch_step(self, *args, **kwargs):
         """Fetch sources if OpenSSL component is needed"""
-        if not all([self.system_ssl['lib'], self.system_ssl['include']]):
+        if not all(self.system_ssl[key] for key in ('bin', 'engines', 'include', 'libs')):
             super(EB_OpenSSL_wrapper, self).fetch_step(*args, **kwargs)
 
     def extract_step(self):
         """Extract sources if OpenSSL component is needed"""
-        if not all([self.system_ssl['lib'], self.system_ssl['include']]):
+        if not all(self.system_ssl[key] for key in ('bin', 'engines', 'include', 'libs')):
             super(EB_OpenSSL_wrapper, self).extract_step()
 
     def install_step(self):
         """Symlink target OpenSSL installation"""
-
-        if all(self.system_ssl[key] for key in ('bin', 'engines', 'include', 'lib')):
+        if all(self.system_ssl[key] for key in ('bin', 'engines', 'include', 'libs')):
             # note: symlink to individual files, not directories,
             # since directory symlinks get resolved easily...
 
@@ -295,9 +299,7 @@ def install_step(self):
             mkdir(lib64_engines_dir, parents=True)
 
             # link existing known libraries
-            ssl_syslibdir = os.path.dirname(self.system_ssl['lib'])
-            lib_files = [os.path.join(ssl_syslibdir, x) for x in self.target_ssl_libs]
-            for libso in lib_files:
+            for libso in self.system_ssl['libs']:
                 symlink(libso, os.path.join(lib64_dir, os.path.basename(libso)))
 
             # link engines library files
@@ -307,7 +309,8 @@ def install_step(self):
 
             # relative symlink for unversioned libraries
             cwd = change_dir(lib64_dir)
-            for libso in self.target_ssl_libs:
+            for libso in self.system_ssl['libs']:
+                libso = os.path.basename(libso)
                 unversioned_lib = '%s.%s' % (libso.split('.')[0], get_shared_lib_ext())
                 symlink(libso, unversioned_lib, use_abspath_source=False)
             change_dir(cwd)
@@ -324,6 +327,9 @@ def install_step(self):
             mkdir(bin_dir)
             symlink(self.system_ssl['bin'], os.path.join(bin_dir, self.name.lower()))
 
+            # install pkg-config files
+            self.install_pc_files()
+
         elif self.cfg.get('wrap_system_openssl'):
             # install OpenSSL component due to lack of OpenSSL in host system
             print_warning("Not all OpenSSL components found in host system, falling back to OpenSSL in EasyBuild!")
@@ -347,6 +353,7 @@ def sanity_check_step(self):
         ssl_dirs = [
             os.path.join('include', self.name.lower()),
             os.path.join('lib', self.target_ssl_engine),
+            os.path.join('lib', 'pkgconfig'),
         ]
 
         custom_paths = {
@@ -358,3 +365,95 @@ def sanity_check_step(self):
         custom_commands = ["ssl_ver=$(openssl version); [ ${ssl_ver:8:3} == '%s' ]" % self.majmin_version]
 
         super(Bundle, self).sanity_check_step(custom_paths=custom_paths, custom_commands=custom_commands)
+
+    def install_pc_files(self):
+        """Install pkg-config files for the wrapper"""
+
+        PC_TEMPLATE = """prefix=%(root)s
+exec_prefix=${prefix}
+libdir=${exec_prefix}/lib64
+includedir=${prefix}/include
+%(enginesdir)s
+
+Name: %(name)s
+Description: %(description)s
+Version: %(version)s
+%(requires)s
+%(libs)s
+%(cflags)s
+"""
+        openssl_components = {
+            'libcrypto': {
+                'name': 'OpenSSL-libcrypto',
+                'description': 'OpenSSL cryptography library',
+                'enginesdir': self.target_ssl_engine,
+            },
+            'libssl': {
+                'name': 'OpenSSL-libssl',
+                'description': 'Secure Sockets Layer and cryptography libraries',
+                'enginesdir': None,
+            },
+            'openssl': {
+                'name': 'OpenSSL',
+                'description': 'Secure Sockets Layer and cryptography libraries and tools',
+                'enginesdir': None,
+            },
+        }
+
+        pc_install_dir = os.path.join(self.installdir, 'lib64', 'pkgconfig')
+        mkdir(pc_install_dir)
+
+        for pc_comp in openssl_components:
+            pc_file = openssl_components[pc_comp]
+
+            pc_file['root'] = self.installdir
+            pc_file['version'] = self.system_ssl['version']
+
+            # component name in system pkg-config
+            pc_name = pc_comp
+            if self.majmin_version == '1.1':
+                # check suffixed names with v1.1
+                pc_name_suffix = pc_name + '11'
+                pc_exists_cmd = "pkg-config --exists %s" % pc_name_suffix
+                if run_cmd(pc_exists_cmd, simple=True, log_ok=False, log_all=False):
+                    pc_name = pc_name_suffix
+
+            # get requires from pkg-config
+            pc_file['requires'] = []
+            for require_type in ['Requires', 'Requires.private']:
+                require_print = require_type.lower().replace('.', '-')
+                requires, _ = run_cmd("pkg-config --print-%s %s" % (require_print, pc_name), simple=False)
+
+                if requires:
+                    # use unsuffixed names for components provided by this wrapper
+                    for wrap_comp in openssl_components:
+                        requires = re.sub(r'^%s[0-9]+$' % wrap_comp, wrap_comp, requires, flags=re.M)
+                    # format requires
+                    requires = requires.rstrip().splitlines()
+                    pc_file['requires'].append("%s: %s" % (require_type, ' '.join(requires)))
+
+            pc_file['requires'] = '\n'.join(pc_file['requires'])
+
+            if pc_comp.startswith('lib'):
+                # add libs and cflags for library components
+                c_lib_name = pc_comp[3:]
+                pc_file['libs'] = "Libs: -L${libdir} -l%s" % c_lib_name
+                pc_file['cflags'] = "Cflags: -I${includedir}"
+                # infer private libs through pkg-config
+                linker_libs, _ = run_cmd("pkg-config --libs %s" % pc_name, simple=False)
+                all_libs, _ = run_cmd("pkg-config --libs --static %s" % pc_name, simple=False)
+                libs_priv = "%s " % all_libs.rstrip()
+                for flag in linker_libs.rstrip().split(' '):
+                    libs_priv = libs_priv.replace("%s " % flag, '')
+                pc_file['libs'] += "\nLibs.private: %s" % libs_priv
+            else:
+                pc_file['libs'], pc_file['cflags'] = '', ''
+
+            # format enginesdir
+            if pc_file['enginesdir']:
+                pc_file['enginesdir'] = 'enginesdir=${libdir}/%s' % pc_file['enginesdir']
+            else:
+                pc_file['enginesdir'] = ''
+
+            pc_path = os.path.join(pc_install_dir, '%s.pc' % pc_comp)
+            write_file(pc_path, PC_TEMPLATE % pc_file)


Test patch: