Repository: cocotb/cocotb

Issue:
Modelsim 32-bit invalid ELF header
Just updated my local cocotb installation from git and ran into an issue:

System:
- Centos 7 (64-bit)
- Intel Modelsim 2020.3 (32-bit)
- Installed anaconda 32-bit system as per instructions:
https://github.com/cocotb/cocotb/wiki/Tier-2-Setup-Instructions 
- Installed cocotb from git: ```pip install -v -e ./cocotb```
- Cocotb version: 3a47227170d872e8bac1de9af3b5547f298d2bdb 


Adder example ouput:
```$ make SIM=modelsim
make results.xml
make[1]: Entering directory `/home/cocotb/examples/adder/tests'
set -o pipefail; MODULE=test_adder TESTCASE= TOPLEVEL="work.adder" \
GPI_EXTRA= TOPLEVEL_LANG=verilog \
/data/tools/intelFPGA_pro/20.3/modelsim_ase/linux/vsim -c  -do sim_build/runsim.do  2>&1 | tee sim_build/sim.log
Reading pref.tcl

# 2020.3

# do sim_build/runsim.do
# Model Technology ModelSim - Intel FPGA Edition vmap 2020.3 Lib Mapping Utility 2020.07 Jul 22 2020
# vmap -c 
# ** Warning: vmap will not overwrite local modelsim.ini.
# Model Technology ModelSim - Intel FPGA Edition vmap 2020.3 Lib Mapping Utility 2020.07 Jul 22 2020
# vmap work sim_build/work 
# Modifying modelsim.ini
# Model Technology ModelSim - Intel FPGA Edition vlog 2020.3 Compiler 2020.07 Jul 22 2020
# Start time: 16:34:55 on Feb 05,2021
# vlog -work work "+define+COCOTB_SIM" -sv -timescale 1ns/1ps -mfcu "+acc" /home/cocotb/examples/adder/tests/../hdl/adder.sv 
# -- Compiling module adder
# 
# Top level modules:
#   adder
# End time: 16:34:55 on Feb 05,2021, Elapsed time: 0:00:00
# Errors: 0, Warnings: 0
# vsim -onfinish exit -pli "/home/cocotb/cocotb/libs/libcocotbvpi_modelsim.so" sim_build/work.adder 
# Start time: 16:34:55 on Feb 05,2021
# Loading sv_std.std
# Loading sim_build/work.adder
# Loading /home/cocotb/cocotb/libs/libcocotbvpi_modelsim.so
#      -.--ns ERROR    cocotb.gpi                         ..s/cocotb_utils.cpp:69   in utils_dyn_open                  Unable to open lib /home/.conda/envs/py3_32/lib/libpython3.7m.a: /home/.conda/envs/py3_32/lib/libpython3.7m.a: invalid ELF header
#      -.--ns INFO     cocotb.gpi                         ../gpi/GpiCommon.cpp:105  in gpi_print_registered_impl       VPI registered
# End time: 16:34:55 on Feb 05,2021, Elapsed time: 0:00:00
# Errors: 0, Warnings: 0
ERROR: results.xml was not written by the simulation!
make[1]: *** [results.xml] Error 1
make[1]: Leaving directory `/home/cocotb/examples/adder/tests'
make: *** [sim] Error 2
```

Also tried some older cocotb version (which gives the same error as in the log above): 
331b879e39c587e94837f73eb05f73c04ec20c6e


This is the most recent cocotb version that runs adder example on modelsim without issues:
50418e54f000316257404134a57557ffb877e2c8


Any suggestions?
Thanks
 


Discussion:
It looks like `cocotb-config --libpython` returns `libpython3.7m.a` and it should `libpython3.7m.so`.
So potentially something here:  https://github.com/cocotb/cocotb/blob/master/cocotb/_vendor/find_libpython/__init__.py

@ktbarrett 
This is a known issue in `find_libpython` (https://github.com/ktbarrett/find_libpython/issues/7). I haven't gotten around to testing RHEL systems yet (https://github.com/ktbarrett/find_libpython/issues/4). It turns out that code wasn't as bulletproof as we would like. I am thinking it probably shouldn't be vendored in so that it can always be as up to date as possible.
@ktbarrett Why not change ".a" to ".so" in those cases ?
'.so's might not exist at the location. Its easiest to just filter those names out entirely. I dont think its a difficult problem, I just haven't gotten around to fixing and testing it yet.
I replicated the issue, the "LDLIBRARY" config var is returning "libpython3.7m.a" and not the ".so" in the conda env =/ That's a pretty blatant misconfiguration, but I can add some safety checks to prevent that, just like is done for the "LIBRARY" config var.
@elgorwi Could you try out this change in `cocotb/_vendor/find_libpython/__init__.py`? https://github.com/ktbarrett/find_libpython/pull/13/commits/95c37b3e2d1d838e5125e69a0df57ff8116ee235

I works on my Debian machine and a CentOS 7 container.
I'll make it an action to vendor in the fix after it makes it into upstream.
>Could you try out this change in cocotb/_vendor/find_libpython/__init__.py? ktbarrett/find_libpython@95c37b3

For me, it seems to work with this change.
Thanks for the patch @ktbarrett. 
Just tried it and everything seems to be working on my system. 


Resolving patch:
diff --git a/cocotb/_vendor/find_libpython/__init__.py b/cocotb/_vendor/find_libpython/__init__.py
--- a/cocotb/_vendor/find_libpython/__init__.py
+++ b/cocotb/_vendor/find_libpython/__init__.py
@@ -32,10 +32,14 @@
 import sys
 import sysconfig
 
+from ._version import version as __version__  # noqa: F401
+
 logger = getLogger("find_libpython")
 
 is_windows = os.name == "nt"
 is_apple = sys.platform == "darwin"
+is_msys = sysconfig.get_platform().startswith("msys")
+is_mingw = sysconfig.get_platform().startswith("mingw")
 
 SHLIB_SUFFIX = sysconfig.get_config_var("SHLIB_SUFFIX")
 if SHLIB_SUFFIX is None:
@@ -49,49 +53,6 @@
     SHLIB_SUFFIX = ".dylib"
 
 
-def linked_libpython():
-    """
-    Find the linked libpython using dladdr (in *nix).
-
-    Calling this in Windows always return `None` at the moment.
-
-    Returns
-    -------
-    path : str or None
-        A path to linked libpython.  Return `None` if statically linked.
-    """
-    if is_windows:
-        return None
-    return _linked_libpython_unix()
-
-
-class Dl_info(ctypes.Structure):
-    _fields_ = [
-        ("dli_fname", ctypes.c_char_p),
-        ("dli_fbase", ctypes.c_void_p),
-        ("dli_sname", ctypes.c_char_p),
-        ("dli_saddr", ctypes.c_void_p),
-    ]
-
-
-def _linked_libpython_unix():
-    libdl = ctypes.CDLL(ctypes.util.find_library("dl"))
-    libdl.dladdr.argtypes = [ctypes.c_void_p, ctypes.POINTER(Dl_info)]
-    libdl.dladdr.restype = ctypes.c_int
-
-    dlinfo = Dl_info()
-    retcode = libdl.dladdr(
-        ctypes.cast(ctypes.pythonapi.Py_GetVersion, ctypes.c_void_p),
-        ctypes.pointer(dlinfo),
-    )
-    if retcode == 0:  # means error
-        return None
-    path = os.path.realpath(dlinfo.dli_fname.decode())
-    if path == os.path.realpath(sys.executable):
-        return None
-    return path
-
-
 def library_name(name, suffix=SHLIB_SUFFIX, is_windows=is_windows):
     """
     Convert a file basename `name` to a library name (no "lib" and ".so" etc.)
@@ -152,14 +113,24 @@ def candidate_names(suffix=SHLIB_SUFFIX):
         Candidate name libpython.
     """
     LDLIBRARY = sysconfig.get_config_var("LDLIBRARY")
-    if LDLIBRARY:
+    if LDLIBRARY and os.path.splitext(LDLIBRARY)[1] == suffix:
         yield LDLIBRARY
 
     LIBRARY = sysconfig.get_config_var("LIBRARY")
-    if LIBRARY:
-        yield os.path.splitext(LIBRARY)[0] + suffix
+    if LIBRARY and os.path.splitext(LIBRARY)[1] == suffix:
+        yield LIBRARY
+
+    DLLLIBRARY = sysconfig.get_config_var("DLLLIBRARY")
+    if DLLLIBRARY:
+        yield DLLLIBRARY
+
+    if is_mingw:
+        dlprefix = "lib"
+    elif is_windows or is_msys:
+        dlprefix = ""
+    else:
+        dlprefix = "lib"
 
-    dlprefix = "" if is_windows else "lib"
     sysdata = dict(
         v=sys.version_info,
         # VERSION is X.Y in Linux/macOS and XY in Windows:
@@ -195,8 +166,6 @@ def candidate_paths(suffix=SHLIB_SUFFIX):
         and may not exist.
     """
 
-    yield linked_libpython()
-
     # List candidates for directories in which libpython may exist
     lib_dirs = []
     append_truthy(lib_dirs, sysconfig.get_config_var("LIBPL"))
@@ -209,7 +178,7 @@ def candidate_paths(suffix=SHLIB_SUFFIX):
     #
     # But we try other places just in case.
 
-    if is_windows:
+    if is_windows or is_msys or is_mingw:
         lib_dirs.append(os.path.join(os.path.dirname(sys.executable)))
     else:
         lib_dirs.append(
@@ -297,6 +266,8 @@ def finding_libpython():
     """
     logger.debug("is_windows = %s", is_windows)
     logger.debug("is_apple = %s", is_apple)
+    logger.debug("is_mingw = %s", is_mingw)
+    logger.debug("is_msys = %s", is_msys)
     for path in candidate_paths():
         logger.debug("Candidate: %s", path)
         normalized = normalize_path(path)
@@ -356,6 +327,10 @@ def main(args=None):
         "--verbose", "-v", action="store_true", help="Print debugging information."
     )
 
+    parser.add_argument(
+        "--version", action="version", version="find_libpython {}".format(__version__)
+    )
+
     group = parser.add_mutually_exclusive_group()
     group.add_argument(
         "--list-all",
diff --git a/cocotb/_vendor/find_libpython/__main__.py b/cocotb/_vendor/find_libpython/__main__.py
new file mode 100644
--- /dev/null
+++ b/cocotb/_vendor/find_libpython/__main__.py
@@ -0,0 +1,5 @@
+from . import main
+
+
+if __name__ == "__main__":
+    main()
diff --git a/cocotb/_vendor/find_libpython/_version.py b/cocotb/_vendor/find_libpython/_version.py
new file mode 100644
--- /dev/null
+++ b/cocotb/_vendor/find_libpython/_version.py
@@ -0,0 +1,5 @@
+# coding: utf-8
+# file generated by setuptools_scm
+# don't change, don't track in version control
+version = '0.2.0'
+version_tuple = (0, 2, 0)


Test patch: