Repository: mesonbuild/meson

Issue:
python extension_module should not link to python library on macOS and Linux
Continuing from https://gitlab.gnome.org/GNOME/pygobject/issues/253#note_307567

`python.extension_module()` needs to be passed `python_dep` explicitly, which makes sense because meson can't always know what python the project wants to use, but it should not explicitly link to the Python `so` or `dylib` (but it should link to the `dll`) since it's a `shared_module()`.

Currently projects need to use `python_dep.partial_dependency(compile_args: true)` on non-Windows platforms, but we should handle this automatically.

CCing @MathieuDuponchelle @tschoonj 


Discussion:
Isn't this playing with fire? What if the python 2 installation you linked against was compiled with UCS2 support, and the one you're using at runtime is compiled with UCS4 support? Isn't this mixing slightly dangerous, and not linking to the interpreter even more so?
I've found some upstream notes here: https://www.python.org/dev/peps/pep-0513/#libpythonx-y-so-1 There are some links to mailing list/bug tracker discussions at the end of that section.
https://github.com/python/cpython/blob/e42b705188271da108de42b55d9344642170aa2b/Lib/distutils/command/build_ext.py#L695

Looks like it depends on the platform and whether Python is build in shared mode (the default on Fedora, not on Debian)
So I guess we should duplicate that logic? 
And Debian patches that logic away: https://sources.debian.org/src/python3.6/3.6.6-4/debian/patches/ext-no-libpython-link.diff/ So there is no way for us to know .. :(

edit: I guess this would work:

```python
def links_against_libpython():
    from distutils.core import Distribution, Extension
    cmd = Distribution().get_command_obj("build_ext")
    return bool(cmd.get_libraries(Extension("dummy", [])))

print(links_against_libpython())
```
Hum. I guess the trend is towards not explicitly linking to libpython on Linux and macOS. I think that would be a good start, and we can change that based on feedback from distros?

We could also enable that for MSVC based on what distutils is doing. We already have a test for this, so we should know if it works.
> Hum. I guess the trend is towards not explicitly linking to libpython on Linux and macOS. I think that would be a good start, and we can change that based on feedback from distros?

:+1:

Resolving patch:
diff --git a/mesonbuild/modules/python.py b/mesonbuild/modules/python.py
--- a/mesonbuild/modules/python.py
+++ b/mesonbuild/modules/python.py
@@ -267,6 +267,12 @@ def get_pkgconfig_variable(self, variable_name, kwargs):
 
 install_paths = sysconfig.get_paths(scheme='posix_prefix', vars={'base': '', 'platbase': '', 'installed_base': ''})
 
+def links_against_libpython():
+    from distutils.core import Distribution, Extension
+    cmd = Distribution().get_command_obj('build_ext')
+    cmd.ensure_finalized()
+    return bool(cmd.get_libraries(Extension('dummy', [])))
+
 print (json.dumps ({
   'variables': sysconfig.get_config_vars(),
   'paths': sysconfig.get_paths(),
@@ -274,6 +280,7 @@ def get_pkgconfig_variable(self, variable_name, kwargs):
   'version': sysconfig.get_python_version(),
   'platform': sysconfig.get_platform(),
   'is_pypy': '__pypy__' in sys.builtin_module_names,
+  'link_libpython': links_against_libpython(),
 }))
 '''
 
@@ -291,6 +298,7 @@ def __init__(self, interpreter, python, info):
         self.version = info['version']
         self.platform = info['platform']
         self.is_pypy = info['is_pypy']
+        self.link_libpython = info['link_libpython']
 
     @permittedKwargs(mod_kwargs)
     def extension_module(self, interpreter, state, args, kwargs):
@@ -304,6 +312,18 @@ def extension_module(self, interpreter, state, args, kwargs):
 
             kwargs['install_dir'] = os.path.join(self.platlib_install_path, subdir)
 
+        # On macOS and some Linux distros (Debian) distutils doesn't link
+        # extensions against libpython. We call into distutils and mirror its
+        # behavior. See https://github.com/mesonbuild/meson/issues/4117
+        if not self.link_libpython:
+            new_deps = []
+            for holder in mesonlib.extract_as_list(kwargs, 'dependencies'):
+                dep = holder.held_object
+                if isinstance(dep, PythonDependency):
+                    holder = interpreter.holderify(dep.get_partial_dependency(compile_args=True))
+                new_deps.append(holder)
+            kwargs['dependencies'] = new_deps
+
         suffix = self.variables.get('EXT_SUFFIX') or self.variables.get('SO') or self.variables.get('.so')
 
         # msys2's python3 has "-cpython-36m.dll", we have to be clever


Test patch: