Repository: conan-io/conan
Pull request: Fix: add preprocessor_definitions to Meson + CC/CXX from build requirements
Changelog: Fix: The new `MesonToolchain` now takes the declared environment variables (`CC`, `CXX`...) from build-requires and profiles to set the variables `c`, `cpp`, `c_ld`, `cpp_ld` etc, into the `conan_meson_native.ini`
Changelog: Fix: Added new `preprocessor_definitions` to new Meson build helper. 
Changelog: Fix: The new `MesonToolchain` now allows adjusting any variable before generating the `conan_meson_native.ini` file.
Docs: https://github.com/conan-io/docs/pull/2139

Closes https://github.com/conan-io/conan/issues/8311


this is mostly to align interface with `CMake`, `Make` and `MSBuild` (feature parity)
important side change: `c_args`, `c_link_args`, `cpp_args` and `cpp_link_args` are now arrays as documented at https://mesonbuild.com/Builtin-options.html#compiler-options:
```
c_args |   | free-form comma-separated list | C compile arguments to use
-- | -- | -- | --
c_link_args |   | free-form comma-separated list | C link arguments to use

cpp_args |   | free-form comma-separated list | C++ compile arguments to use
-- | -- | -- | --
cpp_link_args |   | free-form comma-separated list | C++ link arguments to use
```


Problem statement:
[bug] MesonToolchain does not pick up CC/CXX from build requirements (in build profile)
The compiler paths of a build requirement with a cross compiler that sets the CC/CXX environment variables
are not available in `conan_meson_cross.ini`.

I have a build requirement that sets some compiler variables:
```
def package_info(self):
    self.env_info.CC = os.path.join(self.package_folder, "bin", "aarch64-none-elf-gcc")
    self.env_info.CC_LD = os.path.join(self.package_folder, "bin", "aarch64-none-elf-gcc")
    self.env_info.CXX = os.path.join(self.package_folder, "bin", "aarch64-none-elf-g++")
    self.env_info.CXX_LD = os.path.join(self.package_folder, "bin", "aarch64-none-elf-g++")
    self.env_info.AR = os.path.join(self.package_folder, "bin", "aarch64-none-elf-ar")
```

But the `[binaries]` section of the cross file remains empty.

When I run conan with `CC=abc conan create .... -pr:h ... -pr:b ...`, then `c = 'abc' is present in the `binaries` section.

`CFLAGS/CXXFLAGS` from a build requirement are also not propagated.


### Environment Details (include every applicable attribute)
  * Operating System+version: Fedora Linux 30
  * Compiler+version: native: gcc 9.3, cross: gcc 10.2
  * Conan version: 1.32.1
  * Python version: 3.7.7

### Steps to reproduce (Include if Applicable)

1. Create a conan recipe that packages a compiler (sets `C`/`CXX`/...). Let's name it `crosscompiler/0.1`.
2. Create a host profile that adds the package created in step 1 as a build requirement, let's name the profile: `cross`.
3. Create a build profile for your native compiler. Let's name it `native`.
4. Cross build a meson project (using `MesonToolchain` like: `conan create . meson-project/0.1@ -pr:h cross -pr:b native`

The generated `conan_meson_cross.ini` won't contain the `properties.c` values, set by `crosscompiler/0.1`.
If you'd run `CC=abc conan create . meson-project/0.1@ -pr:h cross -pr:b native`, then it would.


CC'ing @SSE4 for being the author of `MesonToolchain`.

Hints from issue discussion:
@jgsogo do you have an idea why could it happen? e.g. meson tests use CC from the conan profile, and it works flawlessly 
Hi!

The environment from the build requires is populated before entering the `build()` method. The `MesonToolchain` takes all these values from the environment, but they are not available in the `generate()` function.

I'm not sure how we want to pass these variables from now on (ping @memsharded ), we can populate the environment during `generate()` the same way we do during `build()` or we can leverage on https://github.com/conan-io/conan/pull/8266

Not a bug because it's all about experimental features, but something to define and implement for sure.

🤔  With the toolchains and generators, if everything is written down during the `generate()` call, then we no longer need the environment in the `build()` call. 
I would say that if we want to be very transparent and have a better developer experience, the way to go might be to generate virtualenv files for everything in the environment. After a ``conan install`` a developer should be able to build and run, with their native tools (cmake, make, meson), without relying on ``conan build``. The only way is that Conan translate whatever is in its environment, its build_requires, etc into a virtualenv file that can be used. I would say that the developer experience should be something similar to:

```bash
$ git clone .... & cd ...
$ conan install ...
$ (if necessary) source conantoolchainenv.sh
$ meson ... (using Conan generated files with MesonToolchain)
# or
$ cmake .... -DCMAKE_TOOLCHAIN_FILE=conantoolchain.cmake
```

And yes, if the environment becomes something explicit to the build, then the Conan ``build()`` method do not need to change the environment before launching, but instead it should be explicit in some way:

```python
def build(self):
     with activate_env("conantoolchainenv.sh"):
           meson = Meson(self)
           meson.build()
```

This is a very preliminary vision, sure we need to work on it, but the main principles would be:

- Developer should be able to have a native build experience without invoking ``conan build``, achieving the same build as in with ``conan create`` without much pain
- Making the process and the environment more explicit and decoupled, so it is kind of easier to debug things, and probably easy to have greater configurability.

Right now `MesonToolchain` writes a file with values taken from the environment. I don't know if that `conan_meson_cross.ini` file can read the environment in runtime (so it can leverage on the user/conan running `source conantoolchainenv.sh` to populate the environment), but it would be a completely different implementation of that toolchain class.

Before thinking about the local workflow, we need `conan create` to work. As it is implemented now, we need to pass configuration to `generate()` function, it is where `MesonToolchain` is used: 

> I'm not sure how we want to pass these variables from now on (ping @memsharded ), we can populate the environment during `generate()` the same way we do during `build()` or we can leverage on #8266

...or use a completely different approach for MesonToolchain.
Ok, now I understand better the issue. It is not about the transfer between the generate() and the build(), but how the generate() populate things. Which would be related to what @madebr was talking about the new ``conf``, if it was a way of propagating information from the build_requires to the consumers of those build_requires.

I would like if we could try to explicit define the interfaces that will be available for Conan 2.0 about the dependencies, so the ``generate()`` method uses them, of course being the environment one of them. But maybe that doesn't mean to automatically activate the environment for the ``generate()`` method, so it can capture those env-vars and write them to files, because that makes also difficult to separate things defined by Conan, potentially making the generated environment files very noisy.
that seems like `conanfile` has an `env` attribute, which is populated from the `build_requires` (method `_propagate_info`).
this is used at least by the `VirtualEnvGenerator` right now.
however, the build method seems to be very special (see `run_build_method` which does env management).
moreover, it seems like by design, `build_requires` are only applied to the `build` method?
at least I see `self._conanfile.env` is empty, as well as `self._conanfile.build_requires` is also empty during the `generate` method.
addition from #8311 - CC from the profile aren't captured by `MesonToolchain`, they are captured by `meson` executable itself.
it turns out it's problematic to implement `preprocessor_defininitions` because of this.

Commit message: - fix env propagation

Signed-off-by: SSE4 <tomskside@gmail.com>

Patch to review:
diff --git a/conan/tools/meson/toolchain.py b/conan/tools/meson/toolchain.py
index 40c7bd8203a..0e0800d6fa6 100644
--- a/conan/tools/meson/toolchain.py
+++ b/conan/tools/meson/toolchain.py
@@ -28,6 +28,8 @@ class MesonToolchain(object):
     {% if pkgconfig %}pkgconfig = {{pkgconfig}}{% endif %}
 
     [built-in options]
+    preprocessor_definitions = [{% for it, value in preprocessor_definitions.items() -%}
+    '-D{{ it }}="{{ value}}"'{%- if not loop.last %}, {% endif %}{% endfor %}]
     {% if buildtype %}buildtype = {{buildtype}}{% endif %}
     {% if debug %}debug = {{debug}}{% endif %}
     {% if default_library %}default_library = {{default_library}}{% endif %}
@@ -35,10 +37,10 @@ class MesonToolchain(object):
     {% if b_ndebug %}b_ndebug = {{b_ndebug}}{% endif %}
     {% if b_staticpic %}b_staticpic = {{b_staticpic}}{% endif %}
     {% if cpp_std %}cpp_std = {{cpp_std}}{% endif %}
-    {% if c_args %}c_args = {{c_args}}{% endif %}
-    {% if c_link_args %}c_link_args = {{c_link_args}}{% endif %}
-    {% if cpp_args %}cpp_args = {{cpp_args}}{% endif %}
-    {% if cpp_link_args %}cpp_link_args = {{cpp_link_args}}{% endif %}
+    c_args = {{c_args}} + preprocessor_definitions
+    c_link_args = {{c_link_args}}
+    cpp_args = {{cpp_args}} + preprocessor_definitions
+    cpp_link_args = {{cpp_link_args}}
     {% if pkg_config_path %}pkg_config_path = {{pkg_config_path}}{% endif %}
     """)
 
@@ -71,7 +73,9 @@ def __init__(self, conanfile, env=os.environ):
         self._shared = self._conanfile.options.get_safe("shared")
         self._fpic = self._conanfile.options.get_safe("fPIC")
         self.definitions = dict()
-        self._env = env
+        self.preprocessor_definitions = dict()
+        self._env = env.copy()
+        self._env.update(self._conanfile.env)
 
     @staticmethod
     def _to_meson_value(value):
@@ -137,6 +141,10 @@ def _to_meson_cppstd(self, cppstd):
     def _none_if_empty(value):
         return "'%s'" % value if value.strip() else None
 
+    def _env_array(self, name):
+        import shlex
+        return shlex.split(self._env.get(name, ''))
+
     @property
     def _context(self):
         project_options = []
@@ -171,13 +179,12 @@ def _context(self):
             "b_ndebug": self._to_meson_value(self._ndebug) if self._build_type else None,
             # https://mesonbuild.com/Builtin-options.html#compiler-options
             "cpp_std": self._to_meson_cppstd(self._cppstd) if self._cppstd else None,
-            "c_args": self._none_if_empty(self._env.get("CPPFLAGS", '') +
-                                          self._env.get("CFLAGS", '')),
-            "c_link_args": self._env.get("LDFLAGS", None),
-            "cpp_args": self._none_if_empty(self._env.get("CPPFLAGS", '') +
-                                            self._env.get("CXXFLAGS", '')),
-            "cpp_link_args": self._env.get("LDFLAGS", None),
-            "pkg_config_path": "'%s'" % os.getcwd()
+            "c_args": self._to_meson_value(self._env_array('CPPFLAGS') + self._env_array('CFLAGS')),
+            "c_link_args": self._to_meson_value(self._env_array('LDFLAGS')),
+            "cpp_args": self._to_meson_value(self._env_array('CPPFLAGS') + self._env_array('CXXFLAGS')),
+            "cpp_link_args": self._to_meson_value(self._env_array('LDFLAGS')),
+            "pkg_config_path": "'%s'" % os.getcwd(),
+            "preprocessor_definitions": self.preprocessor_definitions
         }
         return context
 
diff --git a/conans/test/functional/toolchains/meson/test_android.py b/conans/test/functional/toolchains/meson/test_android.py
index a4fbd549728..9b38188f8b2 100644
--- a/conans/test/functional/toolchains/meson/test_android.py
+++ b/conans/test/functional/toolchains/meson/test_android.py
@@ -101,12 +101,14 @@ def env(self):
         ar = self._tool('ar')
         cflags = '--target=%s' % self._target
         cxxflags = '--target=%s' % self._target
+        ldflags = '--target=%s' % self._target
 
         return {'CC': cc,
                 'CXX': cxx,
                 'AR': ar,
                 'CFLAGS': cflags,
-                'CXXFLAGS': cxxflags}
+                'CXXFLAGS': cxxflags,
+                'LDFLAGS': ldflags}
 
     def profile(self):
         template = textwrap.dedent("""
diff --git a/conans/test/functional/toolchains/meson/test_ios.py b/conans/test/functional/toolchains/meson/test_ios.py
index a2fcf3fe5fc..5f1bd010fbe 100644
--- a/conans/test/functional/toolchains/meson/test_ios.py
+++ b/conans/test/functional/toolchains/meson/test_ios.py
@@ -71,11 +71,13 @@ def env(self):
         cflags += " -isysroot " + self.xcrun.sdk_path
         cflags += " -arch " + to_apple_arch(self.arch)
         cxxflags = cflags
+        ldflags = cflags
 
         return {'CC': cc,
                 'CXX': cxx,
                 'CFLAGS': cflags,
-                'CXXFLAGS': cxxflags}
+                'CXXFLAGS': cxxflags,
+                'LDFLAGS': ldflags}
 
     def profile(self):
         template = textwrap.dedent("""
diff --git a/conans/test/functional/toolchains/meson/test_preprocessor_definitions.py b/conans/test/functional/toolchains/meson/test_preprocessor_definitions.py
new file mode 100644
index 00000000000..d90f1e0303f
--- /dev/null
+++ b/conans/test/functional/toolchains/meson/test_preprocessor_definitions.py
@@ -0,0 +1,67 @@
+import os
+import textwrap
+
+from conans.test.assets.sources import gen_function_cpp, gen_function_h
+from conans.test.functional.toolchains.meson._base import TestMesonBase
+
+
+class MesonPreprocessorDefinitionsTest(TestMesonBase):
+    _conanfile_py = textwrap.dedent("""
+    from conans import ConanFile, tools
+    from conan.tools.meson import Meson, MesonToolchain
+
+
+    class App(ConanFile):
+        settings = "os", "arch", "compiler", "build_type"
+        options = {"shared": [True, False], "fPIC": [True, False]}
+        default_options = {"shared": False, "fPIC": True}
+
+        def config_options(self):
+            if self.settings.os == "Windows":
+                del self.options.fPIC
+
+        def generate(self):
+            tc = MesonToolchain(self)
+            tc.preprocessor_definitions["TEST_DEFINITION1"] = "TestPpdValue1"
+            tc.preprocessor_definitions["TEST_DEFINITION2"] = "TestPpdValue2"
+            tc.generate()
+
+        def build(self):
+            meson = Meson(self)
+            meson.configure()
+            meson.build()
+    """)
+
+    _meson_build = textwrap.dedent("""
+    project('tutorial', 'cpp')
+    hello = library('hello', 'hello.cpp')
+    executable('demo', 'main.cpp', link_with: hello)
+    """)
+
+    def test_build(self):
+        hello_h = gen_function_h(name="hello")
+        hello_cpp = gen_function_cpp(name="hello",
+                                     preprocessor=["TEST_DEFINITION1", "TEST_DEFINITION2"])
+        app = gen_function_cpp(name="main", includes=["hello"], calls=["hello"])
+
+        self.t.save({"conanfile.py": self._conanfile_py,
+                     "meson.build": self._meson_build,
+                     "hello.h": hello_h,
+                     "hello.cpp": hello_cpp,
+                     "main.cpp": app})
+
+        self.t.run("install . %s" % self._settings_str)
+
+        content = self.t.load("conan_meson_native.ini")
+
+        self.assertIn("[built-in options]", content)
+        self.assertIn("buildtype = 'release'", content)
+
+        self.t.run("build .")
+        self.t.run_command(os.path.join("build", "demo"))
+
+        self.assertIn("hello: Release!", self.t.out)
+        self.assertIn("TEST_DEFINITION1: TestPpdValue1", self.t.out)
+        self.assertIn("TEST_DEFINITION2: TestPpdValue2", self.t.out)
+
+        self._check_binary()


Code review comments:
diff_hunk: @@ -71,11 +71,13 @@ def env(self):
         cflags += " -isysroot " + self.xcrun.sdk_path
         cflags += " -arch " + to_apple_arch(self.arch)
         cxxflags = cflags
+        ldflags = cflags
path: conans/test/functional/toolchains/meson/test_ios.py
text: @user1:
This looks wrong on the surface.

@user1:
according to @author apparently this is correct

@author:
anyway, changed the code to be more accurate