Repository: mesonbuild/meson

Issue:
Don't force uniqueness in target names within a project
Currently, we don't allow you to have two `executable()` or `shared_library()` targets of the same name in the same project. However, this is sometimes needed:

1. [Test and benchmark executables can have the same names](https://github.com/mesonbuild/meson/pull/1867#issuecomment-304692828)
2. [A library and its Python module can have the same name](https://github.com/mesonbuild/meson/issues/2859#issuecomment-354684303)

This restriction seems to have been borrowed from cmake, but there is nothing in our code stopping us from implementing it, as shown by [this PR](https://github.com/mesonbuild/meson/pull/1867), and no other build system has this restriction.

In practice, having two targets of the same type with the same name, and in different subdirs (or if in the same subdir, with different suffixes and/or prefixes) should not be a problem for the build system (unless they are installed into the same destdir or you're building with a flat layout), and is anyway equivalent (in practice) to having a target with the same name in a subproject (which we don't, shouldn't, and can't really disallow).

We should remove this restriction.

CCing @kaimast 
Fix backwards-compatibility issues with slashes in target names
commit af8c188bd1219617dc66a8f5c9cc62f1a200c180

    Don't force target names to be unique everywhere
    
    The only reasons this was added were:
    
    1. Building with a flat layout is impossible
    2. Installation into the same destdir would overwrite the previous file
    
    We now error out on duplicate target names only when the layout is
    flat, and we error out on install if we try to install two files of the
    same filename into the same directory.
    
    This is actually better than the previous behaviour because now we
    explicitly ensure that we never install any two identically-named files
    into the same directory.

commit 6e5919fdb0c9971deddd9bdf675f080e596a1ed6

    Add curdir to the include path of targets with path separators in the name
    
    This is needed for backwards compatibility since in the earlier
    behaviour we were adding the current directory to the include paths
    and we need to keep doing that or targets with path separators in the
    name may fail to build because they can't find headers.
    
    See: https://github.com/mesonbuild/meson/pull/1824

commit d612bf60c80ffa7b46ab7ed530b523f56f05736f
    Create a helper for checking if a string has a path component
    
    This is used in a number of places, and in some places it is incomplete.
    Use a helper to ensure it's used properly.

commit 1556903d434e5a8e56ef89931e72527f2d57c8dc

    Add a helper for maintaining backwards-compat code
    
    We now check the meson_version specified in project() (per subproject),
    and if it is older than the version in which the compatibility-breaking
    change was made, we use the older behaviour and print a warning.
    
    Hence, if you set `meson_version : '>=0.41.0'` in project(), you will
    not be allowed to use path separators in target names.
    
    This allows us to have a consistent warning message for deprecations,
    allows us to track deprecations across releases, and will also make it
    easy to remove the fallback code when we decide to stop supporting
    backwards-compat with a specific old version of Meson.
    
    Will also be useful for https://github.com/mesonbuild/meson/pull/761

commit 6a535bc907180ac080a65211b33130aaaaf40921

    Add an env var to force meson to print a backtrace
    
    This is really useful when debugging test failures. Without a stack
    trace, you have to grep the source code for the error message.
    
    Also set this in run_tests.py.



Discussion:
Having the same name of a library and an executable will fail if you build the project with `msbuild` on windows. I think uniqueness should be enforced even stricter than it currently is.
> Having the same name of a library and an executable will fail if you build the project with msbuild on windows.

That will fail with any backend when you build with MSVC since pdb files use the `name` and not the full filename, so foo.dll and foo.exe write out foo.pdb and clobber each other. Right now we don't detect this case at all, partly because of limitations in ninja (it does not handle rules with multiple outputs well at all).

However, we shouldn't design according to the lowest-common-denominator, otherwise we'd have to assume every project is a flat layout on a case-insensitive layout built with MSVC, which is a setup with a lot of ridiculous problems. A lot of people do not care about MSVC at all, f.ex systemd, Xorg, etc, and would not be happy to be restricted by a platform they will never use.

One of the things we *could* do is warn (on all platforms) when some of these cases occur so people who are writing new libraries are aware of it and don't hit their head on it when they try to port their library to MSVC (for example). Of course, we should error out on the relevant platform.

Another thing we could do, is add a feature such as 

```meson
supported_setups(platform: ['linux', 'osx', 'windows'],
                 layout: ['mirror', 'flat'])
```

and then you will get cross-platform errors about problems in your build files that are relevant to all these platforms or configurations. It will also tell users that they should expect to find problems when they try to build on an unsupported platform, so they can work with upstream to add that platform.
  
@nirbheek mentioned that related discussion is in #1867

TODO:
- Figure out corner cases besides Windows
- Make proof of concept
> Make proof of concept

fwiw, the linked PR also has an implementation.
Yup, but it is broken. I will study it anyway :) It's more or less note for myself because I tend to forget things when I switch contexts.
Discussed in IRC with @nirbheek and @jpakkane 

Things to consider:
- introspection ID must remain unique
- we start with simple name mangling `foo/baz/bar` becomes `foo_baz_bar`, we ignore `foo/baz_bar` vs `foo_baz/bar` for now
To clarify, `Don't force target names to be unique everywhere` is needed for https://github.com/mesonbuild/meson/pull/1824#issuecomment-304588207. Target names with a path meant that the target name was unique in gstreamer for `tests/benchmarks/controller` and `tests/check/controller`, but not anymore once we started using only the basename.

There's no technical reason for this afaict, so pending the Visual Studio backend's test results, I think this should be fine to do.
> The only reasons this was added were

Those are not the only reason. The main reason is that duplicate names are problematic everywhere. As an example when introspecting results it gets very confusing for downstream if there are multiple targets with the same name. Similarly the code rewriter becomes more complicated because you can't just say "add source X to target Y" because Y can mean many different things. We should avoid leaking backend behaviour to the user as much as possible. Any setup we have should either work on all of them or be a hard error immediately whenever that is possible.

The fact that target names must be unique was a deliberate and specific design choice. It follows the principle of least astonishment.



> As an example when introspecting results it gets very confusing for downstream if there are multiple targets with the same name.

How so? The sub directory is sufficient context.

> Similarly the code rewriter becomes more complicated because you can't just say "add source X to target Y" because Y can mean many different things.

Why not say "target Y in subdir Z"?

> We should avoid leaking backend behaviour to the user as much as possible. Any setup we have should either work on all of them or be a hard error immediately whenever that is possible.

Absolutely. I don't think this is a restriction on any backend at all, though.

> The fact that target names must be unique was a deliberate and specific design choice. It follows the principle of least astonishment.

I would argue that since no other build system that I know of has this restriction, it is actually quite astonishing for new users. :)

There's already gstreamer code sitting in a branch right now that will use the same name for a library and a plugin, which will cause an error with Meson (but no other build system). If it's merged, it won't be buildable with Meson at all without this PR.
> I would argue that since no other build system that I know of has this restriction, it is actually quite astonishing for new users.

CMake has the same thing. That's where I took it from. This is the output:

    CMake Error at CMakeLists.txt:5 (add_executable):
      add_executable cannot create target "foo" because another target with the
      same name already exists.  The existing target is an executable created in
      source directory "/home/jpakkane/cmake".  See documentation for policy
      CMP0002 for more details.

CMake is more strict than Meson. It does not permit having two targets with the same name _even if_ they have different types. So you can't build a shared library foo and an executable foo that uses it.

Remembering what a nightmare Visual Studio naming is I totally understand why they have this limitation. It makes things simpler. We don't have the limitation, because this is a thing people want to do and thus we have to do extra somersaults to make it easy for them.

> How so? The sub directory is sufficient context.

For computers, yes. For humans, no.

> There's already gstreamer code sitting in a branch right now that will use the same name for a library and a plugin, which will cause an error with Meson (but no other build system). If it's merged, it won't be buildable with Meson at all without this PR.

That is a thing we should support and need to fix before the next release. On the other hand the combination of target type + name _must_ be unique for a given project. This is what we have always supported and what I have tried to put in all documentation and as advice to people (this may not have been 100% successful but that's a different story).

Building a shared library and plugin in the same dir can lead to clashes but there's nothing we can really do to prevent that. The same issue pops up if someone is crazy enough to build an executable with the prefix `lib` and extension `.so`.
> CMake has the same thing. That's where I took it from.

Cmake is being very silly then, I think. Autotools does not have this restriction, Ninja does not, and XCode and VS do not. :)

I'm not sure if cmake has this restriction because of architectural deficiencies or what, but I don't think we should emulate them in this regard.

>> How so? The sub directory is sufficient context.
> For computers, yes. For humans, no.

I disagree. If you look at the gstreamer code that is failing because of this, with `tests/benchmark/controller` vs `tests/check/controller` the subdirs contain much-needed context compared to just `controller`.

Regardless, arguing that the source directory structure is "too difficult to understand" for someone hacking on the source code's build system is a bit strange. I would assume that the GUI would show the filename and not the target name anyway: "Edit the cflags of `tests/benchmark/controller.exe`", etc. Or it would ask you to select the target from a tree-based structure since projects often have far too many targets to make you select from a list.

> That is a thing we should support and need to fix before the next release.

In gstreamer, plugins and libraries are both built with `library()` because `shared_module()` has the wrong semantics (a plugin is literally just a library, and we also need to build static versions of it), so we actually need targets of the same type to have the same name.

Also, gstreamer tests already share binary names, and stable releases will be broken because of this assumption. We cannot assume that for a target name with path separators the basename is unique enough.

So far I haven't heard any arguments for having the filename of a target not be unique. All I got was:

1. Cmake does this (for some unknown reason)
2. Makes it easier to support automated build file editing (arguably not true)

Anything else?
# [Codecov](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=h1) Report
> Merging [#1867](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=desc) into [master](https://codecov.io/gh/mesonbuild/meson/commit/0f85d9f66d85a6cca7e57dba6a9590a3bdbac57d?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `60.95%`.

[![Impacted file tree graph](https://codecov.io/gh/mesonbuild/meson/pull/1867/graphs/tree.svg?src=pr&token=BplKwTJqLg&width=650&height=150)](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1867      +/-   ##
==========================================
- Coverage   63.01%   62.99%   -0.02%     
==========================================
  Files          58       59       +1     
  Lines       16882    16938      +56     
  Branches     3456     3465       +9     
==========================================
+ Hits        10638    10670      +32     
- Misses       5150     5172      +22     
- Partials     1094     1096       +2
```


| [Impacted Files](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [mesonbuild/modules/windows.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9tb2R1bGVzL3dpbmRvd3MucHk=) | `72.34% <0%> (ø)` | :arrow_up: |
| [mesonbuild/scripts/meson\_install.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9zY3JpcHRzL21lc29uX2luc3RhbGwucHk=) | `0% <0%> (ø)` | :arrow_up: |
| [mesonbuild/backend/xcodebackend.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9iYWNrZW5kL3hjb2RlYmFja2VuZC5weQ==) | `0% <0%> (ø)` | :arrow_up: |
| [mesonbuild/scripts/yelphelper.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9zY3JpcHRzL3llbHBoZWxwZXIucHk=) | `0% <0%> (ø)` | :arrow_up: |
| [mesonbuild/mesonlib.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9tZXNvbmxpYi5weQ==) | `81.4% <100%> (+0.18%)` | :arrow_up: |
| [mesonbuild/mlog.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9tbG9nLnB5) | `83.56% <100%> (+0.46%)` | :arrow_up: |
| [mesonbuild/backend/vs2010backend.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9iYWNrZW5kL3ZzMjAxMGJhY2tlbmQucHk=) | `89.13% <100%> (ø)` | :arrow_up: |
| [mesonbuild/mesonmain.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9tZXNvbm1haW4ucHk=) | `53.72% <50%> (+0.76%)` | :arrow_up: |
| [mesonbuild/interpreter.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9pbnRlcnByZXRlci5weQ==) | `73.62% <62.5%> (-0.22%)` | :arrow_down: |
| [mesonbuild/backend/ninjabackend.py](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree#diff-bWVzb25idWlsZC9iYWNrZW5kL25pbmphYmFja2VuZC5weQ==) | `88.48% <71.42%> (-0.13%)` | :arrow_down: |
| ... and [5 more](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=footer). Last update [0f85d9f...430f041](https://codecov.io/gh/mesonbuild/meson/pull/1867?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> Cmake is being very silly then, I think. Autotools does not have this restriction, Ninja does not, and XCode and VS do not. :)

Suppose we have an executable E that uses a shared library L that is in subdirectory foo. Suppose further that there is a second library called L that is in subdirectory bar. When linking, E gets an rpath that points to foo and everything is fine.

Except that it's not. Suppose that E also links against L2 that is in subdirectory bar. Now it has rpath entries to foo _and_ bar. Depending on the order it might work or the dynamic linker might start resolving E's symbols in L2, because that is the first thing it finds. Now you have a program that fails at random in a way that is _extremely_ difficult to understand if you don't already know what the issue is.

Multiple targets with the same name _seem_ like a good idea and something you want. But the truth is that they are by their nature unreliable and can break in a multitude of ways, all of which are nasty. It's the same as having multiple header files with the same name in one project (or even worse, installed to the system) and thinking that "there's no way these will get mixed up". But they will. Every time. Usually when you least expect it. The design philosophy of Meson has always been to make potentially erroneous situations impossible to express because, while sometimes irritating, it makes a whole class of bugs impossible to create. Just like Rust. We should aim for the same here.

However since there is code that depends on this and the release is coming up we're going to need a saner transition path. How about this:

 - undo the change that removes path segments and their checks
 - update rpath evaluation function to handle files that are in an incorrect directory
 - add BIG HONKING WARNINGS that this is broken, not reliable and will be removed in some future release

This allows us to get a release that does reproducible builds and some breathing room for the actual transition.
The `RPATH` hypothetical you talk about is already possible with subprojects. There is no difference between two targets having the same name in different subprojects and having the same name in the same subproject, so I don't understand why we need to attend to this specially here.

The linker uses the library name as the unique identifier, so you cannot link to two different shared libraries at the same time. If you use an intermediary library to trick it, it will use the first one it finds (as you say). However, we can actually detect this while generating the ninja file and error out, so that would avoid that completely[1].

I completely agree with shutting off broken use-cases at their root, and this mechanism would actually allow that. Wouldn't it? Unlike header files sharing the same name (which already happens in the wild), we can detect and prevent this trivially.

The real use-case for this is two libraries that are used in totally different contexts (one loaded with the runtime linker, another loaded manually with `dlopen()` or two versions of the same library; one with feature X and another with feature Y, used to by two separate sets of executables, etc). These are genuine use-cases, and are already possible with subprojects. Detecting user error in these would be a major plus.

1. Unless someone forces linking with `-lfoo` etc, but then you're twisting the build system's arm and all bets are off; just like someone using `name_prefix:` to force meson to build two libraries with the same name.

Resolving patch:
diff --git a/mesonbuild/build.py b/mesonbuild/build.py
--- a/mesonbuild/build.py
+++ b/mesonbuild/build.py
@@ -310,11 +310,16 @@ def get_subdir(self):
     def get_id(self):
         # This ID must also be a valid file name on all OSs.
         # It should also avoid shell metacharacters for obvious
-        # reasons.
-        base = self.name + self.type_suffix()
-        if self.subproject == '':
-            return base
-        return self.subproject + '@@' + base
+        # reasons. '@' is not used as often as '_' in source code names.
+        # In case of collisions consider using checksums.
+        # FIXME replace with assert when slash in names is prohibited
+        name_part = self.name.replace('/', '@').replace('\\', '@')
+        assert not has_path_sep(self.type_suffix())
+        myid = name_part + self.type_suffix()
+        if self.subdir:
+            subdir_part = self.subdir.replace('/', '@').replace('\\', '@')
+            myid = subdir_part + '@@' + myid
+        return myid
 
     def process_kwargs(self, kwargs):
         if 'build_by_default' in kwargs:
diff --git a/run_unittests.py b/run_unittests.py
--- a/run_unittests.py
+++ b/run_unittests.py
@@ -1117,7 +1117,7 @@ def test_internal_include_order(self):
         incs = [a for a in shlex.split(execmd) if a.startswith("-I")]
         self.assertEqual(len(incs), 9)
         # target private dir
-        self.assertPathEqual(incs[0], "-Isub4/someexe@exe")
+        self.assertPathEqual(incs[0], "-Isub4/sub4@@someexe@exe")
         # target build subdir
         self.assertPathEqual(incs[1], "-Isub4")
         # target source subdir
@@ -1917,6 +1917,15 @@ def test_identical_target_name_in_subproject_flat_layout(self):
         self.init(testdir, extra_args=['--layout=flat'])
         self.build()
 
+    def test_identical_target_name_in_subdir_flat_layout(self):
+        '''
+        Test that identical targets in different subdirs do not collide
+        if layout is flat.
+        '''
+        testdir = os.path.join(self.common_test_dir, '189 same target name flat layout')
+        self.init(testdir, extra_args=['--layout=flat'])
+        self.build()
+
     def test_flock(self):
         exception_raised = False
         with tempfile.TemporaryDirectory() as tdir:
@@ -2601,8 +2610,8 @@ def test_compiler_cpp_stds(self):
     def test_unity_subproj(self):
         testdir = os.path.join(self.common_test_dir, '49 subproject')
         self.init(testdir, extra_args='--unity=subprojects')
-        self.assertPathExists(os.path.join(self.builddir, 'subprojects/sublib/sublib@@simpletest@exe/simpletest-unity.c'))
-        self.assertPathExists(os.path.join(self.builddir, 'subprojects/sublib/sublib@@sublib@sha/sublib-unity.c'))
+        self.assertPathExists(os.path.join(self.builddir, 'subprojects/sublib/subprojects@sublib@@simpletest@exe/simpletest-unity.c'))
+        self.assertPathExists(os.path.join(self.builddir, 'subprojects/sublib/subprojects@sublib@@sublib@sha/sublib-unity.c'))
         self.assertPathDoesNotExist(os.path.join(self.builddir, 'user@exe/user-unity.c'))
         self.build()
 


Test patch:
diff --git a/test cases/failing/17 same name/file.c b/test cases/common/183 same target name/file.c
similarity index 100%
rename from test cases/failing/17 same name/file.c
rename to test cases/common/183 same target name/file.c
diff --git a/test cases/failing/17 same name/meson.build b/test cases/common/183 same target name/meson.build
similarity index 100%
rename from test cases/failing/17 same name/meson.build
rename to test cases/common/183 same target name/meson.build
diff --git a/test cases/failing/17 same name/sub/file2.c b/test cases/common/183 same target name/sub/file2.c
similarity index 100%
rename from test cases/failing/17 same name/sub/file2.c
rename to test cases/common/183 same target name/sub/file2.c
diff --git a/test cases/failing/17 same name/sub/meson.build b/test cases/common/183 same target name/sub/meson.build
similarity index 100%
rename from test cases/failing/17 same name/sub/meson.build
rename to test cases/common/183 same target name/sub/meson.build
diff --git a/test cases/common/189 same target name flat layout/foo.c b/test cases/common/189 same target name flat layout/foo.c
new file mode 100644
--- /dev/null
+++ b/test cases/common/189 same target name flat layout/foo.c	
@@ -0,0 +1 @@
+int meson_test_main_foo(void) { return 10; }
diff --git a/test cases/common/189 same target name flat layout/main.c b/test cases/common/189 same target name flat layout/main.c
new file mode 100644
--- /dev/null
+++ b/test cases/common/189 same target name flat layout/main.c	
@@ -0,0 +1,16 @@
+#include <stdio.h>
+
+int meson_test_main_foo(void);
+int meson_test_subproj_foo(void);
+
+int main(void) {
+    if (meson_test_main_foo() != 10) {
+        printf("Failed meson_test_main_foo\n");
+        return 1;
+    }
+    if (meson_test_subproj_foo() != 20) {
+        printf("Failed meson_test_subproj_foo\n");
+        return 1;
+    }
+    return 0;
+}
diff --git a/test cases/common/189 same target name flat layout/meson.build b/test cases/common/189 same target name flat layout/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/common/189 same target name flat layout/meson.build	
@@ -0,0 +1,11 @@
+project('subdir targets', 'c')
+
+# Idea behind this test is to create targets with identical name
+# but different output files. We can do this by choosing different
+# name_prefix of libraries. Target id does not depend on name_prefix.
+
+main_foo = static_library('foo', 'foo.c', name_prefix : 'main')
+subdir('subdir') # defines subdir_foo
+
+exe = executable('prog', 'main.c', link_with : [main_foo, subdir_foo])
+test('main test', exe)
diff --git a/test cases/common/189 same target name flat layout/subdir/foo.c b/test cases/common/189 same target name flat layout/subdir/foo.c
new file mode 100644
--- /dev/null
+++ b/test cases/common/189 same target name flat layout/subdir/foo.c	
@@ -0,0 +1 @@
+int meson_test_subproj_foo(void) { return 20; }
diff --git a/test cases/common/189 same target name flat layout/subdir/meson.build b/test cases/common/189 same target name flat layout/subdir/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/common/189 same target name flat layout/subdir/meson.build	
@@ -0,0 +1 @@
+subdir_foo = static_library('foo', 'foo.c', name_prefix : 'subdir')
diff --git a/test cases/failing/17 same target/file.c b/test cases/failing/17 same target/file.c
new file mode 100644
--- /dev/null
+++ b/test cases/failing/17 same target/file.c	
@@ -0,0 +1 @@
+int func() { return 0; }
diff --git a/test cases/failing/17 same target/meson.build b/test cases/failing/17 same target/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/failing/17 same target/meson.build	
@@ -0,0 +1,4 @@
+project('same target', 'c')
+
+static_library('foo', 'file.c')
+static_library('foo', 'file.c')