Repository: mesonbuild/meson

Issue:
Regression in installing symlinked source files as regular files
After https://github.com/mesonbuild/meson/issues/3914
systemd installation has regressed.

The source tree has:
./units/basic.target (not installed)
./test/basic.target -> ../units/basic.target

In test/meson.build it does:
                install_data(file,
                             install_dir : testsdir + '/testdata/' + subdir)
To install ./test/basic.target into ./usr/lib/systemd/tests/testdata/basic.target

Previously with v0.47 and earlier, installation of ./test/basic.target into /usr/lib/systemd/tests/testdata/basic.target would result in a regular text file.

Now with master it results in /usr/lib/systemd/tests/testdata/basic.target being a dangling symlink to a non-existant '../units/basic.target'

I believe this is a regression and a change of behavior. It is slightly unexpected that a resolvable source file, started to be copied as a broken dangling symlink. Especially since install_symlink is not used....

Is this an upstream bug or feature? How should this be fixed in meson or in my meson.build? And to be backwards/forwards compatible with older and the upcoming meson versions....


Discussion:
This is a workaround i managed to come with.... but imho this issue is a regression in meson if i have to do this:

```
diff --git a/test/meson.build b/test/meson.build
index 826e684e59..4afa730e3e 100644
--- a/test/meson.build
+++ b/test/meson.build
@@ -192,15 +192,24 @@ if conf.get('ENABLE_RESOLVE') == 1
         '''.split()
 endif
 
+cp_cmd = 'cp @0@ $DESTDIR/@1@'
+
 if install_tests
         foreach file : test_data_files
                 subdir = file.split('/')[0]
                 if subdir == file
                         subdir = ''
                 endif
-
-                install_data(file,
-                             install_dir : testsdir + '/testdata/' + subdir)
+                # because install_data is broken, and instead of copying test
+                # files as real files into target dir it now copies them as
+                # dangling symlinks, if the source file is a symlink to another
+                # file in the tree
+                src = meson.current_source_dir() + '/' + file
+                target = testsdir + '/testdata/' + subdir
+                meson.add_install_script('sh', '-c',
+                                         mkdir_p.format(target))
+                meson.add_install_script('sh', '-c',
+                                         cp_cmd.format(src, target))
         endforeach
 endif
 

```
Mh, I've like the idea that it's a bit hard to support all the symlink use-cases without a `follow_symlinks` options on install. As in this case you want the pointed file to be installed not the link, while in other scenarios you want the link itself.

As things might be installed in a different order, I guess it's not easy to define if a linked path will be eventually installed or not. For sure I'd guard by installing a symlink target that points to somehwere outside the meson scope.
Hmm, interesting. This is indeed a regression. I guess we should emulate the old behaviour in which a dangling symlink is copied as-is, but a real symlink is followed and the file itself is copied.

This behaviour may be unexpected for new users, but we shouldn't regress `install_subdir()` like this.
... also we should ensure that the target of the real symlink is a file inside the source tree, else we should copy it as a dangling symlink.
I really think that "install this source subtree as is" should not resolve any symlinks but instead transfer them as is. If I had realized this when I first implemented this feature I would have made it work like that. I'll go even further and say that having symlinks in your source directory is not a very good idea in any case because it is confusing and also not portable to Windows (not really an issue for systemd but still).

What I'd _really_ want to do is to detect this symlink case, print a warning about that and then change the behaviour to be the sane one in a release or two. Having a non-regular behaviour like that (sometimes things are copied, sometimes not) is not really nice in the long term.
Yes, it's not a great behaviour to have, I agree. Like you said we can't regress, and I also believe we should almost never deprecate behaviour without a way to replace it. How about we make it explicit with a `follow_symlinks:` kwarg? That kwarg will only ever copy symlink targets that are inside the source tree. Then we could make it the default and warn for a few releases and then require that kwarg to be set.
> How about we make it explicit with a follow_symlinks: kwarg? 

You do know what will happen then, right? People will want some symlinks followed and others not. And then you get into a quagmire of complexity and madness.

The important question here is why do people have symlinks in their source tree? What are they trying to achieve with them? Why do they need installing? And most importantly: is there a better way of achieving the same?
> You do know what will happen then, right? People will want some symlinks followed and others not. And then you get into a quagmire of complexity and madness.

Sounds correct to me? You would be able to set `follow_symlinks: false` to disable this behaviour.

> The important question here is why do people have symlinks in their source tree? What are they trying to achieve with them? Why do they need installing? And most importantly: is there a better way of achieving the same?

This is something @xnox should answer in more detail, but my guess is that symlinks are a useful way to keep data that is used in two different places in sync since git doesn't provide any native ways to do so.
in this case there is no subtree. it is just installs one datafile. Yes we have symlinks, in the tree because same units are used for in-tree, system, user, testsuite contexts (different location). I don't know how this should be handled, cause ideally we want the source trees to be usable without building anything too. I guess i should have a dummy generator that will copy the source -> regular file, and then install generated regular file -> destdir? sounds odd as well....
Ideally you do want one location for your unit files and then install them in different locations by calling `install_data()` or `install_subdir()` multiple times. Using symlinks for this means we have no way to deduce your intent.

Is there a good reason why you have it in separate directories at all? Uninstalled use-case?
@nirbheek i cannot comment on that, as i'm not the one maintaining / know well those meson build files in the systemd upstream tree.

Resolving patch:
diff --git a/mesonbuild/minstall.py b/mesonbuild/minstall.py
--- a/mesonbuild/minstall.py
+++ b/mesonbuild/minstall.py
@@ -27,6 +27,10 @@
     # This is only used for pkexec which is not, so this is fine.
     main_file = None
 
+symlink_warning = '''Warning: trying to copy a symlink that points to a file. This will copy the file,
+but this will be changed in a future version of Meson to copy the symlink as is. Please update your
+build definitions so that it will not break when the change happens.'''
+
 selinux_updates = []
 
 def buildparser():
@@ -242,7 +246,15 @@ def do_copyfile(self, from_file, to_file):
             os.remove(to_file)
         print('Installing %s to %s' % (from_file, outdir))
         if os.path.islink(from_file):
-            shutil.copy(from_file, outdir, follow_symlinks=False)
+            if not os.path.exists(from_file):
+                # Dangling symlink. Replicate as is.
+                shutil.copy(from_file, outdir, follow_symlinks=False)
+            else:
+                # Remove this entire branch when changing the behaviour to duplicate
+                # symlinks rather than copying what they point to.
+                print(symlink_warning)
+                shutil.copyfile(from_file, to_file)
+                shutil.copystat(from_file, to_file)
         else:
             shutil.copyfile(from_file, to_file)
             shutil.copystat(from_file, to_file)


Test patch: