Repository: mesonbuild/meson

Issue:
git version
Many projects rely on git generated version. git itself, and there is the popular http://git.savannah.gnu.org/cgit/autoconf.git/tree/build-aux/git-version-gen for autoconf. Other projects use similar scripts.

It would be great if meson had a 'git-version-gen' option in project() 



Discussion:
[This](https://github.com/mesonbuild/meson/wiki/Reference%20manual#vcs_tag) should do what you need.

Interesting, but how does that help to specify the project version?

It can't. Project versions must be written by hand and be explicit. It must also work when Git is not installed. The common approach is to have a sensible version number in `project` and put the git hash in `config.h` or similar so you have access to both.

It's not the same though, git-version-gen allows to generate incremental version for each commit, and avoid having hard-coded version duplication. It's not simply about the git hash.

I'm not sure I understand how that's not covered by the `vcs_tag` command. git commit ids change as soon as you do a rebase or any operation that rewrites history. Even cherry-pick rewrites the commit id. Could you please explain your use-case a bit more? What sort of versions does `git-version-gen` generate? How are they incremental?

By default vcs_tag will output something using `git describe` such as `0.33.0-22-g58359c8` which basically means 22 commits after the 0.33.0 tag, with head at commit 5835...

What is project() version used for? According to reference manual, it's just for later reference with meson.project_version(), but if you use vcs_tag you'll probably not use it.

Similarly what is the project name for? I can't even find a 'dist' target.

Is there a way to list available targets? 

Btw, it would be quite helpeful if the reference manual had function declarations: project(project_name, language1, _language2_ ...)
optional keywords: version, subproject_dir, meson_version, default_options

Meson is quite confusing when coming from autofoo or even plain Makefile, and documentation has a lot to be desired.

A guess a more complete hello-world project (not just http://mesonbuild.com/samples.html or test cases in meson.git), including binary, static library, shared library, tests.. would help a lot.

I wish there would be a meson version of GNU hello: http://git.savannah.gnu.org/cgit/hello.git

Project versions are used when importing subprojects. They allow you you specify version limits similar to `dependency` like this:

```
foo_sp = subproject('foo', version : '>=1.0.0')
```

The wiki has [a more in-depth tutorial](https://github.com/mesonbuild/meson/wiki/IndepthTutorial), which should be what you want.

The function declarations thing is a known issue. Unfortunately Github's wiki formatting is limited so it is hard to make it look similar to e.g. gtk-doc output. 

Is there anything else you need here, @elmarco?
@nirbheek I haven't tried recently, and I have not good knowledge of meson. Afaik, it's not possible to let the version be generated from git describe, like git-version-gen does.
It's almost possible to use an approach similar to git-version-gen, since you should be able to run_command() like that:

    version: run_command(['git-version-gen',
                          '$MESON_SOURCE_ROOT/.tarball_version']).stdout()

However, there is no easy way I can think of to ship .tarball_version... That file should be generated at dist time, and not kept in the build directory after dist, but should be included in dist.
> However, there is no easy way I can think of to ship .tarball_version...

Resolving #2166 would provide one, I think?
I am also interested in having some automagic git tag versioneering, much like https://github.com/pypa/setuptools_scm. It's just so convenient to not have to do boring version bump commits and being able to do a release by tagging a commit in a UI.
Question for @elmarco: if the `vcs_tag()` call would do what it currently does inside a version controlled source tree and would 'magically' transport the version info during a dist step into the tarball, such that if you built the extracted tarball with meson again, it reproduces the info that it stored during the dist, would that be what you are looking for with your `git-version-gen` request?
@axxel that sounds like it would be enough.

fwiw, Eduardo Lima managed to somehow use git-version-gen in spice-gtk:

https://gitlab.freedesktop.org/spice/spice-gtk/blob/master/meson.build#L5

I haven't looked at the details, but my feeling is that this could be simplified by some common meson facilities.
actually, his solution doesn't work, tarball-version isn't shipped, and make dist fails.
Even if it would work, it would not be quite the same thing: the `vcs_tag()` sets up a 'build_always' target that checks on _every_ build if the state of your wc changed. The idea was to make sure you don't accidentally ship uncommitted changes by typically adding a '+' to your version string in that case. Executing some script during the 'global configure' step will not provide that.

Also, I was not proposing to somehow make the meson.project_version() 'automacially' return the vcs version of the current wc, albeit I would see the conceptual benefit of that: You'd had only one kind of version to reason about. It would also naturally fix #3903.

@jpakkane:  would you consider merging something along the lines of:
```
project('name', 'c', version : vcs_tag(template: '1.0.@VCS_TAG@'))
```
It would make the `vcstagger.py` simply rerun the config process as if `meson.build` had changed (instead of generating some output file). The template parameter could have a default value of '@VCS_TAG@'. Putting the information into some `config.h` would then simply use the existing `configure_file()` machinery. The question of how to transport the version info from the wc to a tarball is independent of this new approach. Thinking about it for a few minutes makes me regret to not have come up with this idea when I first implemented the `vcs_tag()`. I might very well be overlooking some fundamental issues with this 'full rerun' approach, though.
@axxel @jpakkane has there been any update for this?
thanks
Since I got no reaction from Jussi to my "would you consider merging...?" question above, I did not further investigate this idea.
I think it would make sense to have it as new function (e.g. `vcs_tag_string()`) that returns the version as a string. I don't think you need a template argument because you can do it as `'1.0.@0@'.format(vcs_tag_string())`.

From an implementation POV, it would be a `build_always` target that spawn `meson --reconfigure` in case the version changed. Note that means each commit would trigger a reconfigure, not sure that's what people here wants.

@jpakkane any objection?
Any system that sets up the vcs tag version via a `meson.build` definition is not really reliable because if you do new commits they don't necessarily trigger a Meson reconfiguration. To get that reliable we'd need to rerun reconfigure on every change, which would be too slow.
Just to make sure I understand your objection correctly: a reconfiguration would be necessary after every WC state change (like commit, checkout, etc.) not on every build attempt. How long does a reconfiguration after a manual change of the version info in the `project()`-line take on a 'large' project?
Worst case that I know of is tens of seconds.

But even worse is that it changes an operation (empty build) from something that is instantaneous to something that is noticeable. This is poor UX.
Tens of seconds sounds not very nice (in and of itself) indeed. But going one step back and looking at the issue from a top level point of view, this cost is what you _have_ to pay for getting an up to date `version` info for your project then, whether you provide it manually by changing the build file or have meson figure it out from the vcs. So if your version template only contains e.g. git tags and no commit hash, then you'd pay the very same price but without the requirement to keep your version info in sync manually (which is, of course, what this whole vcs-tag-feature is all about).

Apart from that: would it not be possible to speed up the reconfiguration by caching data if the only change is known to be the version string?
I just spent an hour going through issue #2166 and playing around with add_dist_script(), add_postconf_script() etc., but none of these really work. I wrote a little `src/getversion.sh` helper:
```sh
#!/bin/sh
set -eux
ROOT=$(dirname $(dirname $(realpath "$0")))
STAMP="$ROOT/.version"

[ ! -e "$ROOT/.git" ] || git describe > "$STAMP"

if [ -e "$STAMP" ]; then
    cat "$STAMP"
else
    echo "ERROR: Neither a git checkout nor $STAMP, cannot determine version" >&2
    exit 1
fi
```

But I can't figure out how to integrate that into meson.build. add_dist_script() does not have access to the git checkout (so can't run`git describe`), and the postconf script does not add the generated ./version file to the dist tarball. This is really annoying -- these days we have git, signed releases, pipelines, and release automation. Doing a release should be "push a signed tag", nothing else; we really shouldn't have to rely on bolted-on "release.sh" scripts any more which hack the build system into poking a version number into a git tracked file.

Did anyone here happen to have a solution for this? It feels to me like one could coerce all these building blocks into something that works, but the meson docs nor google make this really hard to figure out..

I'd even go as far as saying that meson should have some builtin default of taking the release number from git (and storing it in the dist tarball somehow), but at least documenting a supported way would be really nice.

Thanks!
> but none of these really work. add_dist_script() does not have access to the git checkout (so can't run`git describe`), [...]

It should, though? There are 3 documented environment variables:
- `MESON_DIST_ROOT`
- `MESON_SOURCE_ROOT`
- `MESON_BUILD_ROOT`

The first one is, obviously, the temporary staging directory without access to the git checkout. The other two are documented as only set for dist scripts since 0.54.0; they were previously only available in other types of scripts.

Regardless of which type of script is being run, if `MESON_SOURCE_ROOT` is available it points to the git checkout.
> I'd even go as far as saying that meson should have some builtin default of taking the release number from git (and storing it in the dist tarball somehow), but at least documenting a supported way would be really nice.

I do want to eventually add this kind of thing. I have some thoughts about how it might work. The tricky part is probably figuring out how to handle `meson.project_version()` being used, but it may be possible to simply ban it when versions come from git, and expect all such work to be handled via `vcs_tag()` (or realistically speaking, a much-improved `vcs_tag2()`, sigh).
@eli-schwartz : I did find these env variables in the [docs](https://mesonbuild.com/Reference-manual_builtin_meson.html#mesonadd_dist_script), but they didn't seem to help me. On top of the [initial naive broken attempt](https://github.com/martinpitt/umockdev/commit/3cb4214c12902ec932112a3d51bc4ab8aa6660d7) I added a [debug commit](https://github.com/martinpitt/umockdev/commit/4cbaa428c05f99929e5d693329834b66e94fa205) which shows all the env variables that contain "MESON", and the current dir:
```
+ pwd
/var/home/martin/upstream/umockdev/b/meson-private/dist-unpack/umockdev-0.17.13-8-g4cbaa42
+ env
+ grep MESON
MESON_SUBDIR=
MESON_BUILD_ROOT=/var/home/martin/upstream/umockdev/b/meson-private/dist-build
MESONINTROSPECT=/usr/bin/meson introspect
MESON_SOURCE_ROOT=/var/home/martin/upstream/umockdev/b/meson-private/dist-unpack/umockdev-0.17.13-8-gf9f1a87
```
That `MESON_SOURCE_ROOT` is already some non-git copy of the tree; my actual tree is in `/var/home/martin/upstream/umockdev`, and `b/` is the configured build tree (`meson b`).

That's why I was looking into _postconf_hook(), as that runs in the source dir.

(This is meson 0.62.2)
Ah, another thing I tried yesterday:
```diff
--- meson.build
+++ meson.build
@@ -44,7 +44,7 @@ if cc.has_function('__fxstatat', prefix: '#include <sys/stat.h>')
   add_project_arguments('-DHAVE_FXSTATAT', language: 'c')
 endif
 
-meson.add_dist_script(srcdir / 'getversion.sh')
+meson.add_dist_script(srcdir / 'getversion.sh', meson.current_source_dir())
 
 #
 # dependencies
diff --git src/getversion.sh src/getversion.sh
index e2922f1..9c01ed6 100755
--- src/getversion.sh
+++ src/getversion.sh
@@ -1,5 +1,6 @@
 #!/bin/sh
 set -eux
+echo "ARG1: ${1:-}" >&2
 pwd >&2
 env | grep MESON >&2
 ROOT=$(dirname $(dirname $(realpath "$0")))
```
as the documentation says that you can pass arguments to add_dist_script(). But it is not actually getting passed, `$1` is empty.
You're running getversion.sh twice, once as a dist script and once as a project() version kwarg. So I guess it is erroring out after the dist, once it tries testing the dist.
Thanks @eli-schwartz ! Indeed, the script needs to cover these cases as well. I am getting somewhere now [in the branch](https://github.com/martinpitt/umockdev/tree/auto-version), still fighting with getting this to green on all the OS tests. I'll report here once I have a fully working solution, as several other people are interested in this as well.
So, with a rather large amount of stubbornness I [figured it out](https://github.com/martinpitt/umockdev/pull/189). [Here is the commit](https://github.com/martinpitt/umockdev/commit/cf0d7b4cd0c74e99655c1bc57e2ca5f6ea99ba03), if you ignore the bits in tests/, it is generic (i.e. nothing umockdev specific).

Now I can change the release workflow, and finally the release process will be "push a signed tag with the release news in the tag body", and that takes care of the upstream and all Fedora releases.

Resolving patch:
diff --git a/mesonbuild/mdist.py b/mesonbuild/mdist.py
--- a/mesonbuild/mdist.py
+++ b/mesonbuild/mdist.py
@@ -20,8 +20,8 @@
 from dataclasses import dataclass
 from glob import glob
 from pathlib import Path
-from mesonbuild.environment import detect_ninja
-from mesonbuild.mesonlib import (MesonException, RealPathAction, quiet_git,
+from mesonbuild.environment import Environment, detect_ninja
+from mesonbuild.mesonlib import (MesonException, RealPathAction, get_meson_command, quiet_git,
                                  windows_proof_rmtree, setup_vsenv, OptionKey)
 from mesonbuild.msetup import add_arguments as msetup_argparse
 from mesonbuild.wrap import wrap
@@ -102,10 +102,12 @@ def create_dist(self, archives: T.List[str]) -> T.List[str]:
 
     def run_dist_scripts(self) -> None:
         assert os.path.isabs(self.distdir)
-        env = {}
-        env['MESON_DIST_ROOT'] = self.distdir
-        env['MESON_SOURCE_ROOT'] = self.src_root
-        env['MESON_BUILD_ROOT'] = self.bld_root
+        mesonrewrite = Environment.get_build_command() + ['rewrite']
+        env = {'MESON_DIST_ROOT': self.distdir,
+               'MESON_SOURCE_ROOT': self.src_root,
+               'MESON_BUILD_ROOT': self.bld_root,
+               'MESONREWRITE': ' '.join(shlex.quote(x) for x in mesonrewrite),
+               }
         for d in self.dist_scripts:
             if d.subproject and d.subproject not in self.subprojects:
                 continue
@@ -327,9 +329,6 @@ def run(options: argparse.Namespace) -> int:
     b = build.load(options.wd)
     need_vsenv = T.cast('bool', b.environment.coredata.get_option(OptionKey('vsenv')))
     setup_vsenv(need_vsenv)
-    # This import must be load delayed, otherwise it will get the default
-    # value of None.
-    from mesonbuild.mesonlib import get_meson_command
     src_root = b.environment.source_dir
     bld_root = b.environment.build_dir
     priv_dir = os.path.join(bld_root, 'meson-private')


Test patch:
diff --git a/test cases/unit/35 dist script/subprojects/sub/dist-script.py b/test cases/unit/35 dist script/subprojects/sub/dist-script.py
--- a/test cases/unit/35 dist script/subprojects/sub/dist-script.py	
+++ b/test cases/unit/35 dist script/subprojects/sub/dist-script.py	
@@ -2,11 +2,18 @@
 
 import os
 import pathlib
+import shlex
+import subprocess
 import sys
 
 assert sys.argv[1] == 'success'
 
 source_root = pathlib.Path(os.environ['MESON_PROJECT_DIST_ROOT'])
+mesonrewrite = shlex.split(os.environ['MESONREWRITE'])
+rewrite_cmd = ['kwargs', 'set', 'project', '/', 'version', 'release']
+
+subprocess.run([*mesonrewrite, '-s', source_root, *rewrite_cmd], check=True)
+
 modfile = source_root / 'prog.c'
 with modfile.open('w') as f:
     f.write('int main(){return 0;}')
diff --git a/test cases/unit/35 dist script/subprojects/sub/meson.build b/test cases/unit/35 dist script/subprojects/sub/meson.build
--- a/test cases/unit/35 dist script/subprojects/sub/meson.build	
+++ b/test cases/unit/35 dist script/subprojects/sub/meson.build	
@@ -1,4 +1,5 @@
-project('sub', 'c')
+project('sub', 'c',
+  version : 'vcs')
 
 if get_option('broken_dist_script')
   # Make sure we can add a dist script in a subproject, but it won't be run
@@ -8,4 +9,7 @@ else
   # The dist script replace prog.c with something that actually build.
   meson.add_dist_script('dist-script.py', 'success')
   executable('prog', 'prog.c')
+
+  versiontest = find_program('version-test.py')
+  test('dist version replacement', versiontest, args : meson.project_version())
 endif
diff --git a/test cases/unit/35 dist script/subprojects/sub/version-test.py b/test cases/unit/35 dist script/subprojects/sub/version-test.py
new file mode 100644
--- /dev/null
+++ b/test cases/unit/35 dist script/subprojects/sub/version-test.py	
@@ -0,0 +1,5 @@
+#!/usr/bin/env python3
+
+from sys import argv
+
+assert argv[1] == 'release'