Repository: mesonbuild/meson

Issue:
Simplify version extraction from a file
Providing some way to replace the following pattern would be welcome:

```meson
project('foo', 'c',
  version: run_command(
    find_program('python'), '-c', 'import sys;print(open(sys.argv[1]).read())', files('VERSION')
  ).stdout().strip(),
)
```

Drawbacks with this current approach:
- we can't move the `run_command()` in a variable before `project()`, which likely implies this is currently a nasty hack where we're probably not supposed to do that
- some system may have `python3` and not `python`, and the other way around is also true
- using `run_command(find_program('cat'), ...)` instead of `python` would be fatal on Windows
- having a dedicated Python script is quite an overhead, and doesn't address the external subprocess hack to just read a file


**Rationale**

So the current scenario we're in is the following: we are managing our releases through Github, where making a release is basically simply pushing a tag. So we have the following primitive `make-release.sh` script:

```sh
#!/bin/sh

#
# Release process:
# 1. on a clean git state, run this script with the new version as argument
# 2. check the last commit and tag
# 3. git push && git push --tags
#

set -eu

if [ $# -ne 1 ]; then
	echo "Usage $0 <major.minor.micro>"
	exit 1
fi

cd "$(dirname $0)"

if ! git diff-index --quiet HEAD; then
	echo "Git index is not clean"
	exit 1
fi

set -x
VERSION="$1"
echo "$VERSION" > VERSION
git add VERSION
git commit -m "Release $VERSION"
git tag "v$VERSION"
```

The `VERSION` file is basically the channel of communication between this script and `meson.build`

**Alternative considered**

- hot-patching `meson.build` within the release script, but it's not exactly cleaner to say the least
- using `dist` scripts with `meson`; unfortunately we're running git commands at an unfortunate timing, and it involves quite some logistic
- requesting in the release process to patch the `meson.build` with the new version, and then extracting the version with the meson introspection with tools like `jq` in the release script; this creates an annoying dependency to `jq`, and it also makes the release process much more clumsy than just running a script


**How it could look like**

Here are a two usage suggestions that would simplify and make this workflow much more reliable, which I believe are not very intrusive:

- `project('foo', 'c', version: files('VERSION'))` (it's part of the source files after all)
- `project('foo', 'c', version: meson.get_file_content('VERSION'))`
Introduce `meson.read_file` to read a file as a string
This is an RFC patch that should fix #7890

This patch introdocues the ability to read files at configure time, but
has some restrictions:
    - binary files are not supported (I don't think this will prove a
    problem, and if people are wanting to do something with binary
    files, they should probably be shelling out to their own script.
    - Only files outside the build directory allowed. This limitation
      should prevent build loops
Given that reading an arbitrary file at configure time can affect the
configuration in almost arbitrary ways, meson should force a reconfigure
when the given file changes. I made this non-configurable, but this is
probably better decided by someone more familiar with meson's core
principles.


Discussion:
Alternatively:

```meson
project('foo', 'c', version_file: 'VERSION')
```

Specifying both `version` and `version_file` should be a hard error.
@jpakkane, we had discussed this at some length I'm the past, and had come up with both of these solutions. Do your prefer version_file? I'd really like this for Mesa
The thing I like about version_file is that it is more readable and higher level, i.e. "this is what I want" vs "do these steps to achieve something".
Though if the file is always called VERSION, then having to write that seems superfluous. Maybe we could just always check for it and use it if it's there? Having both a version file and `version` kwarg should still be a hard error.

That's a bit too implicit and magical for me. And it would be a backwards compatibility problem for mesa
Adopting something like cmake's [`FILE(READ...`](https://cmake.org/cmake/help/v3.19/command/file.html?highlight=read#read) is less magical and is much more flexible for other uses. I can imagine a fair few scenarios where reading the contents of a file might be useful in meson for configuration and trivially enables the behaviour needed by Mesa

I'd definitely be happy to see the following (from the example above) for my own purposes

```meson
project('foo', 'c', version: meson.get_file_content('VERSION')).strip())
```

cmake's interfaces for files are pretty unpalatable to work with, but I think this is an excellent way to solve the problem in a general way.

@jpakkane I'd be happy to provide a patch for the above. Let me know if that's the route you want to take and I'll push a PR
Just by way of reference for anyone needing to support this on older versions of meson, while "cat" doesn't work on windows, "more" works fine as an acceptable substitute for short files like a versionfile. Therefore one can do the following:

```
project(...., version: run_command(find_program('cat', 'more'),
                files('VERSION')).stdout().strip(), ...)

```
Horrible, yes, but it also works. :-)
For my own use case at work: we need to be able to parse the version from a Debian changelog file in a cross-platform way (yay), so for us it would be a better option to have `meson.get_file_content()` over a `version_file` kwarg.

But feel free to add both, we solved it by using a similar workaround as the description of this issue :slightly_smiling_face: 
> This looks really useful, and generally correct. There's a bunch of type enforcement that should be going on that isn't, which is easy to overlook because so much of the existing code doesn't do it. Thanks for writing the reference-manual documentation. It might be nice to add something to the `How do I..." page as well.

Yes. I think the "how do I" would be useful - including the example from the linked #7890 is probably a good bet. I'll add a section.
> 
> I noticed that `mesonlib.File` objects (returned by the mesonlang `files()` function) aren't supported, but they should be. 
I've not actually used them yet myself, but I'll take a look and add them to the next iteration

> I also wonder if this would be better called `read_file`? I know that's a "what color to paint the bikeshed kind of question".

Yes. I'm not keen on the name either. `read_file` probably *is* better. I just copied from the linked ticket


On Tue Nov 3, 2020 at 11:22 PM GMT, Dylan Baker wrote:
> @dcbaker commented on this pull request.
>
>
>
> > +        '''
> + if len(args) != 1:
> + raise InterpreterException('expected single positional <path> arg')
> + path = args[0]
> + encoding = kwargs.get('encoding', 'utf-8')
> + allow_absolute = kwargs.get('allow_absolute', False)
> + source_dir = self.interpreter.environment.get_source_dir()
> + build_dir = self.interpreter.environment.get_build_dir()
> + if not allow_absolute and os.path.isabs(path):
> + raise InterpreterException('paths must be relative')
> + path = os.path.join(source_dir, path)
> + if mesonlib.path_is_in_root(Path(path), Path(build_dir),
> resolve=True):
> + raise InterpreterException('path must not be in the build tree')
> + try:
> + with open(path, 'rb') as f:
> + data = f.read().decode(encoding)
>
> But you raise an uncaught exception if the decode fails, so no
> dependency will be added. Unless I'm missing something.
>
Yep. I saw it's cleaner just to do this after the block. I was just being
intransigent
My next iteration has this:

```python

        try:
            with open(path, 'r', encoding=encoding) as f:
                data = f.read()
        except UnicodeDecodeError:
            raise InterpreterException(f'decoding failed for {path}')
        # Reconfigure meson when this file changes as it can contain data used
        # by any part of the build configuration (e.g. `project(..., version:
        # meson.read_file('VERSION')` or `configure_file(...)`
        self.interpreter.add_build_def_file(path)
        return data
```
> --
> You are receiving this because you authored the thread.
> Reply to this email directly or view it on GitHub:
> https://github.com/mesonbuild/meson/pull/7931#discussion_r517013110


That looks like what I'd expect. I'm just very cautious putting things in try blocks, I've been bitten many times by a large try block catching things I didn't expect it to.
Ah, you need to quote "FileOrString", it's only declared when type checking
This functionality is useful on its own but if we look at the main usage:

> project('myproj', 'c', 'cpp', version: meson.read_file('VERSION').strip())

This is utilitarian (as in it works) but it is not particularly pretty. Since this is a fairly common thing to want to do, it would be better if the UI was smoother (on the eyes at least).

Some alternative implementations:

- If the string passed in to `version` points to a file on disk (i.e. has value `"VERSION"`) then read the contents of that file as the version (as here)
- add a `version_file` kwarg, and if both `version` and `version_file` are specified Meson reads both and exits with a hard error if they have different values (I'm told this is needed for backwards compatibility for some projects)

In both of these cases the actual functionality of loading files should then either go to the `fs` module or as a method on a `file` object.
Oh, and since strings in Meson are UTF-8, that is the only text encoding that should be supported. Encoding problems should be a hard error.
Note that if this is added to the `fs` module you should be able to do something like:

```meson
version: import('fs').load_file('VERSION').strip()
```

@jpakkane, I disagree on the encoding. This may be useful for wraps, and we had to relax the encoding of configure file inputs because of wraps, and some projects using weird encodings. I'd say let's default to utf-8, but leave support for non-utf-8
Well the Meson way is to only support utf-8 at start, make everything else a hard error and wait for someone to complain with an actual use case.
Based on various discussions (mostly on IRC) it seems that people prefer to have the higher level `version_file` kwarg for `project`, so let's go with that.

- `version_file`s value should be a string, like `version_file: 'VERSION'`
- if both `version` and `version_file` are specified they *must* have the same version value, if they are different then it should be a hard error
- the version file should be only one line, and it may have a newline at the end of it
- the version file should be added to the list of files, who cause reconfigurations when they are edited

Having a separate general purpose mechanism of reading file contents is also ok, but it should be done in a separate PR and the functionality should be in the `fs` module.
> Based on various discussions (mostly on IRC) it seems that people prefer
> to have the higher level `version_file` kwarg for `project`, so let's go
> with that.
>
> - if both `version` and `version_file` are specified they *must* have
> the same version value, if they are different then it should be a hard
> error
> - the version file should be only one line, and it may have a newline at
> the end of it
> - the version file should be added to the list of files, who cause
> reconfigurations when they are edited
>
> Having a separate general purpose mechanism of reading file contents is
> also ok, but it should be done in a separate PR and the functionality
> should be in the `fs` module.
>
Sounds reasonable. I should have some bandwidth to do this in a week or so.

> Based on various discussions (mostly on IRC) it seems that people prefer to have the higher level `version_file` kwarg for `project`, so let's go with that.
> 
>     * `version_file`s value should be a string, like `version_file: 'VERSION'`
> 
>     * if both `version` and `version_file` are specified they _must_ have the same version value, if they are different then it should be a hard error

Just in case it hasn't been suggested yet (sorry if I missed the discussion): why not just allow:
```
project('foo', version: file('VERSION'), ...)
```
as well, without adding a new 'version_file` kwarg?

(edit: arguably `version_file` is slightly more discoverable, but then you have to define the behaviour if both are set and check that)
That works too, though it should be `version: files('VERSION')` as there is no `file` function. Perhaps we should add one?
I'd really prefer the `version : files(...)` to having a new keyword, but if we have the new keyword I'd prefer to make passing both an error. I don't see any legitimate reason to pass both at the same time.

Resolving patch:
diff --git a/mesonbuild/interpreter.py b/mesonbuild/interpreter.py
--- a/mesonbuild/interpreter.py
+++ b/mesonbuild/interpreter.py
@@ -1,4 +1,4 @@
-# Copyright 2012-2019 The Meson development team
+# Copyright 2012-2021 The Meson development team
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
@@ -3169,9 +3169,29 @@ def func_project(self, node, args, kwargs):
         if not self.is_subproject():
             self.build.project_name = proj_name
         self.active_projectname = proj_name
-        self.project_version = kwargs.get('version', 'undefined')
-        if not isinstance(self.project_version, str):
-            raise InvalidCode('The version keyword argument must be a string.')
+        version = kwargs.get('version', 'undefined')
+        if isinstance(version, list):
+            if len(version) != 1:
+                raise InvalidCode('Version argument is an array with more than one entry.')
+            version = version[0]
+        if isinstance(version, mesonlib.File):
+            FeatureNew.single_use('version from file', '0.57.0', self.subproject)
+            self.add_build_def_file(version)
+            ifname = version.absolute_path(self.environment.source_dir,
+                                           self.environment.build_dir)
+            try:
+                ver_data = Path(ifname).read_text(encoding='utf-8').split('\n')
+            except FileNotFoundError:
+                raise InterpreterException('Version file not found.')
+            if len(ver_data) == 2 and ver_data[1] == '':
+                ver_data = ver_data[0:1]
+            if len(ver_data) != 1:
+                raise InterpreterException('Version file must contain exactly one line of text.')
+            self.project_version = ver_data[0]
+        elif isinstance(version, str):
+            self.project_version = version
+        else:
+            raise InvalidCode('The version keyword argument must be a string or a file.')
         if self.build.project_version is None:
             self.build.project_version = self.project_version
         proj_license = mesonlib.stringlistify(kwargs.get('license', 'unknown'))
diff --git a/run_unittests.py b/run_unittests.py
--- a/run_unittests.py
+++ b/run_unittests.py
@@ -5212,6 +5212,12 @@ def test_nested_cmake_rebuild(self) -> None:
             os.utime(str(cmakefile))
         self.assertReconfiguredBuildIsNoop()
 
+    def test_version_file(self):
+        srcdir = os.path.join(self.common_test_dir, '2 cpp')
+        self.init(srcdir)
+        projinfo = self.introspect('--projectinfo')
+        self.assertEqual(projinfo['version'], '1.0.0')
+
 
 class FailureTests(BasePlatformTests):
     '''


Test patch:
diff --git a/test cases/common/2 cpp/VERSION b/test cases/common/2 cpp/VERSION
new file mode 100644
--- /dev/null
+++ b/test cases/common/2 cpp/VERSION	
@@ -0,0 +1 @@
+1.0.0
diff --git a/test cases/common/2 cpp/meson.build b/test cases/common/2 cpp/meson.build
--- a/test cases/common/2 cpp/meson.build	
+++ b/test cases/common/2 cpp/meson.build	
@@ -1,4 +1,4 @@
-project('c++ test', 'cpp')
+project('c++ test', 'cpp', version: files('VERSION'))
 
 cpp = meson.get_compiler('cpp')
 if cpp.get_id() == 'intel'