Experimentations on Bazel: version.txt
Tags: bazel, git
Published: 2021-03-28 17:06:47

We'll try to create a useful rule that create a `version.txt` file with the output of `git describe --always --dirty`:

- If you use git annotated tag for version number, this command will display only the annotated tag when you are on it, else it will provide more info, like the short hash (if no annotated tag), or the number of commit since the last annotated tag, the short hash (prefixed by `g`) and maybe the suffix `dirty` if you have uncommitted changes.
- The `version.txt` could be used as input of other rule to include this metainfo into the build

```sh
# commit previous changes then
❯ git describe --always --dirty
5472a43
# add the first annoted tag
❯ git tag -a "0.1.0" -m ":bookmark: 0.1.0"
❯ git describe --always --dirty
0.1.0
```
{% link https://dev.to/davidb31/for-version-as-git-tag-use-annotated-tag-not-v-prefix-2i4c %}

Now create the rule, `git describe` need to access the content of folder `.git` to run and bazel require to be explicit about inputs and input files should be under the package. So we will create the rule into `./BUILD.bazel` into the parent folder, aka the workspace root folder.

```python
genrule(
    name = "version",
    srcs = [".git"],
    outs = ["version.txt"],
    cmd_bash = "git --git-dir=$(location :.git) describe --always --dirty |tee $@",
)
```
```sh
❯ bazel build //:version
INFO: Analyzed target //:version (1 packages loaded, 2 targets configured).
INFO: Found 1 target...
WARNING: /home/david/src/github.com/davidB/sandbox_bazel/BUILD.bazel:1:8: input '.git' to //:version is a directory; dependency checking of directories is unsound
INFO: From Executing genrule //:version:
0.1.0-dirty
Target //:version up-to-date:
  bazel-bin/version.txt
INFO: Elapsed time: 0.046s, Critical Path: 0.01s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
```

Except the warning seems to work. Commit this code and rerun bazel.

```sh
❯ git add BUILD.bazel
❯ git commit -m ":alembic: create version.txt"
[development b5fea4c] :alembic: create version.txt
 1 file changed, 6 insertions(+)
 create mode 100644 BUILD.bazel

❯ git describe --always --dirty
0.1.0-1-gb5fea4c

❯ bazel build //:version
INFO: Analyzed target //:version (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:version up-to-date:
  bazel-bin/version.txt
INFO: Elapsed time: 0.029s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
```

:-( bazel did NOT detect the change.

What is instead of declare `.git` as input we declare all the content of the folder with the function `glob` ?

```python
genrule(
    name = "version",
    srcs = glob([".git/**"]),
    outs = ["version.txt"],
    cmd_bash = "git --git-dir=$(location :.git) describe --always --dirty |tee $@",
)
```
```sh
❯ bazel build //:version
ERROR: /home/david/src/github.com/davidB/sandbox_bazel/BUILD.bazel:1:8: in cmd_bash attribute of genrule rule //:version: label '//:.git' in $(location) expression is not a declared prerequisite of this rule
ERROR: Analysis of target '//:version' failed; build aborted: Analysis of target '//:version' failed
INFO: Elapsed time: 0.116s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 111 targets configured)
```

Damned `location` if for a single file, and `.git` is not longer listed into srcs. Also because only the files under `.git` folder are symlinked and not the git working directory the command will always detect the working directory as "dirty". In this case maybe the we should request bazel to NOT run the rule into a sandbox, with `local = True` (as an exercice try to create the rule `envinfo_local` by duplicate `envinfo` and adapt it).

```python
genrule(
    name = "version",
    srcs = glob([".git/**"]),
    outs = ["version.txt"],
    cmd_bash = "git --git-dir=.git --no-pager describe --always --dirty |tee $@",
    local = True,
)
```

And test it

```sh
❯ bazel build //:version
INFO: Analyzed target //:version (1 packages loaded, 112 targets configured).
INFO: Found 1 target...
INFO: From Executing genrule //:version:
0.1.0-1-gb5fea4c-dirty
Target //:version up-to-date:
  bazel-bin/version.txt
INFO: Elapsed time: 0.073s, Critical Path: 0.01s
INFO: 2 processes: 1 internal, 1 local.
INFO: Build completed successfully, 2 total actions

❯ git add BUILD.bazel
❯ git commit -m ":alembic: tune version"
❯ git describe --always --dirty
0.1.0-2-ge453fae

❯ bazel build //:version
INFO: Analyzed target //:version (1 packages loaded, 115 targets configured).
INFO: Found 1 target...
INFO: From Executing genrule //:version:
0.1.0-2-ge453fae-dirty
Target //:version up-to-date:
  bazel-bin/version.txt
INFO: Elapsed time: 0.057s, Critical Path: 0.01s
INFO: 2 processes: 1 internal, 1 local.
INFO: Build completed successfully, 2 total actions

❯ git describe --always --dirty
0.1.0-2-ge453fae
```

It's better but we have the unexpected `-dirty` when running through bazel.

If we modify the `cmd_bash` to run `ls -al` before `git` we'll see that the folder where the git command is running is always a sandbox with symlinks to the folder (more symlinks than without `local = True` but symlinks plus other file and folders.

```sh
❯ bazel build //:version
INFO: Analyzed target //:version (1 packages loaded, 118 targets configured).
INFO: Found 1 target...
INFO: From Executing genrule //:version:
total 28
drwxr-xr-x 5 david david 4096 Mar 28 17:51 .
drwxr-xr-x 3 david david 4096 Mar 14 11:08 ..
lrwxrwxrwx 1 david david   61 Mar 28 17:51 .bazelversion -> /home/david/src/github.com/davidB/sandbox_bazel/.bazelversion
lrwxrwxrwx 1 david david   52 Mar 28 17:51 .git -> /home/david/src/github.com/davidB/sandbox_bazel/.git
lrwxrwxrwx 1 david david   55 Mar 28 17:51 .github -> /home/david/src/github.com/davidB/sandbox_bazel/.github
lrwxrwxrwx 1 david david   58 Mar 28 17:51 .gitignore -> /home/david/src/github.com/davidB/sandbox_bazel/.gitignore
lrwxrwxrwx 1 david david   59 Mar 28 17:51 BUILD.bazel -> /home/david/src/github.com/davidB/sandbox_bazel/BUILD.bazel
lrwxrwxrwx 1 david david   63 Mar 28 17:51 WORKSPACE.bazel -> /home/david/src/github.com/davidB/sandbox_bazel/WORKSPACE.bazel
drwxr-xr-x 5 david david 4096 Mar 28 17:51 bazel-out
lrwxrwxrwx 1 david david   59 Mar 28 17:51 exp_genrule -> /home/david/src/github.com/davidB/sandbox_bazel/exp_genrule
drwxr-xr-x 2 david david 4096 Mar 28 17:51 external
drwx------ 3 david david 4096 Mar 28 17:51 local-spawn-runner.1529500632871132789
0.1.0-2-ge453fae-dirty
Target //:version up-to-date:
  bazel-bin/version.txt
INFO: Elapsed time: 0.098s, Critical Path: 0.01s
INFO: 2 processes: 1 internal, 1 local.
INFO: Build completed successfully, 2 total actions
```

Maybe it's time to find an other solution. We can remove the flags `--dirty`, but it means that uncommitted change will be able to generate code with invalid VERSION info, and also to destroy part of the trust into this information to debug,...

## Workspace status

After back in the doc of basel, we can find a section [Workspace status](https://docs.bazel.build/versions/master/user-manual.html#workspace_status)

> Use these options to "stamp" Bazel-built binaries: to embed additional information into the binaries, such as the source control revision or other workspace-related information. You can use this mechanism with rules that support the stamp attribute, such as genrule, cc_binary, and more...

Seams to match our goal, let's try it. For details, why,... please read the documentation linked above, but as overlook we can provide a script to bazel that should generate key/value pair(s) the key starting by `STABLE_` will be stored into a file `bazel-out/stable-status.txt` and other into `bazel-out/volatile-status.txt`. Both files are available to action/rules. Action with attribe `stamp = True` are retriggered if change is detected into `bazel-out/stable-status.txt`.

```sh
# check if the files already exists
❯ cat bazel-out/stable-status.txt
BUILD_EMBED_LABEL 
BUILD_HOST xxxxxxxxx
BUILD_USER david

❯ cat bazel-out/volatile-status.txt
BUILD_TIMESTAMP 1616946664
```

Like for the rest of the tutorial, use a linux/macOs only bash script for workspace_status, `tools/workspace_status.sh` (do not forgot the set it as executable):

```sh
mkdir tools
touch tools/workspace_status.sh
chmod +x tools/workspace_status.sh
```

Define the content of `tools/workspace_status.sh` as:
```sh
#!/bin/bash

echo "STABLE_BUILD_GIT_DESCRIBE $(git --no-pager describe --always --dirty)"
```

For quick test run with the tools and visual check of `stable-status.txt`:

```sh
❯ bazel build //:version --workspace_status_command "tools/workspace_status.sh"
INFO: Analyzed target //:version (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:version up-to-date:
  bazel-bin/version.txt
INFO: Elapsed time: 0.049s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action

❯ cat bazel-out/stable-status.txt
BUILD_EMBED_LABEL 
BUILD_HOST dwaynebox6
BUILD_USER david
STABLE_BUILD_GIT_DESCRIBE 0.1.0-2-ge453fae
```

The version number doesn't include the `-dirty` because dirty flag ignores untracked files like the new file we created (it's not perfect).

So no let modify the version rule to use info from `bazel-out/stable-status.txt`:

```python
genrule(
    name = "version",
    outs = ["version.txt"],
    # alternative: "sed -n 's/STABLE_BUILD_GIT_DESCRIBE //p' bazel-out/stable-status.txt"
    cmd_bash = "grep -Po '^STABLE_BUILD_GIT_DESCRIBE\\s+\\K.*' bazel-out/stable-status.txt |tee $@",
    stamp = True,
)
```

```sh
❯ bazel build //:version --workspace_status_command "tools/workspace_status.sh"
INFO: Analyzed target //:version (1 packages loaded, 1 target configured).
INFO: Found 1 target...
INFO: From Executing genrule //:version:
0.1.0-2-ge453fae-dirty
Target //:version up-to-date:
  bazel-bin/version.txt
INFO: Elapsed time: 0.079s, Critical Path: 0.02s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
```

Before doing a test without uncommitted change, we'll add register `--workspace_status_command "tools/workspace_status.sh"` as a default flags for every `bazel build` calls via `.bazelrc`:

```sh
echo 'build --workspace_status_command "tools/workspace_status.sh"' >>.bazelrc
```

No its time to commit and test if dirty go away.

```sh
❯ git add BUILD.bazel tools/* .bazelrc
❯ git commit -m ":recycle: rewrite version using workspace status"

❯ bazel build //:version
INFO: Analyzed target //:version (1 packages loaded, 1 target configured).
INFO: Found 1 target...
INFO: From Executing genrule //:version:
0.1.0-3-g95b84e9
Target //:version up-to-date:
  bazel-bin/version.txt
INFO: Elapsed time: 0.087s, Critical Path: 0.02s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
```
:tada:

## Conclusion

It was a long experimentation to have this `version.txt` file that we can (re)use in real project. We gain better understanding and experience with `genrule` and we discover `local` and `stamp` attribute.

The sandbox_bazel is hosted on github (not with the same history, due to errors), use tag to have the expected view at end of article: [article/3_versiontxt](https://github.com/davidB/sandbox_bazel/tree/article/3_versiontxt).