CLI commands with google/subcommands
Tags: howto, go, cli
Published: 2022-03-06 14:04:50

---
title: CLI commands with google/subcommands
published: true
description: "How to create CLI commands using google/subcommands ?"
tags: [howto,golang,cli]
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tr7lickxymigwnc6axhq.jpeg
---

How to create CLI programs with multiple commands, flags and subcommands, and do it fast and simply?

[google/subcommands](https://github.com/google/subcommands) makes it a breeze. Let us see how.

(original location: https://osinet.fr/go/en/articles/cli-google-subcommands/)

## The problem scope

First, let us agree on the topics covered here.

Any Go program is - at least in concept - run from a command line, whether it be an actual shell command line or some automation tool, using the `execve(2)` system call.

To define the specific words used in this tutorial, let us consider a typical *NIX command:

```
docker -D container inspect -s some_container
```

- `docker` is the _program_, or _application_
- `-D` is a _global flag_
- `container` is a _root_ or _top-level command_
- `inspect` is a _subcommand_ of the `container` command; more generally, it is
  also a _command_ too, but a level 2 one.
- `-s` is a _local flag_ for the `container` subcommand
- `some_container` is an _argument_ to the `inspect` subcommand

In order to interpret that kind of structure, Go code needs to work from a single global variable, `os.Args`, the first element of which contains the absolute path to the program being run, while the other elements are copies of the arguments passed when starting the program.

To avoid having to redo that kind of very formalized work in every other program, various libraries/modules exist, with the most popular being:

| Module                                                      |     ☆ |    ⑂ | Direct deps. |
|-------------------------------------------------------------|------:|-----:|:------------:|
| [spf13/cobra](https://github.com/spf13/cobra)               | 25400 | 2200 |      5       |
| [urfave/cli](https://github.com/urfave/cli)                 | 17400 | 1500 |      3       |
| [alecthomas/kingpin](https://github.com/alecthomas/kingpin) |  3200 |  240 |      5       |
| [google/subcommands](https://github.com/google/subcommands) |   594 |   51 |      0       |

The first three libraries are by far the best known and the richest in terms of functionality, but that comes with multiple dependencies, sometimes quite a lot (3136 for Cobra 1.3). While this is easily justified in humongous projects like [Docker](https://github.com/moby/moby/blob/master/cli/cobra.go) or [Kubernetes](https://github.com/kubernetes/kubernetes/blob/master/go.mod#L74), such a complexity is not as acceptable for typical projects like microservices or [small, sharp tools](https://pragprog.com/titles/bhcldev/small-sharp-software-tools/) for the CLI.

This is where google/subcommands shines, providing support for most commonly needed command creation tools without requiring any dependency, in a single module around 500 lines only, very easy to use in any application and with a negligible size cost even for the tiniest projects. This tutorial is about how to use it from the simplest to the most advanced scenario.

All along the successive refinement levels in this demo, we are going to be adding features to the same example code, which you can find on https://github.com/fgm/subcommands_demo, with every branch matching the eponymous section in this tutorial, containing 100% of the source code for the level being discussed, as well as a `Makefile` configured to demonstrate just the specifics of that level.

This tutorial covers 100% of the functionality in google/subcommands, hence its overall length ; you will however be able to create a complete application as early as level 1.2 : later levels describe features meant for projects ever more complex and demanding.

## Level 1 : simple commands

We shall start by creating a couple of _top-level commands_, laying out our code in two directories: the project root and `cmd/`, with the following main files :

| Path          | Contents                      |
|---------------|-------------------------------|
| `go.mod`      | the module description file   |
| `main.go`     | the program entry point       |
| `Makefile`    | the build tasks file          |
| `cmd/`        | the commands directory        |
| `cmd/root.go` | the code running the commands |

**A word of advice:** in each branch, the `make` command runs a different default sequence of tasks, meant to illustrate the changes brought by the current branch: you might wish to use that command on your own machine any time you change branch, to see the impact the code changes actually have.

### 1.1 Enabling builtin commands

_The code for that level is available on [level1.1-builtin_commands](https://github.com/fgm/subcommands_demo/tree/level1.1-builtin_commands)._

At this step, the code is just made up of two files: the application entry point in `main.go`:

```go
func main() {
	ctx := context.Background()
	sts := cmd.Execute(ctx) // Runs the designated command on the CLI
	os.Exit(int(sts))       // Returns the command result to the program caller
}
```

…and the code triggering execution of the commands, in `cmd/root.go` :

```go
// Execute sets up the command chain and runs it.
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]subcommands.Command{
		subcommands.CommandsCommand(), // Implement "commands"
		subcommands.FlagsCommand(),    // Implement "flags"
		subcommands.HelpCommand(),     // Implement "help"
	} {
		subcommands.Register(command, "")
	}

	flag.Parse()
	return subcommands.Execute(ctx)
}
```

The above code registers the three optional commands provided by `subcommands` proper, which enables listing existing commands and flags, as well as providing CLI help, then it parses global flags and ends up passing control to the actual command selected by the CLI arguments, and returning its result.

Our application scaffold is now ready and supports commands `commands`, `flags`, et `help`.

```text
$ go run . help
Usage: subcommands_demo <flags> <subcommand> <subcommand args>

      Subcommands:
      commands         list all command names
      flags            describe all known top-level flags
      help             describe subcommands and their syntax
$
```

We can now create our first custom commands.


### 1.2 Writing custom commands

_The code for that level is available on [level1.2-custom_commands](https://github.com/fgm/subcommands_demo/tree/level1.2-custom_commands._

The best practices with custom commands is to have one file per command. This eases code navigation and comparison of commands between each other to ensure consistency of practices. We shall therefore add two files, one for each of the two top-level commands we are adding:

| Path          | Contents                                                         |
|---------------|------------------------------------------------------------------|
| `cmd/top1.go` | the `top1` command, showing a fixed message                      |
| `cmd/top2.go` | the `top2` command, showing a message based on the CLI arguments |

For google/subcommands, a command is provided by a variable, the type of which  implements the [subcommands.Command](https://pkg.go.dev/github.com/google/subcommands#Command) interface. Our two files will therefore be very similar to each other, based on the structure in file `cmd/top2.go` :

```go 
type top2 struct{}

func (cmd *top2) Name() string {
	return "top2"
}

func (cmd *top2) Synopsis() string {
	return "top2 is an example top-level custom command with arguments"
}

func (cmd *top2) Usage() string {
	return fmt.Sprintf("%s arg1 arg2 ...", cmd.Name())
}

func (cmd *top2) SetFlags(fs *flag.FlagSet) {}

func (cmd *top2) Execute(_ context.Context, fs *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	// Notice how the command line arguments are taken on the flag set, not on the variadic.
	fmt.Printf("In %s %v\n", cmd.Name(), fs.Args())
	return subcommands.ExitSuccess
}
```

For now, `top2` just implements `subcommands.Command` and does nothing more:

- `top2.Name()` returns the name of the command
- `top2.Synopsis` returns a single-line summary of the command description
- `top2.Usage()` returns an example of how to use the command, and may spread
  over multiple text lines. The returned value is used by the `help` builtin command.
- `top2.SetFlags()` defines the _local flags_ on the command: none for now.
- `top2.Execute()` is the actual command implementation.
  - the `context.Context` arguments supports passing context values, and using
    timeout and cancellation capabilities, which will be available to subcommands,
    just like in a `http.Handler`.
  - the `flag.FlagSet` holds the _local_ flag definitions, actual values, and
    the command arguments.
    - when running that method, the name of the program, the global flags, as
      well as the command local flags have been parsed, and `fs.Args()` holds
      the command arguments with no extras.
    - if we run the program as `demo top2 hello world`, `fs.Args()` will contain
      `[]string{"hello", "world"}`.
  - the third parameter is not used in this scenario.

Now that we have created our _two_ commands, we can register all _three_ of them with google/subcommands: `top1`, `top2`, and a `1` _alias_ for `top1`. We can perform this by adding instances of the commands to the `subcommands.Register`
call besides `commands`, `flags` and `help`, in `root.go` :

```go 
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]subcommands.Command{
		subcommands.CommandsCommand(),   // Implement "commands"
		subcommands.FlagsCommand(),      // Implement "flags"
		subcommands.HelpCommand(),       // Implement "help"
		&top1{},                         // Our first top-level command, without args
		&top2{},                         // Our second top-level command, with args
		subcommands.Alias("1", &top1{}), // An alias for our top1 command
	} {
		subcommands.Register(command, "")
	}
```

Once this is done, our program supports these two commands, as well as a `"1"` alias for command `top1`.

Many applications will not anything more when it comes to commands handling.

```text
$ go run . commands
commands
flags
help
top1
top2
1
$ go run . top2 hello world
In top2 [hello world]
$
```

### 1.3 Passing non-CLI arguments to commands

_The code for that level is available on [level1.3-non_cli_arguments](https://github.com/fgm/subcommands_demo/tree/level1.3-non_cli_arguments)._

In the previous example, the `top2` command takes arguments passed from the CLI.

Command code may however need extra arguments passed to all commands regardless of the CLI arguments, like a logger, an authentication service, etc.

That is where the variadic argument to `subcommands.Execute` enters into play, taking any number of values of any type, to be received by the `Command.Execute()` implementations on their variadic parameter. We shall be using it to pass two values :

```go 
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]subcommands.Command{
		/* ...snip... */
	flag.Parse()
	return subcommands.Execute(ctx, "meaning", 42)
```

Here they are handled in `top1` :

```go
func (cmd *top1) Execute(_ context.Context, _ *flag.FlagSet, args ...interface{}) subcommands.ExitStatus {
	// The variadic arguments are the ones passed to subcommands.Execute().
	// Unlike the CLI args, they are always a []interface{}.
	fmt.Printf("In %s.\nNon-CLI args: %#v\n", cmd.Name(), args)
	return subcommands.ExitSuccess
}
```

```text
$ go run . top1
In top1.
Non-CLI args: []interface {}{"meaning", 42}
$
```

The salient issue here is how, as a command author, you need to coordinates the types and order of values passed in that variadic arguments, since they are passed as `interface{}` (aka `any` starting with Go 1.18), which will usually mean a type assertion in the `Execute` method to get a ready-to-use value.

### 1.4 Grouping commands

_The code for that level is available on [level1.4-command_groups](https://github.com/fgm/subcommands_demo/tree/level1.4-command_groups)._

When a project starts to grow and get ever more commands, grouping them by purpose may  become useful. This is what the second, `string`, parameter of `subcommands.Register` is for. We use it in `root.go`, to associate each registered command to a given [subcommands.Group](https://pkg.go.dev/github.com/google/subcommands#CommandGroup).

In this example, we shall separate commands provided by google/subcommands, in the `help` group, from our custom commands, in the `top` group.

```go
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		group string
		subcommands.Command
	}{
		{"help", subcommands.CommandsCommand()},  // Implement "commands"
		{"help", subcommands.FlagsCommand()},     // Implement "flags"
		{"help", subcommands.HelpCommand()},      // Implement "help"
		{"top", &top1{}},                         // Our first top-level command, without args
		{"top", &top2{}},                         // Our second top-level command, with args
		{"top", subcommands.Alias("1", &top1{})}, // An alias for our top1 command

	} {
		subcommands.Register(command.Command, command.group)
	}
```

The `help` command now displays our commands in these two different groups :
```text
$ go run . help
Usage: subcommands_demo <flags> <subcommand> <subcommand args>

Subcommands for help:
        commands         list all command names
        flags            describe all known top-level flags
        help             describe subcommands and their syntax

Subcommands for top:
        top1, 1          top1 is an example top-level custom command without arguments
        top2             top2 is an example top-level custom command with arguments
$
```
### 1.5 Adding global or local flags

_The code for that level is available on [level1.5-flags](https://github.com/fgm/subcommands_demo/tree/level1.5-flags)._

Many programs use a few _global_ flags, like a boolean `-v` used to increase the verbosity of all commands ; and some also use _local_ flags allowing individual commands to adjust their own behaviour.

Let us start with the typical global boolean `-v` flag, and implementing it in our two existing commands. We can do this with the stdlib package `flag`, with which google/subcommands has a hidden integration.

To make sure that flag will be available to all commands and subcommands, we add its value to the context. Following best Go practices, we define an [unexported specific context key type](https://medium.com/@matryer/context-keys-in-go-5312346a868d), and a variable of that type, in our `root.go` file :

```go
type verboseKey struct{}

var VerboseKey = verboseKey{}
```

Now, in our `Execute` function, we define that the flag, parse the program CLI arguments, and add the parsed flag value to the context using that key :

```go 
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		/* ...snip... */
  }

	verbose := flag.Bool("v", false, "Be more verbose")
	flag.Parse()
	ctx = context.WithValue(ctx, VerboseKey, *verbose)
```

Commands can now fetch the flag value from the context, as in this example taken from `top1.go` :

```go 
func (cmd *top1) Execute(ctx context.Context, _ *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	if ctx.Value(VerboseKey).(bool) {
		fmt.Printf("In %s.\n", cmd.Name())
	}
```

Since we defined the flag type as boolean, and the key is of a unique unexported type, there can be no collision on the context value even in a large program, which means the type assertion will never fail.

For that _global_ flag, we did not knowingly use google/subcommands at all, and it is correctly recognized by builtin command `flags`, but does not appear in the builtin help provided by the `help` command :

<a id="level15-help" ></a>
```text 
$ go run . help
Usage: subcommands_demo <flags> <subcommand> <subcommand args>

Subcommands for help:
        commands         list all command names
        flags            describe all known top-level flags
        help             describe subcommands and their syntax

Subcommands for top:
        top1, 1          top1 is an example top-level custom command without arguments
        top2             top2 is an example top-level custom command with arguments


Use "subcommands_demo flags" for a list of top-level flags
$ go run . flags
  -v    Be more verbose
$
```

Now, let us define a _local_ flag on command `top1`. Since that flag will be specific to the command, it will be listed after the command name in the command line arguments, preventing a simple use of the `flag` package procedural API like we just did for global flags.

Local flags are defined on each command by implementing their `SetFlags` method to store the parsed flag value on a command instance field.

To that effect, we shall add a field to the `top1` type implementing our command, in the `top1.go` file, to store the parsed flag value, which will be a `string` field in this example.

```go
type top1 struct {
	prefix string
}
```

We then implement the `top1.SetFlags` method to define that flag and store its parsed value on the command instance :

```go
func (cmd *top1) SetFlags(fs *flag.FlagSet) {
	fs.StringVar(&cmd.prefix, "prefix", "", "Add a prefix to the result")
}

func (cmd *top1) Execute(ctx context.Context, _ *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	if ctx.Value(VerboseKey).(bool) {
		fmt.Printf("In %s.\n", cmd.Name())
	}
	fmt.Println(strings.Join(append([]string{cmd.prefix}, "hello"), ": "))
	return subcommands.ExitSuccess
}
```

The `help` command is now aware of our local flag :

```text
$ go run . help top1
top1  -prefix string
        Add a prefix to the result
$
```

When `subcommands.Execute` prepares to invoke our `top1` command instance, it starts by creating a new empty `flag.FlagSet` from the remaining arguments on the current one, which it passed to the `SetFlags` method on the command instance. That defines flags pointing to fields on said command instance, so that parsing will field values on it. These fields are thereafter available on the instance
when eventually running the `top1.Execute` method, allowing the method code to access their values as plain typed fields on the method receiver, as demonstrated on line 40 for `cmd.prefix`.

```text
$ go run . top1 -prefix today
today: hello
$
```

Another possible mechanism enabled by this process would be for the method implementation to just drop the flag, not pointing it anywhere, then extract the value from the `fs *FlagSet` argument in `top1.Execute`, as done below, but that leads to code which is much less readable than the recommended method above.

```go
func (cmd *top1) SetFlags(fs *flag.FlagSet) {
  _ = fs.String("prefix", "", "Add a prefix to the result")
}

func (cmd *top1) Execute(ctx context.Context, fs *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	prefix := fs.Lookup("prefix").Value.(fmt.Stringer).String()
	fmt.Println(strings.Join(append([]string{cmd.prefix}, "hello"), ": "))
	return subcommands.ExitSuccess
}
```

### 1.6 Labeling flags as important

_The code for that level is available on [level1.6-important_flags](https://github.com/fgm/subcommands_demo/tree/level1.6-important_flags)._

Some flags may be frequently used, making them worthy of being reported in the `help` command output.

Let us describe our `-v` global flag as an important one, and our code also include an unimportant boolean `-debug` flag. We can inform google/subcommands of that difference when declaring the flags in our `root.go` file :

```go
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		/* ...snip... */
	}

	debug := flag.Bool("debug", false, "Show debug information")
	verbose := flag.Bool("v", false, "Be more verbose")
	subcommands.ImportantFlag("v")
	flag.Parse()
```

Without the `subcommands.ImportantFlag("v")` statement, the `help` command does not document any top-level flag, as we saw in the [level1.5 example](./#level15-help) above, where the help command mentions that top-level flags exist, but does not list any.

When we add this statement, however, the now _important_ `-v` flag is included in the `help` output, while the ordinary `-debug` flag remains unlisted.

```text 
$ go run . help
Usage: subcommands_demo <flags> <subcommand> <subcommand args>

Subcommands for help:
        commands         list all command names
        flags            describe all known top-level flags
        help             describe subcommands and their syntax

Subcommands for top:
        top1, 1          top1 is an example top-level custom command without arguments
        top2             top2 is an example top-level custom command with arguments


Top-level flags (use "subcommands_demo flags" for a full list):
  -v=false: Be more verbose
$
```

Both flags are listed indiscriminately by command `flags`, since its express purpose is to document flags  :

```text
$ go run . flags
  -debug
        Show debug information
  -v    Be more verbose
$
```

With all the features we have now examined, google/subcommands covers the needs of most small applications. Later levels will mostly be relevant for more complex projects, especially those which require unit testing coverage for commands themselves, instead of just requiring it for the service-level code the commands call upon, which is a more frequent requirement.

## Level 2 : Reusing command code

_The code for that level is available on [level2.1-reuse](https://github.com/fgm/subcommands_demo/tree/level2.1-reuse)._

Until this point we have been using one defined type per command, not counting aliases, as suggested by the minimal  [documentation](https://pkg.go.dev/github.com/google/subcommands#section-readme) provided by the google/subcommands repository. Doing so allows storing command-specific properties of commands, like local flags, on the instance of that type which is registered with google/subcommands.

In practice, however, many commands either do not use any flag, or share whichever flags they use ; and repeating identical code for each of these types is not entirely satisfying. But there is no actual need to use different types for different commands, especially if they share the same structure.

The main reason for different types is the `Command.Execute` method, which has to differ for each command since commands implement different features. Sharing types means using different methods for each command instance.

Various techniques are available for that purpose. For example, using a single `top` type, its `Execute` method could look up a property on the command instance to select and invoke a concrete runner function, as in this fragment :

```go 
// In top.go
type top struct {
	// ...shared fields
	name string // command name
}

// Add the Name, Synopsis, and SetFlags methods to implement subcommands.Command.

// NewTop replaces both NewTop1 and NewTop2.
func NewTop(name string) {
	return &top{name: name}
}

func (t *top) Execute(ctx context.Context, fs *flag.FlagSet, args ...any) {
	switch t.name {
	case "top1":
		Top1Execute(ctx, fs, args) // A function in top1.go
  case "top2":
	  Top2Execute(ctx, fs, args) // A function in top2.go
	default:
		panic("Unexpected command %s, should never happen", t.name)
	}
}
```
```go 
// In root.go,
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		group string
		subcommands.Command
	}{
		{"help", subcommands.CommandsCommand()},    // Implement "commands"
		{"help", subcommands.FlagsCommand()},       // Implement "flags"
		{"help", subcommands.HelpCommand()},        // Implement "help"
		{"top", NewTop("top1")},                    // Our first top-level command, without args
		{"top", NewTop("top2")},                    // Our second top-level command, with args
```

Onw downside with this approach is how it introduces coupling between

- the code in the shared `Execute` method and the various commands.
- the code in the shared `NewTop` factory function, which could requires modifications to accommodate the specifics of various commands

A better model is more direct and prevents such coupling. It keeps per-command factory functions, and uses a function field to store the runner function on the instance. That way the dedicated factory functions are used in `root.go` :

```go 
func Execute(ctx context.Context) subcommands.ExitStatus {
	for _, command := range [...]struct {
		group string
		subcommands.Command
	}{
		{"help", subcommands.CommandsCommand()},    // Implement "commands"
		{"help", subcommands.FlagsCommand()},       // Implement "flags"
		{"help", subcommands.HelpCommand()},        // Implement "help"
		{"top", NewTop1()},                         // Our first top-level command, without args
		{"top", NewTop2()},                         // Our second top-level command, with args
```

The `top.go` contains the reusable `top` type , with various fields to store :

- the constant values for which the `Name`, `Synopsis` and `Usage` will just be getters;
- the local flags shared by both our commands, in this example `prefix`, as declared by `SetFlags`
- a function field storing the runner function for the `Execute` method.

```go 
type top struct {
	name, synopsis, usage string // Reuse support
	prefix                string // Actual features
	run                   func(context.Context, *top, *flag.FlagSet, ...interface{}) subcommands.ExitStatus
}

func (cmd top) Name() string {
	return cmd.name
}

func (cmd top) Synopsis() string {
	return cmd.synopsis
}

func (cmd top) Usage() string {
	return cmd.usage
}

func (cmd *top) SetFlags(fs *flag.FlagSet) {
	fs.StringVar(&cmd.prefix, "prefix", "", "Add a prefix to the result")
}

func (cmd *top) Execute(ctx context.Context, fs *flag.FlagSet, args ...interface{}) subcommands.ExitStatus {
	if cmd.run == nil {
		log.Printf("command %s is not runnable", cmd.name)
		return subcommands.ExitFailure
	}
	return cmd.run(ctx, cmd, fs, args)
}
```

A limitation of function fields vs methods lies in the fact that they do not receive the instance as a hidden first parameter like methods do, meaning that with the same signature, they cannot access the instance, which `Execute` needs to access the local flag values.
The command instance must therefore be passed as an explicit extra argument, hence the `*top` parameter in the function signature, on line 15.

To remain as close as possible to a standard [method expression](https://go.dev/ref/spec#Method_expressions), that parameter should be the first, but it would conflict with the Go usage of having context always be the first parameter of any function, hence the placement as the second parameter.

Commands using the shared type, like `top2`, define a runner function with that signature, which provides the runner with access to the instance as if they were methods; and their factory function assigns that runner to the instance during the instance initialization process, like in this code in `top2.go` :

```go 
func top2Execute(ctx context.Context, cmd *top, fs *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	if ctx.Value(VerboseKey).(bool) {
		fmt.Printf("In %s.\n", cmd.Name())
	}
	fmt.Println(strings.Join(append([]string{cmd.prefix}, fs.Args()...), ": "))
	return subcommands.ExitSuccess
}

func NewTop2() *top {
	const name = "top2"
	return &top{
		name:     name,
		synopsis: fmt.Sprintf("%s is an exemple top-level custom command with arguments", name),
		usage:    fmt.Sprintf("%s arg1 arg2 ...", name),
		prefix:   "",
		run:      top2Execute,
	}
}
```

This process removes coupling between commands and the `Execute` function. An unlimited number of commands can now reuse the shared command type, without having to modify the implementation in `top.Execute`.

```text
$ go run . top1 -prefix today
today: hello
$ go run . top2 -prefix "the answer is" 42
the answer is: 42
$
```

## Level 3 : Commanders
### 3.1 Procedural API vs object API

_The code for that level is available on [level3.1-object_api](https://github.com/fgm/subcommands_demo/tree/level3.1-object_api)._

Up until now, we have been using the public functions exported by google/subcommands :

- `subcommands.Alias` and `subcommands.Register` to register command instances
- `subcommands.ImportantFlag` to label flags as flobal
- `subcommands.CommandsCommand`, `subcommands.FlagsCommand` and `subcommands.HelpCommand` for the builtin help and documentation features

That procedure-based API is actually only a facade simplifying the underlying object API : exepct for `Alias`, which does not depend on a `Commander`, each of these functions is actually a method call to the method with the same name on a default instance of the `subcommands.Commander` type, held in the exported mutable global variable `subcommands.DefaultCommander`, which is initialized when importing `github.com/google/subcommands` by the [`init` pseudo-function](https://github.com/google/subcommands/blob/master/subcommands.go#L462), using the `subcommands.go` fragment below :

```go 
// DefaultCommander is the default commander using flag.CommandLine for flags
// and os.Args[0] for the command name.
var DefaultCommander *Commander

func init() {
	DefaultCommander = NewCommander(flag.CommandLine, path.Base(os.Args[0]))
}

// Register adds a subcommand to the supported subcommands in the
// specified group. (Help output is sorted and arranged by group
// name.)  The empty string is an acceptable group name; such
// subcommands are explained first before named groups. It is a
// wrapper around DefaultCommander.Register.
func Register(cmd Command, group string) {
	DefaultCommander.Register(cmd, group)
}
```

This is but one case of a model frequently used throughout the Go standard library, for instance by packages `flag` (`flag.CommandLine`) or `http` (`http.DefaultClient`).

We can switch to that object model by only changing our `root.go` file, in which we replace every procedural call (except `Alias`) by a method call on that default `Commander` instance :

```go 
func Execute(ctx context.Context) subcommands.ExitStatus {
	commander := subcommands.DefaultCommander

	for _, command := range [...]struct {
		group string
		subcommands.Command
	}{
		{"help", commander.CommandsCommand()},      // Implement "commands"
		{"help", commander.FlagsCommand()},         // Implement "flags"
		{"help", commander.HelpCommand()},          // Implement "help"
		{"top", NewTop1()},                         // Our first top-level command, without args
		{"top", NewTop2()},                         // Our second top-level command, with args
		{"top", subcommands.Alias("1", NewTop1())}, // An alias for our top1 command

	} {
		commander.Register(command.Command, command.group)
	}

	debug := flag.Bool("debug", false, "Show debug information")
	verbose := flag.Bool("v", false, "Be more verbose")
	commander.ImportantFlag("v")
	flag.Parse()

	ctx = context.WithValue(ctx, DebugKey, *debug)
	ctx = context.WithValue(ctx, VerboseKey, *verbose)

	return commander.Execute(ctx, "meaning")

```

The program keeps operating in the exact same way as the previous version.


### 3.2 Using custom commanders

_The code for that level is available on [level3.2-custom_commander](https://github.com/fgm/subcommands_demo/tree/level3.2-custom_commander)._

Our next step will be to create our own `Commander` instance, rather that using the shared mutable instance exported by google/subcommands, so we can keep its initialization and mutations under control. We will therefore create it in the `Execute` function in our `root.go` file :

```go 
func Execute(ctx context.Context) subcommands.ExitStatus {
	// See subcommands.DefaultCommander
	commander := subcommands.NewCommander(flag.CommandLine, os.Args[0])
```

Nothing else deviates from the previous version, but we are now one step closer to a testable code structure, by removing our dependency on the global `subcommands.DefaultCommander`.

Time to finish the work and make `Execute` completely testable by removing all remaining dependencies to other globals.


### 3.3 Creating a testable command structure

_The code for that level is available on [level3.3](https://github.com/fgm/subcommands_demo/tree/level3.3-testable)._

Our code is still encumbered by a number of obstacles before it can be unit tested easily :

| Problem                           | Solution                                     |
|-----------------------------------|----------------------------------------------|
| global standard and error outputs | inject from `main`                           |
| global `os.Args`                  | inject from `main`                           |
| global `flag.CommandLine`         | replace by a custom `flag.FlagSet`           |
| global `flag.Bool`, `flag.Parse`  | replace by equivalent `flag.FlagSet` methods |
| hidden use of global `log.std`    | remplace by a custom `log.Logger`            |
| `flag` calling `os.Exit`          | modify `flag.FlagSet` creation options       |
| command creation in `Execute`     | inject from a factory                        |

Once we apply these changes, our latest version of `main.go` should become the only place where globals are still accessed :

```go 
func main() {
	sts := cmd.Execute(context.Background(), os.Stdout, os.Stderr, os.Args, log.LstdFlags, cmd.Describe)
	os.Exit(int(sts))
}
```

We also need to create a `Describe` factory instantiating our commands in  `root.go`,  so that `Execute` now receives them instead of creating them itself :

```go 
type description struct {
  group   string
  command subcommands.Command
}

func Describe(outW io.Writer, logger *log.Logger) []description {
	return []description{
		{"top", NewTop1(outW, logger)},                         // Our first top-level command, without args
		{"top", NewTop2(outW, logger)},                         // Our second top-level command, with args
		{"top", subcommands.Alias("1", NewTop1(outW, logger))}, // An alias for our top1 command
	}
}
```

…and our new version of `Execute` now receives all that data it needs by injection instead of referencing imported variables or creating instances itself.

```go 
func Execute(ctx context.Context,
	outW io.Writer, // Standard output for command results
	errW io.Writer, // Error output for logs
	args []string,  // CLI args to avoid depending on the flag global
	logFlags int,   // Log flags to make error message testing easier
	describe func(outW io.Writer, logger *log.Logger) []description, // Command registration descriptions
) subcommands.ExitStatus {
	// Do not depend on log.Default().
	logger := log.New(errW, "", logFlags)

	// Create a flag.FlagSet from args to avoid depending on global os.Args.
	// Continue on error to support testing instead of the ExitOnError on flag.CommandLine
	if len(args) < 1 {
		logger.Printf("Expected at least one argument for the program name, got none")
		return subcommands.ExitFailure
	}
	fs := flag.NewFlagSet(args[0], flag.ContinueOnError)

	// Create a custom commander to avoid depending on global flag.CommandLine and os.Args
	commander := subcommands.NewCommander(fs, args[0])

	descriptions := []description{
		{"help", commander.CommandsCommand()}, // Implement "commands"
		{"help", commander.FlagsCommand()},    // Implement "flags"
		{"help", commander.HelpCommand()},     // Implement "help"
	}
	if describe != nil {
		descriptions = append(descriptions, describe(outW, logger)...)
	}
	for _, command := range descriptions {
		commander.Register(command.command, command.group)
	}

	debug := fs.Bool("debug", false, "Show debug information")
	verbose := fs.Bool("v", false, "Be more verbose")
	commander.ImportantFlag("v")

	// Parse must not receive the program name, hence the slice.
	if err := fs.Parse(args[1:]); err != nil {
		// Our logger has been configured above.
		logger.Printf("Error parsing CLI flags: %v", err)
		return subcommands.ExitUsageError
	}

	ctx = context.WithValue(ctx, DebugKey, *debug)
	ctx = context.WithValue(ctx, VerboseKey, *verbose)

	return commander.Execute(ctx, "meaning", 42)
}
```

- 40-41: `outW` and `errW` are the two `io.Writer` passed by `main()`.
- 41,47: `errW` is used to create a `log.Logger`
- 44: `describe` is the command description factory, passed by `main()`
- 55: a new `FlagSet` is created from received parameters, to break the dependency on `os.Args`
- 58: a `Commander` is create from that new `FlagSet` and received arguments, to break the dependency on `flag.CommandLine`, and indirect dependency to `os.Args`
- 66: `outW` and our new logger are passed to `describe` so it can pass them  into the command factory functions, allowing command instances to work on injected writers instead of global variables.
- 72, 73: global flags are defined on the new `FlagSet` instead of global `flag.CommandLine`
- 77: flags parsing is run on the `FlagSet` instead of global `flag.CommandLine`

`Execute` is no longer using any global variable, and passes injected dependencies to the `top1` et `top2` command instances. We have to modify these too, so that they use the injected writer and logger instead of shared globals. This is how it applies in `top1.go` :

```go 
func top1Execute(ctx context.Context, cmd top, fs *flag.FlagSet, _ ...any) subcommands.ExitStatus {
	if ctx.Value(VerboseKey).(bool) {
		cmd.logger.Printf("In %s.\n", cmd.Name())
	}
	if l := fs.NArg(); l != 0 {
		cmd.logger.Printf("%s expects no arguments, called with %d: %v", cmd.Name(), l, fs.Args())
		return subcommands.ExitFailure
	}
	message := "hello"
	if cmd.prefix != "" {
		message = strings.Join(append([]string{cmd.prefix}, message), ": ")
	}
	fmt.Fprintln(cmd.outW, message)
	return subcommands.ExitSuccess
}

func NewTop1(outW io.Writer, logger *log.Logger) *top {
	const name = "top1"
	return &top{
		logger:   logger,
		name:     name,
		outW:     outW,
		prefix:   "",
		run:      top1Execute,
		synopsis: fmt.Sprintf("%s is an exemple top-level custom command without arguments", name),
		usage:    name,
	}
}
```

The code in `top2` is very similar, and the `top` type in `top.go` has been extended as show at lines 34-40 above to store the injected dependencies.

This enables us to establish a [100% test coverage rate](https://github.com/fgm/subcommands_demo/tree/level3.3-testable/cmd) very simply, without needing any mocking tool.

```text
$ go test -race -count=1 -cover ./cmd
ok  	github.com/fgm/subcommands_demo/cmd	0.035s	coverage: 100.0% of statements
$
```

## Level 4 : adding nested subcommands

_The code for that level is available on [level4.1-nesting](https://github.com/fgm/subcommands_demo/tree/level4.1-nesting)._

With more complex applications, some commands will need a hierarchy of subcommands, but google/subcommands does not describe how to implement them, leading to a false impression that it does not support them.

That support is actually a consequence of our use of a custom commander created from received parameters passed to `subcommands.NewCommander` : since flags are now defined on a local `FlagSet` and arguments received, all it takes to implement a command accepting subcommands is to create a `Commander` instance from the arguments not consumed after resolving the command itself.

This is how our new `top3` works, with its `sub31` and `sub32` nested commands, to be found in `top3.go`. Feature-wise, it is similar to `top1` and `top2`, but its runner function `top31Execute` now includes the subcommands handling logic in addition to the base `top3` logic.

```go 
func top3Execute(ctx context.Context, cmd top, topFS *flag.FlagSet, args ...any) subcommands.ExitStatus {
	name := cmd.Name()
	if ctx.Value(VerboseKey).(bool) {
		cmd.logger.Printf("In %s.\n", cmd.Name())
	}
	// Handle command called without subcommands.
	if topFS.NArg() == 0 {
		return top3Internal(ctx, cmd, topFS, args)
	}

	// Create a flag.FlagSet from args to use only remaining args
	// Continue on error to support testing.
	fs := flag.NewFlagSet(cmd.Name(), flag.ContinueOnError)

	// Create a custom commander to restart evaluation below this command.
	commander := subcommands.NewCommander(fs, name)

	descriptions := []description{
		{name, commander.CommandsCommand()}, // Implement "commands"
		{name, commander.FlagsCommand()},    // Implement "flags"
		{name, commander.HelpCommand()},     // Implement "help"
		{name, NewSub31(cmd.outW, cmd.logger)},
		{name, NewSub32(cmd.outW, cmd.logger)},
	}
	for _, command := range descriptions {
		commander.Register(command.command, command.group)
	}

	// Parse must not receive the command name.
	if err := fs.Parse(topFS.Args()); err != nil {
		cmd.logger.Printf("Error parsing %s flags: %v", name, err)
		return subcommands.ExitUsageError
	}

	return commander.Execute(ctx, fs)
}
```

- near the beginning (lines 29-31), the command checks whether it receives arguments, which could be subcommand names. If there are none, it then transfers control to its own internal logic, which performs the work desired for `top3` without a subcommand.
- otherwise, it implements a logic similar to the one in our root `Execute` function :
  - create a local `FlagSet`,
  - create a local `Commander` from that `FlagSet` and available arguments,
  - register subcommands on the `Commander`, including builtin commnds,
  - parse available arguments for local flags and arguments on the nested commands
  - run the subcommand designated by arguments, if any.

```text
$ go run . top3
hello top3
$ go run . top3 commands
commands
flags
help
sub31
sub32
$ go run . top3 help sub31
sub31  -prefix string
    	Add a prefix to the result
$ go run . top3 sub31 -prefix today
today: hello sub31
$
```

## Level 5 : beyond `NewCommander`
### 5.1 Controlling outputs beyond our code

_The code for that level is available on [level5.1-newcommander](https://github.com/fgm/subcommands_demo/tree/level5.1-newcommander)._

Although we have injected everything for our own logic to support unit testing, some error outputs remain, from the `subcommands` and `flag` packages themselves, in error situations, and can be seen when running tests. We could have foreseen it, because `NewCommander` does not take any argument for the standard or output error to use by the instance it returns.

```text
$ go test -race -count=1 -v  ./cmd 2>&1 | grep -vE '(CONT|RUN|PASS|PAUSE)'
flag provided but not defined: -bad
Usage: Test_Execute <flags> <subcommand> <subcommand args>

Subcommands for help:

  ...snip (33 lines total)...

flag provided but not defined: -bad
top1  -prefix string
    	Add a prefix to the result
flag provided but not defined: -bad
c  -prefix string
    	Add a prefix to the result
ok  	github.com/fgm/subcommands_demo/cmd	0.036s
$
```

However just because that redirection is not available at instance creation, it is still available _after_ creation, as we do here in `root.go` for top-level commands and global flags :

```go 
func Execute(ctx context.Context,
	outW io.Writer, // Standard output for command results
	errW io.Writer, // Error output for logs
	args []string, // CLI args to avoid depending on the flag global
	logFlags int, // Log flags to make error message testing easier
	describe func(outW io.Writer, logger *log.Logger) []description, // Command registration descriptions
) subcommands.ExitStatus {
  /* ...snip... */
	fs := flag.NewFlagSet(args[0], flag.ContinueOnError)
	fs.SetOutput(errW)

	// Create a custom commander to avoid depending on global flag.CommandLine and os.Args
	commander := subcommands.NewCommander(fs, args[0])
	commander.Output = outW
	commander.Error = errW

```

Redirecting the additional properties removes most of the parasitic output we were seeing during tests :

```text
$ go test -race -count=1 -v  ./cmd 2>&1 | grep -vE '(CONT|RUN|PASS|PAUSE)'
flag provided but not defined: -bad
flag provided but not defined: -bad
ok  	github.com/fgm/subcommands_demo/cmd	0.045s
$
```

The two remaining uncaptured messages appear to be a bug in version 1.2.0 of google/subcommands, for which [a pull request exists](https://github.com/google/subcommands/pull/39).

We have now covered all the useful features of the google/subcommands package. All that remains is two related sets of features which are unlikely to be used in most practical cases.


### 5.2 Commanders introspection

_The code for that level is available on [level5.2-visit](https://github.com/fgm/subcommands_demo/tree/level5.2-visit)._

The `subcommands.Commander` type includes exported methods allowing code to examine its internal flags and commands data.

Let us create a top-level `visit` command, similar to those we have been writing, and which demonstrates all these methods. It is located in the `visit.go` file.

- The `VisitAll` methods implements the Visitor pattern, allowing introspection of the flags on the `Commander` instances:

```go
func visitAll(commander *subcommands.Commander, w io.Writer) {
	fmt.Fprintln(w, "VisitAll show all the commander flags:")
	tw := tabwriter.NewWriter(w, 0, 0, 1, ' ', tabwriter.Debug)
	fmt.Fprintln(tw, "\tName\tDefault\tValue\tUsage\t")
	commander.VisitAll(func(f *flag.Flag) {
		fmt.Fprintf(tw, "\t%s\t%s\t%s\t%s\t\n", f.Name, f.DefValue, f.Value, f.Usage)
	})
	tw.Flush()
	fmt.Fprintln(w)
}
```
```text
Running "demo visit":
VisitAll show all the commander flags:
 |Name  |Default |Value |Usage                  |
 |debug |false   |false |Show debug information |
 |v     |false   |true  |Be more verbose        |
```

- The `VisitAllImportant` methods is almost identical, but only visits flags labeled as important:
```text
VisitAllImportant only shows the "important" flags:
 |Name |Default |Value |Usage           |
 |v    |false   |true  |Be more verbose |
```

- The `VisitGroups` method implements the Visitor pattern, allowing introspection of the `CommandGroup` instances created from the group names used when registering commands. It is quite limited, as the public methods on the `CommandGroup` type do not provide access to the grouped commands:

```go 
func visitGroups(commander *subcommands.Commander, w io.Writer) {
	fmt.Fprintln(w, "VisitGroups only visits the command groups, not the commands:")
	tw := tabwriter.NewWriter(w, 0, 0, 1, ' ', tabwriter.Debug)
	fmt.Fprintln(tw, "\tName\tLen\t")
	commander.VisitGroups(func(group *subcommands.CommandGroup) {
		fmt.Fprintf(tw, "\t%s\t%d\t\n", group.Name(), group.Len())
	})
	tw.Flush()
	fmt.Fprintln(w)
}
```

```text
VisitGroups only visits the command groups, not the commands:
 |Name |Len |
 |help |3   |
 |top  |5   |
```
- The `VisitCommands` method implement the Visitor pattern in a much more useful way, providing access to the `Command` instances registered on the `Commander`, and from there to their `CommandGroup` and local flags:

```go 
func visitCommands(commander *subcommands.Commander, w io.Writer) {
	fmt.Fprintln(w, "VisitCommands visits the commands themselves:")
	tw := tabwriter.NewWriter(w, 0, 0, 1, ' ', tabwriter.Debug)
	fmt.Fprintln(tw, "\tGroup\tName\tSynopsis\tFlags\t")
	commander.VisitCommands(func(group *subcommands.CommandGroup, c subcommands.Command) {
		fs := flag.NewFlagSet("visit", flag.ContinueOnError)
		c.SetFlags(fs)
		var flags []string
		fs.VisitAll(func(f *flag.Flag) { flags = append(flags, f.Name) })
		fmt.Fprintf(tw, "\t%s\t%s\t%s\t%s\t\n", group.Name(), c.Name(), c.Synopsis(), strings.Join(flags, ", "))
	})
	tw.Flush()
	fmt.Fprintln(w, "(command usage omitted for readability)")
}
```
```text
VisitCommands visits the commands themselves:
 |Group |Name     |Synopsis                                                            |Flags  |
 |help  |commands |list all command names                                              |       |
 |help  |flags    |describe all known top-level flags                                  |       |
 |help  |help     |describe subcommands and their syntax                               |       |
 |top   |top1     |top1 is an exemple top-level custom command without arguments       |prefix |
 |top   |top2     |top2 is an exemple top-level custom command with arguments          |prefix |
 |top   |1        |top1 is an exemple top-level custom command without arguments       |prefix |
 |top   |top3     |top3 is an exemple top-level custom command with nested subcommands |prefix |
 |top   |visit    |demoes commander Visit* functions                                   |       |
(command usage omitted for readability)
```

Because these visitor callbacks do not receive the `Commander` instance, the `visit` command will need to carry an injected commander instance, as commands are not aware of the `Commander` invoking them. We perform this with a new feature in `root.go` :

```go
type CommanderAware interface {
	SetCommander(commander *subcommands.Commander)
}
```
```go 
func Execute(ctx context.Context,
  /* ...snip... */
	for _, command := range descriptions {
		if vc, ok := command.command.(CommanderAware); ok {
			vc.SetCommander(commander)
		}
		commander.Register(command.command, command.group)
	}
```

Our `visit` command is an instance of the `visitCmd` type, which includes a `subcommands.Commander` field, and an associated `visitCmd.SetCommander` setter, making it a `CommanderAware` implementation, allowing the `Execute` function to inject it with the active `Commander` as show on lines 48-49.


### 5.3 Overriding builtin output

_The code for that level is available on [level5.3-explain](https://github.com/fgm/subcommands_demo/tree/level5.3-explain)._

The last and most esoteric feature in google/subcommands is the ability to swap the implementation of the builtin `CommandsCommand`, `FlagsCommand` and `HelpCommand`, using matching function fieled on the `subcommands.Commander` type :

- `Explain` is a `func(io.Writer)` field, initialized in `NewCommander` with the unexported method expression `cdr.writer`
- `ExplainGroup` is a `func(io.Writer, *CommandGroup)` field, initialized in `NewCommander` with unexported function `explainGroup`
- `ExplainCommand` is a `func(w io.Writer, c subcommands.Command)` field, initialized in `NewCommander` with unexported function `explain`

Since these function fields are missing receiver access although they need it, our new `explain` command, defined in `cmd/explain.go`, has to be `CommanderAware`, so it can replace the fields with functions having access to the `Commander` instance, using the one instance it was injected with :

Examples below first show the default builtin output, then the output of a customized version

- `ExplainCommand` : builtin output, then YAML API version:
```text
Running "demo explain":

Demoes overriding ExplainCommand to describe top3.
- Builtin version using unexported explain:
top3  -prefix string
    	Add a prefix to the result
- Custom version in YAML format:
top3:
    flags:
        - name: prefix
          default: ""
          usage: Add a prefix to the result
    synopsis: top3 is an exemple top-level custom command with nested subcommands
    usage: top3
```

- `ExplainGroup` : builtin output, then YAML API version:
```text
Demoes overriding ExplainGroup.
- Builtin version using private explainGroup:
Subcommands for help:
	commands         list all command names
	flags            describe all known top-level flags
	help             describe subcommands and their syntax	
...snip...
- Custom version in YAML format, without access to group contents:
help: 3
top: 6
```

- `Explain` : builtin output, then neutral version suggesting how to write a custom description
```text
Demoes overriding Explain.
- Builtin version using private commander.explain:
Usage: demo <flags> <subcommand> <subcommand args>


Top-level flags (use "demo flags" for a full list):
  -v=false: Be more verbose

- Custom version, build from commander methods:
Use any commander.(Explain|Visit)* methods
```

## Conclusions

- Pros
  - `google/subcommands` is an excellent fit for small, simple projects
  - it is uncommonly lightweight, and very stable due to its having no dependencies at all
  - starting with it does not prevent evolving from a simple version to a multi-level command hierarchy with a mix of local and global flags
  - it does not make it difficult to provide unit test coverage for the commands  themselves instead of limiting one to the service code the commands call upon.
- Cons
  - it does not natively merge CLI flags and environment variables
  - it does not natively include merging configuration files with flags
  - creating complex commands with validation hooks has no direct support: all steps will have to be implemented together in the `Execute` method
  - it does not natively include the _persistent flags_ concept, that is flags defined at one level and also applying to subcommands. These can however be passed manually, for example in the context like we did for global flags.
  - creating multi-level command hierarchies requires a bit more work than with some other tools.

To summarize, that package is likely to be your tool of choice for your next microservice or CLI tool, but not if you are building your next giant monolith.