Project: RabbitMQ (App Definition and Development / Streaming & Messaging)
File: DESIGN.md

RabbitMQ version 3.7 comes with brave new CLI tool to replace `rabbitmqctl`. Some of the issues in the older tool suite we wanted to address: Built-in into RabbitMQ server code Home-grown argument parser Started with `erl` with a lot of installation-dependent parameters Too many commands in a single tool All commands in resided the same module (function clauses) All this made it hard to maintain and extend the tools. The new CLI is different in a number of ways that address the above issues: Standalone . Implemented in Elixir (using Elixir's standard CLI parser) Each command is its own module Extensible Commands can support more output formats (e.g. JSON and CSV) A single executable file that bundles all dependencies but the Erlang runtime Command scopes associate commands with tools (e.g. `rabbitmq-diagnostics` reuses relevant commands from `rabbitmqctl`) Each command is defined in its own module and implements an Elixir (Erlang) behaviour. (See ) Output is processed by a formatter and printer which formats command output and render the output (the default being the standard I/O output). (see ) CLI core consists of several modules implementing command execution process: `RabbitMQCtl`: entry point. Generic execution logic. `Parser`: responsible for command line argument parsing (drives Elixir's `OptionParser`) `CommandModules`: responsible for command module discovery and loading `Config`: responsible for config unification: merges environment variable and command argument values `Output`: responsible for output formatting `Helpers`: self-explanatory Command line arguments are parsed with Parser returns a list of unnamed arguments and a map of options (named arguments) First unnamed argument is a command name. Named arguments can be global or command specific. Command specific argument names and types are specified in the `switches/0` callback. Global argument names are described in [Global arguments] If arguments list is not empty, its first element is considered a command name. Command name is converted to CamelCase and a module with `RabbitMQ.CLI.*.Commands.<CommandName>Command` name is selected as a command module. List of available command modules depend on current tool scope (see ) After the command module is found, effective command arguments are calculated by merging global defaults and command specific defaults for both unnamed and named arguments. A command specifies defaults using the `merge_defaults/2` callback (see ) Arguments are then validated using the `validate/2` callback. `validateexecutionenvironment/2` is another (optional) validation function with the same signature as `validate/2`. It is meant to validate everything that is not CLI arguments, for example, whether a file exists or RabbitMQ is running or stopped on the target node. It is possible to define aliases for commands using an aliases file. The file name can be specified using `RABBITMQCLIALIASES_FILE` environment variable or the `--aliases-file` command lineargument. Aliases can be specified using `alias = command [options]` format. For example: ``` lq = list_queues lqvhost1 = listqueues -p vhost1 lqoff = listqueues --offline ``` with such aliases config file running ``` RABBITMQCLIALIASES_FILE=/path/to/aliases.conf rabbitmqctl lq ``` will be equivalent to executing ```

rabbitmqctl listqueues ``` while ``` RABBITMQCLIALIASESFILE=/path/to/aliases.conf rabbitmqctl lqoff ``` is the same as running ``` RABBITMQCLIALIASESFILE=/path/to/aliases.conf rabbitmqctl listqueues --offline ``` Builtin or plugin-provided commands are looked up first, if that yields no result, aliases are inspected. Therefore it's not possible to override a command by configuring an alias. Just like command lookups, alias expansion happens in the `RabbitMQ.CLI.Core.Parser` module. Aliases can also contain arguments. Command name must be the first word after the `=`. Arguments specified in an alias will precede those passed from the command line. For example, if you specify the alias `passwduser1 = changepassword user1`, you can call it with `rabbitmqctl passwduser1 newpassword`. Combined with the `eval` command, aliases can be a powerful tool for users who cannot or don't want to develop, distribute and deploy their own commands via plugins. For instance, the following alias deletes all queues in a vhost: ``` deletevhostqueues = eval '[rabbitamqqueue:delete(Q, false, false, <<"rabbit-cli">>) || Q <- rabbitamqqueue:list(_1)]' ``` This command will require a single positional argument for the vhost: ``` rabbitmqctl deletevhostqueues vhost1 ``` It is also possible to use a named vhost argument by specifying an underscore variable that's not an integer: ``` deletevhostqueues = eval '[rabbitamqqueue:delete(Q, false, false, <<"rabbit-cli">>) || Q <- rabbitamqqueue:list(_vhost)]' ``` Then the alias can be called like this: ``` rabbitmqctl deletevhostqueues -p vhost1 rabbitmqctl deletevhostqueues vhost <vhost>1 ``` Keep in mind that `eval` command can accept only as named, and it's advised to use positional arguments instead. Numbered arguments will be passed to the eval'd code as Elixir strings (or Erlang binaries). The code that relies on them should perform type conversion as necessary, e.g. by using functions from the . Command is executed with the `run/2` callback, which contains main command logic. This callback can return any value. Output from the `run/2` callback is being processed with the `output/2` callback, which should format the output to a specific type. (see ) Output callback can return an error, with a specific exit code. The `output/2` callback return value is being processed in `output.ex` module by by the appropriate formatter and printer. A formatter translates the output value to a sequence of strings or error value. Example formatters are `json`, `csv` and `erlang` A printer renders formatted strings to an output device (e.g. stdout, file) Errors during execution (e.g. validation failures, command errors) are being printed to `stderr`. If command has failed to execute, a non-zero code is returned. CLI tool is an Elixir Mix application. It is compiled into an escript executable file. This file embeds Elixir, rabbit_common, and rabbitmqcli applications and can be executed anywhere where `escript` is installed (it comes as a part of `erlang`). Although, some commands require rabbitmq broker code and data directory to be known. For example, commands in `rabbitmq-plugins` tool and those controlling clustering (e.g. `forgetclusternode`, `renameclusternode` ...) Those directories can be defined using environment options or rabbitmq environment

In the broker distribution the escript file is called from a shell/cmd script, which loads broker environment and exports it into the script. Environment variables also specify the locations of the enabled plugins file and the plugins directory. All enabled plugins will be searched for commands. Commands from enabled plugins will be shown in usage and available for execution. node (n): atom, broker node name, defaults to `rabbit@<current host>` quiet (q): boolean, if set to `true`, command banner is not shown, defaults to `false` timeout (t): integer, timeout value in seconds* (used in some commands), defaults to `infinity` vhost (p): string, vhost to talk to, defaults to `/` formatter: string, formatter to use to format command output. (see ) printer: string, printer to render output (see ) dry-run: boolean, if specified the command will not run, but print banner only script-name: atom, configurable tool name (`rabbitmq-plugins`, `rabbitmqctl`) to select command scope (see ) rabbitmq-home: string, broker install directory mnesia-dir: string, broker mnesia data directory plugins-dir: string, broker plugins directory enabled-plugins-file: string, broker enabled plugins file longnames (l): boolean, use longnames to communicate with broker erlang node. Should be set to `true` only if broker is started with longnames. aliases-file: string, a file name to load aliases from erlang-cookie: atom, an Environment argument defaults are loaded from rabbitmq environment variables (see ). Some named arguments have single-letter aliases (in parenthesis). Boolean options without a value are parsed as `true` For example, parsing command rabbitmqctl listqueues --vhost myvhost -t 10 --formatter=json name pid --quiet Will result with unnamed arguments list `["list_queues", "name", "pid"]` and named options map `%{vhost: "my_vhost", timeout: 10, quiet: true}` Usage is shown when the CLI is called without any arguments, or if there are some problems parsing arguments. In that cases exit code is `64`. If you want to show usage and return `0` exit code run `help` command rabbitmqctl help Each tool (`rabbitmqctl`, `rabbitmq-plugins`) shows its scope of commands (see ) Each command is implemented as an Elixir (or Erlang) module. Command module should implement `RabbitMQ.CLI.CommandBehaviour` behaviour. Behaviour summary: Following functions MUST be implemented in a command module: usage() :: String.t | [String.t] Command usage string, or several strings (one per line) to print in command listing in usage. Typically looks like `commandname [arg] --option=optvalue` banner(arguments :: List.t, options :: Map.t) :: String.t Banner to print before the command execution. Ignored if argument `--quiet` is specified. If `--dry-run` argument is specified, th CLI will only print the banner. merge_defaults(arguments :: List.t, options :: Map.t) :: {List.t, Map.t} Merge default values for arguments and options (named arguments). Returns a tuple with effective arguments and options, that will be passed to `validate/2` and `run/2` validate(arguments :: List.t, options :: Map.t) :: :ok | {:validation_failure, Atom.t | {Atom.t, String.t}} Validate effective arguments and options. If function returns `{:validation_failure, err}` CLI will print usage to `stderr` and exit with non-zero exit code (typically

run(arguments :: List.t, options :: Map.t) :: run_result :: any Run command. This function usually calls RPC on broker. output(runresult :: any, options :: Map.t) :: :ok | {:ok, output :: any} | {:stream, Enum.t} | {:error, ExitCodes.exitcode, [String.t]} Cast the return value of `run/2` command to a formattable value and an exit code. `:ok` - return `0` exit code and won't print anything `{:ok, output}` - return `exit` code and print output with `format_output/2` callback in formatter `{:stream, stream}` - format with `format_stream/2` callback in formatter, iterating over enumerable elements. Can return non-zero code, if error occurs during stream processing (stream element for error should be `{:error, Message}`). `{:error, exitcode, strings}` - print error messages to `stderr` and return `exitcode` code There is a default implementation for this callback in `DefaultOutput` module Most of the standard commands use the default implementation via `use RabbitMQ.CLI.DefaultOutput` Following functions are optional: switches() :: Keyword.t Keyword list of switches (argument names and types) for the command. For example: `def switches(), do: [offline: :boolean, time: :integer]` will parse `--offline --time=100` arguments to `%{offline: true, time: 100}` options. This switches are added to global switches (see ) aliases() :: Keyword.t Keyword list of argument names one-letter aliases. For example: `[o: :offline, t: :timeout]` (see ) usage_additional() :: String.t | [String.t] Additional usage strings to print after all commands basic usage. Used to explain additional arguments and not interfere with command listing. formatter() :: Atom.t Default formatter for the command. Should be a module name of a module implementing `RabbitMQ.CLI.FormatterBehaviour` (see ) scopes() :: [Atom.t] List of scopes to include command in. (see ) More information about command development can be found in Commands can be organized in scopes to be used in different tools like `rabbitmq-diagnostics` or `rabbitmq-plugins`. One command can belong to multiple scopes. Scopes for a command can be defined in the `scopes/0` callback in the command module. Each scope is defined as an atom value. By default a command scope is selected using naming convention. If command module is called `RabbitMQ.CLI.MyScope.Commands.DoSomethingCommand`, it will belong to `myscope` scope. A scope should be defined in snakecase. Namespace for the scope will be translated to CamelCase. When CLI is run, a scope is selected by a script name, which is the escript file name or the `--script-name` argument passed. A script name is associated with a single scope in the application environment: Script names for scopes: `rabbitmqctl` - `:ctl` `rabbitmq-plugins` - `:plugins` `rabbitmq-diagnostics` - `:diagnostics` This environment is extended by plugins `:scopes` environment variables, but cannot be overridden. Plugins scopes can override each other, so should be used with caution. So all the commands in the `RabbitMQ.CLI.Ctl.Commands` namespace will be available for `rabbitmqctl` script. All the commands in the `RabbitMQ.CLI.Plugins.Commands` namespace will be available for `rabbitmq-plugins` script. To add a command to `rabbitmqctl`, one should either name it with the

prefix or add the `scopes()` callback, returning a list with `:ctl` element The CLI supports extensible output formatting. Formatting consists of two stages: formatting - translating command output to a sequence of lines printing - outputting the lines to some device (e.g. stdout or filesystem) A formatter module performs formatting. Formatter is a module, implementing the `RabbitMQ.CLI.FormatterBehaviour` behaviour: format_output(output :: any, options :: Map.t) :: String.t | [String.t] Format a single value, returned. It accepts output from command and named arguments (options) and returns a list of strings, that should be printed. formatstream(outputstream :: Enumerable.t, options :: Map.t) :: Enumerable.t Format a stream of return values. This function uses elixir Stream [https://hexdocs.pm/elixir/Stream.html] abstraction to define processing of continuous data, so the CLI can output data in realtime. Used in `list_*` commands, that emit data asynchronously. `DefaultOutput` will return all enumerable data as stream, so it will be formatted with this function. A printer module performs printing. Printer module should implement the `RabbitMQ.CLI.PrinterBehaviour` behaviour: init(options :: Map.t) :: {:ok, printer_state :: any} | {:error, error :: any} Init the internal printer state (e.g. open a file handler). finish(printer_state :: any) :: :ok Finalize the internal printer state. printoutput(output :: String.t | [String.t], printerstate :: any) :: :ok Print the output lines in the printer state context. Is called for `{:ok, val}` command output after formatting `val` using formatter, and for each enumerable element of `{:stream, enum}` enumerable printok(printerstate :: any) :: :ok Print an output without any values. Is called for `:ok` command return value. Output formatting logic is defined in `output.ex` module. Following rules apply for a value, returned from `output/2` callback: `:ok` - initializes a printer, calls `print_ok` and finishes the printer. Exit code is `0`. `{:ok, value}` - calls `format_output/2` from formatter, then passes the value to printer. Exit code is `0`. `{:stream, enum}` - calls `format_stream/2` to augment the stream with formatting logic, initializes a printer, then calls `print_output/2` for each successfully formatted stream element (which is not `{:error, msg}`). Exit code is `0` if there were no errors. In case of an error element, stream processing stops, error is printed to `stderr` and the CLI exits with nonzero exit code. `{:error, exitcode, msg}` - prints `msg` to `stderr` and exits with `exitcode` Some commands require information about the server environment to run correctly. Such information is: rabbitmq code directory rabbitmq mnesia directory enabled plugins file plugins directory Enabled plugins file and plugins directory are also used to locate plugins commands. This information can be provided using command line arguments (see ) By default it will be loaded from environment variables, same as used in rabbitmq broker: | Argument name | Environment variable | |-|-| | rabbitmq-home | RABBITMQ_HOME | | mnesia-dir | RABBITMQMNESIADIR | | plugins-dir | RABBITMQPLUGINSDIR | | enabled-plugins-file | RABBITMQENABLEDPLUGINS_FILE | | longnames | RABBITMQUSELONGNAME | | node | RABBITMQ_NODENAME | | aliases-file | RABBITMQCLIALIASES_FILE | | erlang-cookie | RABBITMQERLANGCOOKIE |