Package: m3-cli
The Maestro3 Command Line Interface (CLI) is a unified tool to manage your
Maestro services and resources. With just one tool to download and configure,
you can control multiple resources Maestro provides from the command line and
automate them through scripts.Synopsism3 <command> [parameters]Use m3 command help for information on a specific command. The synopsis for each
command shows its parameters and their usage. Required parameters are marked
with the star (*) character.InstallationPre-requisitesThe installation of m3 cli assumed that you have Python3.10 and pip installed.
Use the following commands to install the tools in case they are not installed.WindowsGuide for python3 installationLinuxGuide for python3 installationMacGuide for python3 installationInstallationNOTE: Due to the best Python practices, it is recommended to use virtual environment to protect against dependency breakage. You can perfrom m3cli tool installation without this step. Need to mention that in case you want to install the to the created virtual environment you will need to activate the virtual environment with installed M3CLI before using it.The guide how to install and activate virtualenv you can find below for the
supported platforms:WindowsGuide for virtualenv installationCreate:virtualenv -p python3 venvActivate:venv\Scripts\activate.batLinux/MacGuide for virtualenv installationCreate:virtualenv -p python3 .venvActivate:source .venv/bin/activateTo install m3-cli tool use the command listed below installation command.NOTE: Replace tool version with the version number you need (seeCHANGELOG.mdorsetup.py).pip install m3-cliConfigurationBefore using m3 cli execute commnadm3 accessto set up all needed settings.In case you want to configure all needed settings manually, please set up the
following environmnet variables:M3SDK_ACCESS_KEY: Specifies an M3 access key associated with Maestro3 user;M3SDK_SECRET_KEY: Specifies the secret key associated with the access key.
This is essentially the "password" for the access key.M3SDK_ADDRESS: Specifies the address of the Maestro3 environment.Or you can set credentials non-interactively usingm3 accesscommand with
specified parameters:m3 access --access_key <access_key> --secret_key <_secret_key> --api_address <api_address>After this adefault.crfile with the access parameters you provided will be
created in a.m3clifolder inside your home directory.Obtaining credentialsTo get your credentials please login to the Maestro3 and follow the guide below:Switch to My Account page:2. Start the 'CLI/SDK Access' wizard:3. Specify the expiration time for the credentials:4. Obtain your credentials and export access key to environment variable
M3SDK_ACCESS_KEY, and secret key to M3SDK_SECRET_KEYOptional configurationThe following environment variables could be used to override default values m3
cli uses:M3CLI_CONFIGURATION_FOLDER_PATH: Specifies the path to the directory
containing files with the actual commands and parameters
definitions (commands_def.json, commands_help.py). The default value:
internal application path.M3SDK_VERSION: In case you need to specify SDK version. The default version
is "3.2.80".M3CLI_DEBUG: in case you need debug mode set to this env variable value
True. The default value is "False".LOG_PATH: in case you need to store the m3cli log file by the custom path.Usage examplesAfter you have installed the tool and set the credentials as environment
variables, the m3 cli is ready to be used.NOTE: The development of the Maestro3 CLI is still in progress. Examples below contain demo data. Will be updated.To get information about the available commands/parameters just run them3as
it is displayed below:Root help contains data about all available commands:Command--helpcontains abbreviated data about parameters:Use--full-helpcommand for all available options:Command result in--tableoutput mode (default output mode) :Command result in--jsonoutput mode (default output mode) :Command result in--fulloutput mode (default output mode) :--verboseparameter writes command result to the terminal.Developers GuideThe Maestro3 CLI tool is designed to provide dynamic command line interface
based on commands configuration declared incommands_def.jsonfile.The tool DOES NOT perform any business logic related to management of resources Maestro3 provides. For now :)The sequence diagram is displayed below.Commands definition file:The commands definition file (commands_def.json) is a file that defines a set
of commands, their groupings, and parameters of commands that can be executed via CLI.
Here is a full example of attributes found in the definition file:{
    "groups": [
        "A name of a group of related commands"
    ],
    "domain_parameters": {
        "domain-parameter-name-1": {
            "alias": "param_alias",
            "api_param_name": "The name of the parameter that the server accepts to form the response",
            "help": "Parameter description",
            "required": true,
            "validation": {
                "type": "string"
            }
        }
    },
    "commands": {
        "command-name-1": {
            "api-action": "REQUIRED. Mapping for M3API command",
            "help_file": true,                      // A Flag. If commands help stored into the file
            "help": "Explanation of what command does",
            "alias": "Alias for the command name. Could be used instead of command-name-1",
            "integration_request": true,            // A Flag. Specify if the request will be processed
            "integration_response": true,           // A Flag. Specify if the response will be processed
            "integration_suffix": "",               // if specified, m3cli should build the plugin name according to the ${command_name}_${integration_suffix}.py pattern
            "groups": [
                "group1",                           // A name of the group of commands to which this command relates
                "cli-<command-name>-help",          // Defines a unidirectional relation to the <command-name> command
                "email-<notification_type>-group",  // Defines a relation to the Maestro notifications of the type <notification_type>
            ],
            "parameters": {                         // Optional
                "inherited-param-from-domain": {
                    "parent": "Name of the domain parameter to inherit properties."
                },
                "param-name-1": {
                    "parent": "name",               // The name of the parent
                    "alias": "Alias for the parameter name. Could be used instead of param-name-1",
                    "api_param_name": "The name of the parameter that the server accepts to form the response",
                    "help": "REQUIRED. Explanation of what the parameter means",
                    "required": true,               // Set the parameter as a REQUIRED
                    "secure": true,                 // Hide the value of the parameter in logs. Allowed values: [true, false]
                    "validation": {                 // REQUIRED. Set of validation rules.
                        "type": "string/object",           // REQUIRED. The type of parameter. Allowed values: ['string', 'number', 'list', 'object', 'date', 'bool', 'file']
                        "allowed_values": [],       // A list of allowed values. Applicable to param of types 'string' and 'list'
                        "regex": "Regular Expression. Applicable to params of types: 'string', 'file'",
                        "regex_error": "A meesage to show if the regex check failed. Applicable to params of types: 'string', 'file'",
                        "properties": {},           // jsonschema validation rules. Applicable to param of type: 'object';
                        "min_value": 0,             // number validation rule. Applicable only to type: 'number',
                        "max_value": 7,             // number validation rule. Applicable only to type: 'number',
                        "max_size_bytes": 1024,     // file validation rule. Applicable only to type: 'file',
                        "file_extensions": ['.txt'] // file validation rule. Applicable only to type: 'file'
                    },
                    "case": "upper/lower"           // convert a value to upper case automatically
                }   
            },
            "output_configuration": {               // REQUIRED. Contains configuration for the output.
                "response_table_headers": ["header-1", "header-2"], // Required. Contains list of attributes to display in output.
                "none": true,                       // Replace the response from server to "The command has been executed successfully"
                "nullable": true,                   // Set the flag 'nullable' in order to prevent hiding zeros and False values of number and boolean attributes.
                "multiple_table": true,             // A flag. If the response consists of several tables
                // The structure of the "response_table_headers" if the "multiple_table" was specified
                "response_table_headers": [
                  {
                    "display_name": "Instance price model:",  // The display name of a table 
                    "name": "instancePriceModel",             // The name of the table that the server sends in the response 
                    "headers": []                             // Contains list of attributes to display in output.
                  }
                ],
                "headers_customization": {                      // Optional. Used to alter appearance of data in columns.
                  "the_name_of_header": {                       // The header name
                    "header_display_name": "Custom name of the header", // Custom name of the header
                    "disable_numparse": true,                   // A flag. Disables automatic conversion of string-values to numbers.
                    "prevent_list_formatting": true             // A flag. Disables automatic alignment of list values to a column.
                  }
                },
              "unmap_key": "Contains 'key' to extract response if needed."
            }
        }
    },
    "version": "major.minor"                            // see the versioning rules below
}NOTE: Commands help values can be stored into the file namedcommands_help.py. The format of the such storing type is Python native. Example:run-instance = """
Use this command to run instance.

Examples:
1. Describes all available instances for the certain tenant in the specified region
    m3 run-instance -tn <tenant-name> -r <region> -iname <instance-name> -shname <shape-name> -key <key-name> -count <number-of-instances>
"""Related commands section in "full-help" contentsThe related commands section is constructed automatically based on thegroupsattribute of each command.
The related commands look like this:Related commands:

1. Deletes a schedule from your tenants schedule library:
        m3 delete-schedule -r <region> -tn <tenant> -n <name>There are two ways a command can be added to a list of related commands of the other command.The first approach requires to declare a group of commands in thegroupattribute in the root of thecommands_deffile,
and then assign it to all commands that are in this group. All commands within the group share a bidirectional relation with each other.
Consider the next example:{
  "groups": ["group_A"],
  "commands": {
    "first_command": {
      "groups": ["group_A"],
      // ...
    },
    "second_command": {
      "groups": ["group_A"],
      // ...
    }
  } 
}With this configuration thefirst_commandwill be in the list of related commands of thesecond_commandand vice versa.The second approach allows us to define a unidirectional relation between commands (auxiliary commands). We make one command related to another
by adding the name of the other command to the list of groups of this command with a special prefix and suffix (cli-and-help).
Consider the next example:{
  "commands": {
    "first_command": {
      "groups": ["cli-second-command-help"],
      // ...
    },
    "second_command": {
      // ...
    }
  } 
}With this configuration we will see thefirst_commandin the list of related commands of thesecond_command.
At the same time thesecond_commandwill not be in the list of related commands of thefirst_command.Default command parameter valuesThe default values for the command parameters can be stored
in them3.propertiesfile....
tenant-name = SFTL-SLCTL
region = SFTL-OPENSTACK-SLCTL
...You can create severalm3.propertiesfiles in different directories with
different default values. To select the required parameters change the
current working directory to the directory with the appropriatem3.propertiesfile.Other option is to store the default values for the command
parameters in thedefault.crfile.{
    ...
    "tenant-name": "SFTL-SLCTL",
    "region": "SFTL-OPENSTACK-SLCTL",
    ...
}The default parameter values from them3.propertiesfile take precedence
over the values from thedefault.crfile.Integration request/responseFrom version 2.1.0 there is possibility to process CLI input and Server output
manually using custom Python code. To use this feature you need to create plugin
with an appropriate properties:Create the directory namedpluginsby the path that were specified as a
value for the env variableM3CLI_CONFIGURATION_FOLDER_PATH;Create simple python module (filename.py) and put it into the directory
namedpluginsmentioned above;To change the CLI input create a method inside your script with
namecreate_custom_requestthat will receive one parameterrequest. For
example:def create_custom_request(request):
    return requestTo change the CLI output create a method inside your script with
namecreate_custom_responsethat will receiverequestandresponseparameters. For
example:def create_custom_response(request, response):
    return responseFrom the version 3.41.7 there is possibility not to hide the boolean fields if
the value is 'False'. To use this feature you should follow the next step:Addnullablefield with the valuetrueto theoutput_configurationsection of thecommands_def.jsonfile. For example:"output_configuration": {
        "nullable": true,
        "response_table_headers": [
          "region",
          "nativeName",
          "cloud",
          "active",
          "hidden"
        ]
      }In example above, if the value of fieldactivewill befalse, the value in
the table column will not be hidden.Command output formattingIn case CLI receives a response with items which have an empty attribute, this attribute will be dropped
from the response (in table view only).To add custom formatting to a response attribute, add theheaders_customizationparameter tooutput_configuration."output_configuration": {
    "response_table_headers": ["header-1", "header-2"],
    "headers_customization": {
      "header-1": {
        "disable_numparse": true,
        "prevent_list_formatting": true
      }
    }
}Thedisable_numparsesetting disables automatic conversion of string-values to numbers.
Theprevent_list_formattingsetting disables automatic alignment of list values to a column.Interactive optionsInteractive optionsis a feature that enables a command to fetch additional
parameters from the server and either prompt a user to interactively provide values for
these parameters or create a file with additional parameters (avarfile) for future use.To enable a command to fetch additional parameters, add the following attribute to the command's definition:"interactive_options": {
    "option_name": "params", - The name of parameters group 
    "parameters_handler": "GET_SERVICE_VARIABLES_INFO", - The name of server handler the returns the list of additional parameters
    "validation_handler": "VALIDATE_SERVICE_VARIABLES", - The name of server handler the validates the list of additional parameters
}If this feature is enabled, the command will get additional parameters via theparameters_handler,
ask the user to fill the values for these parameters or to provide a varfile, validate the additional parameters
and send them back to the server as a part of the original request.To enable a command to create a varfile, add the following attribute to the command's definition:"interactive_options": {
    "parameters_handler": "GET_SERVICE_VARIABLES_INFO", - The name of server handler the returns the list of additional parameters
    "generate_varfile": true, - Marks the command as a generator of a file with additional parameters fetched with the "parameters_handler"
}The varfile is a JSON file, and it contains all the additional parameters that theparameter_handlerprovides.
The parameters are either filled with default values or have empty values.m3cli autocompleteBy default in m3-cli tool  will be enabled autocomplete
that supports only Unix-based platforms (bash&zsh interpreters).To activate it do a few steps:Create virtual environmentInstall m3-cliCreate SYMLINK to virtual environmentsudo ln -s your_path_to_venv/bin/m3 /usr/local/bin/m3Start new terminal sessionExecute commandsudo m3 enable-autocompleteRestart terminal sessionTo deactivateExecute commandsudo m3 disable-autocompleteRestart terminal sessionNOTE: The type of the return value in the functions described above should be the same as the received parameter (request/response)After successful plugins creation add an appropriate attributes to the
command description in the filecommands_def.json:"integration_request": trueor"integration_response": trueCLI update detection flowOnce a day when any of the CLI commands is invoked, CLI performs a check for an available update.
It includes making ahealth-checkcall to the server and using its response to refresh
the data stored in thedefault.crfile about the latest CLI version and distribution links.This data is used between the daily update checks to notify a user of a pending update
while avoiding the need to communicate with the server on each request.An update notification is printed to the user first before the actual response, and contains
instructions of how to update the CLI client on a Linux, Windows or macOS machine.If thehealth-checkcall fails due to a server-side error while CLI performs a daily update check,
a warning is printed informing the user of the error and that CLI is unable to automatically check for updates.To force CLI to check for an update, invoke thehealth-checkcommand manually.CLI development flowUpdate thecommands_def.jsonfile with new commands/parameters that correspond
to the current release and update the version of the file.Update the commands' plugins to modify the models of requests or responses if required.Update the version attribute insetup.py.Update CHANGELOG.md with the changes being made to configuration of commands.Delivery flowChange directory to the project root folder. Filesetup.pyshould be in the
same folder as you arePrepare the dist with the command:python setup.py sdist --formats=zipLogin to epmc-eoos AWS account; (may be changed)Update the filem3-<version>.zipin S3 bucket namedm3-cli-distributionResolving m3cli version for CLISDK Access wizardTests executionWithout Coverage:To run tests just make sure you havem3cliinstalled in your virtual
environment. No other dependencies are required. Execute the following
(add flag-vto see a more detailed output):python-munittestYou should see the output with successful result - something like this:..............................................................................................................................
----------------------------------------------------------------------
Ran137testsin0.259

OKUsing Coverage:To run tests usingCoveragefirst
you have to install the library to the same virtual environment where m3cli is:pipinstallcoverageThen execute the command below:coveragerun-munittest-vIf no tests are failed you can generate a console report:coveragereport--omit"tests*"To generate the report for web-interface use the command:coveragehtml--omit"tests*"You should seehtmlcovfolder in the project's root. Findindex.htmlin
there and open via browser. May your tests always pass successfully!Gitlab CI/CD integrationUse the.gitlab-ci.ymlfor configuringGitlab CI/CD.Use thetox.inifile for automating and standardizing testing with atox.