Package: fair-cli
FAIR Data Pipeline Command Line InterfaceFAIR-CLI forms the main interface for synchronising changes between local and shared remote FAIR Data Pipeline registries, it is also used to instantiate model runs/data submissions to the pipeline. Full documentation of the FAIR Data Pipeline can be found on the projectwebsite.InstallationThe package is installed using Pip:pipinstallfair-cliTo enable tab completion you need to modify your shell:Bash_FAIR_COMPLETE=bash_source fair > ~/.config/.fair-complete.bash
echo '. ~/.config/.fair-complete.bash' >> ~/.bashrczsh_FAIR_COMPLETE=zsh_source fair > ~/.fair-complete.zsh
echo '. ~/.fair-complete.zsh' >> ~/.bashrcFish_FAIR_COMPLETE=bash_source fair > ~/.config/fish/.fair-complete.fish
echo '. ~/.config/fish/.fair-complete.fish' >> ~/.bashrcUninstallationTo uninstall the CLI run:fair purge --all
pip uninstall fairThe User Configuration FileJob runs are configured viaconfig.yamlfiles. Upon initialisation of a project, FAIR-CLI automatically generates a starter configuration file with all requirements in place. To execute a process (e.g. perform a model run from a compiled binary/script) an additional key of eitherscriptorscript_pathmust be provided. Alternatively the commandfair run bashcan be used to append the key and run a command directly.By default the shell used to execute a process isshorbatchfor UNIX and Windows systems respectively. This can be overwritten by assigning the optionalshellkey with one of the following values (where{0}is the script file):ShellCommandbashbash -eo pipefail {0}javajava {0}juliajulia {0}powershellpowershell -command ". '{0}'"pwshpwsh -command ". '{0}'"python2python2 {0}python3python3 {0}pythonpython {0}RR -f {0}shsh -e {0}batch{0}A full description ofconfig.yamlfiles can be foundhere.Available CommandsinitInitialises a new FAIR repository within the given directory. This should ideally be the same location as the.gitfolder for the current project, however during setup an option is given to specify an alternative. The command will ask the user a series of questions which will provide metadata for tracking run authors, and also allow for the creation of a starterconfig.yamlfile. Initialisation will also configure the CLI itself.Custom CLI ConfigurationAfter setup is complete, the current CLI configuration can also be saved using the command:fair init --exportthe created file can then be re-read at a later point during setup. Alternatively, if creating a configuration from scratch the YAML file should contain the following information:namespaces:input:testingoutput:testingregistries:local:data_store:/path/to/local/data_store/,directory:/local/registry/install/directoryuri:http://127.0.0.1:8000/api/origin:data_store:/remote/registry/data/store/path/token:/path/to/remote/tokenuri:https://data.fairdatapipeline.org/api/'user:email:'test@noreply',family_name:'Test'given_names:'Interface'orcid:None,uuid:'2ddb2358-84bf-43ff-b2aa-3ac7dc3b49f1'git:local_repo:/local/repo/pathremote:origindescription:Testing Projectthis file is then read during the initialisation:fairinit--using<cli-config.yamlfile>For integration into a CI workflow, the setup can be skipped by running:fairinit--ciwhich will create temporary directories for some of the required location paths.runThe purpose ofrunis to execute a model/submission run and submit results to the local registry. Outputs of a run will be stored within thecoderunfolder in the directory specified under thedata_storetag in theconfig.yaml, by default this is$HOME/.fair/data/coderun.fairrunIf you wish to use an alternativeconfig.yamlthen specify it as an additional argument:fairrun/path/to/config.yamlYou can also launch a bash command directly, this will be automatically written into theconfig.yaml:fairrun--script'echo "Hello World"'note the command itself must be quoted as it is a single argument.By default the CLI will not allow the user to perform a run if the state of the analysis repository is such that it is behind the git remote, or contains uncommitted changes. To override this behaviour use the--dirtyflag.pullThe commandpullwill update any entries within theconfig.yamlunder theregisterheading creatingexternal_objectanddata_productobjects on the registry and downloading the data to the local data storage. Any data required for a run is downloaded and stored within the local registry. In addition any data products requested that are available on the remote registry are pulled locally.fairpull/path/to/config.yamlstatusThis command displays objects which are awaiting staging or have been staged behaving in a manner similar togit status:fairstatusstaged changes are displayed in green, and unstaged in red.addBefore changes can be pushed to the remote registry they must be staged. This command allows you to stage objects displayed when runningfair statusso that they can be sent to the remote registry. Data products are displayed and staged in the formnamespace:data_product_name@version:fairaddmy_namespace:data_object@v0.1.0pushThepushcommand will push any staged data products to the remote registry:fairpushpurgeThepurgecommand removes setup of the current project so it can bereinitialised:fairpurgeTo remove all configurations entirely (including those global to all projects) run:fairpurge--globalTo remove the data directory itself run:fairpurge--dataWARNING: This is not recommended as the registry may still have entries pointing to this location!Finally to remove everything run:fairpurge--allthis will remove the current repository.fairfolder and the global FAIR directory which also contains the local registry.You can skip any confirmation messages by running:fairpurge--yesregistryBy default the CLI will launch the registry whenever a synchronisation or run is called. The server will only be halted once all ongoing CLI processes (in the case of multiple parallel calls) have been completed.However the user may also specify a manual launch that will override this behaviour, instead leaving the server running constantly allowing them to view the registry in the browser.The commands:fairregistrystartandfairregistrystopwill launch and halt the server respectively.The registry can be installed using the CLI as well by running:fairregistryinstallwith the additional options to specify the installation location, and the data registry repository tag to install from:fairregistryinstall--directory~/.fair/my_registry--versionv1.0-rc5logRuns are logged locally within the local FAIR repository. A full list of runs is shown by running:fairlogThis will present a list of runs in a summary analogous to agit logcall:run 0db35c20946a1ebeaafdc3b30103cd74a57eb6b6Author:Joe Bloggs <jbloggs@noreply.uk>Date:Wed Jun 30 09:09:30 2021NOTEThe SHA for a job isnotrelated to a registry code run identifier as multiple code runs can be executed within a single job.viewTo view thestdoutof a run given its SHA as shown by runningfair loguse the command:fairview<sha>you do not need to specify the full SHA but rather the first few unique characters.Template VariablesWithin theconfig.yamlfile, template variables can be specified by using the notation${{ VAR }}, the following variables are currently recognised:VariableDescriptionDATEDate in the form%Y%m%dDATETIMEDate and time in the form%Y-%m-%sT%H:%M:SDATETIME-%Y%H%MDate and time in custom format (where%Y%H%Mcan be any valid form)USERThe current user as defined in the CLIUSER_IDThe unique identifier for the current userREPO_DIRThe FAIR repository root directoryCONFIG_DIRThe directory containing theconfig.yamlafter template substitutionLOCAL_TOKENThe token for access to the local registrySOURCE_CONFIGPath of the user definedconfig.yamlGIT_BRANCHCurrent branch of thegitrepositoryGIT_REMOTEThe URI of the git repository specified during setupGIT_TAGThe latest tag ongit