Project: CyberArk Conjur (Provisioning / Key Management)
File: spike-cli-dockerless.md

* * - * | Term | Description | | -- | | | Epic | https://github.com/cyberark/conjur/issues/1892 | | Ruby CLI | https://github.com/cyberark/conjur-cli | | Go CLI | https://github.com/cyberark/conjur-cli-go | | Python CLI | https://github.com/cyberark/conjur-api-python3 | | | For building packages for multiple platforms (RPM, DEB, OS X) | | | Build a package from a Ruby gem | | | Packs Ruby application into a single executable for increased portability | This document is one piece in the bigger piece of the Dockerless puzzle. In this document we will discuss possible options for a containerless CLI and present a recommendation for which one we should pursue. In addition, regardless of how Conjur is deployed delivering and officially supporting a CLI that does not require Docker allows our solution to be portable and more easily attainable. The CLI design should answer the following requirements: Core functionality We should support: Configure connection to Conjur (know as `init`, `authn` (`login`, `authenticate`)) Load policy (known as `policy <verb>`) Get/Set secret values (known as `variable values <var_id>` functionalities. Open for discussion: List all resources that I have "read" permissions on (known as `list`) Show resource metadata/annotations (known as `show`) Rotate user and host API key (known as `rotateapikey`) Deployment Provide a manual and automated way to deploy the CLI Require a maximum of 2 steps *Recommendation:* cross-platform deployment (Linux, Windows, macOS) should all have the same flow Currently, we provide 3 CLIs that can be used to interface with Conjur, Ruby, Go, Python. We will explore and differentiate each in terms of their functionality, quality, UX, testing, security and what it is required to deploy them. | | Functionality | | - | | | Ruby CLI | Most expressive coverage | | Go CLI | Implementation is limited with poor architectural design. We could salvage what is currently in-place but it was recommended by the core contributor to rethink the architecture and even perform a complete rewrite. | | Python CLI | Implements half of the desired functionality but more mature than Go CLI. | The following are mappings of functionalities for both Go and Python CLIs. | | Quality | | - | - | | Ruby CLI | Widely used<br />*Status*: Stable | | Go CLI | Not "community" level certified <br />*Status*: POC | | Python CLI | *Status*: Community/alpha level | | | UX | | - | | | Ruby CLI | What we are familiar with. Unintuitive and inconsistent CLI command construction. See list below for more thorough examples | | Go CLI | Similar to Ruby CLI | | Python CLI | More straightforward approach. For example, when setting a Conjur secret, `conjur-cli variable set my_id supersecretvarid`. See list below | The following are examples of just some of flaws in our current Ruby/Go CLI UX. Verb/noun placement Our Ruby and Go CLI commands are current inconsistent in their construction. It is not consistent if the noun or the verb comes first. For example, we currently have `conjur variable` (noun) and `conjur show` (verb) Nested subcommands Nested subcommands lead to a confusing UX. In Conjur, our CLI commands can reach to 2 nested subcommands which makes them hard to understand and hard to

For example `conjur variable value <var_id>` Unintuitive command construction When running `conjur variable value <variableid>`, a secret value will be returned but when looking at just the command, it is unclear what the purpose of the command is. For a more straightforward approach, the command for displaying a secret value could be `conjur get <variableid>` or `conjur show <variable_id>`. Another example of uninuitive command construction would be trying to understand what the difference between `conjur variable value` is and `conjur variable values` without using the `--help` flag or reading documentation. Permitted edge cases We are currently not blocking the chaining of illogical flagging. For example, `conjur policy load --delete --replace foo foo.yml` does not lead to an error. The Python CLI has a more straightforward experience. For example, when setting and getting variables, the user will run ``` conjur-cli variable set myvarid mysupersecretvar conjur-cli variable get myvarid ``` For policy maintaince the outcome of the command is very clear. Unlike the Ruby/Go CLI, there is no chance of an unpredictable outcome because running `replace` and `delete` would lead to an error. ```bash conjur-cli policy apply root ./pathtopolicy.yml conjur-cli policy replace root ./pathtopolicy.yml conjur-cli policy delete root ./pathtopolicy.yml* ``` | | Deployment | | - | | | Ruby CLI | Currently, the Ruby CLI can be deployed in the following ways:<br />1. Deploy to Docker container (not applicable for our use-case)<br />2. Deploy with specific Ruby version (`gem install conjur-cli`)<br />Which means that the CLI is not plug-and-play and has prerequisites | | Go CLI | The CLI is compiled down to a binary. A different binary is need to be compiled for each platform but no other requirements or installers are needed to launch the Go CLI. | | Python CLI | The Python CLI can deployed in the following ways:<br />1. Install CLI via PIP (Python is required on machine)<br />2. Runnable executable (Python is not required on the machine)<br />The CLI is delivered as a , packed with the required runtime and dependencies. For this option there is no need for Python or other prerequisites to be installed on machine. | There are two options for deploying the Ruby CLI: Deploy to Docker container (not applicable for our use-case) Running `gem install conjur-cli` Once this is done, to connect to Conjur, the user will need to run `conjur init` + `conjur authn login` which will generate `.netrc` and `.conjurrc` files which hold user-related and certificate data. To deploy the Ruby CLI, a certain Ruby version needs to be installed on the machine. Because we cannot assume machines have connection to the internet, we will need to either figure out how to either: Create an executable that contains all dependencies. See mini-spike below. Create a package that will install the CLI and all its dependencies. For REHL and Centos, we would supply users with a RPM package. For macOS, we would supply them with PKG and so on. We will need to provide different installation packages for each platform (rpm, pkg, deb, etc). Create zip that will extract all content into a target folder and load CLI and its dependencies. Ruby bumps versions often and it is likely that the newer version of platforms will come with the newer Ruby versions. The newer version of Ruby might break a dependency we relay on to run the CLI. If this is the case, the Ruby CLI will not work, making it difficult to make this Ruby CLI

We can request that users install a certain version but changing Ruby versions is not a non-trival undertaking. This would mean that a specific machine would be dedicated to running the CLI if the user doesn't want a system that they cannot upgrade without problems. A mini-spike was pursued to evaluate how complicated it would be to the pack the conjur-cli rails application to a create a single executable. The project was chosen for this mini-spike. The following conclusions have been reached: The last stable for this packer was over 3 years ago The project itself comes from a single individual and so it is unclear if security was in mind during development The executable that is created is not portable. We tried to build it on different platforms (macOS, Ubuntu 20.04, 16.04, 20.10, Centos8) and the executable resulted in different errors on each one of them The platform setup to pack the executable is flakey and different from platform to platform. The ruby-packer requires ruby version 2.7.1 in the environment which makes us reliant on the packers version of choice. This can be problematic if that version becomes outdated or if there are security issues in that version. In short, we need something to work and work well across the different platforms and this isnt it. The Go CLI is compiled down to a single executable binary. Go gives us the so we don't need to hop to specific platform to compile the code. The Python CLI is , packed with the required runtime and dependencies. For this option there is no need for Python or other prerequisites to be installed on machine. Each executable will need to be bundled on each supported platform. *NOTE*: Many platforms (except for Linux) require signed executables so might be an extra step in the deployment process To use the Python CLI, users can either Install the CLI via PIP Install and run the executable Currently, there is no CLI command to generate the `.conjurrc` and `.netrc` files (`conjur init` and `conjur authn login` respectively). Instead, they can manually create these files or provide the information usually found in their `.netrc` / `.conjurrc` files as command line arguments. For example: ``` ./python-cli-executable-location --insecure -l https://conjur.myorg.com ... -u sigal.sax -p 2ec....gv63 \ variable get foo/bar ``` | | Testing | | - | | | Ruby CLI | Tested very well | | Go CLI | Pipeline maintained but the testing is not at the level it needs to be. Existing functionality will demand additional testing. <br />Considering the limited implemented functionality, new tests will need to be added if we choose to expand. | | Python CLI | Tested very well | None of our CLIs test against other platforms besides Linux. | | Security | | - | -- | | Ruby CLI | See scan | | Go CLI | See scan | | Python CLI | See scan | The CLI should support the following commands: Core - `init`, `authn` (`login`, `authenticate`), `policy (load)`, `list`, `show `, `rotateapikey` (`host` / `user`) and `fetch secrets` Additional we might want to consider - `resource`, `role`, `check`, `help` The following table summarizes the findings criteria detailed above and translates them to numbered

1 - the least work 3 - the most work | | Functionality | Quality | UX | Deployment | Testing | Security | | - | - | - | - | - | - | -- | | Ruby CLI | 1 | 1 | 2 | 3 | 1 | See scan | | Go CLI | 3 | 3 | 3 | 2 | 3 | See scan | | Python CLI | 2 | 2 | 1 | 1 | 2 | See scan | Ruby CLI ~11 weeks Ramp-up on creating packages and create design for CLI (~1 week) Provide package (rpm, pkg, deb, etc) / (CLI + dependencies) (~3 week) Expand current testing if needed (~1 week) Build infrastructure for testing on new platforms (Windows, macOS, REHL environments) (~3 weeks) Run a few CLI commands on new supported platforms (Windows, macOS environments) (~2 weeks) Close open issues (~1 week) Python CLI ~13 weeks Python ramp-up and create design for CLI (~1 week) Implement logic to fill in functionality gaps (7 CLI commands, ~3 weeks) `init`, `authn` (`login`, `authenticate`), `show `, `rotateapikey` (`host` / `user`) Write associated tests for new implementation (~3 weeks) Build infrastructure for testing on new platforms (Windows, macOS environments) (~3 weeks) Run a few of them on different supported platforms (Windows, macOS, REHL environments) (~2 weeks) Close open issues (~1 week) Add documentation Go CLI - Complete rewrite (~18+ weeks) Create design for CLI CLI Ramp up Redesign CLI Implement new CLI commands and logic Write tests for new functionality Might need to redesign and build new infrastructure for environment testing (Windows, macOS, REHL environments) Run a few CLI commands on new supported platforms (Windows, macOS, REHL environments) Add documentation Ruby requires the least amount of work in terms of what is already fuctionally supported but its UX is not intuitive. As part of its deployment, the CLI requires a specific version of Ruby installed on the machine to run the CLI. Machines may not have internet access so we will need to pre-package Ruby and other dependencies and deliver as with an installer. Go CLI would require a full rewrite due to it's limited functionality, test coverage, and architectural flaws. Python has most of the basic actions needed to interface with the Conjur server. We will need to fill in the gaps but is architecturally more stable and well tested. The Python CLI we deliver an executable so the user will not need install additional dependencies. Python CLI because of its better design and it acts as a great point of entry. In terms of a model and architectural design, it is where we want to be so it is a great first step into reforming our APIs. We have *not* reached a decision about which CLI we will pursue. If we pursue the Python CLI, we need to figure out the extent of breaking changes that will be introduced, how they will affect our scripts/automation and what is the extent of the documentation updates that will need to take place (PM/PO) We will need to evaluate how best to deliver the Ruby CLI (single binary, package, zip, etc) The CLI that we choose will require documentation. Ruby CLI has documentation but we will need to update according to platforms we will support. Go CLI documentation can be taken from Ruby CLI because UX is the same. Python CLI hasn't been clearly documented. Therefore we will need to add documentation.