Proposal for new kubecfg design (kubecfg2)

*The attached commit is for prototype/demonstration purposes only and not yet ready to be merged.*

### Intent

The intent of this pull request is to propose a design for the present and future of kubecfg. The conversation from #1167 was the primary input into this design. The idea is to have kubecfg solve two use cases:

1. Provide a great user experience to view the current status of a Kubernetes cluster and to execute simple commands on the cluster.
1. Provide a tool which will accept configuration with generic resources to submit changes to and diff against a Kubernetes cluster.

The problem with the current design is that it is too generic to effectively support (1). The proposed redesign enables custom commands per resource (e.g. kubecfg pods, kubecfg services, etc.) but generic subcommands to satisfy (2) (kubecfg submit, kubecfg diff, kubecfg reconcile, etc.).

### Proposed set of subcommands

*Only a subset of these have been implemented in the attached commit.*

* `kubecfg2` - Parent command.
  * `pods`
    * `list [<id>]` - List pods in a column-oriented view (see [Examples](#examples) below). By default, the first argument passed in acts as a filter by ID.
    * `get <id>` - List info about a pod in a row-oriented view. Designed to show more metadata about a given pod, and over time can be enhanced to make navigating the cluster from a given pod very easy. Currently I'm adding a list of any replication controllers whose selectors match (and therefore who control) the selected pod.
    * `delete <id>` - Delete a pod.
    * `create [-f <filename>]` - In the future we can accept directories or even data from stdin.
  * `rc` - For replication controllers. Happy to use a different abbreviation but replicationControllers seemed too long.
    * `list [<id>]`
    * `get <id>` - RC's `get` command could become extremely rich. For now I've added Pods Status which queries for pods under this replication controller and shows counts for running, waiting and terminated, but in the future you could do diffs against the controller's PodTemplate and show counts for matching and non-matching pods. You could also show the first 3-4 pod ID's per category and then show &lt;X more ...&gt;, making debugging even easier.
    * `create [-f <filename>]`
    * `update [-f <filename>]`
    * `delete <id>` - I have removed the delete/stop/rm replication controller confusion in favor of just `delete` and `resize`, where delete takes an optional --force flag if replicas > 0.
    * `resize <id> <replicas>`
    * *Removed* `rollingupdate <controller> <image> <time>` - I've included this here for parity but I would like to propose taking it out. This is the one command that I don't think belongs in kubecfg ... it's too long-running, fraught with edge cases, and in reality would probably want to take more into consideration (like fluctuations in metrics, % of pods that come up successfully, etc.). I think this level of functionality belongs in an independent client that uses the API that can look at more components in the infrastructure than just kubernetes.
  * `minions`
    * `list [<id>]`
    * `get <id>`
  * `services`
    * `list [<id>]`
    * `get <id>`
    * `create [-f <filename>]`
    * `update [-f <filename>]`
    * `delete <id>`
    * *Postponed* `resolve <id>` - Just an idea, but it could give you a random IP/port combination to connect to. May be useful for quickly locating a node to test for a given service.
  * *Postponed* `submit [<filename>...]` - This command exists to satisfy the second use case above. Reconcile and submit any changes from a given set of config(s), from files or from stdin. If you had an entire directory tree of files that had configs that represented your cluster state, you could use this command to submit them all to either create or update your cluster.
  * *Postponed* `diff [<filename>...]` - A dry-run version of `submit`.
  * `run [-p <port spec>] <image> <replicas> <controller>` - Same as today.

### Examples

Here is what the current commit produces:

*These examples are out of date - see [this comment](https://github.com/GoogleCloudPlatform/kubernetes/pull/1325#issuecomment-56786115) for the latest.*

```
$ kubecfg2
kubecfg2 controls the Kubernetes cluster manager.

Find more information at https://github.com/GoogleCloudPlatform/kubernetes.

Usage:
  kubecfg2 [command]

Available Commands:
  version                   Print version of client and server
  pods                      List, inspect and modify pods
  rc                        List, inspect and modify replication controllers
  help [command]            Help about any command

 Available Flags:
  -a, --auth="/Users/sam/.kubernetes_auth": Path to the auth info file. If missing, prompt the user. Only used if doing https.
      --help=false: help for kubecfg2
  -h, --host="": Kubernetes apiserver to connect to
  -m, --match-version=false: Require server version to match client version

Use "kubecfg2 help [command]" for more information about that command.

```
```
$ kubecfg2 help pods list
List one or more pods. Pass in an ID to filter.

Usage:
  kubecfg2 pods list [<id>] [flags]

 Available Flags:
  -a, --auth="/Users/sam/.kubernetes_auth": Path to the auth info file. If missing, prompt the user. Only used if doing https.
      --help=false: help for list
  -h, --host="": Kubernetes apiserver to connect to
  -m, --match-version=false: Require server version to match client version
  -l, --selector="": Selector (label query) to filter on


Use "kubecfg2 help [command]" for more information about that command.

```
```
$ kubecfg2 pods list
ID                                      IMAGE(S)                HOST            LABELS                                  STATUS
4893ea09-3d0d-11e4-b3ad-0800279696e1    dockerfile/nginx        127.0.0.1/      replicationController=MyNginxController Running
48943371-3d0d-11e4-b3ad-0800279696e1    dockerfile/nginx        127.0.0.1/      replicationController=MyNginxController Running
```
```
$ kubecfg2 pods list 4893ea09-3d0d-11e4-b3ad-0800279696e1
ID                                      IMAGE(S)                HOST            LABELS                                  STATUS
4893ea09-3d0d-11e4-b3ad-0800279696e1    dockerfile/nginx        127.0.0.1/      replicationController=MyNginxController Running
```
```
$ kubecfg2 pods get 4893ea09-3d0d-11e4-b3ad-0800279696e1
ID:                             4893ea09-3d0d-11e4-b3ad-0800279696e1
Image(s):                       dockerfile/nginx
Host:                           127.0.0.1/
Labels:                         replicationController=MyNginxController
Status:                         Running
Replication Controllers:        MyNginxController (2/2 replicas created)
```
```
$ kubecfg2 rc list
ID                      IMAGE(S)                SELECTOR                                REPLICAS
MyNginxController       dockerfile/nginx        replicationController=MyNginxController 2
```
```
$ kubecfg2 rc get MyNginxController
ID:             MyNginxController
Image(s):       dockerfile/nginx
Selector:       replicationController=MyNginxController
Labels:         <none>
Replicas:       2 current / 2 desired
Pods Status:    2 running / 0 waiting / 0 terminated
```

### Other notes

* I am currently using [spf13/cobra](https://github.com/spf13/cobra) but am also considering [codegangsta/cli](https://github.com/codegangsta/cli). The latter may allow us to write out the command definitions in a cleaner way and support bash autocompletion, but the former allows persistent flags. Open to using either one.
* Some of these commands (especially `get` commands that show a lot of info) may result in lots of queries to the server. There are several ways to handle this: (1) don't consider it an issue, (2) offer a "succinct mode" that tries to do as little as possible and (3) over time backport functionality that is particularly useful directly into apiserver as new API endpoints. Especially with (3), you could consider kubecfg a proving ground for new convenience functionality to see whether it's worth implementing in apiserver directly.

### File layout

* cmd/kubecfg2/kubecfg2.go - All cobra and CLI-related code remains in this file/package.
* pkg/kubecfg2/&lt;subcommand&gt;.go - All logic per subcommand lives in one file per subcommand.
* pkg/kubecfg2/kubecfg2.go - Common functions used by cmd/kubecfg2 and pkg/kubecfg2 packages.

### Open Questions
* Can we leave rollingupdate out of this version for the merge? If it's needed for demo purposes, can it be reimplemented somewhere outside of this tool?
* @erictune pointed out that `get` is not the best name for above. `info` and `details` are other options but ideas here are welcome.

Sam and I spoke today, so I wanted to jot some notes.

Looking at the read commands (get, list), I feel like there are 3
orthogonal things going on in each command.

1) What data to fetch - this is a logical "join" across sources (e.g.
getting data about replication controllers to show alongside a pod)

2) Which fields you want to show.

3) How to format the output.


Get and List each have different defaults for these 3 things, which is OK.
But It should *probably* be possible to manipulate all three axes.  I say
probably because Sam raised 2 good points - first, it becomes possible to
accidentally hammer the API and second, it adds some complexity that we
might not need up front.

Still I want to espouse my desire that we build the tool on clean
principles and layers, even if we don't support the full matrix at first.

GET:  Gets a single object
* Default data fetch mode is "full join", where "full" is defined
statically in code.  We do not want to offer arbitrary join syntax, I think.
* Default field selection is built-in.
* Default format is "list" or "long"

LIST: Gets a number of objects
* Default data fetch mode is "simple", just get the object in question
* Default field selection is built-in.
* Default format is "table"

For both commands, it should probably be possible to mix-and-match.
* Fetch mode probably only has two options: "full" and "simple"
* Field selection probably has two modes: "default" and "template"
* Format probably has options for "long", "table", "key-value", "json",
"yaml", "csv", ...

It should be possible for me to ask for GET in simple mode, displaying
specific fields, in table form.  I acknowledge that we may not want to
allow all modes at first, maybe not ever.

Focus should be on great *default* experience, with power and flexibility
coming second.  To that end, push ahead, but leave room for the
orthogonality to come in later.

Sam did I miss anything?

We do still need to settle command structure, flags, singular vs plural,
cached credentials, etc etc - all the little things that, if done right,
nobody will notice (but if done badly will annoy everyone :)

Tim

On Thu, Sep 25, 2014 at 10:01 PM, bgrant0607 <notifications@github.com>
wrote:

> In cmd/kubecfg2/kubecfg2.go:
>
> > +	servicesCmd.AddCommand(servicesGetCmd)
> > +	servicesCmd.AddCommand(servicesCreateCmd)
> > +	servicesCmd.AddCommand(servicesDeleteCmd)
> > +
> > +	return servicesCmd
> > +}
> > +
> > +func NewCmdMinions() *cobra.Command {
> > +	minionsCmd := &cobra.Command{
> > +		Use:   "minions",
> > +		Short: "List, inspect and modify minions",
> > +		Long:  "List, inspect and modify minions.",
> > +		Run:   runHelp,
> > +	}
> > +
> > +	// List command needs to be custom because client.ListMinions() does not
>
> Please file an issue for this. Minions don't have labels right now, but
> will in v1beta3 (#1225
> <https://github.com/GoogleCloudPlatform/kubernetes/pull/1225>).
>
> --
> Reply to this email directly or view it on GitHub
> <https://github.com/GoogleCloudPlatform/kubernetes/pull/1325/files#r18073849>
> .
>