docker/docker — pull request #9232 by joshwilsdon (2014-11-19 07:12:17+00:00)
Title: Proposal: Make API for `/containers/(id)/top` independent of implementation

# Background and Problem

Issue #7205 and discussion therein, describes some problems with the current API with regard to the:

```
GET /containers/(id)/top
```

API endpoint. I think these problems can be boiled down to a single issue which is that
the API includes ps_args which are not bounded and are not in control of the API itself.
The result is different output on different systems even when using the same arguments.
The existing implementation uses `ps` to gather the data but the API does not allow
alternate implementations because it is not fully defined.

This proposal also makes it possible to normalize the behavior across existing hosts,
as different docker hosts with slightly different versions of ps already exhibit minor
differences in response (see for example #8075).

As docker is ported to other platforms which may or may not have the same behavior and
may in fact not even have a `ps` command host-side, I think this will become more of an
issue. I'm making this proposal in attempt to make something which can be supported by
other platforms and also be implemented differently on Linux hosts in the future (Eg.
the /proc reader that was suggested in #7205).

# Existing Behavior

On an Ubuntu 14.04 host with docker 1.3.1 the default fields I get when I use
`docker top ` with no ps_args are:

* UID
* PID
* PPID
* C
* STIME
* TTY
* TIME
* CMD

Which is the equivalent to running `ps -ef` on the host.

The API docs for v1.15 also refer to `aux` as an example for ps_args. With that
the fields are:

* USER
* PID
* %CPU
* %MEM
* VSZ
* RSS
* TTY
* STAT
* START
* TIME
* COMMAND

which actually has very little overlap with the default set.

What I'd like to propose is that the `ps_args` parameter go away, to be replaced
by a `fields` parameter which defines the ordered set of fields that should be in
the output. The possible fields should be clearly defined and a finite set. I've
attempted to err here on the side of including fewer fields as it's much easier
to add them later than to include them now and take them away later.

I've also attempted to move away from implementation-specific formatting for
things like 'start_time' and 'cputime' which on Linux are formatted as "MmmDD"
or "HH:MM" for 'start_time' (depending how old) and '[DD-]hh:mm:ss' for
'cputime'. These are both confusing formats and also lose information that
implementations may otherwise have. For example, on the same Linux system
/proc//sched contains this information with microsecond precision.

# Proposal

I propose that initially the set of allowed 'fields' would include:

| field | type | description |
| ----------- | --------------------------- | --------------------------------------------------------------------------------------------------------- |
| c | integer | processor utilization over process lifetime |
| comm | string | command name (only the executable name) |
| command | string | command with all its arguments |
| cputime | number | cumulative CPU time for the process in seconds, with or without decimals |
| gid | integer | effective group ID number of the process as a decimal integer |
| lwp | integer | light weight process (thread) ID of the dispatchable entity |
| nice | integer | nice value. This ranges from 19 (nicest) to -20 (not nice to others) |
| pcpu | number | cpu utilization of the process as a percentage of its run time, with or without decimals |
| pid | integer | a number representing the process ID |
| pgid | integer | process group ID or, equivalently, the process ID of the process group leader |
| pmem | integer | ratio of the process's resident set size to the physical memory on the machine, expressed as a percentage |
| ppid | integer | parent process ID |
| psr | integer | processor that process is currently assigned to |
| rgid | integer | real group ID |
| rss | integer | resident set size, the non-swapped physical memory that a task has used in KiB |
| ruid | integer | real user ID |
| start_time | number | time the process started (seconds since 1970-01-01 00:00:00 UTC), with or without decimals |
| state | string (see 'state fields') | normalized state of the process |
| state_flags | string (see 'state fields') | implementation-specific process state flags |
| tty | string | controlling tty (terminal) |
| uid | integer | effective user ID |
| vsz | integer | virtual size of the process in KiB |

So the default for `fields` would be:

```
fields=uid,pid,ppid,c,start_time,tty,cputime,command
```

which contains all the data the client would require in order to generate the
existing default v1.15 output of 'docker top', whether the client would want to
display it this way or in some more useful fashion.

I'd also define a special option:

```
fields=all
```

which would be identical to all the fields:

```
fields=c,comm,command,cputime,gid,lwp,nice,pcpu,pid,pgid,pmem,ppid,psr,rgid,rss,ruid,start_time,state,state_flags,tty,uid,vsz
```

NOTE:

* The fields list purposefully does not include an analog to the "USER" ps
field, because it doesn't seem likely that most implementations (the current
one doesn't for example) will display the user names here from the guest's
context, and would fallback to UID anyway.

## state fields

In order to make the `state` and `state_flags` fields more generic and in
attempt to make them more portable these fields should be defined as follows
for `state` (`ps state` is included here for reference only):

| State |`ps state` | Meaning |
| --------------- | --------- | --------------------------------------------------------------------- |
| uninterruptible | D | uninterruptible sleep (usually IO) |
| running | R | running or runnable (on run queue) |
| sleep | S | interruptible sleep (waiting for an event to complete) |
| stopped | T | stopped, either by a job control signal or because it is being traced |
| zombie | Z | defunct ("zombie") process, terminated but not reaped by its parent |

And the `state_flags` should be a comma separated list of strings representing
It is possible that some implementations will not support these flags. As such,
clients should not fail when this list is empty. Supported values at this point
should include (`ps stat` here represents the Linux character used to indicate
this flag with the `stat` field, again only for reference):

| Flag | `ps stat` | Meaning |
| --------- | --------- | ------------------------------------------------------------- |
| high | < | high-priority (not nice to other users) |
| low | N | low-priority (nice to other users) |
| locked | L | has pages locked into memory (for real-time and custom IO) |
| leader | s | is a session leader |
| threads | l | is multi-threaded (using CLONE_THREAD, like NPTL pthreads do) |
| foreground | + | is in the foreground process group |

So an example `ps aux` output that included:

```
syslog 948 0.0 1.6 258300 17260 ? Ssl Nov06 0:08 rsyslogd
root 1037 0.0 0.0 0 0 ? S< Nov06 0:00 [dm_bufio_cache]
root 1114 0.0 0.0 14540 956 tty4 Ss+ Nov06 0:00 /sbin/getty -8 38400 tty4
```

The `state` field for all three of these problems would be 'sleep' and the
`state_flags` fields would be:

* leader,threads
* high
* foreground,leader

respectively.

# Backward compatibility (optional)

For the initial version (in order to support existing usage) ps_args could still
be supported but deprecated and defined in the above terms. Eg.

```
ps_args=aux
```

would be translated into:

```
fields=uid,pid,pcpu,pmem,vsz,rss,tty,stat,stime,time,command
```

however a specific limited set of ps_args values would need to be chosen here.

My own bias would be toward keeping compatibility only for older clients. Ie. have
the code use the old behavior for v1.15 and older clients and the new behavior only
for v1.16 and higher clients.

# Responses

Current responses look like (copied from the v1.15 API doc):

```
{
"Titles":[
"USER",
"PID",
"%CPU",
"%MEM",
"VSZ",
"RSS",
"TTY",
"STAT",
"START",
"TIME",
"COMMAND"
],
"Processes":[
["root","20147","0.0","0.1","18060","1864","pts/4","S","10:06","0:00","bash"],
["root","20271","0.0","0.0","4312","352","pts/4","S+","10:07","0:00","sleep","10"]
]
}
```

and I propose to change the output to something like:

```
{
"Processes":[
{
"uid": 0,
"pid": 20147,
"pcpu": 0.0,
"pmem": 0.1,
"vsz": 18060,
"rss": 1864,
"tty": "pts/4",
"state": "stopped",
"state_flags": "",
"start_time": 1416305160,
"time": 0.0001,
"command": "bash"
}, {
"uid": 0,
"pid": 20271,
"pcpu": 0.0,
"pmem": 0.0,
"vsz": 4312,
"rss": 352,
"tty": "pts/4",
"state": "stopped",
"state_flags": "foreground",
"start_time": 1416305220,
"time": 0.0002,
"command": "sleep 10"
}
]
}
```

which makes it easier for the client to know the which field is which in the
response. Combined with 'fields=all' this also allows a client to determine
which fields a given implementation/server-version supports.

I would also suggest the addition of an additional status code:

- **422** – invalid fields

for the case where invalid values are requested for `fields`.

# CLI changes

In order to support this new API, I propose the CLI interface to this endpoint be
modified from:

```
docker ps [ps ARGS]
```

to:

```
docker ps [FIELDS]
```

with the fields matching those defined by the API. The output here would be the requested
fields in the order they were requested. Eg:

```
$ sudo docker top 8601afda2b pid tty state state_flags cpu_time command
pid tty state state_flags cpu_time command
16623 ? S leader 0.03 sleep 99999
```

# Implementation notes

The initial implementation here in docker would be straight-forward since all these
fields map to ps fields and the initial implementation could be splitting apart the
fields from:

```
ps -Ho
```

With some translation on the server side. Specifically:

* cputime would need to be turned into a number from the 'HH:MM:SS' default format of 'cputime'
* start_time would need to be turned into a number from the default format of 'start_time'
* state would need to be a simple table-translation of single-character state -> string
* state_flags would be a simple table-translation of all the special 'stat' characters -> array of strings

Other implementations however would now be much simpler as we could for
example read data out of /proc and generate objects with the specified fields
without a problem.