sched: separate thread statistics

"class thread" is big and messy, and as we add more per-thread statistics
in the following patch, will become even messier. This patch simply moves
the members related to statistics together to the end of the class.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
sched: per-thread switch statistics

We currently have a "sched_switch" tracepoint which can count (e.g.,
using trace.py) the frequency of context switches in the entire system.
This patch further adds a per-thread counter for the number of times
that this thread was switched in.

The performance cost of the extra counter is very small (a non-atomic
increment to a variable in the same cache-line already touched for the
cpu-time statistics), and tests/misc-ctxsw.so shows no measurable slowdown
of context switches.

Note that dividing the thread's runtime (t->thread_clock()) by the number
of switches results in another interesting statistic - the average length
of time this thread ran before being switched.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
top.py: add "-s" option for context-switch statistics

This patch adds to top.py the ability to show per-thread statistics
on the number of times it was context-switched in, and the average
cpu time it got per switch, over the last refresh period. For
example:

   ID CPU  %CPU    TIME     sw     sw/s us/sw NAME
  478   0   2.9    0.39    630     38.0   763 >>>>>>>java.so

"sw", 630, is the total number of times this thread was ever switched
in. "sw/s", 38.0, is the number of switches per second in the last
refresh period. In each of the time slices in the last refresh period,
the thread ran for an average of 763 microseconds before being either
preempted or voluntarily giving up the CPU (by waiting or yielding).

To reduce users' information overload, the switching information is not
displayed by top.py by default, and the user needs to pass the "-s"
(or "--switches" option) to enable it.

This feature can be useful in understanding the scheduling behavior of
a workload and the function of individual threads. For example, in one
particular Cassandra read-only workload on one CPU, I see the virtio-net-rx
thread being switched in 13,000 times a second running only 2 microseconds
each; I then see 12 Java worker threads each being scheduled only 30 times
a second and doing a fairly large amount of work (around 800 microseconds);
I then see dozens of other Java threads each scheduled very frequently
(over 600 times a second) and doing very little work (20 microseconds) -
presuably the latter threads just read a request and dispatch it to the
worker threads.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
scripts: clean spaces behavior to follow a bit of pep8

Signed-off-by: William Dauchy <william@gandi.net>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
scripts: clean brackets in run.py

no need for them; make the thing more "pythonic"

Reviewed-by: Tomasz Grabiec <tgrabiec@gmail.com>
Signed-off-by: William Dauchy <william@gandi.net>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Ad-hoc command execution

Enable ad-hoc command execution for CLI. With the previous commit which
enabled talking to a different instance (13a7576), this makes it possible to
execute remote commands while still using one's tools and shell.

For example (the executable name will be changed in future commits):

```
export OSV_API=http://192.168.122.89:8000
./cli dmesg | grep "foobar"
```

Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Properly parse string command line

For the interactive shell, split the line to arguments properly (treating
quotes properly). cli() now accepts a list or a string.

Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Refactor function names for consisitency

Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Improve ls command

Add -l, -r and -t flags.

Closes #488.

Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Fix table format padding

Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http server: List all fs API

This API allows calling http://localhost:8000/fs/df/ without file system
parameter.

It would go over all avaialbe file systems and return them as a list.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
aarch64: dtb: read amount of memory from device tree

start making use of device trees for reading the amount of memory.

FDT (Flattened Device Tree) is made available to the guest as a
DTB (Device Tree Blob).

Reading of the dtb requires some reshuffling.

The DTB is moved from its initial location to its final place in
physical memory at the location OSV_KERNEL_BASE, overwriting
the preboot code (preboot.S), and also overwriting the command
line and the like.
Therefore we need to parse the command line before the DTB move.

Some adaptation of the memory setup code is also needed,
to account for a dynamic amount of memory.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>