Fix thread::thread_clock() on a running thread

The thread_clock() function, returning the amount of CPU time spent by a
thread, under-estimated when called for a different running thread. In that
case, it only accounted for full time slices. This underestimate was
usually forgivable (the load balancer wakes up 10 times a second, so keeps
time slices under 0.1 seconds), but as explained in issue #366, we noticed
this problem as idle percentage in "top.py" hovering between 95%-105%,
instead of being at 100% as expected.

The problem in estimating a the cpu time used by a thread running on a
different CPU, is that we need a consistent set of two different values
(running_since and total_cpu_time), but each may become visible at a
different time, and at an unknown order. Additionally, running_since is
currently per-cpu, not per-thread.

Rather than adding additional memory fences to the context-switch path,
I went with the solution in this patch. It is not pretty, but it is the
most efficient solution I could find (surprisingly, I couldn't even measure
a nanosecond of slowdown of the context switch benchmark).
If anybody can suggest a prettier solution with similar performance,
please let me know.

The good performance of this patch happens because all it adds in to a
context switch is a single write, when switching into a thread. There are
no memory fences, or locked instructions, and the written memory is in a
cache-line which should be in the cache anyway. Additionally, there is no
extra code when switching out of a thread.

The idea is that when switching in a thread, we atomically write a
64-bit thread field comprised of both running_since and total_cpu_time,
32 bits of each. To make 32 bits enough we give up 10 bits of accuracy
(so thread_clock() on a running thread is accurate up to a microsecond),
and 22 bits of range (so the range is 2200 seconds - way more than our
context switches). This code, cpu_estimator_set(), is very simple and quick
and I couldn't even measure one nanosecond of slowdown to the context switch.

Fixes #366.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Update mgmt.git head

The failing "make clean" is annoying, and solved in the newer mgmt.git.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
net_device_options
disable msleep
Revert "disable msleep"

This reverts commit a24f287945f8c970c3e7a1e230971dd576a0a497.
Add trace_in_pcblookup_hash_locked

Signed-off-by: Asias He <asias@cloudius-systems.com>
_ifn->if_hwassist = CSUM_IP

Signed-off-by: Asias He <asias@cloudius-systems.com>
Revert "_ifn->if_hwassist = CSUM_IP"

This reverts commit e915e760519588fb71267a84a01d847dfe7f44c1.
LIST count
Revert "LIST count"

This reverts commit fefd64f49d4542eb96913b3d15d18a76fd7fbf28.
rcu hashtable

This generic hash table does not follow STL conventions, since using an
iterator is difficult where things can change under your feet.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: use rcu_hashtable in net channel classifier

Reduce overhead in creating a tcp connection from O(existing connection count)
to O(1).

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Add mac option to Xen
Route cache patch

Signed-off-by: Asias He <asias@cloudius-systems.com>