net: fix poll() abort on net channels

While the net channel poller list (a vector<pollreq>) is managed via rcu,
do_poll() modifies the pollreq without protection when it clears
pollreq::_poll_thread.  This can cause net_channel::wake_pollers() to wake
a null thread.

Fix by converting pollreq::_poll_thread to a sched::thread_handle (which has
internal rcu protection).

Seen running wrk against a static tomcat page (the home page).

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
balloon: protect against ill-sized moves

Carl noticed that it is legal for a java program (think for instance, somebody
using JNI), to move stuff manually from the middle of the balloon. In that case,
we would have a move that is smaller than the size of the object. We haven't put
much tought on how to handle this case - if at all. But it is certainly better
to fault there than to have a silent weird behaviour.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests/misc-lfring.cc: Delete the test for ring_mpsc.

We are about to kill ring_mpsc - delete the corresponding tests.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
ring_spsc/mpsc: Fix pop() method failing after the ring index wraps around

- Fixed ring_spsc: "empty" check should be "_beg == _end" and not "_beg >= _end".
- Deleted ring_mpsc since it's not currently used anywhere.

Fixes #225.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: added a test verifying the fix for issue #225

Running 'make check' after the fix:
...
TEST tst-ring-spsc-wraparound.so OK  (8.817 s)
...

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
balloon: remove obsolete assert

I am currently being able to trigger this assert. It comes from the old days
before we found the opportunistic copying bug. Right now, it *is* possible to
have the balloons list to be empty, and still some old VMAs lying around.

Just remove the assertion.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
move malloc pools to mempool.cc

No reason to expose it in the headers.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
run: don't redirect port 8080 by default

Conflicts with Jenkins.

Add an option instead.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
loader: remove unused declaration

Removed an unused declaration, which is unnecessary and causes a warning
in Eclipse.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
zfs: Boost ZFS mount time by fixing zfs_sync functionality

The main reason behind the slowness on ZFS mount times is that zfs_sync() was
definitely broken. From a ZFS perspective, sync should wait for the underlying
pool to commit all the dirty data.

Previously, a bunch of work was being handed off to the next OSv instance, thus
slowing down the mount process. When tracing the exactly location of the hang,
I found that the actual bottleneck was during the load of an existent SPA,
spa_load(). Going further, I discovered that ZIL replays (if any) are necessary
for the success of spa_load(). zfs_sync() should flush out all the ZIL records
so that there is no need to replay afterwards. That wasn't happening though.

The original implementation in BSD shows that spa_sync_allpools() should be
called when a mount point wasn't specified, i.e. NULL. However, it works
differently in OSv, this clause doesn't even exist.
spa_sync_allpools() will basically sync all pools.

ZFS mount time before the patch:
SSD -> ZFS mounted: 1943.29ms, (+1691.20ms)
Spinning Disk -> ZFS mounted: 13048.46ms, (+12773.31ms) - My old laptop
is incredible horrible!!!

ZFS mount time after the patch:
SSD -> ZFS mounted: 520.28ms, (+160.36ms)
Spinning Disk -> ZFS mounted: 1596.60ms, (+1281.39ms)

About 10 times faster.

It might be better to fix the path that does a sync on the specified file
system, however, it's not something simple, and requires a deep investigation
on the requeriments needed from both the ZFS and the VFS side.

You can see in zfs_sync() that the function call to vfs_stdsync() was removed
as it isn't yet implemented in our VFS layer.
This function basically calls fsync on each opened vnode, which in turn, calls
zil_commit on the correspondent znode.

I have implemented vfs_stdsync(), but that's not enough to fix the sync problem
on the pool. This patch changes zfs_sync() to do sync on the present pools,
in our case, one.

We're fixing a real bug here in the zfs_sync() behavior, and also nicely
boosting the ZFS mount time.

'make check' succeeded.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mgmt: update to latest

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mgmt: update to latest

* Remove web module from mgmt

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
balloon: add size information to memcpy fixup

I have recently introduced size calculation to memcpy fixup as a mean to make
sure that our moves are sane. However, that can't be a static function like I
have implemented.  This is because the rcx register holds number of iterations,
not number of bytes.  Therefore, we also need to include size information in
the fixup and make that a member function.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
balloon: improve tracing

Add more tracepoint information to the balloon to help us find bugs
more easily.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
x64: fix exception stack restoring

Exiting from the second exception (exception_depth == 2) should restore
the exception stack to percpu_exception_stack[0], which was the value
used before the second exception happened.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
x64: Fix nested exception support

Commit 89464d6a037dd06ac5ca37f0c70a65786193866a added support for
nested exceptions. One of the reason these are needed are when using
tracepoints keeping backtraces using backtrace_safe(). If the tracepoint
happens while handling an exception, the call to backtrace_safe() might
itself cause an exception, and while handling *that* nested exception
on the per-cpu nested exception stack, we need a second nested exception
stack (that second stack will only ever be used to print a "nested too deep"
error).

However, the existing code had an off-by-one error in its test, and
actually aborted on the nested exception, when exception_depth=1,
rather than setting the exception stack to precpu_exception_stack[1]
as it should, and only aborting when exception_depth reaches 2.

So despite the aforementioned commit, we did not actually support nested
exceptions.  Turning on malloc tracing with backtraces, with the kernel
arguments "--trace=memory_malloc --trace-backtrace", quickly resulted in an
abort. With this patch, that tracing no longer aborts.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>