net: fix poll() abort on net channels

While the net channel poller list (a vector<pollreq>) is managed via rcu,
do_poll() modifies the pollreq without protection when it clears
pollreq::_poll_thread.  This can cause net_channel::wake_pollers() to wake
a null thread.

Fix by converting pollreq::_poll_thread to a sched::thread_handle (which has
internal rcu protection).

Seen running wrk against a static tomcat page (the home page).

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
balloon: protect against ill-sized moves

Carl noticed that it is legal for a java program (think for instance, somebody
using JNI), to move stuff manually from the middle of the balloon. In that case,
we would have a move that is smaller than the size of the object. We haven't put
much tought on how to handle this case - if at all. But it is certainly better
to fault there than to have a silent weird behaviour.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests/misc-lfring.cc: Delete the test for ring_mpsc.

We are about to kill ring_mpsc - delete the corresponding tests.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
ring_spsc/mpsc: Fix pop() method failing after the ring index wraps around

- Fixed ring_spsc: "empty" check should be "_beg == _end" and not "_beg >= _end".
- Deleted ring_mpsc since it's not currently used anywhere.

Fixes #225.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: added a test verifying the fix for issue #225

Running 'make check' after the fix:
...
TEST tst-ring-spsc-wraparound.so OK  (8.817 s)
...

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
balloon: remove obsolete assert

I am currently being able to trigger this assert. It comes from the old days
before we found the opportunistic copying bug. Right now, it *is* possible to
have the balloons list to be empty, and still some old VMAs lying around.

Just remove the assertion.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
move malloc pools to mempool.cc

No reason to expose it in the headers.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
run: don't redirect port 8080 by default

Conflicts with Jenkins.

Add an option instead.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
loader: remove unused declaration

Removed an unused declaration, which is unnecessary and causes a warning
in Eclipse.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
zfs: Boost ZFS mount time by fixing zfs_sync functionality

The main reason behind the slowness on ZFS mount times is that zfs_sync() was
definitely broken. From a ZFS perspective, sync should wait for the underlying
pool to commit all the dirty data.

Previously, a bunch of work was being handed off to the next OSv instance, thus
slowing down the mount process. When tracing the exactly location of the hang,
I found that the actual bottleneck was during the load of an existent SPA,
spa_load(). Going further, I discovered that ZIL replays (if any) are necessary
for the success of spa_load(). zfs_sync() should flush out all the ZIL records
so that there is no need to replay afterwards. That wasn't happening though.

The original implementation in BSD shows that spa_sync_allpools() should be
called when a mount point wasn't specified, i.e. NULL. However, it works
differently in OSv, this clause doesn't even exist.
spa_sync_allpools() will basically sync all pools.

ZFS mount time before the patch:
SSD -> ZFS mounted: 1943.29ms, (+1691.20ms)
Spinning Disk -> ZFS mounted: 13048.46ms, (+12773.31ms) - My old laptop
is incredible horrible!!!

ZFS mount time after the patch:
SSD -> ZFS mounted: 520.28ms, (+160.36ms)
Spinning Disk -> ZFS mounted: 1596.60ms, (+1281.39ms)

About 10 times faster.

It might be better to fix the path that does a sync on the specified file
system, however, it's not something simple, and requires a deep investigation
on the requeriments needed from both the ZFS and the VFS side.

You can see in zfs_sync() that the function call to vfs_stdsync() was removed
as it isn't yet implemented in our VFS layer.
This function basically calls fsync on each opened vnode, which in turn, calls
zil_commit on the correspondent znode.

I have implemented vfs_stdsync(), but that's not enough to fix the sync problem
on the pool. This patch changes zfs_sync() to do sync on the present pools,
in our case, one.

We're fixing a real bug here in the zfs_sync() behavior, and also nicely
boosting the ZFS mount time.

'make check' succeeded.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mgmt: update to latest

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mgmt: update to latest

* Remove web module from mgmt

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
balloon: add size information to memcpy fixup

I have recently introduced size calculation to memcpy fixup as a mean to make
sure that our moves are sane. However, that can't be a static function like I
have implemented.  This is because the rcx register holds number of iterations,
not number of bytes.  Therefore, we also need to include size information in
the fixup and make that a member function.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
balloon: improve tracing

Add more tracepoint information to the balloon to help us find bugs
more easily.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
x64: fix exception stack restoring

Exiting from the second exception (exception_depth == 2) should restore
the exception stack to percpu_exception_stack[0], which was the value
used before the second exception happened.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
x64: Fix nested exception support

Commit 89464d6a037dd06ac5ca37f0c70a65786193866a added support for
nested exceptions. One of the reason these are needed are when using
tracepoints keeping backtraces using backtrace_safe(). If the tracepoint
happens while handling an exception, the call to backtrace_safe() might
itself cause an exception, and while handling *that* nested exception
on the per-cpu nested exception stack, we need a second nested exception
stack (that second stack will only ever be used to print a "nested too deep"
error).

However, the existing code had an off-by-one error in its test, and
actually aborted on the nested exception, when exception_depth=1,
rather than setting the exception stack to precpu_exception_stack[1]
as it should, and only aborting when exception_depth reaches 2.

So despite the aforementioned commit, we did not actually support nested
exceptions.  Turning on malloc tracing with backtraces, with the kernel
arguments "--trace=memory_malloc --trace-backtrace", quickly resulted in an
abort. With this patch, that tracing no longer aborts.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
dhcp: Fix dhcp issue on VBOX

I've seen OSv can not get dhcp address in Virtual Box.

On Linux, the dhcp client sends dhcp options in both DISCOVER and
REQUEST packets. However, OSv only sends the options in DISCOVER
packet.

Fix the issue by making OSv behave the same with Linux dhcp client.

Though other people (Vlad) did not this issue, it won't hurt to send
options in both packets.

This patch fixes the dhcp issue for me.

Reviewed-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build: fix building in directories containing the string 'debug'

An unrestrained grep ate the gcc include directory.

Fix by grepping for the right things.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
trace: fix bad interaction between debug mode, templates, ld, and 'asm goto'

A call to safe_load() when serializing a const char* trace argument
generates a reference to a label from .text.fixup to a template function
(safe_load()).  If this function is out-of-line, then ld can eliminate it,
as duplicate copies of templates are removed (except one).  This causes a
linker error.

Fix by marking the function (and template functions that call it) as always
inline.

Fixes #217.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
pci: Fix header type check

Header Type is 8-bit. The bit 7 is used to identify a multi-function
device. The lower bit 1 and bit 0 are used to identify the header type.

   00b is defined for Normal devices
   01b is defined for PCI-to-PCI bridges
   10b is defined for PC Card Standard

Fix the check by masking the lower 2 bits only.

This patch fixes the device probe on VMware Workstation.

Signed-off-by: Asias He <asias@cloudius-systems.com>