sched: support for waiting for multiple objects of different types

Waiting is complicated; you can wait for a condition variable, a timer,
wait queues and signals (when we have them), semaphores, and more.

When you need to wait for multiple events, things become even more
complicated as some sort of abstraction is needed.  poll() for example
supports waiting for multiple objects that conform to the fd interface,
or for a timeout.

To allow more flexible waiting, this patch adds a poll-like interface
to the scheduler, except that the interfaces are all compile time.

We add a new function wait_for() that accepts a variable number of waitable
objects.  A waitable object is an object that supports the three methods:

 - poll(): check if waiting is complete
 - arm(): prepare for waiting
 - disarm(): finish waiting

to make a waitable object usable with wait_for, these functions have to be
supplied.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: make timer a waitable object suitable for wait_for()

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
core: add signal catching mechanism

Add a signal catching class (signal_catcher).

Use signal_catcher::wait() to wait for a signal.
Use sched::thread::wait_for(..., signal_catcher&, ...) to wait for a signal
in addition to other events.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: document state machine

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: wake_lock()

This adds a facility to wake a thread, but with the intention that it will
acquire a certain lock after waking, and while the waker holds the lock.
This is implemented using the regular wait morphing code (send_lock() and
receive_lock()), but with additional mutual exclusion to allow regular
wake()s in parallel.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
core: add waitqueue support

A waitqueue is an object on which multiple threads can wait; other threads
can wake up either one or all waiting threads.  A waitqueue is associated
with an external mutex which the user must supply for both wait and wake
operations.

Waitqueues differ from condition variables in three respects:
- waitqueues do not contain an internal mutex.  This makes them smaller, and
  reduces lock acquisitions.  On the other hand the waker must hold the
  associated mutex, whereas this is not required with condition variables.
- waitqueues support sched::thread::wait_for()

waitqueues support wait morphing and do not cause excess lock contention,
even with wake_all().

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
tests: add tests for wait_for() with timer objects

Reviewed-by: Nadav Har'El  <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: remove sockbuf::test_fn()

This was used to ensure all socket-using code was converted to C++, but not
cleaned up later.

Clean it up now.

Reviewed-by: Nadav Har'El  <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: convert sockbuf waiting to use waitqueues

Add sockbuf::sb_cc_wq for using instead of msleep(&sb->sb_cc).  This paves
the way for lockless wakeups for net channels, as the wait primitive is now
thread::wait_for() instead of msleep().  In addition this reduces lock
acquisition and improves netperf bandwidth by about 10%.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: wait_for() support for predicates

Allow arbitrary predicates to be waited on.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
tests: add test for wait_for(predicate)

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Merge branch 'waitqueue2'

Lightweight waitqueue and wait_for() primitive.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>