rcu: make read_by_owner() const

Like iterators, the fact that an rcu pointer is not changable doesn't
mean anything about the pointed-to data.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: add assignment operator to thread_handle

Useful when stored in a container.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: add operator bool to thread_handle

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: add operator== to thread_handle

Useful for removing a handle from a container.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: convert sowakeup() helpers to inline functions

Easier to read and modify.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: remove some unused locking helpers

Less code to churn later.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: change the sowakeup_locked() family to have conventional locking semantics

sowakeup_locked() and a few related functions expect to be entered with the lock
held, and drop it when exiting.  This makes it hard to understand and
modify the code.

Change them to have conventional semantics.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: unify socket::so_rcv and socket::so_snd locks

bsd has split the locks for the send and receive socket buffers.  However,
for tcp and espcially with net channels, this is problematic:

 - a tcp packet contain infomation about both streams, so both locks have
   to be taken anyway (bsd tries to avoid this when unneeded, but it often is
   needed in chatty protocols, to ack just-received data and send our reply).

 - tcp is serialized anyway by the prococol layer mutex, so these locks only
   increase lock traffic without improving concurrency.

 - it is rare for tcp to be active on multiple threads simultaneously;
   and the contention between the stack itself and the application is solved
   by net channels

 - net channels want all packet consumers to sleep on a single mutex, but
   with two mutexes, this is impossible (a tcp writer also needs to consume
   incoming packets, as they can increase the window size and enable it to
   start sending).

 - it prevents us from unifying the socket layer locking with the tcp layer
   locking.

Fix all this by merging the two locks into a single socket lock.

The flip side is that parallel udp send/receive will suffer.  We will have
to address this later with specialized locking for udp.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: convert inpcb lock to a mutex

A read/write lock is just overhead if it is always taken for write, as the
inpcb lock is when the tcp protocol is used.  It also prevents merging with the
socket lock, so convert it.

The loser is udp, which can support parallelism on a single socket.  We'll
have to use separate data structures for tcp and udp in the future.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: remove socket lock, use protocol layer connection lock instead

Having a protocol layer lock and a socket lock (mainly used for protecting
the socket buffers) is counterproductive, since the stack is traversed in
two different directions: incoming packets first undergo protocol processing
and then reach the socket layer, while outgoing data first goes through the
socket layer and then protocol processing.  The upshot is that the locking
rules are complicated, and since the protocol connection lock is held most
of the time, no real concurrency is gained.

It is also problematic for net channels, since the user layer wants to sleep
on a single lock.  This needs to be the socket lock (which protects the buffers
we are watching), but we can't acquire the procotol lock from within the socket
lock due to the locking rules, so we must drop and re-acquire the socket lock.

Instead of all this spaghetti, use a single lock for both layers.  The lock
is recorded in the socket at pru_attach() time and then the normal locking
macros are used.

The important path have been converted to avoid taking both locks, but some
paths remain; they should still work since our mutex is recursive.

The original locking looks like this (from outer lock to inner lock):

- global protocol rwlock (protects the tcp and udp connection hash table)
  (inpcbinfo::ipi_lock)
- protocol-level (tcp/udp) rwlock (protects tcp state and udp "state")
  (inpcb::inp_lock)
- socket buffer lock (protects separate so_rcv and so_snd socket buffers;
   receive lock also protects entire socket) (sockbuf::sb_mtx)
- socket interleave prevention rwlock (sockbuf::sb_rwlock, already replaced)

The challenges a net channel implementation faces from these locks are:

- when reading or writing to a socket via a syscall, we sleep (in sbwait()
  on the socket buffer lock (sockbuf::sb_mtx). But:
    - since there is no longer async packet processing, sbwait() must also
      wait on the net channel itself
    - net channels support only a single consumer
    - so a recv() blocked in sbwait(so_rcv) and a send() blocked in
      sbwait(so_snd) must communicate; a packet processed by one of them
      can affect the other
  this requires the unification of send two sb_mtxs.  Note for tcp they never
  made much sense, since a received packet affects both, unless this is a pure
  send or pure receive stream.  But a pure send or pure receive stream wouldn't
  suffer any contention anyway.
- the lock order is inverted for app-driven processing
    notice how the original locking takes the protocol level lock and then
    the socket buffer lock.  This appears fine if you dispatch a packet up
    the stack.  But we're working the other round, sleeping on the socket
    lock in sbwait(), now we need to take inp_lock in order to process the
    packet?
  this results in unification of the socket lock and protocol lock.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: add net channel support

This adds a net_channel class the connects a packet producer with a
packet consumer.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: packet classifier

Add a structure that holds a hash table of net channels, and is able
to classify tcp/ipv4 packets and dispatch them via the channel.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: add a classifier object to ifnet

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: don't allow state-changing packets in net channels

Let them take the normal path for not to avoid complicating the locking
rules.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: split ether_input_internal()

Split ether_input_internal() to a packet validation and preprocessing stage,
and a dispatch state.  The first stage will be useful in our net channel
implementation, since the packet has already been dispatched and only
needs some preprocessing.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: split ip_input()

Split ip_input() into a validation and preprocessing stage and a dispatch
stage.  The preprocessing stage will be useful to us with net channels,
as the packet has already been dispatched via the classifier.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: change tcp_do_segment() to have more conventional locking

tcp_do_segment() unlocks the inp, making it difficult (or inefficient) to run
it in a loop.  Change it not to unlock the inp and adjust callers.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: consume net channel packets in sbwait()

While waiting for the socket buffers to be filled up, also wait for
net channel packets and consume them if available.

Since net channels support only a single consumer, only allow one thread
to wait for the net channel.  If that thread awakens, it wakes the other
waiting threads so they can compete for the channel.  This should happen
rarely.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: add classifier helpers to ifnet

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
net: tcp-level net channel processing

Add hooks for setting up and tearing down the net channel when the
connection enters and leaves TCPS_ESTABLISHED, respectively.  Define
the tcp packet processing callback.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>