Update mgmt for upload API

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
sched: reimplement incoming_wakeup_queue

This patch implements lockfree_queue (which is used as incoming_wakeup_queue)
so that it doesn't need exchange or compare_exchange operations.

The idea is to use a linked list but interleave actual objects stored in the
queue with helper object (lockless_queue_helper) which are just pointer to the
next element. Each object in the queue owns the helper that precedes it (and
they are dequeued together) while the last helper, which does not precede any
object is owned by the queue itself.

When a new object is enqueued it gains ownership of the last helper in the
queue in exchange of the helper it owned before which now becomes the new
tail of the list.

Unlike the original implementation this version of lockfree_queue really
requires that there is no more than one concurrent producer and no more than
one concurrent consumer.

The results oftests/misc-ctxs on my test machine are as follows (the values
are medians of five runs):

before:
colocated: 332 ns
apart: 590 ns

after:
colocated: 313 ns
apart: 558 ns

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com> 
Signed-off-by: Pawel Dziepak <pdziepak@quarnos.org>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
sched: relax memory ordering in cpu_set::fetch_clear

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com> 
Signed-off-by: Pawel Dziepak <pdziepak@quarnos.org>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
ring_spsc: Use bitwise AND instead of modulo for calculating the entry index

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
ring_spsc: Use size() method instead of duplicating its code

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
virtio-vring: fix kick() to return a correct value.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
virtio-vring: rework the interface of get_buf_finalize()

New interface allows to send a single doorbell to the host per
buffers bulk.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
virtio-net: coalesce doorbells to the host when finalizing Tx buffers.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
virtio-net: use indirect buffer descriptors

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
virtio-net: Move the Tx related code into the txq class.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
virtio-net: make Tx drop-less.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
nway_merger helper class implementation

This class allows efficiently merge n sorted containers.
It allows both a single-call merging with a merge() method and
the iterator-like semantincs with a pop() method.
In both cases the merged stream/next element are streamed to
the output iterator.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Added a test for nway_merger with ring_spsc as a base sorted container.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cpu_queue class

  This class will represent a single per-CPU Tx queue.

  These queues will be subject to the merging by the nway_merger class in
  order to address the reordering issue. Therefore this class will
  implement the following methods/classes:
   - push(val)
   - empty()
   - front(), which will return the iterator that implements:
       - operator *() to access the underlying value
   - erase(it), which would pop the front element.

  If the producer fails to push a new element into the queue (the queue is
  full) then it may start "waiting for the queue": request to be woken when the
  queue is not full anymore (when the consumer frees some some entries from the
  queue):

   - push_new_waiter() method.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
xmitter class

  This class is a heart of a per-CPU Tx framework.
  Except for a constructor it has two public methods:
   - xmit(buff): push the packet descriptor downstream either to the HW or into
     the per-CPU queue if there is a contention.

   - poll_until(cond): this is a main function of a worker thread that will
     consume packet descriptors from the per-CPU queue(s) and send them to
     the output iterator (which is responsible to ensure their successful
     sending to the HW channel).

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
virtio-net: use per-CPU-Tx framework

This involves the introduction of an xmit iterator and a worker thread.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
virtio-net: some white spaces, 80 columns and comments spelling fixes.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
vmxnet3: Fix RX rings initialization

2nd ring of RX queue was filled with descriptors
and immediately cleared. This lead to inability of driver
to receive packets loger than 16K.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
vmxnet3: Fix long RX packets processing

Function vmxnet3::rxq_eof() may exit in the middle
of multi-descriptor RX packet processing. In this case
it is expected to resume processing of the same packet
on next invocation.

Fixed by making corresponding pointers vmxnet3_rxqueue class
members instead of stack variables.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
vmxnet3: Drop memory allocation error handling on RX path

Packet discard logic on RX path is incorrect because
it doesn't take into account that incoming packet may be spread
over a few rings.

At the same time discard logic is called in case of newbuf() failure
only and our newbuf() never fails since we have no memory allocation
errors, so discard logic may be safely dropped and this is an
excellent fix for the original problem.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>