Use file offset in msync instead of vma address

Current code erroneously uses vma address to lookup write cache pages instead
of file offsets. This patch fixes it.

Fixes #452.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
trace: fix 'trace.py download'

Broken by 3730c3953ca0cbae940f41134f73764ec1abb9d1.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
trace: make 'trace.py download' convert the buffers to tracefile by default

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tst-seek: convert to boost UTF

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tst-seek: add test case which checks that seek actually works

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Move tracepoint_count functions into the kernel

Before this patch, trace-count.hh was purly a header file, and when
we used it in a shared object (such as httpserver.so), the code of
the tracepoint hit() function ended up part of the shared object.

Because the hit() function often runs in preemption-disabled context,
this caused a bunch of problems. One problem we already worked around
was that on the first call, it might call a function for the first
time and cause a (possibly sleeping) symbol resolution. But another
bug we never solved was that pages from the shared object might be
evicted, in which case suddenly hit() will cause a page fault.

So this patch moves the hit() implementation out of line, from
trace-count.hh to a new trace-count.cc, i.e., into the kernel's code.
While we're at it, I moved the other inline functions out of line too.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: test more cases for misc-memcpy.so

Vlad had recently hypothesized that we may be lagging behind of redis because
the memcpy addresses for redis are usually misaligned, both relative to the
CPU word size, and to itself.

I am adding tests for those cases in misc-memcpy.so

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: gtod

I have seen gettimeofday - and the pvclock functions in general - show up
quite heavily in some profiles (some 4 - 5 %).

Although this is of course normal an expected for a system that is doing a lot
of time operations, I wanted to make sure that we are not going anything too
wasteful in there.

I wrote this simple benchmark to attest that, since we seem to have none so
far. The good news is that we are quite faster than a CentOS guest, albeit
2 x slower than the host (hard to avoid that).

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
fix netchannel
memory: Global page buffer pool

Currently, we maintain a percpu page buffer pool for page allocator.
When the pool is empty or full, the global mutex free_page_ranges_lock
is take to refill or unfill the pool.

This patch introduces a global page buffer pool in addition to the
percpu page buffer pool.

1) L1-pool (Percpu page buffer pool)

if nr < max * 1 / 4
   refill

if nr > max * 3 / 4
   unfill

2) L2-pool (Global page buffer pool)

if nr < max * 1 / 4
   refill

if nr > max * 3 / 4
   unfill

When L1-pool needs refill or unfill, it moves a batch of pages from or to
L2-pool.

When L2-pool needs refill or unfill, it move a batch of pages from or to
global free page list.

Some notes:

- On fast path, both L1 and L2 allocation/deallocation require no lock.

- Atomic operations amortized to a large number of pages.
  Pages are exchanged in batch (currently 32 pages) between L1 and L2
  and global pool.

- Percpu threads are added for L1-pool refill/unfill. This looks
  over-killing, but there seems no other better way to do it.
  One option is to use async::run_later interface, but the refill/unfill
  operation will block, thus blocking other tasks on the same cpu.
  Another option is to use core/percpu-worker.cc interface. Again, it
  will block other tasks. For now, I am using per-cpu thread approach.

- nr_cpus(L1 pool) + 1(L2 pool) additional threads are added compared to
  current allocator. These threads help filling the pool more
  aggressively: at 1/4 - 3/4 level, compared to 0 - 1 level.

Functional Tests:

I used /tests/misc-malloc.so to stress memory allocation/deallocation.

for i in `seq 100`;
   do echo "Run $i"; echo "";
   sudo scripts/run.py -nv -c2 -m8g -e "/tests/misc-malloc.so";
done

Signed-off-by: Asias He <asias@cloudius-systems.com>
ci
do not use wait_page_batch
drop need_refill and need_unfill
drop if 0
no wait_page_batch
no wait_page_batch