build.mk: introduce *_env variables to make system

According to http://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html,
the versions of standard libraries have a strong matchup with versions
of compilers, so linkage errors occur when using libstdc++ in *external*
submodule when using GCC 4.9.0.

As there is such a mixup in the build system, the environments of
standard libraries in link time should be switchable to support GCC
whose version doesn't match up with external's.

*_env variables are introduced. To build a image with standard libraries
in the host, run `make build_env=host'. For fine-grained settings, use
gcc_lib_env and cxx_lib_env.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Boqun Feng <boqun.feng@linux.vnet.ibm.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: initial malloc benchmark

Currently covering three cases: 1 thread allocation, n threads allocation,
and n threads allocations with frees in a different cpu. We should improve
with more cases.

This test was designed to run on Linux as well. Let's keep it this way.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
vfs: enable support for DEBUG_VFS

At the moment, defining DEBUG_VFS fails OSv compilation. The patch
ensures OSv compiles and correct debug logs are emitted.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Prasad Joshi <prasadjoshi.linux@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu, x64: get maximum supported phys/virt bits from cpuid

Later patches will use reserved bit, so we want to be sure that it is
available.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mmu, x64: fix getting/setting address in a pte

Currently address we get from pte includes reserved bits and setting
zeroes available and reserved bits.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mmu, x64: optimize detection of PROT_NONE mem region using PT reserved bit

Java uses accesses to PROT_NONE region to stop threads sometimes, so it
worthwhile to be able to catch this as fast as possible without taking
vma_list_mutex. The patch does it by setting reserved bit on all ptes in
PROT_NONE VMA which causes RSVD bit to be set in a page fault error code.
It is enough to check it to know that access is to a valid VMA, but
permission is lacking.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mmu: rename intermediate_page() to intermediate_page_post()

The function should be called intermediate_page_post(), otherwise it is
never called.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mmu: remove reference to noexistent hw_pte_ref class

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mmu,x64: optimize detection of PROT_READ mem region by checking cow bit in a pte

Java uses PROT_READ page to synchronize threads, so it worthwhile to be
able to catch this as fast as possible without taking vma_list_mutex.
The patch does it by checking that pte is not marked as cow during write
fault on present pte since cow or PROT_READ are the only reasons for pte
to be write protected. The problem is that to get pte we need to walk
the page table, but access to a page table is currently protected by
vma_list_mutex. The patch uses RCU to free intermediate page table levels
which makes it possible to get to pte without taking vma_list_mutex lock.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
tls: move more details about initialization to arch

the thread_control_block structure needs to be different
between x64 and AArch64;
For AArch64's implementation for local execution, try to
match the layout in glibc and the generated code.

Do not align .tdata and .tbss sections with .tdata : ALIGN(64)
or it will affect the TLS loads.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Cc: Glauber Costa <glommer@cloudius-systems.com>
Cc: Will Newton <will.newton@linaro.org>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
disable rt lock
no arp lock
core: add handle_incoming_wakeups tracepoints

When you see this in `perf kvm top`:

 20,93%  [guest.kernel]    [g] _ZN5sched3cpu23handle_incoming_wakeupsEv

You may want to know where does it come from. For that you can trace
'handle_incoming_wakeups' tracepoint and display call sites with
`trace prof`:

$ trace prof
(100.00%, #32722) All
 log
 sched::cpu::handle_incoming_wakeups()
 |-- (96.18%, #31472) sched::cpu::do_idle()
 |    sched::cpu::idle()
 |    sched::thread::main()
 |
 \-- (3.82%, #1250) sched::cpu::reschedule_from_interrupt(bool)
      |-- (2.79%, #914) sched::cpu::schedule()
      |    |-- (1.94%, #634) sched::thread::wait()
      |    |    |-- (1.19%, #390) sched::thread::do_wait_until
      |    |    |    |    waiter::wait(sched::timer*) const
      |    |    |    |    condvar::wait(lockfree::mutex*, sched::timer*)
no timer lock
get rid of unnecessary recursive locking
disable routing lock

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
java: add -XX:+UseMembar
mempool: make mempool_cpuid() inline

In some runs callq to mempool_cpuid shows up in 'perf kvm top' profile.
mempool: erase page range using an iterator rather than a reference

free_page_ranges is an intrusive set. erasing via a reference requires
iteration over reference equal_range under the hood, which means
traversing the tree to the leafs. Whereas erasing via an iterator
requires no such lookups so should be faster.
mempool: prevent across-CPU page stealing

This addresses the race which is mentioned in the comment. If the
thread doing add_page() is migrated before the preempt_lock is taken
it may feed the pool with a page coming from different CPU's page
buffer.

The fix is to embrace the body with migration-lock.