mmu: separate lock for vpopulate and friends

When the debug allocator is active, it will call vpopulate() to manage the
page table.  This causes a deadlock, as vpopulate() takes vma_list_mutex,
while other operations that hold vma_list_mutex may allocate memory.

Fix by using a separate mutex for the debug range.  This is safe since the
page table root is pre-allocated, and any lower page tables will be either
in the vma list range, or the debug range, but not both.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
virtio: fix virtio-blk under debug allocator

The debug allocator can allocate non-contiguous memory for large requests,
but since b7de9871b31 it uses only one sg entry for the entire buffer.

One possible fix is to allocate contiguous memory even under the debug
allocator, but in the future we may wish to allow discontiguous allocation
when not enough contiguous space is available.  So instead we implement
a virt_to_phys() variant that takes a range, and outputs the physical
segments that make it up, and use that to construct a minimal sg list
depending on the input.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Merge tag 'v2p-debug-v1' of https://github.com/avikivity/osv

This patchset fixes bad interaction between the debug allocator and virtio.

After this patchset, 'make check' still fails, but this is due to genuine
use-after-free problems in the vfs, so will be fixed later.

Fixes #240.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
vfs: leak: Release refcnts hold by main_task structure on shutdown

Whenever you change the working directory through either fchdir or chdir,
a file structure holding refcnts to the underlying mount point is set to
the field t_cwdfp of main_task. If t_cwdfp was previously set, then the
older data will be freed and replaced by the newer one.
The problem exists because these refcnts are never released afterwards.

Changes introduced by this patch:
1) Create a function and use lambda to reuse as much code as possible
(Suggested by Glauber Costa).

2) Create vfs_exit that will use the function previously created to
free up main_task resources, and call unmount_rootfs afterwards.
Needless to say, vfs_exit replaces the function unmount_rootfs at the
shutdown procedure.

After applying the changes described above, the refcnt leaks on the root
mount point are gone.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
trace: implement __lt__ for Trace

Python3 no longer supports __cmp__, we need to define __lt__ to make
the objects comparable. This fixes `trace extract` on GDB linked with
python3.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
runtime: implement getgrnam function

Enhance our runtime with the getgrnam function. Although we have no groups,
we will recognize group "nobody" for compatibility and return that as group
0

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
libtsm: Add scroll callback

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
vga: hw scroll support

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
vga: rename private members

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
rlimit: set rss/data maximum size

Same as we currently have for AS, show that value as unlimited.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
apps: update to latest

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build-osv-release: Add OpenJDK 8 base image

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: compile tst-mmap with "-z now"

Compile tst-mmap.so with "-z now", so that symbols are resolved at load
time instead of when needed.

The problem was that tst-mmap uses wait_until, which runs the functions
sched::thread::wait() and sched::thread::stop_wait() while preemption is
disabled. In about 1 in 10 runs of tst-mmap.so, this caused an assertion-
failure crash (symbol resolution caused sleeping while preemption is disabled).

Fixes #256.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
libc: Implement stubbed clock()

Nadav says:
The clock() function is stubbed. We can easily implement it by calling
clock_gettime(CLOCK_PROCESS_CPUTIME_ID).

Closes #275.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
runtime: stub initgroups

empty implementation will suit us.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
runtime: basic prctl functionality

First of all, we need the symbol. But so far it is fine returning EINVAL to
everybody except for PR_SET_DUMPABLE. There are situations (see man prctl) in
which this flag is automatically cleared. Some programs (nginx here) will then
proceed to set it to make sure it is on. The flag itself is meanginless to us,
since we are not producing per-thread coredumps, and when we do dump, we will
dump regardless of this. But we need the call to suceed.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
af_local: allow ioctl on unix sockets

The current code does not allow ioctls on unix sockets, but real world
applications - nginx in my example will issue them. I am starting by
implementing FIONBIO to set blocking/nonblocking mode.

I notice here that we already have a flag in the read and write functions, and
they seem to handle that flag: the internal pipe_buffer implementation does
check and act on it.

So let's just flip it.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu: export function to measure the amount of mmaped area

It will be used for procfs compatibility. Applications may want to know how much
memory is potentially available through mmap mappings (not necessarily allocated).

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
sched: export total sys and user time statistics

This is one of the statistics that shows up in /proc/self/stat under Linux, but
this is generally interesting for applications. Since we don't have kernel mode
and userspace mode, it is very hard to differentiate between "time spent in
userspace" and "kernel time spent on behalf of the process". Therefore, we will
present system time as always 0. If we wanted, we could at least differentiate
clearly osv-specific threads as system time, but there is no need to go through
the trouble now.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
linux: provide a /proc/$$/stat compatibility file

MongoDB wants it. In general, I am providing the information that is easy
to get, and ignoring the ones which are not - with the exception of process
count, that seemed easy enough to implement.

This is the kind of thing Mongo does with it:

2014-04-15T09:54:12.322+0000 [clientcursormon] mem (MB) res:670160 virt:25212
2014-04-15T09:54:12.323+0000 [clientcursormon]  mapped (incl journal view):160
2014-04-15T09:54:12.324+0000 [clientcursormon]  connections:0

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>