gdb: add osv load-elf command

Sometimes one needs to load external ELF file which loader.py cannot
find automatically. For example when running an application which is
built outside OSv.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
bsd: Implement some of the kobj functions

FreeBSD implements the kobj functions through the use of vnode operations,
whereas I simplified things by making use of system calls.
Otherwise, a bunch of code would have to be ported and adapted.
I'm only implementing here those needed by the function spa_config_load.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
zfs: Re-enable spa_config_load

After the patch 'bsd: Implement kobj function set', spa_config_load can now be
re-enabled. spa_config_load is called during the spa (storage pool allocator)
initialization phase, i.e. spa_init(), and it's intended to populate the SPA
namespace with information stored in the pool configuration files.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: Make misc-fs-stress work on ZFS on Linux

ZFS on Linux doesn't support the O_DIRECT flag.

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
scripts: Add script for benchmarking ZFS mount time

The script basically stresses the ZFS dataset with misc-fs-stress, and
lately measures ZFS mount time.

Output example:
$ sudo ./scripts/benchs/zfs_mount.py --samples 3
Running ZFS mount time benchmark, total samples: 3
unit of time: millisecond (ms)
id	time	min	max	avg	stdev
--	----	---	---	---	-----
1	103.0	103.0	103.0	103.0	0.0
2	73.5	73.5	103.0	88.2	10.4
3	76.8	73.5	103.0	84.5	9.6

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
zfs: Improve mount time when previous instance has performed I/O

This patch considerably improves ZFS mount time when the previous OSv
instance has performed some I/O.
I tried the same type of benchmark on ZFS on Linux, and the results have
shown that apparently it's not affected by the same problem.

After tracing and investigation, the result showed that the slowness exists
at spa_load_impl procedure, whose purpose is to load an existent storage
pool. Back in time, it was discussed in our mailing list about things being
unfinished in the storage pool, being carried over to the next OSv instance,
and consequently slowing down the next ZFS mount.

The overall problem is quite complex, and requires further investigation to
discover everything about. I'm here coming up with a temporary solution which
consists of syncing all the pools when the last ZFS dataset is being unmounted.

Result:

:: BEFORE ::
$ sudo ./scripts/benchs/zfs_mount.py --samples 10
Running ZFS mount time benchmark, total samples: 10
unit of time: millisecond (ms)
id	time	min	max	avg	stdev
--	----	---	---	---	-----
1	2417.8	2417.8	2417.8	2417.8	0.0
2	1329.3	1329.3	2417.8	1873.6	384.8
3	2949.5	1329.3	2949.5	2232.2	519.8
4	1296.9	1296.9	2949.5	1998.4	570.7
5	3337.5	1296.9	3337.5	2266.2	700.1
6	3399.8	1296.9	3399.8	2455.1	746.4
7	3358.2	1296.9	3399.8	2584.1	750.4
8	2946.7	1296.9	3399.8	2629.5	710.9
9	3002.2	1296.9	3399.8	2670.9	679.2
10	1169.5	1169.5	3399.8	2520.8	773.2

:: AFTER ::
$ sudo ./scripts/benchs/zfs_mount.py --samples 10
Running ZFS mount time benchmark, total samples: 10
unit of time: millisecond (ms)
id	time	min	max	avg	stdev
--	----	---	---	---	-----
1	103.0	103.0	103.0	103.0	0.0
2	73.5	73.5	103.0	88.2	10.4
3	76.8	73.5	103.0	84.5	9.6
4	91.8	73.5	103.0	86.3	8.7
5	120.0	73.5	120.0	93.0	14.4
6	84.3	73.5	120.0	91.6	13.5
7	162.7	73.5	162.7	101.7	26.2
8	103.9	73.5	162.7	102.0	24.5
9	80.0	73.5	162.7	99.6	24.0
10	127.8	73.5	162.7	102.4	24.2

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Fix spelling of "below" in comments.

Signed-off-by: Don Marti <dmarti@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
elf: move arch-specific relocation enums to arch

add a per-arch jump slot define for pltgot relocation,
add basic rela relocations for aarch64.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
elf: replace non-ascii 'fi' character combination with 'fi'

replace a mysterious UTF-8 'fi' character combination with two
ASCII 'fi' characters.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
elf: aarch64: relocate pltgot

this adds basic support for AArch64
object::relocate_pltgot()
object::resolve_pltgot()

including support for lazy binding.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Add entries to .gitignore

Pekka is suspicious:

  OK, so why are we adding the above to to global osv.git ignore list?
  Neither of them are generated by the OS build system. If they bother
  your workflow, you can hide them in your local git configuration...

Raphael explains:

  The same applies to many files there :-) And it was already discussed
  that .gitignore could afford files that would make the OSv experience
  better.

  tracefile is always generated by the trace tool, and it just doesn't
  make sense to show it, imagine every new people to OSv needlessly
  having to add tracefile to their local git configuration.

Glauber wins:

  I agree with Raphael. We have discussed it before. This criteria of
  "generated by the build" is totally arbitrary. The real thing that
  matters is how many people are likely to have to do it locally.

  Those are quite pervasive files.

Agreed-with-Raphael-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
shrinkers: wake_waiters should be called when target is zero

__do_reclaim calls byte_until_normal() to get the target (amount
of memory to be freed), however, when there is no pressure in the
system, zero will be returned.

Going further, either ballooning or registered shrinkers will be
used to aliviate pressure, respectively.

On the latter case, _shrinker_loop will be called to go through
all the registered shrinkers asking each of them to release memory.
The problem is that our code currently assumes target being greater
than zero to wake the waiters after memory has been released, and
that will eventually block everything.

My solution is to call _oom_blocked.wake_waiters() even when target
is zero given that __do_reclaim wakes up even when there is no
memory pressure. Change was tested on memory-hog, which is a good
project to test and play with the shrinker infrastructure.
For those interested, follow: https://github.com/dmarti/memory-hog/

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Reported-by: Don Marti <dmarti@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
elf: move arch-specific object rela relocs to arch

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
elf: move arch-specific object jump slot relocs to arch

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
elf: move arch-specific initial dynamic segment relocs

move the initial arch-specific relocations done in get_init
into the arch-specific directories.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
drivers: Add null device

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mmu: introduce memory areas

This patch introduces memory areas. They are going to be used to determine
which memory allocator is responsible for a particular object.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mempool: use memory areas to determine the object allocator

Two new memory areas are introduced: early and mempool. Large allocations,
physically contiguous allocations and MMIO mappings happen in main memory
area.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mempool: remove non_mempool_obj_offset

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Merge branch 'memarea'

"This series is the first step in implementing memory allocator enhancements
described by Avi in #87. Virtual memory starting from address 0xffff800000000000
is divided into memory areas. Each of the object allocators (currently: mempool,
large and early) return addresses to memory block only in its own memory area.
When memory block is freed bits [44:46] of its virtual address are used to
identify the responsible allocator.

phys_to_virt() makes things a bit tricky, since it is not possible to determine
to which memory area should the returned virtual address belong to. Current
solution is to return addresses in memory_area::main (used also by large memory
allocator) and make alloc_phys_contiguous_aligned() use memory::malloc_large()
only."

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>