README: Add a short intro to OSv

Add on the top of README.md a short introduction to what OSv is.
If someone gets to our github page, https://github.com/cloudius-systems/osv,
and scrolls down, it's strange that we only explain how to build OSv,
without first mentioning what it is.

Fixes #148

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
scripts/test: Add option to run all test cases in a single OSv instance

Previously, scripts/test.py had no option to do that. It launched an OSv
instance for each test case.
Terribly slow PCs like mine took a bunch of time to run all test cases
through 'make check'.

Then let's take advantage of testrunner.so which will use a single OSv instance
to run all test cases, consequently boosting the speed considerably.
Let's also change testunner.so to conform our needs, e.g. blacklist.

To run this fast check, do: scripts/test.py --single;

Results show that this option is about 2.5x faster than the current one.

By now, let's not use this approach as the default version given that its
output has to be better formatted.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Fix tests/testrunner.cc coding style

Start using spaces instead of tabs and surround all single-line
control statements with curly braces.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
sched: keep track of thread's runtime

This will be used later to determine for how long have a thread been running.
It can easily be updated right before we call ran_for(), reusing its interval
parameter.

Fixes #135

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
sched: reserve some thread ids

This patch reserves some thread ids, that are kept unused. This is so we can
construct values that reuse the thread public id and add it together with other
information and still fit in 32-bits.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
pthread: add support for pthread_getcpuclockid

This patch implements a simplified version of pthread_getcpuclockid
that should be enough for our needs.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tests: add test for memmove string library functionality

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Exile spinlock to a separate file

In very early OSv history, the spinlock was used in the mutex's
implementation so it made sense to put it in mutex.cc and mutex.h.

But now that the spinlock is all that's left in mutex.cc (the real mutex
is in lfmutex.cc), rename this file spinlock.cc. Also, move the spinlock
definitions from <osv/mutex.h> to a new <osv/spinlock.h>, so if someone
wants to make the grave mistake of using a spinlock - they will at least
need to explicitly include this header file.

Currently, the only remaining user of the spinlock is the console.
Using a spinlock (and not a mutex) in the console allows printing debug
messages while preemption is disabled. Arguably, this use-case is no
longer important (we have tracepoints), so in the future we can consider
dropping the spinlock completely.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
OSv Coding Style

This document describe OSv coding style.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build.mk: Fix dependency

A previous patch renamed mutex.cc to spinlock.cc. This fixes the build.mk
dependency to make the code compile again... Sorry about that.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
gdb: improve 'osv zfs' command

This patch improves the command by adding useful info for debugging ZFS
in general, and also addresses some stylistic issues.

The new output is as follow:
(gdb) osv zfs
:: ZFS TUNABLES ::
	zil_replay_disable:       0
	zfs_nocacheflush:         0
	zfs_prefetch_disable:     0
	zfs_no_write_throttle:    0
	zfs_txg_timeout:          5
	zfs_write_limit_override: 0
	vdev_min_pending:         4
	vdev_max_pending:         10
:: ARC SIZES ::
	Actual ARC Size:        122905056
	Target size of ARC:     1341923840
	Min Target size of ARC: 167740480
	Max Target size of ARC: 1341923840
		Most Recently Used (MRU) size:   670961920 (50.00%)
		Most Frequently Used (MFU) size: 670961920 (50.00%)
:: ARC EFFICIENCY ::
Total ARC accesses: 42662
	ARC hits: 41615 (97.55%)
		ARC MRU hits: 12550 (30.16%)
			Ghost Hits: 0
		ARC MFU hits: 29045 (69.79%)
			Ghost Hits: 0
	ARC misses: 1047 (2.45%)
Prefetch workload ratio: 0.0097%
Prefetch total:          412
	Prefetch hits:   20
	Prefetch misses: 392
Total Hash elements: 1053
	Max Hash elements: 1053
	Hash collisions:   13
	Hash chains:       11

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Refactor timerfd.cc

In his review of timerfd.cc, Avi asked that I simplify the implementation
by having a single "timerfd" object (instead of two I had - timerfd_file
and timerfd_object), and by using a single mutex instead of the complex
combination of mutexes and atomic variable.

This new version indeed does this. It should be easier to understand this
code, and it is 30 lines shorter.

The performance of this code is slightly inferior to the previous one -
in particular poll() now locks and unlocks a mutex - but this should be
negligible in practice.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
config.json: convert tabs to spaces

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
modules: support nested module dependencies

Currently importing module from a module definition would fail because
we cannot call import module with the same name (module.py)
recursively, __import__ will complain that we removed 'module' from
sys.modules.

There is a simple solution to this problem, we can use runpy.run_path()
which works like a charm.

In addition to this we cache loaded modules so that we don't
have to load the file twice.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
modules: make presence of either manifests optional

No need to create an empty bootfs.manifest anymore.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
modules: cleanup resolve.py

No functional changes, just renames to more adequate names.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
modules: make java a separate OSv module

This patch makes java files are copied to the guest image only when
'java' modue is included.

Modules can pull it explicitly by stating:

  require('java')

or implicitly, by creating api.run_java() run configurations.
In future we could consider moving api.run_java() into a java
meta-module.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
modules: introduce higer level API to aid manifest generation

When plain manifests are not enough this is a concise alternative
with improved expresiveness. It allows to declare exclude and include
patterns. It's python based.

Example:

  m = FileMap()
  m.add('${OSV_BUILD_PATH}/tests').to('/tests') \
     .include('**/*.so') \
     .exclude('host/**')

Declared mappings can be saved in manifest form or be subject
of further processing. To save in manifest format:

  save_as_manifest(m, 'my.manifest')

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
modules: allow for file mapping declaration in module.py

If module has 'usr_files' or 'bootfs_files' declared then
their value will be interpreted as FileMaps and appended
to appropriate manifests.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
java: add all JRE files to the image by default

Fixes issue with JVM failing when started with a debugger
with the following message:

  NPT ERROR: Cannot find nptInitialize

Missing openjdk files in usr.manifest were a fertile source
of issues. This patch aims at making them less likely and
adding all files   except blacklisted files to the image.

This patch skips two files from JRE which are broken links
and inclusion of which would cause manifest upload failure:
 - jre/lib/audio/default.sf2
 - jre/lib/security/cacerts

These should be fixed incrementally.

Reported-by: Oded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>