loader: add --delay option

Delay boot by the specified number of seconds, so that slow log collectors
such as ec2 can catch up.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
scripts: add alternative script to create EC2 AMIs

This script creates AMIs from snapshots instead of running instances, and
does not require uploading and converting the volume.  As a result it is
much, much faster.  The downside is that it must be run on ec2 itself, and
cannot be run on a local host.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Swagger Definition: app.json should use type

The currect way of defining return type, and parameter type is with the "type"
keyword.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
x64: Make DUMMY_HANDLER less dumb

It's nice to at least see the RIP where the invalid opcode exception
happened at...

Reviewed-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Mention scripts/setup.py in Ubuntu section of README

The setup.py script works on Ubuntu, and without it
the install will be missing some dependencies.

Signed-off-by: Don Marti <dmarti@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
linux: Fix futex() command bitmask

The "op" argument passed to futex() also include flags so we need to
mask the command bits.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
linux: Wire up uname system call

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tools/uush: add cd builtin and ls, mkdir binaries

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Sever: Add power off API

While the shutdown API attempts to do a clean power off, it sometimes fails and halt the shutdown process.
It is sometimes desireable to to a power off, which is force, unsafe shuting down of the system.

This adds a power off functionality to the API.

to test do:
http://localhost:8000/os/poweroff

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Benchmark for measuring TLS access performance

This patch adds a tool, tests/misc-tls.cc, for quanitifying issue #555.
I.e., this tool can help us measure how much slower is access to TLS
variables in an OSv application (compiled as shared object) compared
to a Linux application.

The results of this benchmark on my laptop is as follows: In a simple
loop accessing a global variable, the iteration length was 2 ns.
In Linux, switching to a thread-local variable did not noticably change
the iteration length. But in OSv, changing the variable to a thread-local
one dramatically increases the iteration length to 27 nanoseconds.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Add options 'R' and 'a' to command ls

cli: Add option to recursively list subdirectories, fix option 'a'

Signed-off-by: Nehal J Wani <nehaljw.kkd1@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Add command poweroff

Signed-off-by: Nehal J Wani <nehaljw.kkd1@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Improve speed and scalability of __tls_get_addr()

As explained in issue #555, applications which use thread-local variables
are noticably slowed down by our slow implementation of __tls_get_addr().

The compiler calls this function in shared-library code which uses thread
local variables. Our implementation used a mutex, which is slow (relative
to an ordinary variable access), and non-scalable (two threads cannot
concurrently access their own thread-local variable!). In this patch we
switch to RCU, which is significantly faster and does not prevent
concurrent use.

This patch improves the tests/misc-tls.so benchmark from 27ns to 8ns.
__tls_get_addr() is still significantly slower than without it (2ns),
so there is still room for improvement - but it's a good start.

I also measured this patch to improve by 17% the performance of an actual
application which makes heavy use of a thread-local variable. The same
application also had abysmal performance when run on two threads
(2-thread performance was 10 times slower than one thread), and this
problem was improved as well.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Further improve __tls_get_addr() speed

After a previous sped up __tls_get_addr() (measured with tst/misc-tls.so)
from 27ns to 8ns, this patch further speeds it up to 4ns. Subtracting the
test iteration time for a non-tls variable (2ns), we are left with a 2ns
overhead for accessing a TLS variable, which is probably small enough to
tolerate.

In this patch, we reorganize the code called in __tls_get_addr() to make
sure the fast path is indeed as fast as possible. We notice that in the
fast path, we just use the module number and don't even need to find
the elf::object pointer - and the previous code organization, which always
found the elf::object, was wasting time doing this. Furthermore, we can
use more inline code (the previous code had 3 layers of out-of-line
functions calling each other). As mentioned above, these changes together
reduce the __tls_get_addr() overhead from 6ns to 2ns.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server Testing: update get file status test to reflect /etc/hosts permissions

/etc/hosts permission was changed from 666 to 664, which break the tests
that relay on those permission to check the file status API.

This update the test to compare to the updated permissions.

Suggested-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server: Modify the image in the testing

The testing require an image that contains java application to it can test the
java API. Now that the java based CLI is no longer supported the image that
will be used is jetty.

In addition, to simplify specific testing with and without ssl, make check was
broken in to two task check-ssl and check-http.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server Test: switch from shutdown to power off

When running the tests, it is better to use power off to the running image, to
make sure that the tests would end.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
libc/ctype: Migrate to upstream musl

The files under libc/ctype were added in commit 5cd0168 ("add the string
library from musl") and they have not been modified since. The imported
version is unknown but the difference to our musl git submodule are
cosmetic.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
libc: Use time/getdate.c from upstream musl

The difference is minimal so switch to upstream musl:

[penberg@nero osv]$ diff -u libc/time/getdate.c musl/src/time/getdate.c
--- libc/time/getdate.c	2014-08-07 21:58:30.627818727 +0300
+++ musl/src/time/getdate.c	2014-08-07 22:04:44.906664048 +0300
@@ -6,8 +6,6 @@

 int getdate_err;

-char *strptime(const char *__restrict s, const char *__restrict f, struct tm *__restrict tm);
-
 struct tm *getdate(const char *s)
 {
 	static struct tm tmbuf;

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Fix crash in tests/misc-mutex.so

tests/misc-mutex.so would consistently crash while testing the spinlock.
The problem is that after locking the spinlock, preemption remains
disabled. Then, when the test program continues to call spinlock::unlock()
for the first time, we attempt to resolve this symbol which is not allowed
in preemption-disabled time. We can have a similar problem if another
piece of the program text needs to be paged-in while preemption is disabled.

The solution is to use OSV_ELF_MLOCK_OBJECT on this shared-object.
It ensures that all symbols are resolved during load, instead of
when first used (possibly while preemption is disabled), and all the
executable always remains in memory.

Fixes #556

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>