Mention scripts/setup.py in Ubuntu section of README

The setup.py script works on Ubuntu, and without it
the install will be missing some dependencies.

Signed-off-by: Don Marti <dmarti@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
linux: Fix futex() command bitmask

The "op" argument passed to futex() also include flags so we need to
mask the command bits.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
linux: Wire up uname system call

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
tools/uush: add cd builtin and ls, mkdir binaries

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Sever: Add power off API

While the shutdown API attempts to do a clean power off, it sometimes fails and halt the shutdown process.
It is sometimes desireable to to a power off, which is force, unsafe shuting down of the system.

This adds a power off functionality to the API.

to test do:
http://localhost:8000/os/poweroff

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Benchmark for measuring TLS access performance

This patch adds a tool, tests/misc-tls.cc, for quanitifying issue #555.
I.e., this tool can help us measure how much slower is access to TLS
variables in an OSv application (compiled as shared object) compared
to a Linux application.

The results of this benchmark on my laptop is as follows: In a simple
loop accessing a global variable, the iteration length was 2 ns.
In Linux, switching to a thread-local variable did not noticably change
the iteration length. But in OSv, changing the variable to a thread-local
one dramatically increases the iteration length to 27 nanoseconds.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Add options 'R' and 'a' to command ls

cli: Add option to recursively list subdirectories, fix option 'a'

Signed-off-by: Nehal J Wani <nehaljw.kkd1@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
cli: Add command poweroff

Signed-off-by: Nehal J Wani <nehaljw.kkd1@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Improve speed and scalability of __tls_get_addr()

As explained in issue #555, applications which use thread-local variables
are noticably slowed down by our slow implementation of __tls_get_addr().

The compiler calls this function in shared-library code which uses thread
local variables. Our implementation used a mutex, which is slow (relative
to an ordinary variable access), and non-scalable (two threads cannot
concurrently access their own thread-local variable!). In this patch we
switch to RCU, which is significantly faster and does not prevent
concurrent use.

This patch improves the tests/misc-tls.so benchmark from 27ns to 8ns.
__tls_get_addr() is still significantly slower than without it (2ns),
so there is still room for improvement - but it's a good start.

I also measured this patch to improve by 17% the performance of an actual
application which makes heavy use of a thread-local variable. The same
application also had abysmal performance when run on two threads
(2-thread performance was 10 times slower than one thread), and this
problem was improved as well.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Further improve __tls_get_addr() speed

After a previous sped up __tls_get_addr() (measured with tst/misc-tls.so)
from 27ns to 8ns, this patch further speeds it up to 4ns. Subtracting the
test iteration time for a non-tls variable (2ns), we are left with a 2ns
overhead for accessing a TLS variable, which is probably small enough to
tolerate.

In this patch, we reorganize the code called in __tls_get_addr() to make
sure the fast path is indeed as fast as possible. We notice that in the
fast path, we just use the module number and don't even need to find
the elf::object pointer - and the previous code organization, which always
found the elf::object, was wasting time doing this. Furthermore, we can
use more inline code (the previous code had 3 layers of out-of-line
functions calling each other). As mentioned above, these changes together
reduce the __tls_get_addr() overhead from 6ns to 2ns.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server Testing: update get file status test to reflect /etc/hosts permissions

/etc/hosts permission was changed from 666 to 664, which break the tests
that relay on those permission to check the file status API.

This update the test to compare to the updated permissions.

Suggested-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server: Modify the image in the testing

The testing require an image that contains java application to it can test the
java API. Now that the java based CLI is no longer supported the image that
will be used is jetty.

In addition, to simplify specific testing with and without ssl, make check was
broken in to two task check-ssl and check-http.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server Test: switch from shutdown to power off

When running the tests, it is better to use power off to the running image, to
make sure that the tests would end.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
libc/ctype: Migrate to upstream musl

The files under libc/ctype were added in commit 5cd0168 ("add the string
library from musl") and they have not been modified since. The imported
version is unknown but the difference to our musl git submodule are
cosmetic.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
libc: Use time/getdate.c from upstream musl

The difference is minimal so switch to upstream musl:

[penberg@nero osv]$ diff -u libc/time/getdate.c musl/src/time/getdate.c
--- libc/time/getdate.c	2014-08-07 21:58:30.627818727 +0300
+++ musl/src/time/getdate.c	2014-08-07 22:04:44.906664048 +0300
@@ -6,8 +6,6 @@

 int getdate_err;

-char *strptime(const char *__restrict s, const char *__restrict f, struct tm *__restrict tm);
-
 struct tm *getdate(const char *s)
 {
 	static struct tm tmbuf;

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Fix crash in tests/misc-mutex.so

tests/misc-mutex.so would consistently crash while testing the spinlock.
The problem is that after locking the spinlock, preemption remains
disabled. Then, when the test program continues to call spinlock::unlock()
for the first time, we attempt to resolve this symbol which is not allowed
in preemption-disabled time. We can have a similar problem if another
piece of the program text needs to be paged-in while preemption is disabled.

The solution is to use OSV_ELF_MLOCK_OBJECT on this shared-object.
It ensures that all symbols are resolved during load, instead of
when first used (possibly while preemption is disabled), and all the
executable always remains in memory.

Fixes #556

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
ec2 script: allow name to be specified

"test-ami" may be already used - or whatever name we end up choosing.
Allow that to be user-specifiable.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
pci: separate generic code from x64-specific code

this patch moves the common pci definitions and the prototypes for
the architecture-dependent pci functions from drivers/pci.hh to
arch/common/pci.hh, and puts (most of) the x64-specific PCI stuff
in arch/x64/pci.cc.
Prepare arch-pci.hh with some x64-specific definitions, and build
the new arch/x64/pci.o module.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
pci: remove use of processor in pci-function.hh

there is no need to get the processor namespace just by
including pci-function.hh, which can be processor independent.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
pci: enable bars issuing pci command

for architectures other than x64 we need to reprogram the
BAR registers as we have no magical good values already
programmed by the "firmware" (qemu...).
It is also possible that there will be the need to reprogram
them on x64 in the future.
Trying to do that without explicitly enabling the I/O space
and memory space using a PCI command leads, in the case of
QEMU, to the reprogramming to be silently ignored.

There is therefore some minor restructuring of the pci
device / function code to account for a slightly different
procedure for the BARs, where we now:

enable access to I/O and mem, then
for each bar
    we first read the bar size, then
    call some arch-dependent code to fix up the final BAR reg,
    so that archs have a chance to reprogram it, then
    we proceed to assign the final address to the bar.

Signed-off-by: Claudio Fontana <claudio.fontana@huawei.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>