Add ncurses module

A requirement for libedit, which is required by the cli.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Add libedit module

Required by the cli.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Add lua module

Required by cli. Downloads and builds Lua 5.2.

Lua's original Makefile comes without a target to create a shared-object.
For this, this build extract the sources to `src/` which contains a Makefile
that creates the shared-object.

Also adds the following modules:

* LuaSocket
* LPeg
* LuaJSON

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Add cli module code

cli.c embeds Lua as the engine of the CLI. It makes it dynamic and scriptable.
The added benefit is that Lua maintains its own state, which enables us to
treat it as a single entity that can be reset when needed.

The cli takes a command with libedit and passes it to cli() in cli.lua. If an
error occures (assert() or error()), it is being taken care of with a proper
message.

Some utilities are available, like table/column printing (see table_print())
and getopt_long()-like method (see echo commands for usage example).

It is possible to use this cli locally (outside OSv), but you need to build
its dependencies first: modules/{ncurses,libedit,lua}.

Commands are placed under commands/ folder.
More documentation is placed under docs/, currently.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Build code for cli module

Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Some documents created while working on the CLI

Signed-off-by: Or Cohen <orc@fewbytes.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu: file-mapped vma flag

Add flag bit to mark if a vma region is file mapped or not + add gdb
script to query individual vma entry.

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
libc: snprintf for "infinite" lengths

Fixes #111.

The standard C snprintf() has an interesting design flaw: snprintf()
takes a "size_t" length, and returns an "int". Unfortunately, size_t
is 64-bit while int is 32-bit, so it is theoretically possible for
snprintf() to succeed writing over 4 gigabytes, and not be able to return
the actual size it wrote!
This is why Posix mandates that when snprintf() is given a length that
cannot fit in 32 bits, it must immediately fail and set errno=EOVERFLOW.
Our implementation, copied from Musl, did this.

However, it turns out that neither glibc nor gcc's builtin snprintf
goes along with this decision, as the test in this patch demonstrates.
We can snprintf with maximum lengths = 2^64-1 or 2^40, and it works well.
This patch tests that we do this like Linux (the test can be run on both
OSv and Linux), and removes the EOVERFLOW case to be more like Linux.

Interestingly, this issue caused bug #111. With -D_FORTIFY_SOURCE=2,
ncurses's use of sprintf() was converted to __sprintf_chk() with a "len"
of -1 (2^64-1). Our implementation of this function called snprintf()
with that len - and failed because our implementation refused to work
when given len=-1.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Add __longjmp_chk()

When compiled with -D_FORTIFY_SOURCE=1 (this is the default in Ubuntu),
a program that uses longjmp() can end up calling __longjmp_chk() instead.
This patch adds a trivial implementation of this missing function.

Ideally, __longjmp_chk() should do some sanity checks on the "env" we're
jumping to, namely that it actually points to a stack frame earlier on
our stack. This patch is functionally-correct, but includes no such checks.
Glibc does some checks, but they are also far from being bulletproof (they
do little more than checking the env address is higher than the current
stack pointer).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
command line: don't wait for disowned threads

If we run

        run.py -e "command1 & command2"

This runs command1 and command2 in parallel. Only when the two commands
return from main(), OSv will shut down.

But for some service background threads, delaying the shutdown is not
a feature, it's a bug. For example if you run,

        run.py -e "libhttpserver.so & /tests/tst-chdir.so"

the guest will never exit, despite tst-chdir.so finishing in less than a
second, because libhttserver never completes.

It's even more annoying now that we automatically add "libhttpserver.so &"
to every command line - now even a 'run.py -e /tests/tst-chdir.so' will
never complete.

This patch adds a new alternative to "&": "&!". This syntax is inspired
by zsh, where &! means creating a *disowned* background process - one which
the shell doesn't track, and doesn't worry about when the shell exits.

With this patch,

        run.py -e "libhttpserver.so &! /tests/tst-chdir.so"

Will shutdown as soon as chdir.so finishes - no matter if libhttpserver.so
finished (it never will...).

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
osvinit: fix linking of osvinit.so

The placement of -l is significant. We should put dynamic libraries
last otherise they will not be recorded as dependencies which leads to
symbol resolution errors like this one:

Failed looking up symbol _ZN4YAML6detail9node_data12empty_scalarE (YAML::detail::node_data::empty_scalar)

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mgmt: update to latest

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
loader: always run httpserver, and other stuff from /init

Some modules, especially httpserver, have a command line which we *always*
want to run, regardless of run.py's "-e" paramter. For example, if the
user builds an image with "make image=httpserver,memcached", the default
is to run both, but if someone uses run.py -e to run memcached with
special parameters, httpserver is not run.

This patch fixes this annoyance by adding a new mechanism to run a
command line for each module on boot time:

The idea is that files in the directory "/init/" specify command lines that
will be run, at filename alphabetical order, before the "-e" command line.
The contents of each /init/* file is our usual command line syntax
(supporting quotes, ";", "&" and "&!").

With this patch, the "httpserver" module no longer has a classic command-line
setting. Rather, it includes a file /init/10-httpserver, with the contents

        /libhttpserver.so &!

This command line will *always* be run on boot, regardless of whether or
not a command line is specified with run.py's "-e" option. The "&!" means
that if the main command line finishes, OSv will shut down, without waiting
for libhttpserver.so to finish (that would never happen).

A "--noinit" kernel option is provided to override the /init/* files
and not run them.

Fixes #358.

Reviewed-by: Tomasz Grabiec <tgrabiec@gmail.com>
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Merge branch 'lua-cli'

From Or:

  This patch set adds an initial version of the Lua based CLI to have the
  development continue upstream.

  It also adds the dependant (osv) modules: lua, ncurses and libedit.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Move osvinit into osv.git

This moves the osvinit code from mgmt.git to osv.git while preserving
commit history.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
modules/osvinit: Fix module configuration

Point the module configuration to modules/osvinit instead of the
mgmt git submodule.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>