ec2 scripts: optionally support small instances

There are 2 types of HVM AMIs on Amazon:

1. Windows based that support all instances types
2. Linux based that support instances larger than medium only

Linux based AMIs suit OSv better because their default seciurity
policy exactly fits OSv model (SSH access) and doesn't require
reconfiguration.

Windows based AMIs require default security policy reconfiguration
in order to allow SSH access but from other hand allow running OSv
on free-tier eligible micro instances which is good for enthusiasts
and initial evaluations.

In order to support both options this patch extends release scripts
with ability to choose between Windows and Linux based templates
during AMIs creation.

Signed-off-by: Dmitry Fleytman <dmitry@daynix.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
apps: update for netperf jenkins integration

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
ahci: Relax _cmd_table and _recv_fis alignment

AHCI spec says cmd_table need to be aligned to 128 bytes, respect it.

AHCI spec does not specify the alignment requirement of _cmd_list and
_recv_fis. To be safe, we align to their size, i.e. 1024 and 256 bytes.

This fixes the issue reported by Roman Shaposhnik.

   $ capstan run -p vbox
   Created instance: i1398297574
   OSv v0.07
   Assertion failed: !(reinterpret_cast<uintptr_t>(ret) & (align - 1))
   (/home/penberg/osv/core/mempool.cc: alloc_phys_contiguous_aligned:
   1378)

   [backtrace]
   0x398316 <memory::alloc_phys_contiguous_aligned(unsigned long,
   unsigned long)+214>
   0x3542bb <ahci::port::setup()+283>
   0x3549ae <ahci::port::port(unsigned int, ahci::hba*)+142>
   0x354a5f <ahci::hba::scan()+111>
   0x354d34 <ahci::hba::hba(pci::device&)+148>
   0x354e07 <ahci::hba::probe(hw::hw_device*)+119>
   0x33e65c
   <hw::driver_manager::register_driver(std::function<hw::hw_driver*
   (hw::hw_device*)>)+188>
   0x33c8c3 <hw::device_manager::for_each_device(std::function<void
   (hw::hw_device*)>)+51>
   0x33e3de <hw::driver_manager::load_all()+78>
   0x20edf5 <do_main_thread(void*)+629>
   0x3ff0a5 <sync+69>
   0x3a3eaa <thread_main_c+26>
   0x361585 <thread_main+7>

Signed-off-by: Asias He <asias@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
malloc: Fix small allocations with alignment > size.

When a small allocation is requested with large alignment, we ignored
the alignment, and as a consequence posix_memalign() or
alloc_phys_contiguous_aligned() could crash when it failed to achieve
the desired alignment. This is not a common case (usually, size >= alignment,
and the new C11 aligned_alloc() even supports only this case), but still
it might happen, and we saw it in cloudius-systems/capstan#75.

When size < alignment, this patch changes the size so we can achieve the
desired alignment. For small alignments, this means setting size=alignment,
so for example to get an alignment of 1024 bytes we need at least 1024-byte
allocation. This is a waste of memory, but as these allocations are rare,
we expect this to be acceptable. For large alignments, e.g., alignment=8192,
we don't need size=alignment but we do need size to be large enough so we'll
use malloc_large() (malloc_large() already supports arbitrarily large
alignments).

This patch also adds test cases to tst-align.so to test alignments larger
than the desired size.

Fixes #271 and cloudius-systems/capstan#75.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
clock: Add fill_tv(), duration to timeval conversion function

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
libc: Move alarm implementation to itimer class, Add setitimer/getitimer

Move static variables for alarm and related function to itimer class.
itimer class offers interfaces to implement setitimer/getitimer, so added setitimer/getitimer.
Now alarm rewrite as a just a wrapper function to call setitimer.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
vfs: move dentry to its new location during file rename

If dentry has elevated reference count during rename (a file is opened
for instance) it is not destroyed and still hashed with old path. As a
result renamed file can still be accessed by its old name.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
test: add test for renaming of opened file.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
vfs: remove dentry of a destination after rename()

If dentry has elevated reference count during rename (a file is opened
for instance) it is not destroyed and remains in the dentry hash. As a
result old file is accessible by destination path instead of new one.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
test: add test for renaming to opened file.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
vfs: remove dentry of an unliked file from dentry cache

If dentry has elevated reference count during unlink (a file is opened
for instance) it is not destroyed and remains in the dentry hash.
As a result unlinked file is still accessible.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
test: add test for removing of an opened file.

Signed-off-by: Gleb Natapov <gleb@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
memory: add utility RAII classes for aligned physically contiguous memory

phys_ptr<T>: unique_ptr<> for physical memory
make_phys_ptr(): allocate and initialize a phys_ptr<>
make_phys_array(): allocate a phys_ptr<> referencing an array

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
ahci: use phys_ptr<> for physically allocated memory

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
trace: fix string conversion error on python 2.7.5

On python 2.7.5 python complains about struct() not accepting unicode
argument. Fix by adding explicit conversion.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
trace: disable port number conversion in tcpdump mode

tcpdump port number conversions vary from system to system. Turn of the
conversions to have consistent output which is also simpler to assert
for in tests.

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
trace: introduce TIMED_TRACEPOINT macro

This macro defines a tracepoint pair which allows to time a code
block.

It also defines a helper class which uses RAII pattern to invoke the
tracepoints. This is especially useful when the code we want to time
has gazillioon of return points and we want to cover them all.

The macro requires a corelation argument to support call nesting.

Example:

TIMED_TRACEPOINT(tcp_do_segment, "so=%x", struct socket*);

void tcp_do_segment()
{
   time_tcp_do_segment{so};

   ...
}
tests: introduce tracing smoke test

The test verifies that:
 * samples are collected by OSv
 * extraction via `trace extract` succeeds
 * `trace summary` shows expected tracepoint names
 * listing via `trace list` shows expected tracepoints
 * network packet parsing works

This test is run as part of standard 'make check'.

Note: this single test can be executed as easily as:

 $ scripts/test.py --test tracing_smoke_test
Advertise need for 'python-dpkt' system module

It's needed by TCP-related tracing commands, as of now:

 trace.py list --tcpdump
 trace.py tcpdump
trace: fix recommendation given by error message in case dpkt is missing

Since we adveritse use of a system package in README, we should also
recommend the same here.