Add SCOPE_LOCK(mutex) macro

Add a macro SCOPE_LOCK(mutex) which locks the given mutex and unlocks
it when the scope ends (this uses RAII, so the mutex will correctly get
unlocked even when the scope is exited via return or exception).

This does the same as C++11's std::lock_guard, but far less verbose:
To use std::lock_guard with a mutex m, one nees to do something like
std::lock_guard<mutex> guard(m);
where the mutex's type needs to be repeated, and a name needs to be
invented for the guard which will likely not be used again. This
macro makes these things unnecessary, and one just writes
SCOPE_LOCK(m);

Note that WITH_LOCK(m) { ... } should usually be preferred over SCOPE_LOCK.
However, SCOPE_LOCK can come in handy in some cases, for example adding a
lock to a function without reindenting it.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Fix loader.cc parallel running bug

When running a command in the background, do_main_thread() passes the
command line in a std::vector pointer to a new pthread. Unfortunately,
soon afterwards the vector can go out of scope and the result is a
crash. Fix this oversight.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Elf: Serialize shared-object load and unload

Our current dynamic-linker code (elf.cc) is not thread safe, and all sort
of disasters can happen if shared objects are loaded, unloaded and/or used
concurrently. This and the following patches solve this problem in stages:

The first stage, in this patch, is to protect concurrent shared-library
loads and unloads. (if the dynamic linker is also in use concurrently,
this will still cause problems, and will be solved in the next patches).

Library load and unload use a bunch of shared data without protection,
so concurrency can cause disaster. For example, two concurrent loads can
pick the same address to map the objects in. We solve this by using a mutex
to ensure only one shared object is loaded or unloaded at a time.

Instead of this coarse-grain locking, we could have used finer-grained
locks to allow several library loads to proceed in parallel, protecting
just the actual shared data. However the benefits will be very small
because with demand-paged file mmaps, "loading" a library just sets up
the memory map, very quickly, and the object will only be actually read
from disk later, when its pages get used.

Fixes #175.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Elf: Fix rare load/unload race

The previous patch fixed most of the concurrent shared-object load/unload
bugs except one: We still have a rare race between loading and unloading the
*same* library.

In other words, it is possible that just when a library's reference count
went down to 0, and its destruction will promptly start (the mutex is not
yet taken), get_library() gets called to load the same library. We cannot
return the already-loaded one (because it is in the process of being
destructed). So in that case we need to ignore the library we found,
and load it again. Luckily, std::weak_ptr's notion of "expired" shared
pointers (that already went down to 0, and can't be lock()ed) and its
atomic implementation thereof, makes fixing this problem easy and natural.

The existing code already had an assert() in place to protect against this
race (and it was so rare we never actually saw it), but this patch should
fix this race once and for all.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Elf: Fix shared-object load concurrent with dynamic linker use

This patch addresses the bugs of *use* of the dynamic linker - looking
up symbols or iterating the list of loaded objects - in parallel with new
libraries being loaded with get_library().

The underlying problem is that we have an unprotected "_modules" vector
of loaded objects, which we need to iterate to look up symbols, but this
list of modules can change when a new shared object is loaded.

We decided *not* to solve this problem by using the same mutex protecting
object load/unload: _mutex. That would make boot slower, as threads using
new symbols are blocked just because another thread is concurrently loading
some unrelated shared object (not a big problem with demand-paged file
mmaps). Using a mutex can also cause deadlocks in the leak detector,
because of lock order reversal between malloc's and elf'c mutexes: malloc()
takes a lock first and then backtrace() will take elf's lock, and on the
other hand elf can take its lock and then call malloc taking malloc's lock.

Instead, this patch uses RCU to allow lock-free reading of the modules
list. As in RCU, writing (adding or removing an object from the list)
manufactures a new list, defering the freeing of the old one, allowing
reads to continue using the old object list.

Note that after this patch, concurrent lookups and get_library() will
work correctly, but concurrent lookups and object *unload* still will
still not be correct because we need to defer an object's unloading from
memory while lookups are in progress. This will be solved in a following
patch.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Elf: Refactor shared-object deleter

In the current code, when a shared-object's reference count drops to zero,
shared_ptr<object> calls delete on this object, which calls the object's
destructor, which (see file::~file) calls program::remove_object() to
remove this object from the program's _files list.

This order was always awkward and unexpected, and I even had a comment
in ~file noting that "we don't delete(ef) here - the contrary, delete(ef)
calls us".

But for the next patch, we can no longer live with this order, which is
not just awkward but also wrong: In the next patch we'll want a shared
object to be immediately removed from the program (so it is no longer
found when resolving symbols, etc.), but the actual deletion of this
object to be delayed till a more convenient time. For this to work, we
cannot have the deletion of the object start before its removal from the
program: We need the opposite - the removal of the object from the program
will delete the object.

Luckily, it's easy to do this properly using a shared_ptr deleter lambda.
This tells shared_ptr to call remove_object(), not delete, when the
reference count drops to zero.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Elf: Fix shared-object unload concurrent with dynamic linker use

After the above patches, one race remains in the dynamic linker: If an
object is *unloaded* while some symbol resolution or object iteration
(dl_iterate_phdr) is in progress, the function in progress may reach
this object after it is already unmapped from memory, and crash.

Therefore, we need to delay unmapping of objects while any object
iteration is going on. We need to allow the object to be deleted from
the _modules and _files list (so that new calls will not find it) but
temporarily delay the actual freeing of the object's memory.

The cleanest way to achieve this would have been to increment each
modules' reference in the RCU section of modules_get(), so they won't
get deleted while still in use. However, this will signficantly slow down
users like backtrace() with dozens of atomic operations. So we chose
a different solution: keep a counter _modules_delete_disable, which
when non-zero causes all module deletion to be delayed until the counter
drops back to zero. with_modules() now only needs to increment this
single counter, not every separate module.

Fixes #176.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Elf: Fix also _module_index_list

Fix also concurrent use of _module_index_list (for the per-module TLS).
Use a new mutex _module_index_list mutex to protect it. We could
probably have done something with the RCU instead, but just adding a
new mutex is a lot easier.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>