rcu_hashtable: fix bad erase()

erase() first shrinks the hash table if needed, then erases the element
given by the iterator.  However, as elements are copied during shrinking
(to satisfy rcu), we erase the element in the *old* table, rather than
the new (shrunk) one.  The effect is that the element is not removed.

Found using wrk vs. tomcat, with the net channel rcu hashtable patch
applied.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Revert "Revert "net: use rcu_hashtable in net channel classifier""

This reverts commit 181d26253d5e, which itself reverted 181d26253d.

With the rcu hashtable fixed (16ab9520eb), we can use rcu_hashtable in the
net channel classifier again.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
bsd: avoid #define malloc() when possible

bsd uses a 3-argument variant of malloc() instead of the standard single-
argument malloc().  This causes problems when including standard headers
along with bsd headers.

Fix by using a function overload instead.  A #define is only used in C code.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
memory: allow using debug allocator in parallel with standard allocator

Minor changes, allowing dbg::malloc() to be used in parallel with ::malloc().
This allows use-after-free debugging when the misbehaving type is known or
suspected.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
mempool: fix page_range_allocator::_bitmap resize

When resizing a dynamic_bitset some memory may be freed. In such case it is
not safe to attempt to merge unallocated block since the bitmap is in
undefined state. Instead, the memory free is deferred to a moment when the
resize is done and bitmap is reinitialized with up to date values.

page_range_allocator::_bitmap uses now custom allocator to get rid of
numerous checks agains _nobitmap in allocation and deallocation code.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Revert "Revert "mempool: use new page range allocator""

This reverts commit 0ff46442658062554221fc22f7bf2225b2f9b4b0.

Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
memory: debug allocator helper

Add a helper template that defines the appropriate new/delete operators for
the debug allocator.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Merge branch dbg-alloc

Small debug allocator improvements.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
build: remove .SECONDARY

Somehow this confuses make, causing it to ignore the explicit dependency
between libstatic-thread-variable.so and tst-static-thread-variable.so,
breaking a parallel build.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>
Fix 'sign-compare' error in unordered_ring_mpsc.hh

Fixes the following error when compiling osv on my Archlinux machine
with gcc 4.9:

  In file included from tst-unordered-ring-mpsc.cc:12:0:
  unordered_ring_mpsc.hh:40:22: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
               if (_idx >= _ring.rings.size()) {
                      ^
  unordered_ring_mpsc.hh:48:33: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
               do {} while (++_idx < _ring.rings.size() && !_ring.rings[_idx].pop(_element));

Reviewed-by: Tomasz Grabiec <tgrabiec@gmail.com>
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build: change libgcc_s.a to libgcc.a

Fixes the following error with gcc 4.9 (build_env=host)
  osv/include/api/sched.h:125: undefined reference to `__popcountdi2'

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Reported-by: Minoru KANATSU <icc.pot.tyew272@gmail.com>
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build: fix tst-static-thread-variable.so dependencies (again)

Commit 7a69d0d8 and 99d80368 introduced:

  tests/tst-static-thread-variable.so: COMMON += -L./tests -lstatic-thread-variable

However make variables are inherited by prerequisites, which will cause
libstatic-thread-variable.so being added as its dependencies itself,
resulting in this error:

  ./tests/libstatic-thread-variable.so: file not recognized: File truncated

Add 'private' keyword [1] to surpass the inheritance.

[1]: https://www.gnu.org/software/make/manual/html_node/Suppressing-Inheritance.html

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
scripts: encode string before use it as buffer

In python3, string and bytearray are now different.

This patch fixes:
  Adding /usr/lib/jvm/java/lib/tools.jar...
  Traceback (most recent call last):
    File "osv/scripts/upload_manifest.py", line 192, in <module>
        main()
      File "osv/scripts/upload_manifest.py", line 180, in main
        upload(osv, manifest, depends)
      File "osv/scripts/upload_manifest.py", line 120, in upload
        cpio_send(link)
      File "osv/scripts/upload_manifest.py", line 80, in cpio_send
        s.sendall(data)
    TypeError: 'str' does not support the buffer interface.

Signed-off-by: Zifei Tong <zifeitong@gmail.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Merge branch 'archlinux-build-fixes'
Adding libtools.so library

The libtools will hold code that is not part of the kernel but is
required by both tools and the API.

As a first step it holds the logic for lsroute and remove it from
kernel.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server: Write a text to a file

This adds the ability to write a text to a file.
The new op code WRITE will write the content to a file, if the overwrite
flag is set to true, the file will be truncated, if not, the text will
be append.

Calling PUT /file/f.txt?op=WRITE&overwrite=true&content=hello
will write hello to /f.txt overwrite its content if exist.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server: code generation improve error handling

This series changes the somehow cryptic code generation error messages
to more meaningful ones.

It catches all exception and print an error message with the current
file being parsed.

It also tries to identify if a missing attribute causes an error and
report the missing attribute with its whereabout.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Revert "Adding libtools.so library"

This reverts commit 049b1a9bb60e8a9d179c11afd0b74601e81b29f3.

It breaks aarch64 build.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Http Server: Implicitly call json formatting for json handlers

Most of the API method return json type object. This patch remove the
need to explicitly call the json formatter when returning the result.

To do so a wrapper for the return type was added, it uses autoboxing to
translate return types to their json formatted representation.

The set_handler method with default return type, is set to return json
and to accept lambda expression that return the wrapper.

After the change it is possible to call:
getOSversion.set_handler([](const_req req)
    {
        return osv::version();
    });

The return string will be translated to json string.
No need to call the formatter anymore and no need to set the return type
to json.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
build: fix build/release link generation

Since there's no rule for the link, it needs to be forced every build.

Signed-off-by: Avi Kivity <avi@cloudius-systems.com>