string: add fixups for memcpy operations

When we start using the JVM balloon, our memcpy could fail for valid reasons
when the JVM is moving memory that is now in an unmapped region. To handle that,
register a fixup that will trigger a JVM call when the fault happens. If all goes
well, we will be able to continue normally.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu: make operate quantifiable.

operate so far operates in a page range and at the very most sets a success
flag somewhere. I am here extending the API to allow it to return how much
data it manipulated.

So as an example, if we fault in 2Mb in an empty range, it will return 2 << 20.
But if fault in the same 2Mb in a range that already contained some sparse 4k
pages, we will return 2 << 20 - previous_pages.

That will be useful to count memory usage in certain VMAs.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu: account evacuated size

This will be useful when we shrink, so we know how much memory we newly
released for system consumption.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
semaphore: allow extending the interface

Following an early suggestion from Nadav, I am trying to use semaphores for the
balloon instead of keeping our own queue. For that to work, I need to have a bit
more functionality that may not belong in the main balloon class. Namely:

1) I need to query for the presence of waiters (and maybe in the future for the
number of waiters)

2) I need a special post that would allow me to make sure that we are almost posting
at most as much we're waiting for, and nothing more.

This patch transforms the post method in an unlocked version (and exposes a
trivial version that just locks around it) and make other changes necessary to allow
subclassing

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mempool: shrink memory when no longer used.

This patch introduces the memory reclaimer thread, which I hope to use to
dispose of unused memory when pressure kicks in. "Pressure" right now is
defined to be when we have only 20 % of total memory available. But that can be
revisited.

The way it will work is that each memory user that is able to dispose of its
memory will register a shrinker, and the reclaimer will loop through them.
However, the current "loop through all" only "works" because we have only one
shrinker being registered. When other appears, we need better policies to drive
how much to take, and from whom.

Memory allocation will now wait if memory is not available, instead of
aborting.  The decision of aborting should belong to the reclaimer and no one
else.

We should never expect to have an unbounded and more importantly, all opaque,
number of shrinkers like Linux does. We have control of who they are and how
they behave, so I expect that we will be able to make a lot better decisions
in the long run.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mmu: implement a new JVM vma

After carrying on some testing, I quickly realized that the old fixup-only
solution I was attempting for the ballooning was not really flying. The reason
for that, is that we would take a fault, figure out the fixup address, and
return.  If that wasn't a JVM fault, we were forced to take another fault
(since we were already out of fault context).

Once demand paging is a reality, the vast majority of the faults are for non
balloon addresses, so we were effectively doubling our number of page faults
for no reason. I have decided to go with the VMA (+fixups for instruction
decoding) route after all. This is way more efficient and it seems to be
working fine.

The JVM vma is really close to the normal anonymous VMA. Except that it can
never hold pages, and its fault handler calls into the JVM balloon facilities
for decoding.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
JVM ballon driver

This patch implements the JVM balloon driver, that is responsible for borrowing
memory from the JVM when OSv is short on memory, and giving it back when we are
plentiful. It works by allocating a java byte array, and then unmapping a large
page-aligned region inside it (as big as our size allows).

This array is good to go until the GC decides to move us. When that happens, we
need to carefuly emulate the memcpy fault and put things back in place.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
jvm_balloon: control shrinker activation / deactivation

There are restrictions on when and how a shrinker can run. For instance, if we
have no balloons inflated, there is nothing to deflate (the relaxer should,
then, be deactivated). Or also, when the JVM fails to allocate memory for an
extra balloon, it is pointless to keep trying (which would only lead to
unnecessary spins) until *at least* the next garbage collection phase.

I believe this behavior of activation / deactivation ought to be shrinker
specific. The reclaiming framework will only provide the infrastructure to do
so.

In this patch, the JVM Balloon uses that to inform the reclaimer when it makes
sense for the shrinker or relaxer to be called.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
java: memory pressure monitor

The best possible criteria for deflating balloons is heap pressure: Whenever
there is pressure in the JVM, we should give back memory so pressure stops.

To accomplish that, we need to somehow tap into the JVM. This patch register
a MXBean that will send us notifications about collections. We will ignore
minor collections and act upon major collections by deflating any existing
balloons.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
jvm: insert probe

To find out which vmas hold the Java heap, we will use a technique that is very
close to ballooning (in the implementation, it is effectively the same)

What we will do is we will insert a very small element (2 pages), and mark the
vma where the object is present as containing the JVM heap. Due to the way the
JVM allocates objects, that will end up in the young generation. As time
passes, the object will move the same way the balloon moves, and every new vma
that is seen will be marked as holding the JVM heap.

That mechanism should work for every generational GC, which should encompass
most of the JDK7 GCs (it not all). It shouldn't work with the G1GC, but that
debuts at JDK8, and for that we can do something a lot simpler, namely: having
the JVM to tell us in advance which map areas contain the heap.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
mm: Count total memory used by the JVM heap

To make informed reclaim decisions, we need to have as much relevant
information as possible about our reclaim targets. Specifically, it
is useful to know how much memory is currently used by the JVM heap.

The reasoning behind this is that if pressure is coming from the heap,
ballooning will harm us, instead of helping us.

Note: This is really just a first approximation. Ideally, total memory
shouldn't matter, but rather memory delta since a last common event.
But counting memory is the initial first step for both.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
jvm_balloon: disable balloon upon jvm memory pressure.

The biggest problem I am seeing with the balloon is that right now the only
time we call the balloon is when we're seeing memory pressure. If pressure is
coming from the JVM, we can livelock in quite interesting ways. We need to
detect that and disable the ballon in those situations, since ballooning when
the pressure comes from the JVM will only trash our workloads.

It's not yet working reliably, but this is the direction I plan to start from.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
jvm: set max_heap to all available memory.

We respect -Xmx when instructed by the user, but when that is left blank, we
set that to be all remaining memory that we have. That is not 100 % perfect
because the JVM itself will use some memory, but that should be good enough of
an estimate. Specially given that some of the memory currently in use by OSv
could be potentially freed in the future should we need it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>