gdb: update flag translation

We have added a new mmap_small flag, so translation need to be updated.
's' is taken, so let's go for 'm'.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: fix move-to-itself bug

While working in the reclaim policy, I came across an unhandled balloon case
that seem to happen in some certain low memory conditions. The balloon will be
moved to the same address it was before. While this seems weird at first, it is
perfectly plausible if we think that we can have a space before the balloon almost
a page in size. If the JVM is only shifting the elements slightly, they will end up
at the same ballooned region.

The problem with that is that evacuate() is called first - the vma is still in place,
so it will be found, and it will call delete. When the jvm fault handler executes,
it will also call delete and we will have a double free.

This patch avoids the delete, but the problem is actually more intricate than
that: if we just follow the normal path, we will remove the newly created
address from the candidates list right after we insert it (because the
addresses are the same). Also, if the address is *not* the same, but falls in
the same region (this is theoretical), evacuate will be called and split the region
in two. Our code to handle finish_move would have to be significantly complicated
to deal with this fact. This patch also handles this theoretical case by making
sure that the whole region is always disposed at once (by finishing the old balloon
manually before evacuate is called)

Why I am using a new mmu flag: when we search for the old vma, we need to
identify which type of vma we are dealing with. If it is an anonymous or
ballooned area. I use an extra flag to avoid a type field or anything like
that.

Couldn't we just take the vma out of the list before we delete it? In this case
the region would be empty, and we could assume that was a ballooned region.
That would also be a solution, but because we are copying the flag from the
previous vma, we would have to fallback to a default flag set. The solution
of marking the JVM vma and leaving it in place allows the flag information to
be carried on.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: use a shared ptr for the balloon object

The old explicit pattern would break if the vma that holds the balloon
was already destroyed. Use a shared pointer to avoid the complications from it

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
reclaimer: implement a stronger data structure for waking sleepers

After Avi's last comments, here is a new take for the reclaimer's wake queue.
It does not abuse semaphore's in any way, so there is no need to extend it.
(After this patch is applied, the semaphore extension patch can be reverted)

In this new approach, we will not only check for free memory as a scalar, but
will instead transverse the remaining page ranges (notice that in case of
fragmentation this can be a *large* set) and try to find a range that would
allow our allocation to happen.

Also, because we are implementing both the wait and the "post" methods, the
waking of the reclaimer happens atomically and the problem that Tomek described
(wake before waiters is set) no longer exists.

Finally, another advantage of this approach is that we no longer need to keep
track of the amount of memory we have freed when making OOM decisions: if we
can wake someone up, we can continue. Otherwise, we OOM. This will be specially
helpful for the FS shrinkers, since they may not always be able to synchronously
return a meaningful number accurately (due to, for instance, writeback).

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
jvm_balloon: better define boundaries for the balloon.

Right now, we are starting the jvm with the maximum amount of memory we can.
This is, however, terribly bad. The reason is that if that if we go to reclaim
during a heap allocation, this will easily crash the system. That will be just
for granted if that allocation turns out to be done on behalf of the GC. In
that case, no other Java threads will make progress, meaning the balloon cannot
be activated.

Even if the balloon is no longer part of the shrinker infrastructure, it may
still be a problem, because we may need to use it to deflate memory. The best
solution is to just limit heap usage to 90 % of the available memory, which is
the reclaim criteria.

Also, it is pointless to activate the balloon for very low memory conditions.
So if our total memory is less than 1Gb, just leave it alone.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
java: heuristically determine the heap regions at startup

We can determine which mappings from the JVM are heap areas in a more
robust way by flipping a flag when we start JVM creation, and flipping
it back when we are finished. Doing this early is more robust and should
work better with a bigger set of GCs.

Although this is a heuristic, Calle agrees this is safe, and it has
been tested and confirmed to work with the following options:

    -XX:+UseSerialGC
    -XX:+UseG1GC
    -XX:+UseParallelGC
    -XX:+UseParallelOldGC
    -XX:+UseParNewGC
    -XX:+UseConcMarkSweepGC
    -XX:+UseConcMarkSweepGC

which pretty much covers all relevant GCs

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
jvm_balloon: remove the probe

New method does not rely on physical memory accounting, so the probe
is useless.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
jvm balloon: does not take the balloon lock during fault

The lock itself is no longer needed - it was in the past before the balloon was
directly put in the vma. But besides that, it is leading to subtle deadlocks:
let's say a new balloon is being inserted into the list, which needs to be done
with lock held. If Garbage Collection kicks in at this very same time and
happens to move a balloon, it will stop in the lock. This means that all java
threads will stop, waiting for the GC to finish. All it takes is a single JNI
call with the lock taken to achieve a deadlock. In particular, deleting a balloon
involves destroying a reference, so that is enough to trigger it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
jvm_balloon: handle partial writes

There are situations during which some Garbage Collectors will copy a large
object in parallel, using various threads, each being responsible for a part of
the object. In particular, that easily happens under high memory pressure for
the Parallel and ParallelOld collectors.

If such copy happens, the simple balloon move algorithm will break. However,
because offset 'x' in the source will always be copied to offset 'x' in the
destination, we can still calculate the final destination object.  The problem
is that we cannot open the new balloon yet. Since the JVM believes it is
copying only a part of the object, the destination may (and usually will)
contain valid objects, that need to be themselves moved somewhere else before
we can install our object there.

Also, we can't close the object fully when someone writes to it: because a part
of the object is now already freed, the JVM may and will go ahead and copy
another object to this location.

This is a quite tricky problem, but the patch bellow solves it by keeping the
source balloon open and counting the number of partial bytes copied. It makes
the assumption that the JVM needs to always copy the whole object to a different
location, meaning that when it does, the balloon can be closed.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
map_jvm: update the balloon object on address reuse

If we are reusing an address, it might be that we are doing for the same
balloon - move-to-self scenario, or it might be that simply there is another
balloon that is moved to the address where previously another balloon was held,
but before we had the chance to close it. In the later case, the balloon object
- which is used to compute offsets in the object is wrong, and we will copy
things to unpredictable locations. Need to update it correctly.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
zfs: add ability to control amount to be reclaimed

Currently, although arc_evict() can be told a size up until it should reclaim,
the high level reclaimer is always too aggressive, reclaiming up to a
parametrized value. There are situations in which we would just like to reclaim
enough to make room for something. For those cases, I am introducing here a
function that allow us to inform ZFS what is the size we need to see evicted
from the caches.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: no longer make it part of the shrinker infrastructure.

The current ballooning mechanism has a very serious design flaw, that became
obvious once the tests with it advanced: if we wait until memory is short to
release memory, it may be that the JVM itself may block. In particular, the
blocking thread can be running the Garbage Collector. More likely, it can
start running the GC to open up space for an object so big as the balloon.

I don't believe this is something we should fix with a quick hack, and I am
then pushing for a redesign: The solution is to use a reservation system, that
works by ballooning beforehand when the amount of reserved memory goes too low.

This system works by noting that when we balloon, we are effectively limiting
the maximum amount of memory java will ever use. We already have accounts of
both the amount of free memory and the amount of memory allocated to the heap.
With those figures, we can easily find out when the amount of used memory +
a full heap would go over the amount of available memory. When that happens,
we balloon.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
mmu: return vma size instead of evacuated pages

As we move towards a reservation system, the actual memory that we have freed
matters little. Return instead the size of the vma that was passed to OSv

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: no longer use recent statistics

We are moving the balloon out of the reclaimer path, so those does not
really make a lot of sense.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: use a separate thread the for the balloon

Synchronously ballooning unfortunately won't work: as usual the garbage
collector can run in order to release memory for the balloon, and then we will
block.

Since we can't make the balloon worker synchronous, we set its priority to a
really high number, that would guarantee that Java threads won't run before it.
This will help in really fast allocation situations. Durin those, between the
notification and the time this thread gets to run, the JVM may have very well
exhausted all the memory remaining.

Notice that this value is guaranteed to be lower than any JVM created thread
ever will get. This is since for JVM threads, priority will only be set through
the setpriority interface, wich will yield 0.2 for its maximum prio.
jvm_balloon: fix braino

small logic invertion: it is when we *do* copy that we should break. Was not important
up to now because we were always executing this loop once. It is important now because
we will execute it multiple times.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: close the balloons earlier if the regions intersect

Instead of just looking at which region our start address falls at, we can
iterate through all the vmas that are affected by us installing a new vma, and
then close the region earlier.

This design has one major big advantage: split() will never happen, and if that
is the case, the balloon becomes a lot more predictable - which at this point
is a blow of sanity in the whole thing

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
mempool: transform reclaimer thread into a normal thread

The balloon is no longer synchronous, and now lives in its own
thread. We used to have the requirement that the reclaimer needed
to have a full pthread API, but that is no longer. Make it now a
normal thread.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
reclaimer: add tracepoint for waiters

Useful for debugging, specially for the backtrace

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
mempool.cc: print the amount of memory available when we oom

We consider ourselves to be "out of memory" when an allocation can not be
satisfied, not when we reach 0 free memory. If we are heavily fragmented, those
two situations can be quite departed from each other. I have never printed the
amount of free memory at oom because abort() could not handle a format. Now it
can.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>