jvm_balloon: handle partial writes

There are situations during which some Garbage Collectors will copy a large
object in parallel, using various threads, each being responsible for a part of
the object. In particular, that easily happens under high memory pressure for
the Parallel and ParallelOld collectors.

If such copy happens, the simple balloon move algorithm will break. However,
because offset 'x' in the source will always be copied to offset 'x' in the
destination, we can still calculate the final destination object.  The problem
is that we cannot open the new balloon yet. Since the JVM believes it is
copying only a part of the object, the destination may (and usually will)
contain valid objects, that need to be themselves moved somewhere else before
we can install our object there.

Also, we can't close the object fully when someone writes to it: because a part
of the object is now already freed, the JVM may and will go ahead and copy
another object to this location.

This is a quite tricky problem, but the patch bellow solves it by keeping the
source balloon open and counting the number of partial bytes copied. It makes
the assumption that the JVM needs to always copy the whole object to a different
location, meaning that when it does, the balloon can be closed.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
map_jvm: update the balloon object on address reuse

If we are reusing an address, it might be that we are doing for the same
balloon - move-to-self scenario, or it might be that simply there is another
balloon that is moved to the address where previously another balloon was held,
but before we had the chance to close it. In the later case, the balloon object
- which is used to compute offsets in the object is wrong, and we will copy
things to unpredictable locations. Need to update it correctly.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
zfs: add ability to control amount to be reclaimed

Currently, although arc_evict() can be told a size up until it should reclaim,
the high level reclaimer is always too aggressive, reclaiming up to a
parametrized value. There are situations in which we would just like to reclaim
enough to make room for something. For those cases, I am introducing here a
function that allow us to inform ZFS what is the size we need to see evicted
from the caches.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: no longer make it part of the shrinker infrastructure.

The current ballooning mechanism has a very serious design flaw, that became
obvious once the tests with it advanced: if we wait until memory is short to
release memory, it may be that the JVM itself may block. In particular, the
blocking thread can be running the Garbage Collector. More likely, it can
start running the GC to open up space for an object so big as the balloon.

I don't believe this is something we should fix with a quick hack, and I am
then pushing for a redesign: The solution is to use a reservation system, that
works by ballooning beforehand when the amount of reserved memory goes too low.

This system works by noting that when we balloon, we are effectively limiting
the maximum amount of memory java will ever use. We already have accounts of
both the amount of free memory and the amount of memory allocated to the heap.
With those figures, we can easily find out when the amount of used memory +
a full heap would go over the amount of available memory. When that happens,
we balloon.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
mmu: return vma size instead of evacuated pages

As we move towards a reservation system, the actual memory that we have freed
matters little. Return instead the size of the vma that was passed to OSv

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: no longer use recent statistics

We are moving the balloon out of the reclaimer path, so those does not
really make a lot of sense.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: use a separate thread the for the balloon

Synchronously ballooning unfortunately won't work: as usual the garbage
collector can run in order to release memory for the balloon, and then we will
block.

Since we can't make the balloon worker synchronous, we set its priority to a
really high number, that would guarantee that Java threads won't run before it.
This will help in really fast allocation situations. Durin those, between the
notification and the time this thread gets to run, the JVM may have very well
exhausted all the memory remaining.

Notice that this value is guaranteed to be lower than any JVM created thread
ever will get. This is since for JVM threads, priority will only be set through
the setpriority interface, wich will yield 0.2 for its maximum prio.
jvm_balloon: fix braino

small logic invertion: it is when we *do* copy that we should break. Was not important
up to now because we were always executing this loop once. It is important now because
we will execute it multiple times.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: close the balloons earlier if the regions intersect

Instead of just looking at which region our start address falls at, we can
iterate through all the vmas that are affected by us installing a new vma, and
then close the region earlier.

This design has one major big advantage: split() will never happen, and if that
is the case, the balloon becomes a lot more predictable - which at this point
is a blow of sanity in the whole thing

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
balloon: fix move to self with partial copy

A similar bug that has been already fixed with move-to-self for the normal
balloon case also happens with partial copies. Fix them here, as early in the
code path as we can (that last part, to avoid useless spent cycles). Reminding
again, "move to self" happens not when the JVM move the object to the same
address - which would be stupid, but to an address that is close enough to the
old address to fall in the same mapping.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
mempool: transform reclaimer thread into a normal thread

The balloon is no longer synchronous, and now lives in its own
thread. We used to have the requirement that the reclaimer needed
to have a full pthread API, but that is no longer. Make it now a
normal thread.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
reclaimer: add tracepoint for waiters

Useful for debugging, specially for the backtrace

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
mempool.cc: print the amount of memory available when we oom

We consider ourselves to be "out of memory" when an allocation can not be
satisfied, not when we reach 0 free memory. If we are heavily fragmented, those
two situations can be quite departed from each other. I have never printed the
amount of free memory at oom because abort() could not handle a format. Now it
can.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
reclaimer: wipe out conditionals

One of the reasons we are being overly aggressive with ZFS reclaim is that once
we reach the reclaimer, unless we take ourselves away from the pressure
situation, we will keep reclaiming until we do. That happens even if we don't
need any extra memory. This is particularly problematic given the way ZFS
reclaimer is designed: if it is called twice in a row, it changes its mode of
operation from conservative (ARC_RECLAIM_CONS) to aggressive
(ARC_RECLAIM_AGGR), and will shrink the cache to the current cache usage.

Since we will wake the reclaimer every time we have reason to believe memory is
under pressure, there is no need for the reclaimer itself to keep looping.

After this patch is applied, the reclaimer is still called sequentially if the
allocation rate is high (because we will wake it many times), but will be left
alone if it isn't.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
reclaimer: only wake reclaimer when memory is used by OSv itself

The current waking criteria - to wake up the reclaimer when memory is overall
short - won't really work when we have ballooning enabled: in that situation,
the whole of memory will most likely be taken by the heap. This means that we
will constantly be bellow the threshold and will reclaim at every opportunity.

OSv memory usage won't grow, and we won't be placing memory when it is most
needed.  What we really need is to keep waking the reclaimer when memory usage
caused by OSv is high, but for the balloon policy itself we will need our own
calls.

stats::jvm_heap() will return 0 when we have no balloon, therefore that case
will keep seeing no differences.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
bsd_shrinkers: create specialized ARC shrinker

It will give us more freedom to apply ARC shrinking strategies. It is
particularly important given the fact that ARC has two modes of shrinking
(aggressive and conservative) and we may not always want to trigger a policy
change.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
shrinkers: allow soft and hard mode

There are some shrinkers, like the ARC, that may become very agressive when
called more than once in a row. When we have no waiters, we can be more relaxed
and just go for a best effort shrinking.

This patch introduce a soft mode shrinking that will do just that. In the
future, we may introduce another threshold smaller than the minimum in which we
can go in hard mode even before we have waiters. For now, this will do.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
mmu: remove lock guard in balloon_fault

The fault method itself is always called with the lock held. Locking
it again will only slow us down.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
bsd: also use jvm figures in kmem_used calculations

We are defined kmem_used() as total - free. However, as the name implies, this
field is supposed to report on the amount of kernel memory used, not total phys
memory.

Granted, this situation is trickier for us to determine, because we're running
everything together. But in some situations, this can lead to an interesting
performance deterioration: if the JVM touches all the memory for the heap, this
will lead to a very large memory usage, even though the ARC itself is using very
little of it. The ARC will then shrink, and will never have the chance of grow
to the sizes it should. Including the jvm_heap figures will allow the ARC to
have a better idea about what happens.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
OSvMonitor: change criteria for removing balloons

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>