Revert "When at the limit, evict "data" before "metadata""

This reverts commit 8d6fe7406f6adf6ef188eed937ac0c4bde3e6eec.
Revert "Redefine the arc_meta_used and arc_size values"

This reverts commit 993f0aab7116c8689b2744642646d542a86af75e.
Prioritize "metadata" in arc_get_data_buf

When the arc is at it's size limit and a new buffer is added, data will
be evicted (or recycled) from the arc to make room for this new buffer.
As far as I can tell, this is to try and keep the arc from over stepping
it's bounds (i.e. keep it below the size limitation placed on it).

This makes sense conceptually, but there appears to be a subtle flaw in
its current implementation, resulting in metadata buffers being
throttled. When it evicts from the arc's lists, it also passes in a
"type" so as to remove a buffer of the same type that it is adding. The
problem with this is that once the size limit is hit, the ratio of
"metadata" to "data" contained in the arc essentially becomes fixed.

For example, consider the following scenario:

    * the size of the arc is capped at 10G
    * the meta_limit is capped at 4G
    * 9G of the arc contains "data"
    * 1G of the arc contains "metadata"

Now, every time a new "metadata" buffer is created and added to the arc,
an older "metadata" buffer(s) will be removed from the arc; preserving
the 9G "data" to 1G "metadata" ratio that was in-place when the size
limit was reached. This occurs even though the amount of "metadata" is
far below the "metadata" limit. This can result in the arc behaving
pathologically for certain workloads.

To fix this, the arc_get_data_buf function was modified to evict "data"
from the arc even when adding a "metadata" buffer (unless it's at the
"metadata" limit). In addition, arc_evict now more closely resembles
arc_evict_ghost; such that when evicting "data" from the arc, it may
make a second pass over the arc lists and evict "metadata" if it cannot
meet the eviction size the first time around.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Ignore "data" headers in "arc_meta_used" value

Currently the "arc_meta_used" value contains all of the arc_buf_t's,
arc_buf_hdr_t's, dnode_t's, dmu_buf_impl_t's, arc "metadata" buffers,
etc. allocated on the system. The value is then used in conjunction with
"arc_meta_limit" to pro-actively throttle arc "metadata" buffers
contained in the arc's regular lists (i.e. non-ghost lists). There's a
couple problems that can arise as a result:

    1. As the "arc_meta_limit" is reached, new arc "metadata" buffers
       that are added will force an eviction from either the arc's mru
       list or the mfu list. Since *all* arc_buf_hdr_t's are contained
       in "arc_meta_used", even the ghost list entries will take up
       space in "arc_meta_used" and _may_ never be pruned. Given a
       "metadata" intensive workload (e.g. rsync, mkdir, etc.), it is
       possible for the ghost list headers to completely consume
       "arc_meta_used" and leave no room for "useful" metadata;
       causing atrocious arc hit rates (i.e. terrible performance).

    2. Since *all* arc_buf_hdr_t's are accounted for in "arc_meta_used",
       this means that even "data" buffers will take up a small amount
       of space in "arc_meta_used". Given a significant amount of "data"
       buffers (or a small enough "arc_meta_limit"), this can have a
       noticeable impact on the amount of arc "metadata" that can be
       cached. What makes it worse, even if we evict "metadata" from the
       arc (as we do now when over "arc_meta_limit"), these "data" buffer
       headers will never be touched.

This patch is intended to address problem (2) described above. Instead
of using "arc_meta_used" to contain *all* arc_buf_hdr_t's, arc_buf_t's,
etc., now it only contains those headers that accompany arc "metadata".
This way, if *all* of the arc's "metadata" buffers were evicted,
"arc_meta_used" will shrink to zero (even if "data" buffers are still
contained in the arc).

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Remove "arc_meta_used" from arc_adjust calculation

Using "arc_meta_used" to determine if the arc's mru list is over it's
target value of "arc_p" doesn't seem correct. The size of the mru list
and the value of "arc_meta_used", although related, are completely
independent. Buffers contained in "arc_meta_used" may not even be
contained in the arc's mru list. As such, this patch removes
"arc_meta_used" from the calculation in arc_adjust.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Allow "arc_p" to drop to zero or grow to "arc_c"

Setting a limit on the minimum value of "arc_p" has been shown to have
detrimental effects on the arc hit rate for certain "metadata" intensive
workloads. Specifically, this has been exhibited with a workload that
constantly dirties new "meatadata" but also frequently touches a "small"
amount of mfu data (e.g. mkdir's).

What is seen is that the new anon data throttles the mfu list to a
negligible size (because arc_p > anon + mru in arc_get_data_buf), even
though the mfu ghost list receives a constant stream of hits. To remedy
this, arc_p is now allowed to drop to zero if the algorithm deems it
necessary.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Remove arc_p adjustment dampener in arc_adapt

It's unclear why adjustments to arc_p need to be dampened as they are in
arc_adjust. With that said, it's removal significantly improves the arc's
ability to "warm up" to a given workload. Thus, I'm removing it until
its usefulness is better understood.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Prune from "metadata" from ghost lists

Work in progress, needs better commit message.

To try and prevent arc headers from completely consuming
"arc_meta_used", this commit changes arc_adjust_meta to more closely
align with arc_adjust; "arc_meta_limit" is used similarly to "arc_c".