Ignore "data" headers in "arc_meta_used" value

Currently the "arc_meta_used" value contains all of the arc_buf_t's,
arc_buf_hdr_t's, dnode_t's, dmu_buf_impl_t's, arc "metadata" buffers,
etc. allocated on the system. The value is then used in conjunction with
"arc_meta_limit" to pro-actively throttle arc "metadata" buffers
contained in the arc's regular lists (i.e. non-ghost lists). There's a
couple problems that can arise as a result:

    1. As the "arc_meta_limit" is reached, new arc "metadata" buffers
       that are added will force an eviction from either the arc's mru
       list or the mfu list. Since *all* arc_buf_hdr_t's are contained
       in "arc_meta_used", even the ghost list entries will take up
       space in "arc_meta_used" and _may_ never be pruned. Given a
       "metadata" intensive workload (e.g. rsync, mkdir, etc.), it is
       possible for the ghost list headers to completely consume
       "arc_meta_used" and leave no room for "useful" metadata;
       causing atrocious arc hit rates (i.e. terrible performance).

    2. Since *all* arc_buf_hdr_t's are accounted for in "arc_meta_used",
       this means that even "data" buffers will take up a small amount
       of space in "arc_meta_used". Given a significant amount of "data"
       buffers (or a small enough "arc_meta_limit"), this can have a
       noticeable impact on the amount of arc "metadata" that can be
       cached. What makes it worse, even if we evict "metadata" from the
       arc (as we do now when over "arc_meta_limit"), these "data" buffer
       headers will never be touched.

This patch is intended to address problem (2) described above. Instead
of using "arc_meta_used" to contain *all* arc_buf_hdr_t's, arc_buf_t's,
etc., now it only contains those headers that accompany arc "metadata".
This way, if *all* of the arc's "metadata" buffers were evicted,
"arc_meta_used" will shrink to zero (even if "data" buffers are still
contained in the arc).

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Remove "arc_meta_used" from arc_adjust calculation

Using "arc_meta_used" to determine if the arc's mru list is over it's
target value of "arc_p" doesn't seem correct. The size of the mru list
and the value of "arc_meta_used", although related, are completely
independent. Buffers contained in "arc_meta_used" may not even be
contained in the arc's mru list. As such, this patch removes
"arc_meta_used" from the calculation in arc_adjust.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Allow "arc_p" to drop to zero or grow to "arc_c"

Setting a limit on the minimum value of "arc_p" has been shown to have
detrimental effects on the arc hit rate for certain "metadata" intensive
workloads. Specifically, this has been exhibited with a workload that
constantly dirties new "meatadata" but also frequently touches a "small"
amount of mfu data (e.g. mkdir's).

What is seen is that the new anon data throttles the mfu list to a
negligible size (because arc_p > anon + mru in arc_get_data_buf), even
though the mfu ghost list receives a constant stream of hits. To remedy
this, arc_p is now allowed to drop to zero if the algorithm deems it
necessary.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Remove arc_p adjustment dampener in arc_adapt

It's unclear why adjustments to arc_p need to be dampened as they are in
arc_adjust. With that said, it's removal significantly improves the arc's
ability to "warm up" to a given workload. Thus, I'm removing it until
its usefulness is better understood.

Signed-off-by: Prakash Surya <surya1@llnl.gov>