Allow arc_evict_ghost() to only evict meta data

The default behavior of arc_evict_ghost() is to start by evicting
data buffers.  Then only if the requested number of bytes to evict
cannot be satisfied by data buffers move on to meta data buffers.

This is ideal for honoring arc_c since it's preferable to keep the
meta data cached.  However, if we're trying to free memory from the
arc to honor the meta limit it's a problem because we will need to
discard all the data to get to the meta data.

To avoid this issue the arc_evict_ghost() is now passed a fourth
argumented describing which buffer type to start with.  The
arc_evict() function already behaves exactly like this for a
same reason so this is consistent with the existing code.

All existing callers have been updated to pass ARC_BUFC_DATA so
this patch introduces no functional change.  New callers may
pass ARC_BUFC_METADATA to skip immediately to evicting meta
data leaving the normal data untouched.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Evict meta data from ghost lists + l2arc headers

When the meta limit is exceeded the ARC evicts some meta data
buffers from the mfu+mru lists.  Unfortunately, for meta data
heavy workloads it's possible for these buffers to accumulate
on the ghost lists if arc_c doesn't exceed arc_size.

To handle this case arc_adjust_meta() has been entended to
explicitly evict meta data buffers from the ghost lists in
proportion to what was evicted from the mfu+mru lists.

If this is insufficient we request that the VFS release
some inodes and dentries.  This will result in the release
of some dnodes which are counted as 'other' metadata.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Set zfs_arc_min to 4MB

Decrease the mimimum ARC size from 1/32 of total system memory
(or 64MB) to a much smaller 4MB.

1) Large systems with over a 1TB of memory are being deployed
   and reserving 1/32 of this memory (32GB) as the mimimum
   requirement is overkill.

2) Tiny systems like the raspberry pi may only have 256MB of
   memory in which case 64MB is far too large.

The ARC should be reclaimable if the VFS determines it needs
the memory for some other purpose.  If you want to ensure the
ARC is never completely reclaimed due to memory pressure you
may still set a larger value with zfs_arc_min.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Enable l2arc reclaim

Currently all l2arc headers count against the meta limit and
the only way to release them is to cleanup an l2arc device.
This means that given a large enough l2arc device the headers
can accumulate until they push out all other meta data.

While we absolutely don't want to commonly discard these headers,
because it invalidates any blocks written to the l2arc device,
it should be possible to reclaim them.  If memory is desperately
needed by an application it's preferable to reclaim this memory
and pay the performance penalty.

To do this a l2arc_evict_headers() function has been added.  It
will attempt to reclaim N bytes of l2arc headers by repeatedly
advancing the write hand and evicting blocks from the l2arc.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Fix l2arc_write_done/l2arc_evict race

A GPF may occur if an l2arc buffer is evicted before the write
completes for the l2arc buffer.  The l2arc_write_done() function
unconditionally starts walking the list at the write head so we
must verify that l2arc_evict() has not removed it from the list.

This long standing issue was exposed by allowing arc_adjust() to
evict entries from the l2arc due to memory pressue.  Prior to
this change buffers would only be evicted immediately in front
of the write hand or during l2arc cleanup.  In neither case could
we wrap around remove the write head as part of l2arc_evict().

PID: 3182   TASK: ffff88020df28080  CPU: 0   COMMAND: "z_null_int/0"
  [exception RIP: l2arc_write_done+128]
 6 [ffff880210e97c58] zio_done at ffffffffa0597e0c [zfs]
 7 [ffff880210e97cd8] zio_done at ffffffffa05982a7 [zfs]
 8 [ffff880210e97d58] zio_done at ffffffffa05982a7 [zfs]
 9 [ffff880210e97dd8] zio_execute at ffffffffa05948a3 [zfs]
10 [ffff880210e97e18] taskq_thread at ffffffffa041d8d8 [spl]
11 [ffff880210e97ee8] kthread at ffffffff81090d26
12 [ffff880210e97f48] kernel_thread at ffffffff8100c14a

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>