Fix l2arc_write_done/l2arc_evict race

A GPF may occur if an l2arc buffer is evicted before the write
completes for the l2arc buffer.  The l2arc_write_done() function
unconditionally starts walking the list at the write head so we
must verify that l2arc_evict() has not removed it from the list.

This long standing issue was exposed by allowing arc_adjust() to
evict entries from the l2arc due to memory pressue.  Prior to
this change buffers would only be evicted immediately in front
of the write hand or during l2arc cleanup.  In neither case could
we wrap around remove the write head as part of l2arc_evict().

PID: 3182   TASK: ffff88020df28080  CPU: 0   COMMAND: "z_null_int/0"
  [exception RIP: l2arc_write_done+128]
 6 [ffff880210e97c58] zio_done at ffffffffa0597e0c [zfs]
 7 [ffff880210e97cd8] zio_done at ffffffffa05982a7 [zfs]
 8 [ffff880210e97d58] zio_done at ffffffffa05982a7 [zfs]
 9 [ffff880210e97dd8] zio_execute at ffffffffa05948a3 [zfs]
10 [ffff880210e97e18] taskq_thread at ffffffffa041d8d8 [spl]
11 [ffff880210e97ee8] kthread at ffffffff81090d26
12 [ffff880210e97f48] kernel_thread at ffffffff8100c14a

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Set zfs_arc_min to 4MB

Decrease the mimimum ARC size from 1/32 of total system memory
(or 64MB) to a much smaller 4MB.

1) Large systems with over a 1TB of memory are being deployed
   and reserving 1/32 of this memory (32GB) as the mimimum
   requirement is overkill.

2) Tiny systems like the raspberry pi may only have 256MB of
   memory in which case 64MB is far too large.

The ARC should be reclaimable if the VFS determines it needs
the memory for some other purpose.  If you want to ensure the
ARC is never completely reclaimed due to memory pressure you
may still set a larger value with zfs_arc_min.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Limit L2ARC header footprint

Currently there is not mechanism in place to reclaim or limit the number
of L2ARC headers contained in RAM. This means that given a large enough
L2ARC device, the headers can accumulate until they push out all other
data, and can potentially cause an out-of-memory event on the system.

While we'd rather not discard these headers, because it invalidates any
blocks written to the L2ARC device, we also don't want these headers to
push out all other (potentially more useful) data on the system and/or
cause the system to hit a out-of-memory event.

As a fist attempt to fix this issue, a new "l2arc_hdr_limit" module
parameter has been introduced. This option allows an administrator to
set a limit on the cumulative size that L2ARC headers can grow to on the
system. By default this limit is 1/8th of the total ARC size.

To do this a l2arc_evict_headers() function has been added. It will
attempt to reclaim N bytes of l2arc headers by repeatedly advancing the
write hand and evicting blocks from the l2arc.

In addition, since we're not pruning from the L2ARC devices based on
"arc_meta_limit" or "arc_meta_used", these headers were completely taken
out of the "arc_meta_used" calculation.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Don't prune from ARC when over "arc_meta_limit"

Work in progress. I don't have the time to write a detailed enough
commit message to describe this change, but I'll be sure to do so before
it lands in the upstream code.

Signed-off-by: Prakash Surya <surya1@llnl.gov>