zfsonlinux port of work by Christopher Siden to address illumos issue 4631

The original commit comments:

4631 zvol_get_stats triggering too many reads
Reviewed by: Adam Leventhal ahl@delphix.com
Reviewed by: Matthew Ahrens mahrens@delphix.com
Reviewed by: Sebastien Roy sebastien.roy@delphix.com

Original author: George Wilson

From the comments on issue 4631:

"The problem is that when a dataset is last closed (e.g. a filesystem is
unmounted), we need to get rid of all its state in the DMU (dbufs, dnodes,
etc).  This needs to be coordinated with the ARC (because the arc bufs have
pointers to these data structures too).  We end up evicting all the data out of
the ARC.

This results in very bad performance for things like "zfs get all" if some
filesystems/zvols/snapshots are not mounted (or for zvols, are not open /
exported by comstar), because every snapshot's objset will be opened (to get
the zpl/zvol props), read, and then closed (which will be the last close,
evicting all the data from the ARC).  When the next snapshot is opened, it
probably shares all its relevant blocks with the previous snapshot, but we just
explicitly evicted all of those blocks from the ARC.  So we end up repeatedly
reading in the same blocks from disk.

The fix is to change the interaction between the DMU and ARC so that when the
DMU is shutting down an objset, we do not evict the data from the ARC.  Instead
we simply coordinate the destruction of the DMU's data with the ARC.  The only
case where we actually need to explicitly evict from the ARC is when
dbuf_rele_and_unlock() determines that the administrator has requested that it
not be kept in memory, via the primarycache/secondarycache properties.  In this
case, we evict the data from the ARC by its blkptr_t, the same way as when a
block is freed we explicitly evict it from the ARC."

The resulting (significant positive) change in performance of zfs list -t all operations
with large number of zvols and snapshots is easily observed.