3744 zfs shouldn't ignore errors unmounting snapshots

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Porting notes:

zfs_unmount_snap() significantly differed from its Illumos counterpart,
so the changes to this function had to be done by hand. In addition,
zfs_unmount_snap() contained a possible memory leak because strfree()
relies on strlen() to determine which SLAB cache to use to free memory,
so the function was reworked to fix it.
3740 Poor ZFS send / receive performance due to snapshot hold / release processing

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Porting notes:

1. 13fe019870c8779bf2f5b3ff731b512cf89133ef introduced a merge conflict
in dsl_dataset_user_release_tmp where some variables were moved outside of
the preprocessor directive.

2. dea9dfefdd747534b3846845629d2200f0616dad made the previous merge
conflict worse by switching KM_SLEEP to KM_PUSHPAGE. This is notable
because this commit refactors the code, adding a new KM_SLEEP
allocation. It is not clear to me whether this should be converted tO
KM_PUSHPAGE.

3. We had a merge conflict in libzfs_sendrecv.c because of copyright
notices.
3836 zio_free() can be processed immediately in the common case

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>

Porting notes:

1. This fixed a typo in a comment in zio_impl.h that was added in illumos 3236
zio nop-write. That commit will not be merged until a ztest failure that it is
known to make more frequent is identified and fixed, so the hunk that modified
that comment was dropped.
3894 zfs should not allow snapshot of inconsistent dataset

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Gordon Ross <gwr@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>
3888 zfs recv -F should destroy any snapshots created since the incremental source

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Peng Dai <peng.dai@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Ported-by: Richard Yao <ryao@gentoo.org>

Porting notes:

1. Commit 1fde1e37208c2f56c72c70a06676676f04b65998 wrapped a declaration in
dsl_dataset_modified_since_lastsnap in ASSERTV(). The ASSERTV() has been
removed for consistency with Illumos.

2. The exported symbol dsl_dataset_modified_since_lastsnap() was renamed
to dsl_dataset_modified_since_snap().
3875 panic in zfs_root() after failed rollback

Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Gordon Ross <gwr@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>
3834 incremental replication of 'holey' file systems is slow

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Ported-by: Richard Yao <ryao@gentoo.org>
3973 zfs_ioc_rename alters passed in zc->zc_name

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>
Ported-by: Richard Yao <ryao@gentoo.org>
3955 ztest failure: assertion refcount_count(&tx->tx_space_written) + delta <= tx->tx_space_towrite

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Ported-by: Richard Yao <ryao@gentoo.org>
3949 ztest fault injection should avoid resilvering devices
3950 ztest: deadman fires when we're doing a scan
3951 ztest hang when running dedup test
3952 ztest: ztest_reguid test and ztest_fault_inject don't place nice together

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Ported-by: Richard Yao <ryao@gentoo.org>

Porting notes:

The deadman thread was removed from zdb in ZFSOnLinux by Brian Behlendorf, so
the deadman thread changes were dropped. We probably should look into
reintroducing that feature.
4046 dsl_dataset_t ds_dir->dd_lock is highly contended

Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Ported-by: Richard Yao <ryao@gentoo.org>

Porting notes:

This commit removed dsl_dataset_namelen in Illumos, but that appears to have
been removed from ZFSOnLinux in an earlier commit.
Illumos #4045 write throttle & i/o scheduler performance work

4045 zfs write throttle & i/o scheduler performance work

1. The ZFS i/o scheduler (vdev_queue.c) now divides i/os into 5 classes: sync
read, sync write, async read, async write, and scrub/resilver.  The scheduler
issues a number of concurrent i/os from each class to the device.  Once a class
has been selected, an i/o is selected from this class using either an elevator
algorithem (async, scrub classes) or FIFO (sync classes).  The number of
concurrent async write i/os is tuned dynamically based on i/o load, to achieve
good sync i/o latency when there is not a high load of writes, and good write
throughput when there is.  See the block comment in vdev_queue.c (reproduced
below) for more details.

2. The write throttle (dsl_pool_tempreserve_space() and
txg_constrain_throughput()) is rewritten to produce much more consistent delays
when under constant load.  The new write throttle is based on the amount of
dirty data, rather than guesses about future performance of the system.  When
there is a lot of dirty data, each transaction (e.g. write() syscall) will be
delayed by the same small amount.  This eliminates the "brick wall of wait"
that the old write throttle could hit, causing all transactions to wait several
seconds until the next txg opens.  One of the keys to the new write throttle is
decrementing the amount of dirty data as i/o completes, rather than at the end
of spa_sync().  Note that the write throttle is only applied once the i/o
scheduler is issuing the maximum number of outstanding async writes.  See the
block comments in dsl_pool.c and above dmu_tx_delay() (reproduced below) for
more details.

This diff has several other effects, including:

 * the commonly-tuned global variable zfs_vdev_max_pending has been removed;
use per-class zfs_vdev_*_max_active values or zfs_vdev_max_active instead.

 * the size of each txg (meaning the amount of dirty data written, and thus the
time it takes to write out) is now controlled differently.  There is no longer
an explicit time goal; the primary determinant is amount of dirty data.
Systems that are under light or medium load will now often see that a txg is
always syncing, but the impact to performance (e.g. read latency) is minimal.
Tune zfs_dirty_data_max and zfs_dirty_data_sync to control this.

 * zio_taskq_batch_pct = 75 -- Only use 75% of all CPUs for compression,
checksum, etc.  This improves latency by not allowing these CPU-intensive tasks
to consume all CPU (on machines with at least 4 CPU's; the percentage is
rounded up).

--matt

APPENDIX: problems with the current i/o scheduler

The current ZFS i/o scheduler (vdev_queue.c) is deadline based.  The problem
with this is that if there are always i/os pending, then certain classes of i/
os can see very long delays.

For example, if there are always synchronous reads outstanding, then no async
writes will be serviced until they become "past due".  One symptom of this
situation is that each pass of the txg sync takes at least several seconds
(typically 3 seconds).

If many i/os become "past due" (their deadline is in the past), then we must
service all of these overdue i/os before any new i/os.  This happens when we
enqueue a batch of async writes for the txg sync, with deadlines 2.5 seconds in
the future.  If we can't complete all the i/os in 2.5 seconds (e.g. because
there were always reads pending), then these i/os will become past due.  Now we
must service all the "async" writes (which could be hundreds of megabytes)
before we service any reads, introducing considerable latency to synchronous i/
os (reads or ZIL writes).

Notes on porting to ZFS on Linux:

- zio_t gained new members io_physdone and io_phys_children.  Because
  object caches in the Linux port call the constructor only once at
  allocation time, objects may contain residual data when retrieved
  from the cache. Therefore zio_create() was updated to zero out the two
  new fields.

- vdev_mirror_pending() relied on the depth of the per-vdev pending queue
  (vq->vq_pending_tree) to select the least-busy leaf vdev to read from.
  This tree has been replaced by vq->vq_active_tree which is now used
  for the same purpose.

- vdev_queue_init() used the value of zfs_vdev_max_pending to determine
  the number of vdev I/O buffers to pre-allocate.  That global no longer
  exists, so we instead use the sum of the *_max_active values for each of
  the five I/O classes described above.

- The Illumos implementation of dmu_tx_delay() delays a transaction by
  sleeping in condition variable embedded in the thread
  (curthread->t_delay_cv).  We do not have an equivalent CV to use in
  Linux, and we just need to sleep for a specified time interval, so this
  port just calls msleep().

- These tunables are added as module parameters:

  zfs_dirty_data_max_percent
  zfs_delay_min_dirty_percent
  zfs_dirty_data_max
  zfs_dirty_data_max_max
  zfs_dirty_data_sync
  zfs_delay_scale

  The latter four have type unsigned long, whereas they are uint64_t in
  Illumos.  This accommodates Linux's module_param() supported types, but
  means they may overflow on 32-bit architectures.

References:
        http://www.illumos.org/issues/4045
        illumos/illumos-gate@69962b5647e4a8b9b14998733b765925381b727e

Ported-by: Ned Bass <bass6@llnl.gov>
4168 ztest assertion failure in dbuf_undirty
4169 verbatim import causes zdb to segfault
4170 zhack leaves pool in ACTIVE state

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>
Ported-by: Richard Yao <ryao@gentoo.org>
3829 fix for 3740 changed behavior of zfs destroy/hold/release ioctl

Reviewed by: Matt Amdur <matt.amdur@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Use bitfields for boolean values in znode_t

znode_t uses 1312 bytes in length on Linux 3.9.6 amd64. Switching to
bitfields for boolean values reduces the size to 1296 bytes. Further
reductions should be possible if padding were removed, but saving 16
bytes per struct can translate into megabytes of savings when hundreds
of thousands of znodes are cached. While the savings is relatively
small in comparison to memory used, the small efficiency gain is
practically free.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Remove unused option -r from zpool

Signed-off-by: Richard Yao <ryao@gentoo.org>
Implement -t option to zpool import for temporary pool names

Specifying newname to a pool import currently changes the pool name. We
implement option -t. Specifying option -t in conjuction with newpool to
zpool import will make the name change temporary. In specific, the
original pool name will be used in all label updates and therefore is
retained upon export.

This is useful in any situation where maintenance on a system's root
pool is done from another system. A good example is environments using
virtualization, where it is more desireable to do rollback of the root
dataset of a guest's root pool than it is to rollback of the zvol
hosting the guest's root pool when an issue prevents the guest from
booting properly. Other examples include maintenance on root pools
hosted on iSCSI and operating system installations on USB keys that are
used to rescue non-bootable sytems.

Signed-off-by: Richard Yao <ryao@gentoo.org>
26126 panic system rather than corrupting pool if we hit bug 26100

Ported-by: Richard Yao <ryao@gentoo.org>