Clear PG_writeback for sync I/O error case

Commit 2b2861362f7dd09cc3167df8fddb6e2cb475018a accidentally
introduced this issue by only conditionally registering the
commit callback in the async case.

The error handing code for the dmu_tx_assign() failure case
relied on there always being a registered commit callback to
clear the PG_writeback bit.  Since that is no longer strictly
true for the synchronous case we must explicitly invoke the
callback.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #961
Make sure free ZIO sizes are correct for gang blocks.

Currently, the PSIZE of the block pointer is used as the I/O size for
all free ZIOs. This is incorrect for gang header blocks, for which
PSIZE is the logical size of the gang block data.

This patch fixes the issue by using SPA_GANGBLOCKSIZE as the I/O size
for "free gang header block" ZIOs.
Export avl_is_empty from zavl.

Currently, zavl users cannot call avl_is_empty since it is not
exported. This patch fixes that.
Add TRIM support.

The code builds a map of regions that were freed. On every write the
code consults the map and eventually removes ranges that were freed
before, but are now overwritten.

Freed blocks are not TRIMed immediately. There is a tunable that defines
how many txg we should wait with TRIMming freed blocks (64 by default).

There is a low priority thread that TRIMs ranges when the time comes.
During TRIM we keep in-flight ranges on a list to detect colliding
writes - we have to delay writes that collide with in-flight TRIMs in
case something will be reordered and write will reached the disk before
the TRIM. We don't have to do the same for in-flight writes, as
colliding writes just remove ranges to TRIM.

NOTE: this patch only adds the core infrastucture for TRIM. It doesn't
implement actual discard requests on vdevs. In other words, the patch
in itself doesn't actually do anything (yet).

NOTE: you need DKIOCTRIM support in SPL for this patch to build.

Signed-off-by: Etienne Dechamps <etienne.dechamps@ovh.net>
Fix frame size for trim_map_free_locked().

Currently, compilation fails because trim_map_free_locked() exceeds the
frame size. This is because we're allocating a large structure (zio_t)
on the stack.

This patch fixes the issue by only allocating the fields we need from
the zio structure.
Don't test for gang blocks when populating the trim map.

Since 2fd4ce81a6748b46728127212a1ba384c008696f, gang block free zios
have correct size, so there's no need to skip them. Besides, the test
fails with a NULL dereference bug on RAID-Z because io_bp is NULL in
this case.
Don't TRIM after the SPA has been frozen.

Currently, the TRIM module doesn't check for spa_freeze_txg when
issuing TRIM requests. In a normal run this isn't an issue since
vdev_config_sync() doesn't wake the TRIM thread if the SPA is frozen.

However, when closing a frozen pool, trim_thread_destroy() gets called,
which wakes the TRIM thread, which results in a TRIM commit. As a
result, all TXGs until spa_syncing_txg - txg_limit get TRIMmed, which
includes spa_freeze_txg.

As a result, when reopening, ZFS tries to read data from the frozen
TXG, and fails.

This patch fixes the issue by taking spa_freeze_txg into account when
commiting TRIMs.
Add TRIM support for disk vdevs using Linux DISCARD interface.

We introduce a new vdev_disk_io_trim() function inspired from the Linux
blkdev_issue_discard() function and the pre-existing vdev_disk_physio()
function. The new function takes care of splitting discard requests
into smaller ones if necessary according to maximum discard limits.
Add TRIM support for file vdevs.

This adds TRIM support for file vdevs, using the newly-added VOP_SPACE
compatibility interface which translates to Linux
fallocate(FALLOC_FL_PUNCH_HOLE). When a TRIM request is issued to a
file vdev, ZFS will "punch a hole" (deallocate space) inside the
underlying file. Note that few Linux filesystems support hole punching
(currently OCFS2, XFS, and ext4).

Note that for this to work, you need to build against a kernel that
supports file hole punching (i.e. Linux >= 2.6.38). In addition, for
this to work in userspace (e.g. ztest), the installed userspace
kernel headers (i.e. /usr/include/linux) must be up-to-date.

NOTE: you need VOP_SPACE() support in SPL for this patch to build.
Fix assertion failure due to stale inflight writes.

ztest runs with TRIM enabled on a backing filesystem that doesn't
support hole punching revealed the following race condition:

 1. A write is issued to a vdev and added to the TRIM map inflight
    writes tree.
 2. The very first TRIM request for this vdev is issued.
 3. The TRIM request fails with EOPNOTSUPP, as expected. vdev_notrim
    is set.
 4. The write completes. trim_map_write_done() is called.
 5. trim_map_write_done() sees that vdev_notrim is set, and returns
    immediately.
 6. As a result, the write stays in the TRIM map inflight writes tree
    forever.
 7. When the vdev is closed, trim_map_destroy() tries to destroy the
    inflight writes tree, which triggers an assertion failure because
    it isn't empty.

This patch fixes the issue by removing the check on vdev_notrim in
trim_map_write_done().
Fix assertion failure due to inflight frees.

ztest runs with TRIM enabled revealed a race condition which occurs when
a vdev close is pending while there are TRIM requests in flight. When
the TRIM ZIOs complete, the SPA lock is released. The SPA lock is then
immediately taken by the vdev close path, which calls trim_map_destroy()
before trim_map_vdev_commit_done() has a chance to run.
trim_map_destroy() then tries to destroy the inflight free tree, which
fails with an assertion failure because it is non-empty.

This patch fixes the issue by making trim_map_destroy() call
trim_map_vdev_commit_done() before destroying the TRIM map.
Add ZIO TRIM statistics.

This patch adds a new kstat page: zio_trim, which indicates the total
number of bytes TRIMmed, and the number of successful, unsupported and
failed TRIM requests. This makes it easy to verify if TRIM is actually
working or not.