Comment the lack of real_LZ4_uncompress()

Added several comments regarding the removal of real_LZ4_uncompress()
which exists in the reference implementation but has been removed here
since it's not used.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Illumos #4936 fix potential overflow in lz4

4936 lz4 could theoretically overflow a pointer with a certain input
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Reviewed by: Keith Wesolowski <keith.wesolowski@joyent.com>
Approved by: Gordon Ross <gordon.ross@nexenta.com>
Ported by: Tim Chase <tim@chase2k.com>

References:
  https://illumos.org/issues/4936
  https://github.com/illumos/illumos-gate/commit/58d0718

Porting notes:

This fixes the widely-reported "20-year-old vulnerability" in
LZO/LZ4 implementations which inherited said bug from the reference
implementation.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2429
Return default value on numeric properties failing the "head check.

Updates 962d52421236fc9cd61d59b4f18cff3276077da9.

The referenced fix to get_numeric_property() caused numeric property
lookups to consider the type of the parent (head) dataset when checking
validity but there are some cases in the caller expects to see the
property's default value even when the lookup is invalid.

One case in which this is true is change_one() which is part of the
renaming infrastructure.  It may look up "zoned" on a snapshot of a volume
which is not valid but it expects to see the default value of false.

There may be other, yet unidentified cases in which zfs_prop_get_int()
is used on technically invalid properties but which expect the property's
default value to be returned.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Turbo Fredriksson <turbo@bayour.com>
Closes #2320
Document the optional "device" argument for "zpool split"

Most ZFS implementations seemed to have missed this bit of documentation.
The additional text is based on FreeBSD's man page.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2416
Fix comment spelling errors.

Signed-off-by: Garrison Jensen <garrison.jensen@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2402
Preserve asize when last mirror child promoted to top-level vdev

If the smaller of 2 different sized child vdev's of a mirrored vdev is
detached, and the pool has the autoexpand property set to off, as the
remaining larger vdev is promoted to a top level vdev it fails to retain
the asize of the original top level mirror vdev and therefore partially
autoexpands.

This partially autoexpanded state leaves the new vdev too large to
re-mirror by adding the smaller vdev back in, and the pool fails to
utilize the space until next imported.

If the autoexpand property is set to on, the child vdev grows
in size after it has been promoted to a top level vdev as expected.

This commit causes the remaining child mirror to retain the asize of its
old parent mirror vdev if the autoexpand property is set to off,
this allows the smaller vdev to be re-added if required the vdev
can then be told to expand if required by the usual using zpool online -e.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Andrew Barnes <barnes333@gmail.com>
Signed-off-by: George Wilson <george.wilson@delphix.com>
Closes #1208
Include sys/taskq.h in linux/vfs_compat.h

We should have included sys/taskq.h directly because we use the taskq
code here, but we instead had files that included sys/taskq.h also
include sys/kmem.h, which happened to include sys/taskq.h. sys/kmem.h no
longer does this, so we must define the include as we should
have done in the first place.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Use Linux kzalloc()/kfree() for superblocks

We store the bdi in the superblock. This is accessed using virt_to_page
in bit_waitqueue(), which does not work on kernel virtual memory. We
switch to kzalloc()/kfree() to fix this.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Mark DMU transaction processing with PF_FSTRANS

It was thought that we could ensure forward progress by avoiding memory
allocations during transaction processing and avoiding KM_SLEEP in the
few instances in which we could not.

Recent tests involving L2ARC on systems without swap have shown that
this is not enough. VMWare's bencharmk on a system with 8GB of RAM and
200GB of L2ARC will reliably deadlock when atime is enabled, but will
run fine when atime is not enabled. This was initially thought to be a
mix of an extreme case of ARC contention and atime updates somehow
blocking on transaction group commit, but further examination showed
that only 137MB of RAM were in use by L2ARC's headers and additional
testing showed no deadlocks occurred with 24GB of RAM when atime was
enabled.

Subsequent review of the code revealed that the only change was that
direct reclaim no longer started transactions to perform atime updates.
The logical conclusion is that direct reclaim can occur in locations
that hold locks that block an open transaction through some unforeseen
dependency chain. This patch attempts to address that by modifying the
KM_SLEEP avoidance mechanism to be used in all transaction processing,
rather than just operations on zvols. We also take the opportunity to do
some cleanup by replacing PF_NOIO with PF_FSTRANS, which is the proper
Linux flag for this.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Add add_prop_list_default helper

Adding to a property list only if there is no existing value is used
twice. Once by zpool create -R and again by zpool import -R. Now that
zpool create -t and zpool import -t also need it, lets refactor it into
a helper function to make the code more readable.

Signed-off-by: Richard Yao <ryao@gentoo.org>
zpool import -t should not update cachefile

zpool import's -t parameter is intended for use with -R when operating
on pools that belong to other systems. Like -R, pools imported in this
way should not update the cachefile unless explicitly requested. The
initial implementation allowed the cachefile to be updated when -R was
not used. This went uncaught during testing because -R had implicitly
disabled use of the cachefile.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Implement -t option to zpool create for temporary pool names

Creating virtual machines that have their rootfs on ZFS on hosts that
have their rootfs on ZFS causes SPA namespace collisions when the
standard name rpool is used. The solution is either to give each guest
pool a name unique to the host, which is not always desireable, or boot
a VM environment containing an ISO image to install it, which is
cumbersome.

26b42f3f9d03f85cc7966dc2fe4dfe9216601b0e introduced `zpool import -t
...` to simplify situations where a host must access a guest's pool when
there is a SPA namespace conflict. We build upon that to introduce
`zpool import -t tname ...`. That allows us to create a pool whose
in-core name is tname, but whose on-disk name is the normal name
specified.

This simplifies the creation of machine images that use a rootfs on ZFS.
That benefits not only real world deployments, but also ZFSOnLinux
development by decreasing the time needed to perform rootfs on ZFS
experiments.

Signed-off-by: Richard Yao <ryao@gentoo.org>
ZFS_IOC_RECV should only call zvol_create_minors() on zvols

Signed-off-by: Richard Yao <ryao@gentoo.org>
zfs send -p send properties only for snapshots that are actually sent

... as opposed to sending properties of all snapshots of the relevant
filesystem.  The previous behavior results in properties being set on
all snapshots on the receiving side, which is quite slow.
More details are here:
  http://thread.gmane.org/gmane.comp.file-systems.openzfs.devel/346

Behavior of zfs send -R is not changed.
send_iterate_fs: orderly iterate snapshots

The previous commit depends on ordered iteration of the snapshots,
otherwise the code won't be able to correctly detect snapshots
that fall into the requested snapshot range.

This commit can be considered a partial cherry-pick of commit
freebsd/freebsd@4995789cde5 where ordered iteration was introduced
to fix a different problem.
We might want to pick up the complete fix eventually.
Mark SoftNAS SP2 Release

Signed-off-by: Richard Yao <ryao@gentoo.org>
zp->z_sync_cnt should not use atomic operations on Linux

On most UNIX systems, the VFS will notify the filesystem about all
open() and close() operations on files and ZFS will take advantage of
this to track the number of synchronous opens in zp->z_sync_cnt. This
affects the zil_commit behavior. However, Linux only notifies us about
the last close on a file, which means that concurrent synchronous opens
will leave zp->z_sync_cnt set to a non-zero value on last close. This
means that all subsequent opens receive synchronous behavior even if
they had not asked for it.

We address this by modifying zfs_open()/zfs_close() to use
zp->z_sync_cnt on Linux as a flag variable that is set on synchronous
open and cleared on synchronous close.

Signed-off-by: Richard Yao <ryao@gentoo.org>