Switch KM_SLEEP to KM_PUSHPAGE

This warning indicates the incorrect use of KM_SLEEP in a call
path which must use KM_PUSHPAGE to avoid deadlocking in direct
reclaim.  See commit b8d06fca089fae4680c3a552fc55c512bfb02202
for additional details.

  SPL: Fixing allocation for task txg_sync (6093) which
  used GFP flags 0x297bda7c with PF_NOFS set

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #973
Illumos #1948: zpool list should show more detailed pool info

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Albert Lee <trisk@nexenta.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Eric Schrock <eric.schrock@delphix.com>

References:
  https://www.illumos.org/issues/1948

Ported by:	Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #685
Illumos #2703: add mechanism to report ZFS send progress

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

References:
  https://www.illumos.org/issues/2703

Ported by: Martin Matuska <martin@matuska.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
zvol_probe should return NULL when the device isn't found.

Previously we returned ERR_PTR(-ENOENT) which the rest of the kernel
doesn't expect and as such we can oops.

Signed-off-by: Chris Wedgwood <cw@f00f.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #949
Closes #931
Closes #789
Closes #743
Closes #730
Implement .commit_metadata hook for NFS export

In order to implement synchronous NFS metadata semantics ZFS
needs to provide the .commit_metadata hook.  All it takes there
is to make sure changes are committed to ZIL.  Fortunately
zfs_fsync() does just that, so simply calling it from
zpl_commit_metadata() does the trick.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #969
Force 4K blocksize when testing ext2 on zvol.

Currently, mkfs.ext2 on zconfig.sh zvols tries to use a 8K blocksize,
probably because by default zvol exposes an optimal I/O size of 8K.

Unfortunately, a ext2 blocksize of 8K is not supported by the kernel,
so the resulting filesystem is unmountable.

This patch fixes the issue by making sure the blocksize is 4K. We have
to use -F to force it else mkfs.ext2 won't allow us to use a blocksize
smaller than the optimal I/O size.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #979
Modify vdev_elevator_switch() to use elevator_change()

As of Linux 2.6.36 an elevator_change() interface was added.
This commit updates vdev_elevator_switch() to use this interface
when available, otherwise it falls back to the usermodehelper
method.

Original-patch-by: foobarz <sysop@xeon.(none)>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #906
Create threads in detached state in userspace.

Currently, thread_create(), when called in userspace, creates a
joinable (i.e. not detached thread). This is the pthread default.

Unfortunately, this does not reproduce kthreads behavior (kthreads
are always detached). In addition, this contradicts the original
Solaris code which creates userspace threads in detached mode.

These joinable threads are never joined, which leads to a leakage of
pthread thread objects ("zombie threads"). This in turn results in
excessive ressource consumption, and possible ressource exhaustion in
extreme cases (e.g. long ztest runs).

This patch fixes the issue by creating userspace threads in detached
mode. The only exception is ztest worker threads which are meant to be
joinable.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #989
Fix VOP_CLOSE() in userspace.

Currently, for unknown reasons, VOP_CLOSE() is a no-op in userspace.
This causes file descriptor leaks. This is especially problematic with
long ztest runs, since zpool.cache is opened repeatedly and never
closed, resulting in resource exhaustion (EMFILE errors).

This patch fixes the issue by making VOP_CLOSE() do what it is supposed
to do.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #989
Fix ztest vdev file paths.

Currently, in several instances (but not all), ztest generates vdev
file paths using a statement similar to this:

    snprintf(path, sizeof (path), ztest_dev_template, ...);

This worked fine until 40b84e7aec6392187722e61e5a4a853b530bf60f, which
changed path to be a pointer to the heap instead of an array allocated
on the stack. Before this change, sizeof(path) would return the size of
the array; now, it returns the size of the pointer instead.

As a result, the aforementioned sprintf statement uses the wrong size
and truncates the vdev file path to the first 4 or 8 bytes (depending
on the architecture). Typically, with default settings, the file path
will become "/tmp/zt" instead of "/test/ztest.XXX".

This issue only exists in ztest_vdev_attach_detach() and
ztest_fault_inject(), which explains why ztest doesn't fail right away.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #989
Temporarily disable the reguid test.

Currently, ztest fails with the following error:

    error: Pool 'ztest' has encountered an uncorrectable I/O failure
    and the failure mode property for this pool is set to panic.

We know how to fix it (see issue #939), but it may take some time
before we get around to merging the fix, which has some heavy
dependencies.

In the mean time, it is not ideal to be unable to use ztest just
because of a small isolated issue, so this patch works around the
problem by disabling the reguid test. This is just a temporary hack to
keep ztest usable.

The reguid test will be enabled again when the proper fix is merged.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #997
Illumos #3129, #3130

illumos/illumos-gate@d6afdce20f8481c95471dd821bc8ec0dbde66213
Illumos changeset: 13794:7c5e0e746b2c

3129 'zpool reopen' restarts resilvers
3130 ztest failure: Assertion failed:
     0 == dmu_objset_destroy(name, B_FALSE) (0x0 == 0x10)

Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Matt Ahrens <matthew.ahrens@delphix.com>
Reviewed by: Christopher Siden <chris.siden@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/3129
  https://www.illumos.org/issues/3130

Ported by: Etienne Dechamps <etienne.dechamps@ovh.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #994
Illumos #2399: zfs manual page does not document use of "zfs diff".

illumos/illumos-gate@3b8be6bf4fd2c744dfb8b5ce2a6c85ad0a2c8f75
Illumos changeset: 13773:00c2a08cf1bb

2399 zfs manual page does not document use of "zfs diff"

Reviewed by: Joshua M. Clulow <josh@sysmgr.org>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Approved by: Robert Mustacchi <rm@joyent.com>

Ported-by: Etienne Dechamps <etienne.dechamps@ovh.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #940
Illumos #3100: zvol rename fails with EBUSY when dirty.

illumos/illumos-gate@2e2c135528b3edfe9aaf67d1f004dc0202fa1a54
Illumos changeset: 13780:6da32a929222

3100 zvol rename fails with EBUSY when dirty

Reviewed by: Christopher Siden <chris.siden@delphix.com>
Reviewed by: Adam H. Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Eric Schrock <eric.schrock@delphix.com>

Ported-by: Etienne Dechamps <etienne.dechamps@ovh.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #995
Switch KM_SLEEP to KM_PUSHPAGE

This warning indicates the incorrect use of KM_SLEEP in a call
path which must use KM_PUSHPAGE to avoid deadlocking in direct
reclaim.  See commit b8d06fc for additional details.

  SPL: Fixing allocation for task txg_sync (6093) which
  used GFP flags 0x297bda7c with PF_NOFS set

Signed-off-by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1002
Illumos #1950: ztest backwards compatibility testing option.

illumos/illumos-gate@420dfc9585ff67e83ee7800a7ad2ebe1a9145983
Illumos changeset: 13571:a5771a96228c

1950 ztest backwards compatibility testing option

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Eric Schrock <eric.schrock@delphix.com>

Ported-by: Etienne Dechamps <etienne.dechamps@ovh.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Fix mmap() usage in ztest.

illumos/illumos-gate@ad135b5d644628e791c3188a6ecbd9c257961ef8
Illumos changeset: 13700:2889e2596bd6

Note that this is only a partial port of the aforementioned Illumos
changeset.

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <gwilson@delphix.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Dan Kruchinin <dan.kruchinin@gmail.com>
Approved by: Eric Schrock <Eric.Schrock@delphix.com>

Ported to zfsonlinux by: Etienne Dechamps <etienne.dechamps@ovh.net>
Use dynamic file descriptor numbers in ztest.

Currently, ztest expects to get 3 and 4 as the file descriptors for
data and random files, respectively. This is quite fragile and breaks
easily if ztest is run with these file descriptors already opened
(e.g. in a complex shell script).

This patch fixes the issue by removing the assumptions on the file
descriptor numbers that open() returns.

For the random file (/dev/urandom), the new code doesn't rely on a
shared file descriptor; instead, it reopens the file in the child.

For the data file, the new code writes the file descriptor number into
a "ZTEST_FD_DATA" environment variable so that it can be recovered
after the execv() call.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Minimize ztest stack frame size

To ensure ztest behaves as similarly as possible to the kernel
implementation of ZFS we attempt to honor the kernel stack limits.
This includes keeping the individual stack frame sizes under 1K
in size.  We currently use gcc to detect and enforce this limit.

Therefore to get this building cleanly with full debugging enabled
the stack usage in the following functions has been reduced by
moving the buffer to the heap.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Replace tempnam() with mkstemp()

The use of tempnam() is racy and it should be avoided in favor of
mkstemp().  According to the Linux tempnam(3) man page.

  "Although tempnam() generates names that are difficult to guess,
  it is nevertheless possible that between the time that tempnam()
  returns a pathname, and the time that the program opens it, another
  program might create that pathname using open(2), or create it as
  a symbolic link.  This can lead to security holes.  To avoid such
  possibilities, use the open(2) O_EXCL flag to open the  pathname.
  Or better yet, use mkstemp(3) or tmpfile(3)."

This issue was flagged by gcc.

  ztest.o: In function `setup_data_fd': cmd/ztest/ztest.c:5822:
  warning: the use of `tempnam' is dangerous, better use `mkstemp'

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>