Obtain advisory lock on ZED PID file

ZED uses an advisory lock on its state file to protect against
multiple instances running concurrently.  However, work is planned
to move this state information into the kernel, and ZED will still
need to protect against starting multiple instances.

This commit adds an advisory lock on the PID file to protect against
starting multiple instances.  A lock failure can be overridden with
the "-f" (force) command-line option.  The advisory lock on the state
file is being retained for as long as the state information is stored
in the state file.

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #2756
Cleanup struct zed_conf vars in zed_conf_destroy

Reset struct zed_conf file descriptors to -1 after close(),
and pointers to NULL after free().

Signed-off-by: Chris Dunlap <cdunlap@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #2756
Amend Dracut module to export ZFS root on shutdown

Make use of Dracut's ability to restore the initramfs on shutdown and
pivot to it, allowing for a clean unmount and export of the ZFS root.
No need to force-import on every reboot anymore.

Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #2195
Issue #2476
Issue #2498
Issue #2556
Issue #2563
Issue #2575
Issue #2600
Issue #2755
Issue #2766
Illumos 5176 - lock contention on godfather zio

5176 lock contention on godfather zio
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Alex Reece <alex.reece@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Richard Elling <richard.elling@gmail.com>
Reviewed by: Bayard Bell <Bayard.Bell@nexenta.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  https://www.illumos.org/issues/5176
  https://github.com/illumos/illumos-gate/commit/6f834bc

Porting notes:

Under Linux max_ncpus is defined as num_possible_cpus().  This is
largest number of cpu ids which might be available during the life
time of the system boot.  This value can be larger than the number
of present cpus if CONFIG_HOTPLUG_CPU is defined.

Ported by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2711
Add an example for 'zfs bookmark' to the Example section.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2762
Fix CPU_SEQID use in preemptible context

Commit e022864 introduced a regression for kernels which are built
with CONFIG_DEBUG_PREEMPT.  The use of CPU_SEQID in a preemptible
context causes zio_nowait() to trigger the BUG.  Since CPU_SEQID
is simply being used as a random index the usage here is safe. To
resolve the issue preempt is disable while calling CPU_SEQID.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ned Bass <bass6@llnl.gov>
Closes #2769
Improve VERIFY() error in dmu_write()

This is a debug patch designed to ensure an error code is logged
to the console when this VERIFY() is hit.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ned Bass <bass6@llnl.gov>
Issue #1440
Add a stern warning about dedup

Users intending to use dedup should be clearly advised about
its memory requirements and the risks involved.

Thanx to Sachiru for comments and suggestions.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2754
Install header during post-build rather than post-install.

New versions of dkms clean up the build directory after installing.

It appears that this was always intended, but had rm -rf "/path/to/build/*"
(note the quotes), which prevented it from working.

Also, the build step is already installing stuff into the directory where
these files go, so installing our stuff there as part of build rather than
install makes sense.

Signed-off-by: Tom Prince <tom.prince@clusterhq.com>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2776
Point dkms build at installed source tree, rather than build directory.

Signed-off-by: Tom Prince <tom.prince@clusterhq.com>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2776
=> 1099: turbo/iscsi - This is iSCSI sharing for ZoL (shareiscsi).
* Supports the following iSCSI implementations (in this order of discovery):
  + IET         http://iscsitarget.sourceforge.net
  + STGT        http://stgt.berlios.de
  + SCST        http://scst.sourceforge.net
    + Requires that SCST was compiled with /sys support (the default).
  + LIO         http://linux-iscsi.org
* Will refuse to unshares an active target (one with sessions).
* Supports the following options to the 'shareiscsi' property:
  + name/iqn    Full iSCSI Qualified Name (IQN), including identifier.
                This is generated by iscsi_generate_target():
                + Uses the content of an optional /etc/iscsi_target_id file:
                  Example: iqn.YYYY-MM.tld.domain
                + If this file doesn't exist, it uses todays year, month and
                  domain name to generate the iqn.
                + The dataset name is appended at the end of the iqn, with
                  slashes replaced with dots.
                  => <iqn>:<dataset>
  + lun         LUN (0-16384)
                Default: 0 (1 for STGT)
  + type        Share mode (fileio, blockio, nullio, disk, tape)
                STGT: ssc, pt
                Default: blockio (disk for STGT)
  + iomode      IO mode (wb, wt, ro)
                STGT: rdwr, aio, mmap, sg, ssc
                Default: wt (rdwr for STGT)
  + blocksize   Logical block size (512, 1024, 2048, 4096)
                Default: Volume blocksize, 4096 if not usable.
                NOTE: Currently not supported for STGT (doesn't seem to be
                      an option for it in tgtadm).
  + initiator   Allow only this initiator to bind to target.
                Currently only availible for LIO and STGT.
  + authname    Global user to use in binds on targets.
  + authpass    Password for global user.
* If not called with a 'name/iqn' value, then force setting it.
  This so that the IQN doesn't change every month (when it is
  regenerated again). It will be generated by iscsi_generate_target()
  (see above).

NOTE:
+ Moved nfs.c:foreach_nfs_shareopt() to libshare.c:foreach_shareopt()
  so that it can be (re)used in smb.c and iscsi.c.
+ Use the list_{create,insert}() etc for keeping tabs of the linked lists
  instead of using a home-made version.
+ A half second delay had to be added in lib/libzfs/libzfs_mount.c:zfs_unmount()
  after the successful unshare. This to avoid 'dataset busy' when destroying
  recursivly.
+ Fix some gcc-4.8.1 compile fixes. Thanx to munix9.
+ Fix error in manpage.
+ Don't set last char in domainname to NULL willy-nilly. Only if newline.
+ Instead of blindingly generate an IQN, use opts->name if it's set.
+ Split iSCSI implementation into their own separate source files
+ The 'initiator', 'authname' and 'authpass' option might have some issues:
  It will make the compatibility between the different iSCSI implementations
  questionable - can't switch between them easily (it will ONLY be availible
  in ZoL). I COULD make the option be silently ignored (instead of forcibly
  rejected if not availible).

  But that still doesn't solve the (possible) problem between ZoL and
  OpenZFS/Illumos (setting the option, will possibly introduce problems when
  importing the pool on something else than ZoL).

  So some more discussion might be needed..

Misc iSCSI fixes/cleanups/attempts to get 'zfs rename' to work.

!! Doesn't - a 'zfs rename' works, but most often than not, the rename
!! IOCTL hangs. Not every time and not always on the first dataset being
!! renamed... Seems a little to random to me.
=> 1476: turbo/smbfs_registry-shares - Rewrite of the Libshare/SMBFS
* Add support for options to 'sharesmb'.
  Fixes: #1182
  + Move nfs.c:foreach_nfs_shareopt() to libshare.c:foreach_shareopt()
    so that it can be (re)used in smb.c (and later iscsi.c).
  + Call net(8) with the guest_ok option.
  + Rewrite the sharesmb part of the zfs(8) manpage.
    + Add more examples
    + Inform about the (new) options to sharesmb.
  + Add STDERR_VERBOSE to libzfs_run_process() so that we
    can catch any errors from net(8).
  + Don't call smb_retrieve_shares() after enabling a share.
    No point, it's done when really needed anyway.
  + Support 'y', 'yes' and 'true' for guest_ok.
  * If sharesmb=on, just exit cleanly directly at the top of
    get_smb_shareopts_cb().
  * Remove param 2 and 3 from smb_enable_share_one(). That is
    availible in the first param - the impl_share struct.
  * Extra debugging added in smb_enable_share_one() - print
    all our options just before we call net(8).
* Add support for optional, extra script /sbin/zfs_share_smbfs
  to run after shareing is done.
  + Add an example zfs_share_smbfs script
* The option sharesmb should not be inherited by childs.
  Samba is perfectly able to show child file systems...
* Use dataset name in comment, not the smb share name.
* Use REGISTRY SHARES instead of USER SHARES
  + These are a lot more configurable (more possibility to customize the share).
  + Remove the 'acl' option and replace it with 'writeable' instead.
  + On some machines, there is a "global" share. This isn't really a share,
    but the possibility to modify global options. So first get a line of
    _real_ shares using 'net conf listshares' and _then_ cyckle through
    them retrieving info with 'net conf showshare <share>'.
  + Add an example on how to modify a share after ZFS share and what man
    pages to use for more info.
* Speedups (less share retreivals) and make sure the 'net' command is executable.
  + Since smb_shares is global, we only retreive shares if it's NULL
    in smb_retrieve_shares(). It should limit the number of times we
    fork() and read shares (which takes time).
  + If the net command don't exist (is executable), don't even register
    the smb fstype in libshare_smb_init().
  + access() fix - check if NET_CMD_PATH is _executable_ not just 'exists'.
* Run cstyle.pl to fix any code style problems.
* Output error if 'net' can't talk to samba.
* Improve error message if 'net' isn't executable.
* Move the checks from smb_available() to libshare_smb_init()
  to avoid doing them multiple times.
* Convert homemade linked list to the ZFS versions list_* functions (which uses a link_t).
* Script to list SMB shares added.
* Comment out a call to zfs_unshare_smb(). Don't seem to be needed.
  This because zfs_unshare_smb() is called twice if setting 'sharesmb=off' - once
  'somewhere else' and once here (this one AFTER the share have been unshared). That
  makes this one fail! Don't know if this is correct, but it works to comment out this...
=> 2790: turbo/sharenfs_rewrite - Rewrite of nfs.c to keep options per host separated.

Example: Setting sharenfs to

  "rw=@192.168.69.8,no_subtree_check,no_root_squash,ro=@192.168.69.45,no_subtree_check,root_squash"

will create the option string

  sec=sys,ro,no_subtree_check,no_root_squash,mountpoint,no_subtree_check,no_root_squash,no_subtree_check,root_squash

which is then run on _each host_!. Part of this is "default" options.

Instead, go through the option string, look for hosts and then make
sure options for each host is kept apart by using a linked list
(maintained by the ZFS list_* functions). Also, the default options
should only be added to the list of options if setting sharenfs to
"on".

Closes: #2773

Update nfs.c with fixes that makes it compatible with the iSCSI and SMB patches.
+ Remove foreach_nfs_shareopt() - was renamed to libshare.c:foreach_shareopt()
+ Remove foreach_nfs_host() - no longer needed.
=> 2087: turbo/initramfs - Initramfs scripts for ZoL.

Initramfs scripts for ZoL.
* Put in /usr/share/initramfs-tools/ or /etc/initramfs-tools/.
* With pull request #1476 (not yet merged) comes a verbose warning
  if /usr/bin/net doesn't exist or isn't executable. Just create
  a dummy...
* If user haven't specified pool, but bootfs then use the first part
  of the bootfs path as pool name.
* Add support for booting of a ZFS snapshot.
  Do this by cloning the snapshot into a dataset. If this the resulting
  dataset already exists, destroy it. Then set mountpoint=/ on that
  dataset and mount it on root.
  + If snapshot does not exist, use base dataset (the part before '@')
    as boot filesystem instead.
* Add support for more kernel command line arguments (ideas taken from
  the Fedora/Redhat dracut script):
  + (zfs_force|zfs.force|zfsforce)=(on|yes|1) (ignore case).
  + root=zfs:AUTO => try to find rootfs automatically.
  + root=zfs:<dataset>
  + root=ZFS=<dataset>
* Remove the existing '-f' option to 'zpool import', instead let
  this be controlled on the kernel command line.
* Do not force-set mountpoint=/ - should (??) not be nessesary.
* Support mounting a rootfs with mountpoint=legacy
* Support both RPM based and DEB based system by removing the logic
  from the dracut script and instead use the logic in scripts/zfs-initramfs/scripts/zfs
  for both Debian GNU/Linux (etc) and RedHat/Fedora (etc).
* Only run the local-* script(s) if/when it/they exists.
* Don't auto import pools when loading module - keep better control of the imports.
* If called with only 'rpool' (or only 'zfs-bootfs'), then fake it by setting 'root=zfs:AUTO'
  (to avoid duplication of code etc) so that it will be auto detected later.
  + If pool is specified and we're auto discovering, don't go through ALL pools, only the one we specified.
* Rewrite the auto detection of bootfs in the auto detector.
* Only try to import pool if it haven't already been imported (in the auto detector).
* Add support for only having 'root=pool/dataset' option.
* Remove the use of setting 'ZFS_RPOOL=rpool'. No longer nessesary.
* Check if /scripts/local-* is either file or directory - should be dir, but just to be safe check both.
* Make wait_for_udev wait for 10 seconds.
* Add a configurable ZFS_INITRD_POST_MODPROBE_SLEEP (set in /etc/default/zfs) after the modprobe.
* Merge https://github.com/zfsonlinux/zfs/pull/2196.
  Remove lines that contain only a hyphen (match '^-$' instead of '-').
* Add a 'default' file to go in '/etc/default/zfs'.
* Support mounting additional filesystems (such as /var and /usr etc)
  in the initrd using ZFS_INITRD_ADDITIONAL_DATASETS set in /etc/defaults/zfs.
* Add exceptions to pool imports.
  + Both for init and initrd script.
* Make sure that the zpool.cache file is actually copied onto the initrd!!
* Include /etc/modprobe.d/{zfs,spl}.conf in the initrd if they exist.
* Export all pools found after the modprobe - zfs_autoimport_disable=1'
  don't seem to be working in >= 0.6.3.
  + This snippet will go away as soon as I can figure out why it doesn't,
    it's ugly!!
* Only try /dev/disk/by-* in the initrd if USE_DISK_BY_ID is set.
  + Try this first, then fallback to using the cache file if that exist.
=> 2106: turbo/fixes.init_total_rewrite - Base scripts on the init scripts included with Debian GNU/Linux

Base scripts on the init scripts included with Debian GNU/Linux
then take code from the already existing ones, trying to merge
them into one.
+ NFS server service is just called 'nfs' in Redhat based system, and
  nfs-kernel-server on Debian GNU/Linux based.
+ Merge https://github.com/zfsonlinux/zfs/pull/2148 - Inform OpenRC that ZFS uses mtab.
+ Stop after umountfs
+ When stopping, don't check for LOCKDIR/zfs-mount - it won't be mounted,
  so the file won't exist making the script not run.
+ Add a configurable ZFS_INITRD_POST_MODPROBE_SLEEP used in the initrd to sleep after the modprobe.
+ The import function, do_import(), imports pools by name instead of '-a' [all].
  + Test all '/dev/disk/by-*' dirs for import.
+ Fallback on importing the pool using the cache file (if it exists) if the
  'by-id' didn't work.
+ Add exceptions to pool imports.
=> 2726: turbo/debug_documentation - Remove *BSD example on how to retrieve debug log with working Linux commands.
=> 2727: turbo/zol-bug-report - Initial version of a ZoL "report issue" script.
=> 1082: turbo/issue-1082 - Support 'out of tree' builds

* Support 'out of tree' builds by adding the 'src' and 'obj' variables
  and removing the absolute path to the build results to the Makefile.in
  files below the module directory.

* For VPATH builds (addition for #1082):
  + Add '../../spl' to spl source directory list to search
  + Do a find in the found spl source for spl_config.h

Closes: #1082
=> 2376: turbo/issue-2033 - Exclude the /usr/lib/modules/*/modules.* files
                            they are regenerated at module pkg install by depmod.
=> 2512: turbo/issue-2448 - Default to --with-blkid in RPM packaging
                          - Make linking with and finding libblkid required

Default to --with-blkid in RPM packaging

Support for libblkid has been in the ZoL for years but has been
disabled by default in the packaging.  When libblkid is disabled
the 'zpool create' command will not warn you if you're going to
overwrite an existing filesystem.  For this reason the default
behavior has been changed to require libblkid.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #2448

Make linking with and finding libblkid required (so that creating
pools on devices with existing filesystem will get a warning).
Use '--without-blkid' to explicitly disable this.

Signed-off-by: Turbo Fredriksson <turbo@bayour.com>
Closes: #2448