MFC r255428:

Pass -n (do not emit comments) when saving mtree information for future
mergemaster(8) runs.
MFC r254330 -- allow users to run SYS_PD mode JBOD with a non-default MAXPHYS
MFC r255412:

In r243868, the error message buffer errmsg have been changed from
an on-stack array to a pointer and therefore sizeof(errmsg) would
become 4 or 8 bytes depending on the architecture.

Fix this by using ERRMSGL in place of sizeof().

Submitted by:	J David <j.david.lists@gmail.com>
MFC of 254995:

A performance problem was reported in PR kern/181226:

    I have 25TB Dell PERC 6 RAID5 array. When it becomes almost
    full (10-20GB free), processes which write data to it start
    eating 100% CPU and write speed drops below 1MB/sec (normally
    to gives 400MB/sec). The revision at which it first became
    apparent was http://svnweb.freebsd.org/changeset/base/249782.

The offending change reserved an area in each cylinder group to
store metadata. The new algorithm attempts to save this area for
metadata and allows its use for non-metadata only after all the
data areas have been exhausted. The size of the reserved area
defaults to half of minfree, so the filesystem reports full before
the data area can completely fill. However, in this report, the
filesystem has had minfree reduced to 1% thus forcing the metadata
area to be used for data. As the filesystem approached full, it
had only metadata areas left to allocate. The result was that
every block allocation had to scan summary data for 30,000 cylinder
groups before falling back to searching up to 30,000 metadata areas.

The fix is to give up on saving the metadata areas once the free
space reserve drops below 2%. The effect of this change is to use
the old algorithm of just accepting the first available block that
we find. Since most filesystems use the default 5% minfree, this
will have no effect on their operation. For those that want to push
to the limit, they will get their crappy block placements quickly.

Submitted by:  Dmitry Sivachenko
Fix Tested by: Dmitry Sivachenko
PR:            kern/181226

MFC of 254996:

In looking at block layouts as part of fixing filesystem block
allocations under low free-space conditions (-r254995), determine
that old block-preference search order used before -r249782 worked
a bit better. This change reverts to that block-preference search order.
MFC r255459:
  Remove documentation describing functionality which geom(4) does not,
  in fact, provide.
MFC r255439:
Call free() on the pointer returned from malloc().
MFC r255048:

  - Fix LOCAL_MTREE so it properly handles multiple files and quotes
    its value into submakes

PR:		conf/179466
Submitted by:	Garrett Cooper <yaneurabeya@gmail.com>
Sponsored by:	EMC / Isilon Storage Division
MFC r254713: fbt: drop a local write-only variable
MFC r254445,254711: zfs: inline and remove zfs_vnode_lock
MFC r254714: zfs: do not reject any operations on a pool just because
it's a boot pool
MFC r255261: watch: Do not mess up the tty modes on early error.

Record the initial state earlier, so it is always safe to restore it.

One way this happens is if watch(8) is started by a user that does not have
access to /dev/snp. The result is "staircase effect" during later commands.

PR:		bin/153052
MFC: r254337
Fix several performance related issues in the new NFS server's
DRC for NFS over TCP.
- Increase the size of the hash tables.
- Create a separate mutex for each hash list of the TCP hash table.
- Single thread the code that deletes stale cache entries.
- Add a tunable called vfs.nfsd.tcphighwater, which can be increased
  to allow the cache to grow larger, avoiding the overhead of frequent
  scans to delete stale cache entries.
  (The default value will result in frequent scans to delete stale cache
   entries, analagous to what the pre-patched code does.)
- Add a tunable called vfs.nfsd.cachetcp that can be used to disable
  DRC caching for NFS over TCP, since the old NFS server didn't DRC cache TCP.
It also adjusts the size of nfsrc_floodlevel dynamically, so that it is
always greater than vfs.nfsd.tcphighwater.

For UDP the algorithm remains the same as the pre-patched code, but the
tunable vfs.nfsd.udphighwater can be used to allow the cache to grow
larger and reduce the overhead caused by frequent scans for stale entries.
UDP also uses a larger hash table size than the pre-patched code.
MFC r255119:
Fix SES_ENABLE_PASSTHROUGH kernel option, unexpectedly broken during driver
overhaul.
MFC r255534:
Add more references.
MFC r253643:
Following r222950, revert unintentional change cls -> class in argument name
in r245264.  Aside from non-uniformity, that again confused C++ compilers.
MFC r253806:
Allow three IOCTLs to be used on suspended pool, restoring state that
existed before IOCTL code refactoring merged change 4445fffb from illumos
at r248571.

This change allows `zpool clear` to be used again to recover suspended pool.
It seems the only was supposed by the code to restore pool operation after
reconnecting lost disks that were required for data completeness.  There
are still cases where `zpool clear` command can just safely stuck due to
deadlocks inside ZFS kernel part, but probably that is better then having
no chances to recover at all.
MFC r253990:
Make ZFS to use separate thread to handle SPA_ASYNC_REMOVE async events.
Existing async thread is running only on successfull spa_sync() completion,
that is impossible in case of pool loosing required (last) disk(s).  That
indefinite delay of SPA_ASYNC_REMOVE processing made ZFS to not close the
lost disks, preventing GEOM/CAM from destroying devices and reusing names
on later disk reattach.

In earlier version of the patch I've tried to just run existing thread
immediately, unrelated to spa_sync() completion, but that exposed number
of situations where it could stuck due to locks held by stuck spa_sync(),
that are required for other kinds of async events.

Experiments with OpenIndiana snapshot confirmed that they also have this
issue with lost disks reattach.
MFC r253991:
Make `zpool clear` to reopen also reconnected cache and spare devices.
Since `zpool status` reports about such kinds of errors, it is strange
that they are not cleared by `zpool clear`.
MFC r253993:
Block reporting of ZFS features for suspended pools.

Before executing any subcommand, zpool tool fetches pools configuration from
the kernel.  Before features support was added, kernel was regenerating that
configuration based on data always present in memory.  Unfortunately, pool
features list and activity counters are not such. They are stored in ZAP,
that normally resides in ARC, but under heavy memory pressure may be swapped
out.  If pool is suspended at this point, there is no way to recover it back
since any zpool command will stuck.

This change has one predictable flaw: `zpool upgrade` always wish to upgrade
suspended pools, but fortunately it can't do it due to the suspension.
MFC r255363:
Micro-optimize cpu_search(), allowing compiler to use more efficient inline
ffsl() implementation, when it is available, instead of homegrown iteration.

On dual-E5645 amd64 system (2x6x2 cores) under heavy I/O load that reduces
time spent inside cpu_search() from 19% to 13%, while IOPS increased by 5%.