3537 add kstat_waitq_enter and friends

These kstat interfaces are required to port
"Illumos #3537 want pool io kstats" to ZFS on Linux.

kstat_waitq_enter()
kstat_waitq_exit()
kstat_runq_enter()
kstat_runq_exit()

Additionally, zero out the ks_data buffer in __kstat_create() so
that the kstat_io_t counters are initialized to zero.
kstat to use private lock by default

While porting Illumos #3537 I found that ks_lock member of kstat_t
structure is different between Illumos and SPL. It is a pointer to
the kmutex_t in Illumos, but the mutex lock itself in SPL.
Apparently Illumos kstat API allows consumer to override the lock
if required. With SPL implementation it is not possible anymore.

Things were alright until the first attempt to actually override
the lock. Porting of Illumos #3537 introduced such code for the
first time.

In order to provide the Solaris/Illumos like functionality we:
  1. convert ks_lock to "kmutex_t *ks_lock"
  2. create a new field "kmutex_t ks_private_lock"
  3. On kstat_create() ks_lock = &ks_private_lock

Thus if consumer doesn't care we still have our internal lock in use.
If, however, consumer does care she has a chance to set ks_lock to
anything else before calling kstat_install().

The rest of the code will use ks_lock regardless of its origin.
Emulate illumos interface cv_timedwait_hires()

Needed for Illumos #3852. This interface is supposed to support a
variable-resolution timeout with nanosecond granularity.  This
implementation just rounds up to jiffie resolution, as this was the most
expedient solution, and nanosecond resolution is rarely needed for
real-world performance tuning.  Add flags from sys/callo.h as these are
used to control the behavior of cv_timedwait_hires().  Specifically,

CALLOUT_FLAG_ABSOLUTE
    Normally, the expiration passed to the timeout API functions is
    an expiration interval. If this flag is specified, then it is
    interpreted as the expiration time itself.

CALLOUT_FLAG_ROUNDUP
    Roundup the expiration time to the next resolution boundary. If this
    flag is not specified, the expiration time is rounded down.

References:
    https://www.illumos.org/issues/3582
    illumos/illumos-gate@0689f76
Add kpreempt() compatibility macro

This is needed for the Illumos #4045 write throttle patch.  It is used
in the arc eviction code to avoid blocking all arc activity by sitting on
arcs_mtx too long.
Simplify hostid logic

There is plenty of compatibility code for a hw_hostid
that isn't used by anything. At the same time, there are apparently
issues with the current hostid logic. coredumb in #zfsonlinux on
freenode reported that Fedora 17 changes its hostid on every boot, which
required force importing his pool. A suggestion by wca was to adopt
FreeBSD's behavior, where it treats hostid as zero if /etc/hostid does
not exist

Adopting FreeBSD's behavior permits us to eliminate plenty of code,
including a userland helper that invokes the system's hostid as a
fallback.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Honor __GFP_NORETRY in spl_kmem_cache_alloc()

The Open Solaris man page states that KM_NOSLEEP will "Return NULL
immediately if memory is not available". However, we currently do not
honor this behavior. We map KM_NOSLEEP to a Linux equivalent that
contains __GFP_NORETRY, which is meant to prevent this behavior. We
current this by checking for __GFP_NORETRY in spl_kmem_cache_alloc() and
spl_cache_refill() If it is set, exit the busy loop and return with
whatever we have done.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Call SLAB object constructors and destructors on demand

The SPL's SLAB code was written with an optimization to preconstruct
objects before they are used. This is semantically differs from the
Solaris behavior where objects are constructed and deconstructed on
demand. A consequence of preallocation is that buffers that should be
zeroed by constructors will contain stale data whenever the buffer is
reused. I discovered this when experimenting with the conversion of
important ARC memory allocations to SLAB allocation.

We switch to on-demand construction and deconstruction to correct this.

Signed-off-by: Richard Yao <ryao@gentoo.org>
kmem_alloc/vmem_alloc: Yield CPU before retrying memory allocations

Solaris and Linux both have kswapd for paging out memory. However, the
Linux kernel also has a concept of direct reclaim, where an allocation
that is low on memory will attempt to free memory. Unfortunately, we can
have hundreds of threads doing this when the system is under heavy load.
This can cause discernible lags and will leave the system in a state
where it appears to be deadlocked.

We set __GFP_NORETRY to prevent the kernel's own routines from retrying
allocations repeatedly. We then try the allocation, yield the CPU if it
fails and then try again. This should allow us to perform more
gracefully when hundreds of threads block on memory allocations during
heavy system load.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Change spl_kmem_cache_expire default setting to 2

This makes us better emulate the Solaris behavior.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Remove unused flags variable from __cv_init()

GCC 4.8.1 complained about an unused flags variable when building
against Linux 2.6.26.8:

/var/tmp/portage/sys-kernel/spl-9999/work/spl-9999/module/spl/../../module/spl/spl-condvar.c:
In function ‘__cv_init’:
/var/tmp/portage/sys-kernel/spl-9999/work/spl-9999/module/spl/../../module/spl/spl-condvar.c:39:6:
error: variable ‘flags’ set but not used
[-Werror=unused-but-set-variable]
  int flags = KM_SLEEP;
        ^
	cc1: all warnings being treated as errors

Signed-off-by: Richard Yao <ryao@gentoo.org>
Replace current_kernel_time() with getnstimeofday()

current_kernel_time() is used by the SPLAT, but it is not meant for
performance measurement. We modify the SPLAT to use getnstimeofday(),
which is equivalent to the gethrestime() function on Solaris.
Additionally, we update gethrestime() to invoke getnstimeofday().

Signed-off-by: Richard Yao <ryao@gentoo.org>
Force emergency allocations to trigger under low memory conditions

A concept of an emergency allocation was introduced last year to permit
allocations that took excessive periods of time to switch to physical
memory to make forward progress. Emergency allocations currently occur
by setting a flag, telling kmem_cache_alloc() to try again and then
doing the allocation. This is useful, but it might not always trigger
because we do not permit allocations in the spl_cache_grow() to
retry. This permits allocation failures to fail before the timer for
emergency allocations can trigger. This will result in a return to
kmem_cache_alloc(), which will then repeat the process. The result is
that we can loop infinitely without any forward progress under low
memory conditions because the emergency allocations will not trigger.

We change the code to force the worker to retry indefinitely. This
guarentees that the emergency allocations will trigger under low memory
conditions.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Preallocate memory for spl_kmem_alloc_t in spl_kmem_cache_t

Signed-off-by: Richard Yao <ryao@gentoo.org>
Use pages for emergency allocations when possible

A concept of an emergency allocation was introduced last year to permit
allocations that took excessive periods of time to switch to physical
memory to make forward progress.

Unfortunately, the Linux kernel's physical memory allocator does SLAB
allocation for all allocations less than or equal to 8192. It is
possible for Linux's SLAB allocator to invoke direct reclaim when
allocating new slabs. This can cause swap to deadlock. Additionally,
memory fragmentation is able to cause allocations to hang whenever an
allocation is made from a cache that uses slabs that span multiple
pages. In specific, whenever the allocation is greater than 256-bytes.

Whenever an emergency object is small enough to fit into a single page
and we allocate a page for the emergency allocation instead of going
through kmalloc(). If the object and its ske header cannot fit together
in a single page, we allocate two non-contiguous pages. Larger object
sizes are still at risk, but there is little we can do for them.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Declare get_vmalloc_info_fn only when needed

A couple of definitions were outside a preprocessor conditional
statement when they should have been inside it. This caused build
failures for some people.

Closes zfsonlinux/spl#291

Reported-by: Aaron Fineman <abyxcos@mnetic.ch>
Signed-off-by: Richard Yao <ryao@gentoo.org>