Simplify hostid logic

There is plenty of compatibility code for a hw_hostid
that isn't used by anything. At the same time, there are apparently
issues with the current hostid logic. coredumb in #zfsonlinux on
freenode reported that Fedora 17 changes its hostid on every boot, which
required force importing his pool. A suggestion by wca was to adopt
FreeBSD's behavior, where it treats hostid as zero if /etc/hostid does
not exist

Adopting FreeBSD's behavior permits us to eliminate plenty of code,
including a userland helper that invokes the system's hostid as a
fallback.

Signed-off-by: Richard Yao <ryao@cs.stonybrook.edu>
Return immediate success from spl_cache_grow upon success

Consumers of the SPL invoke spl_kmem_cache_alloc to allocate memory from
caches. When a cache is out of memory, spl_kmem_cache_alloc() will
invoke spl_cache_refill(), which will invoke spl_cache_grow().

spl_cache_grow() will give the job of doing allocations to a task queue.
It will then call wait_event_timeout() to wait for completion. If
completion is too slow, it will time out. This gives us the option of
either telling spl_cache_refill() that we succeeded and there should be
memory or we did not succeed. Currently, we return ENOMEM
unconditionally, which forces us to do another iteration to figure out
that we really should have returned 0 for success. This is
unnecessary, so lets return 0 immediately upon success.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Do not defer emergency allocations

A concept of an emergency allocation was introduced last year to permit
allocations that took excessive periods of time to switch to physical
memory to make forward progress. Emergency allocations currently occur
by setting a flag, telling kmem_cache_alloc() to try again and then
doing the allocation. This is unnecessary, so we restructure the code
to do the emergency allocations immediately.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Honor __GFP_NORETRY in spl_kmem_cache_alloc()

The Open Solaris man page states that KM_NOSLEEP will "Return NULL
immediately if memory is not available". However, we currently do not
honor this behavior. We map KM_NOSLEEP to a Linux equivalent that
contains __GFP_NORETRY, which is meant to prevent this behavior. We
current this by checking for __GFP_NORETRY in spl_kmem_cache_alloc() and
spl_cache_refill() If it is set, exit the busy loop and return with
whatever we have done.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Do not block on IO or book-keeping when attempting to grow caches

Growing cache can block on swap operations, which will force all
allocations to be emergency allocations until the swap operations have
finished. Each emergency allocation will wait a maximum of 1 second,
which means that we will spend a fairly substantial amount of time
waiting when growing blocks on swap operations and swap happens to be a
zvol. We disallow ___GFP_IO to prevent this.

Additionally, __GFP_WAIT appears to tell the kernel that is okay to do
book keeping. That could potentially make swap operations that depend on
spl_kmem_cache_alloc() to wait longer, so we disallow that as well.

Signed-off-by: Richard Yao <ryao@gentoo.org>
kmem_alloc/vmem_alloc: Yield CPU before retrying memory allocations

Solaris and Linux both have kswapd for paging out memory. However, the
Linux kernel also has a concept of direct reclaim, where an allocation
that is low on memory will attempt to free memory. Unfortunately, we can
have hundreds of threads doing this when the system is under heavy load.
This can cause discernible lags and will leave the system in a state
where it appears to be deadlocked.

We set __GFP_NORETRY to prevent the kernel's own routines from retrying
allocations repeatedly. We then try the allocation, yield the CPU if it
fails and then try again. This should allow us to perform more
gracefully when hundreds of threads block on memory allocations during
heavy system load.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Use Linux SLAB allocator for spl_kmem_alloc_t objects

The Linux kernel's virtual memory subsystem has a single global lock.
Contention for this lock suffers from fairness issues where allocations
that depend on it could be starved for an indefinite period of time.
The SPL SLAB allocator is implemented on top of the Linux kernel's
virtual memory allocator. It attempts to mitigate such problems by doing
allocations in bulk.  However, other things will use the same interfaces
for virtual memory allocations, which can be a source of contention.

When an attempt at an allocation is done and a cache is full, the SPL
SLAB allocator will allocate a temporary spl_kmem_alloc_t object before
doing any actual allocations. This increases contention on the global
lock and slows down allocations. This also adds a potential source of
internal fragmentation in the virtual memory address space immediately
prior to an allocation, which is problematic on 32-bit systems.

We switch to the kernel's physical memory slab allocator. The objects
themselves have the property that they are small and are used
infrequently, which should permit the kernel's slab allocator to satisfy
all allocations from a single page. This provides us with O(1)
allocations, reduces contention and also eliminates a source of internal
fragmentation of the virtual memory address space.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Call SLAB object constructors and destructors on demand

The SPL's SLAB code was written with an optimization to preconstruct
objects before they are used. This is semantically differs from the
Solaris behavior where objects are constructed and deconstructed on
demand. A consequence of preallocation is that buffers that should be
zeroed by constructors will contain stale data whenever the buffer is
reused. I discovered this when experimenting with the conversion of
important ARC memory allocations to SLAB allocation.

We switch to on-demand construction and deconstruction to correct this.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Linux 3.8 compat: Use kuid_t/kgid_t when required

When CONFIG_UIDGID_STRICT_TYPE_CHECKS is enabled uid_t/git_t are
replaced by kuid_t/kgid_t, which are structures instead of integral
types. This causes any code that uses an integral type to fail to build.
The User Namespace functionality introduced in Linux 3.8 requires
CONFIG_UIDGID_STRICT_TYPE_CHECKS, so we could not build against any
kernel that supported it.

We resolve this by converting between the new kuid_t/kgid_t structures
and the original uid_t/gid_t types.

Original-patch-by: DHE
Rewrite-by: Richard Yao <ryao@gentoo.org>