Set spl_kmem_cache_reclaim=0 to default

Reinstate the correct default behavior of returning the number of objects
in the cache for reclaim.  This behavior was disabled in recent releases
to do occasional reports of spinning in shrink_slabs().  Those issues have
been resolved and can no longer can be reproduced.  See commit 376dc35.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: DHE <git@dehacked.net>
Issue #358
Closes #379
Set spl_kmem_cache_slab_limit=16384 to default

For small objects the Linux slab allocator should be used to make the most
efficient use of the memory.  However, large objects are not supported by
the Linux slab and therefore the SPL implementation is preferred.  A cutoff
of 16K was determined to be optimal for architectures using 4K pages.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: DHE <git@dehacked.net>
Issue #356
Closes #379
random_get_pseudo_bytes() need not provide cryptographic strength entropy

Perf profiling of dd on a zvol revealed that my system spent 3.16% of
its time in random_get_pseudo_bytes(). No SPL consumers need
cryptographic strength entropy, so we can reduce our overhead by
changing the implementation to utilize a fast PRNG.

The Linux kernel did not export a suitable PRNG function until it
exported get_random_int() in Linux 3.10. While we could implement an
autotools check so that we use it when it is available or even try to
access the symbol on older kernels where it is not exported using the
fact that it is exported on newer ones as justification, we can instead
implement our own pseudo-random data generator. For this purpose, I have
written one based on a pseudo-random number generator proposed in a
paper by Sebastiano Vigna that itself was based on work by the late
George Marsaglia.

http://arxiv.org/pdf/1402.6246v2.pdf

Profiling the same benchmark with a variant of this patch that did not
disable interrupts showed that time spent in random_get_pseudo_bytes()
dropped to 0.06%. That is a factor of 50 improvement.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Revert "Add PF_NOFS debugging flag"

This reverts commit eb0f407a2b9089113ef6f2402ebd887511315b43.
Rework generic memory allocation interfaces

On Solaris, kernel memory allocations are virtual with few exceptions.
The low level memory allocator is the vmem interface. It is a low level
allocator that partitions memory into arenas from which objects are
allocated. The kmem_cache interface is a front end for this. The generic
kmem interface is a HOARD-style allocator implemented on top of that.
Additionally, the memory allocations have modifier flags. KMEM_SLEEP is
the default. It is the flag where no bits are set and allocations are
permitted to block. Setting additional bits modifies the behavior.

The SPL's memory allocation interfaces are intended to mimic those of
Solaris. However, the original implementation did not match what was
done on Solaris. In specific, we had the following differences arise:

1. The kmem interface was mapped to Linux's physical memory allocator,
kalloc(). That has no strict analog on Solaris in that it allocates
general purpose memory without any page table manipulation. This has the
consequence of causing internal memory fragmentation, which can lead to
allocations randomly blocking.

2. The vmem interface was mapped to Linux's virtual memory allocator,
valloc(). That is inferior to what is available on Solaris in many ways.
In specific, if allocations cannot be satisfied, it will spin until they
can be, which causes deadlocks. In addition, there is a single global
spinlock, which causes parallelization issues.

3. The kmem flags are mapped to gfp flags via preprocessor directives.
That approach requires that the two be semantically compatible, which is
not the case. Solaris' KMEM_SLEEP is equivalent to Linux's GFP_KERNEL.
On Solaris, KMEM_SLEEP is represented as no bits set, with additional
bits being added to modify it. On Linux, no bits set is illegal, so
GFP_KERNEL represents a bit position that is then modified by additional
bits.  This is a subtle incompatibility that makes it difficult to write
code for both platforms.

This patch attempts to address these issues in a few ways. First, it
unifies the kmem and vmem interface implements by mapping vmem to kmem.
It uses vmalloc() for allocations that exceed 2 pages while it uses
kalloc() for allocations that are 2 pages or less, which we assume can
safely be satisfied. This ensures that large allocations are not made
that risk causing large internal fragmentation while small allocations
are not blocked on a global spinlock. All vmem_* functions are mapped to
their kmem_* equivalents via preprocessor directives. Second, it
abandons the preprocessor-based kmem to gfp flag transition in favor of
a translation function called kmem_flags_convert(). It is modelled after
the one in the mainline XFS port, which shares UNIX heritage with ZFS.

These two changes appear to create the most semantically equivalent
mapping possible on Linux.  The result is the elimination of concerns
regarding proper use of generic interfaces when writing portable code,
which posed problems for the development of things like sgbuf. The only
exception involves code using Linux's wake_up_bit, which does not exist
in Solaris code. Any code that does use it should use the native Linux
allocators.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Set PF_MEMALLOC_NOIO when doing virtual memory allocations

PF_MEMALLOC_NOIO was added in Linux 3.9. __GFP_IO is removed from page
allocation whenver it is set. ZFS' cousin XFS began using it in Linux
3.15. We should be using it too in the interest of safety.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Use Linux kzalloc to allocate kmem_cache_t objects

wake_up_bit() is called on a word inside kmem_cache_t objects.  This
calls virt_to_page() on the address of the memory. That is incompatible
with virtual memory, so we must switch to Linux's memory allocator.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Remove kmem_alloc_node interface

This interface has no external consumers and does not exist on Solaris.
Lets remove it.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Allow NULL pointers to be passed to strfree()

Linux has no kernel thread local storage code, but the SPL provides code
to implement that. This requires that consumers call tsd_exit() on the
off chance that it is being used to avoid memory leaks. Unfortunatley,
that has the side effect of passing NULL to strfree(). In the past, that
would result in kfree(NULL), which the Linux kernel happily accepted,
but the generic memory allocation rework means that this is now
kmem_free(NULL), which causes a NULL pointer dereference. We add a
branch to check for NULL and do nothing when we see it. This enables the
TSD hack to continue working.

When Linux gains thread local storage infrastructure to avoid the need
for tsd_exit(), we can revert this in favor of printing a stack trace
whenever we see a NULL pointer passed to strfree().

Signed-off-by: Richard Yao <ryao@gentoo.org>
spl-kmem should not use taskq interface

This is a prerequisite for the Illumos taskq import.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Introduce macros for manipulating PF_FSTRANS

Signed-off-by: Richard Yao <ryao@gentoo.org>
KM_SLEEP kmem_cache allocations should not block high priority allocations

Signed-off-by: Richard Yao <ryao@gentoo.org>
spl-kmem: Use smp_mb__after_clear_bit()

We should use smp_mb__after_clear_bit() to ensure that threads being
woken will always see that a bit has been cleared.

Signed-off-by: Richard Yao <ryao@gentoo.org>
kmem_cache: Call constructor/destructor on each alloc/free

This has a few benefits. First, it fixes a regression that "Rework
generic memory allocation interfaces" appears to have triggered in
splat's slab_reap and slab_age tests. Second, it makes porting code from
Illumos to ZFSOnLinux easier. Third, it has the side effect of making
reclaim from slab caches that specify reclaim functions an order of
magnitude faster. The splat slab_reap test usually took 30 to 40
seconds. With this change, it takes 3 to 4.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Merge branch 'kmem-rework' into dehacked

Conflicts:
	module/spl/spl-kmem.c
Cleanup vn_rename() and vn_remove()

zfsonlinux/spl#bcb15891ab394e11615eee08bba1fd85ac32e158 implemented
Linux 3.6+ support by adding duplicate vn_rename and vn_remove
functions. The new ones were cleaner, but the duplicate functions made
the codebase less maintainable. This adds some compatibility shims that
allow us to retire the older vn_rename and vn_remove in favor of the new
ones on old kernels. The result is a net 143 line reduction in lines of
code and a cleaner codebase.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Remove obsolete HAVE_PATH_IN_NAMEIDATA autotools check

The vn_rename() and vn_remove() cleanup rendered this obsolete, so lets
kill it.

Signed-off-by: Richard Yao <ryao@gentoo.org>