Safer debugging and assertion macros.

Spl's debugging and assertion macros macro used the typical do/while(0)
form for if/else friendliness, however, this limits their use in contexts
where a do loop is not valid; such as within another multi-statement
style macro.

The following macros have been converted to not use do/while(0):
	PANIC, ASSERT, ASSERTF, VERIFY, VERIFY3_IMPL

PANIC has been converted to a wrapper around the new spl_PANIC() function.

The other macros have been converted to use the "&&" operator for the
branch-predicition conditional and also to use spl_PANIC().

The __ASSERT() macro was not touched.  It is only used by the debugging
infrastructure and that code, including this macro, will be retired when
the tracepoint patches are merged.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #367
Revert "Add PF_NOFS debugging flag"

This reverts commit eb0f407a2b9089113ef6f2402ebd887511315b43.
Rework generic memory allocation interfaces

On Solaris, kernel memory allocations are virtual with few exceptions.
The low level memory allocator is the vmem interface. It is a low level
allocator that partitions memory into arenas from which objects are
allocated. The kmem_cache interface is a front end for this. The generic
kmem interface is a HOARD-style allocator implemented on top of that.
Additionally, the memory allocations have modifier flags. KMEM_SLEEP is
the default. It is the flag where no bits are set and allocations are
permitted to block. Setting additional bits modifies the behavior.

The SPL's memory allocation interfaces are intended to mimic those of
Solaris. However, the original implementation did not match what was
done on Solaris. In specific, we had the following differences arise:

1. The kmem interface was mapped to Linux's physical memory allocator,
kalloc(). That has no strict analog on Solaris in that it allocates
general purpose memory without any page table manipulation. This has the
consequence of causing internal memory fragmentation, which can lead to
allocations randomly blocking.

2. The vmem interface was mapped to Linux's virtual memory allocator,
valloc(). That is inferior to what is available on Solaris in many ways.
In specific, if allocations cannot be satisfied, it will spin until they
can be, which causes deadlocks. In addition, there is a single global
spinlock, which causes parallelization issues.

3. The kmem flags are mapped to gfp flags via preprocessor directives.
That approach requires that the two be semantically compatible, which is
not the case. Solaris' KMEM_SLEEP is equivalent to Linux's GFP_KERNEL.
On Solaris, KMEM_SLEEP is represented as no bits set, with additional
bits being added to modify it. On Linux, no bits set is illegal, so
GFP_KERNEL represents a bit position that is then modified by additional
bits.  This is a subtle incompatibility that makes it difficult to write
code for both platforms.

This patch attempts to address these issues in a few ways. First, it
unifies the kmem and vmem interface implements by mapping vmem to kmem.
It uses vmalloc() for allocations that exceed 2 pages while it uses
kalloc() for allocations that are 2 pages or less, which we assume can
safely be satisfied. This ensures that large allocations are not made
that risk causing large internal fragmentation while small allocations
are not blocked on a global spinlock. All vmem_* functions are mapped to
their kmem_* equivalents via preprocessor directives. Second, it
abandons the preprocessor-based kmem to gfp flag transition in favor of
a translation function called kmem_flags_convert(). It is modelled after
the one in the mainline XFS port, which shares UNIX heritage with ZFS.

These two changes appear to create the most semantically equivalent
mapping possible on Linux.  The result is the elimination of concerns
regarding proper use of generic interfaces when writing portable code,
which posed problems for the development of things like sgbuf. The only
exception involves code using Linux's wake_up_bit, which does not exist
in Solaris code. Any code that does use it should use the native Linux
allocators.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Set PF_MEMALLOC_NOIO when doing virtual memory allocations

PF_MEMALLOC_NOIO was added in Linux 3.9. __GFP_IO is removed from page
allocation whenver it is set. ZFS' cousin XFS began using it in Linux
3.15. We should be using it too in the interest of safety.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Use Linux kzalloc to allocate kmem_cache_t objects

wake_up_bit() is called on a word inside kmem_cache_t objects.  This
calls virt_to_page() on the address of the memory. That is incompatible
with virtual memory, so we must switch to Linux's memory allocator.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Remove kmem_alloc_node interface

This interface has no external consumers and does not exist on Solaris.
Lets remove it.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Allow NULL pointers to be passed to strfree()

Linux has no kernel thread local storage code, but the SPL provides code
to implement that. This requires that consumers call tsd_exit() on the
off chance that it is being used to avoid memory leaks. Unfortunatley,
that has the side effect of passing NULL to strfree(). In the past, that
would result in kfree(NULL), which the Linux kernel happily accepted,
but the generic memory allocation rework means that this is now
kmem_free(NULL), which causes a NULL pointer dereference. We add a
branch to check for NULL and do nothing when we see it. This enables the
TSD hack to continue working.

When Linux gains thread local storage infrastructure to avoid the need
for tsd_exit(), we can revert this in favor of printing a stack trace
whenever we see a NULL pointer passed to strfree().

Signed-off-by: Richard Yao <ryao@gentoo.org>
spl-kmem should not use taskq interface

This is a prerequisite for the Illumos taskq import.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Introduce macros for manipulating PF_FSTRANS

Signed-off-by: Richard Yao <ryao@gentoo.org>
KM_SLEEP kmem_cache allocations should not block high priority allocations

Signed-off-by: Richard Yao <ryao@gentoo.org>
spl-kmem: Use smp_mb__after_clear_bit()

We should use smp_mb__after_clear_bit() to ensure that threads being
woken will always see that a bit has been cleared.

Signed-off-by: Richard Yao <ryao@gentoo.org>
kmem_cache: Call constructor/destructor on each alloc/free

This has a few benefits. First, it fixes a regression that "Rework
generic memory allocation interfaces" appears to have triggered in
splat's slab_reap and slab_age tests. Second, it makes porting code from
Illumos to ZFSOnLinux easier. Third, it has the side effect of making
reclaim from slab caches that specify reclaim functions an order of
magnitude faster. The splat slab_reap test usually took 30 to 40
seconds. With this change, it takes 3 to 4.

Signed-off-by: Richard Yao <ryao@gentoo.org>