Squashed commit of the following:

commit e94cdf521f87f18610131a9d194b8672ec406d23
Author: Neeti Desai <neetid@codeaurora.org>
Date:   Tue Jun 11 00:14:21 2013 +0000

    msm: Allow lowmem to be non contiguous and mixed.

    Any image that is expected to have a lifetime of
    the entire system can give the virtual address
    space back for use in vmalloc.

    Change-Id: I81ce848cd37e8573d706fa5d1aa52147b3c8da12
    Signed-off-by: Neeti Desai <neetid@codeaurora.org>

commit ced362c0acfcefa0964d01f85dd025a1e70da8e9
Author: Paul Reioux <reioux@gmail.com>
Date:   Mon Sep 2 14:29:47 2013 -0500

    drivers/tty/smux_private.h: SLAB/SLOB compatibility fixup

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit 6b7c40f2563437dce5530d0ad0ecc1d3d9a3052a
Author: Paul Reioux <reioux@gmail.com>
Date:   Mon Sep 2 14:22:25 2013 -0500

    Revert "mm: micro-optimise slab to avoid a function call"

    This reverts commit 381760eadc393bcb1bb328510ad75cf13431806d.

commit 7dcdd2bd9c3fae17f134ce086d75885184e07261
Author: Glauber Costa <glommer@parallels.com>
Date:   Tue Dec 18 14:22:46 2012 -0800

    sl[au]b: always get the cache from its page in kmem_cache_free()

    struct page already has this information.  If we start chaining caches,
    this information will always be more trustworthy than whatever is passed
    into the function.

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Frederic Weisbecker <fweisbec@redhat.com>
    Cc: Greg Thelen <gthelen@google.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: JoonSoo Kim <js1304@gmail.com>
    Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
    Cc: Mel Gorman <mel@csn.ul.ie>
    Cc: Michal Hocko <mhocko@suse.cz>
    Cc: Pekka Enberg <penberg@cs.helsinki.fi>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Suleiman Souhlal <suleiman@google.com>
    Cc: Tejun Heo <tj@kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

    backported to Android Linux 3.4 by faux123
    Signed-off-by: Paul Reioux <reioux@gmail.com>

    Conflicts:
    	include/linux/memcontrol.h
    	mm/slab.h

commit 1e77893e32fb00d774e098eeb4067a6e9ac22cb4
Author: Glauber Costa <glommer@parallels.com>
Date:   Tue Dec 18 14:22:31 2012 -0800

    slab: annotate on-slab caches nodelist locks

    We currently provide lockdep annotation for kmalloc caches, and also
    caches that have SLAB_DEBUG_OBJECTS enabled.  The reason for this is that
    we can quite frequently nest in the l3->list_lock lock, which is not
    something trivial to avoid.

    My proposal with this patch, is to extend this to caches whose slab
    management object lives within the slab as well ("on_slab").  The need for
    this arose in the context of testing kmemcg-slab patches.  With such
    patchset, we can have per-memcg kmalloc caches.  So the same path that led
    to nesting between kmalloc caches will could then lead to in-memcg
    nesting.  Because they are not annotated, lockdep will trigger.

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Cc: Christoph Lameter <cl@linux.com>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Frederic Weisbecker <fweisbec@redhat.com>
    Cc: Greg Thelen <gthelen@google.com>
    Cc: Johannes Weiner <hannes@cmpxchg.org>
    Cc: JoonSoo Kim <js1304@gmail.com>
    Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
    Cc: Mel Gorman <mel@csn.ul.ie>
    Cc: Michal Hocko <mhocko@suse.cz>
    Cc: Pekka Enberg <penberg@cs.helsinki.fi>
    Cc: Rik van Riel <riel@redhat.com>
    Cc: Suleiman Souhlal <suleiman@google.com>
    Cc: Tejun Heo <tj@kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 86535eac4b07f7f520b3ce99e7a05ca3d65ef7af
Author: Lai Jiangshan <laijs@cn.fujitsu.com>
Date:   Tue Dec 11 16:01:05 2012 -0800

    slub, hotplug: ignore unrelated node's hot-adding and hot-removing

    SLUB only focuses on the nodes which have normal memory and it ignores the
    other node's hot-adding and hot-removing.

    Aka: if some memory of a node which has no onlined memory is online, but
    this new memory onlined is not normal memory (for example, highmem), we
    should not allocate kmem_cache_node for SLUB.

    And if the last normal memory is offlined, but the node still has memory,
    we should remove kmem_cache_node for that node.  (The current code delays
    it when all of the memory is offlined)

    So we only do something when marg->status_change_nid_normal > 0.
    marg->status_change_nid is not suitable here.

    The same problem doesn't exist in SLAB, because SLAB allocates kmem_list3
    for every node even the node don't have normal memory, SLAB tolerates
    kmem_list3 on alien nodes.  SLUB only focuses on the nodes which have
    normal memory, it don't tolerate alien kmem_cache_node.  The patch makes
    SLUB become self-compatible and avoids WARNs and BUGs in rare conditions.

    Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Minchan Kim <minchan.kim@gmail.com>
    Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
    Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
    Cc: Rob Landley <rob@landley.net>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Jiang Liu <jiang.liu@huawei.com>
    Cc: Kay Sievers <kay.sievers@vrfy.org>
    Cc: Greg Kroah-Hartman <gregkh@suse.de>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Wen Congyang <wency@cn.fujitsu.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 4cfc1a792ed0f9ea3c8c3ec42a6cdc9f13e81475
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Nov 28 16:23:16 2012 +0000

    mm/sl[aou]b: Common alignment code

    Extract the code to do object alignment from the allocators.
    Do the alignment calculations in slab_common so that the
    __kmem_cache_create functions of the allocators do not have
    to deal with alignment.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit a3a653c7843526f48331ac0ab6a7281bb626986d
Author: Arnd Bergmann <arnd@arndb.de>
Date:   Fri Oct 5 16:55:20 2012 +0200

    mm/slob: use min_t() to compare ARCH_SLAB_MINALIGN

    The definition of ARCH_SLAB_MINALIGN is architecture dependent
    and can be either of type size_t or int. Comparing that value
    with ARCH_KMALLOC_MINALIGN can cause harmless warnings on
    platforms where they are different. Since both are always
    small positive integer numbers, using the size_t type to compare
    them is safe and gets rid of the warning.

    Without this patch, building ARM collie_defconfig results in:

    mm/slob.c: In function '__kmalloc_node':
    mm/slob.c:431:152: warning: comparison of distinct pointer types lacks a cast [enabled by default]
    mm/slob.c: In function 'kfree':
    mm/slob.c:484:153: warning: comparison of distinct pointer types lacks a cast [enabled by default]
    mm/slob.c: In function 'ksize':
    mm/slob.c:503:153: warning: comparison of distinct pointer types lacks a cast [enabled by default]

    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Arnd Bergmann <arnd@arndb.de>
    [ penberg@kernel.org: updates for master ]
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 2fe20fb142fad4c421d6c194d231bc1bcd26e546
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Mon Oct 22 09:04:31 2012 -0300

    mm/slob: Use free_page instead of put_page for page-size kmalloc allocations

    When freeing objects, the slob allocator currently free empty pages
    calling __free_pages(). However, page-size kmallocs are disposed
    using put_page() instead.

    It makes no sense to call put_page() for kernel pages that are provided
    by the object allocator, so we shouldn't be doing this ourselves.

    This is based on:
    commit d9b7f22623b5fa9cc189581dcdfb2ac605933bf4
    Author: Glauber Costa <glommer@parallels.com>
    slub: use free_page instead of put_page for freeing kmalloc allocation

    Cc: Christoph Lameter <cl@linux-foundation.org>
    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: Matt Mackall <mpm@selenic.com>
    Acked-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 8e9246ea192d82568864f8eec76b19d2f16eb019
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Nov 28 16:23:09 2012 +0000

    slab: Use the new create_boot_cache function to simplify bootstrap

    Simplify setup and reduce code in kmem_cache_init(). This allows us to
    get rid of initarray_cache as well as the manual setup code for
    the kmem_cache and kmem_cache_node arrays during bootstrap.

    We introduce a new bootstrap state "PARTIAL" for slab that signals the
    creation of a kmem_cache boot cache.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 59bd482a468007c1e071380c3015f27e94dc8ba0
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Nov 28 16:23:07 2012 +0000

    slub: Use statically allocated kmem_cache boot structure for bootstrap

    Simplify bootstrap by statically allocated two kmem_cache structures. These are
    freed after bootup is complete. Allows us to no longer worry about calculations
    of sizes of kmem_cache structures during bootstrap.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 1c303a4c619c245845c369f4a4ff6d4aa03c6abc
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Nov 28 16:23:07 2012 +0000

    mm, sl[au]b: create common functions for boot slab creation

    Use a special function to create kmalloc caches and use that function in
    SLAB and SLUB.

    Acked-by: Joonsoo Kim <js1304@gmail.com>
    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 5139da19e81586d05f6a98dbacf2ecf0308f552b
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Nov 28 16:23:01 2012 +0000

    slab: Simplify bootstrap

    The nodelists field in kmem_cache is pointing to the first unused
    object in the array field when bootstrap is complete.

    A problem with the current approach is that the statically sized
    kmem_cache structure use on boot can only contain NR_CPUS entries.
    If the number of nodes plus the number of cpus is greater then we
    would overwrite memory following the kmem_cache_boot definition.

    Increase the size of the array field to ensure that also the node
    pointers fit into the array field.

    Once we do that we no longer need the kmem_cache_nodelists
    array and we can then also use that structure elsewhere.

    Acked-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 0b0d7e20c4ad5a137dc8d9b26cc6cefd1801f45d
Author: Randy Dunlap <rdunlap@infradead.org>
Date:   Tue Nov 6 17:10:10 2012 -0800

    mm: fix slab.c kernel-doc warnings

    Fix new kernel-doc warnings in mm/slab.c:

    Warning(mm/slab.c:2358): No description found for parameter 'cachep'
    Warning(mm/slab.c:2358): Excess function parameter 'name' description in '__kmem_cache_create'
    Warning(mm/slab.c:2358): Excess function parameter 'size' description in '__kmem_cache_create'
    Warning(mm/slab.c:2358): Excess function parameter 'align' description in '__kmem_cache_create'
    Warning(mm/slab.c:2358): Excess function parameter 'ctor' description in '__kmem_cache_create'

    Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
    Cc:	Christoph Lameter <cl@linux-foundation.org>
    Cc:	Pekka Enberg <penberg@kernel.org>
    Cc:	Matt Mackall <mpm@selenic.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 58cff5069b9f617250408626f0797539ed6d9de7
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Nov 28 16:23:00 2012 +0000

    slub: Use correct cpu_slab on dead cpu

    Pass a kmem_cache_cpu pointer into unfreeze partials so that a different
    kmem_cache_cpu structure than the local one can be specified.

    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 2b77a3ac15cfe1c6402ebb0be3a6bee382419849
Author: Glauber Costa <glommer@parallels.com>
Date:   Wed Oct 17 15:36:51 2012 +0400

    slab: Ignore internal flags in cache creation

    Some flags are used internally by the allocators for management
    purposes. One example of that is the CFLGS_OFF_SLAB flag that slab uses
    to mark that the metadata for that cache is stored outside of the slab.

    No cache should ever pass those as a creation flags. We can just ignore
    this bit if it happens to be passed (such as when duplicating a cache in
    the kmem memcg patches).

    Because such flags can vary from allocator to allocator, we allow them
    to make their own decisions on that, defining SLAB_AVAILABLE_FLAGS with
    all flags that are valid at creation time.  Allocators that doesn't have
    any specific flag requirement should define that to mean all flags.

    Common code will mask out all flags not belonging to that set.

    Acked-by: Christoph Lameter <cl@linux.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit bad3c0a55ef60309f1bcf5c4aa2628b95df1afa6
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Fri Oct 19 09:33:12 2012 -0300

    mm/sl[aou]b: Move common kmem_cache_size() to slab.h

    This function is identically defined in all three allocators
    and it's trivial to move it to slab.h

    Since now it's static, inline, header-defined function
    this patch also drops the EXPORT_SYMBOL tag.

    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: Matt Mackall <mpm@selenic.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit deb9093c105fffb1d4a6374256ac5bd2570abf1c
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Fri Oct 19 09:33:11 2012 -0300

    mm/slob: Use object_size field in kmem_cache_size()

    Fields object_size and size are not the same: the latter might include
    slab metadata. Return object_size field in kmem_cache_size().
    Also, improve trace accuracy by correctly tracing reported size.

    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: Matt Mackall <mpm@selenic.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 8884f4c738bf4efa00dcd5cf3c08e129d366c909
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Fri Oct 19 09:33:10 2012 -0300

    mm/slob: Drop usage of page->private for storing page-sized allocations

    This field was being used to store size allocation so it could be
    retrieved by ksize(). However, it is a bad practice to not mark a page
    as a slab page and then use fields for special purposes.
    There is no need to store the allocated size and
    ksize() can simply return PAGE_SIZE << compound_order(page).

    Cc: Pekka Enberg <penberg@kernel.org>
    Cc: Matt Mackall <mpm@selenic.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 62cdeed10822685ad4d5764c9e7e2a7a7b971aef
Author: David Rientjes <rientjes@google.com>
Date:   Tue Sep 25 12:53:51 2012 -0700

    mm, slob: fix build breakage in __kmalloc_node_track_caller

    On Sat, 8 Sep 2012, Ezequiel Garcia wrote:

    > @@ -454,15 +455,35 @@ void *__kmalloc_node(size_t size, gfp_t gfp, int node)
    >  			gfp |= __GFP_COMP;
    >  		ret = slob_new_pages(gfp, order, node);
    >
    > -		trace_kmalloc_node(_RET_IP_, ret,
    > +		trace_kmalloc_node(caller, ret,
    >  				   size, PAGE_SIZE << order, gfp, node);
    >  	}
    >
    >  	kmemleak_alloc(ret, size, 1, gfp);
    >  	return ret;
    >  }
    > +
    > +void *__kmalloc_node(size_t size, gfp_t gfp, int node)
    > +{
    > +	return __do_kmalloc_node(size, gfp, node, _RET_IP_);
    > +}
    >  EXPORT_SYMBOL(__kmalloc_node);
    >
    > +#ifdef CONFIG_TRACING
    > +void *__kmalloc_track_caller(size_t size, gfp_t gfp, unsigned long caller)
    > +{
    > +	return __do_kmalloc_node(size, gfp, NUMA_NO_NODE, caller);
    > +}
    > +
    > +#ifdef CONFIG_NUMA
    > +void *__kmalloc_node_track_caller(size_t size, gfp_t gfpflags,
    > +					int node, unsigned long caller)
    > +{
    > +	return __do_kmalloc_node(size, gfp, node, caller);
    > +}
    > +#endif

    This breaks Pekka's slab/next tree with this:

    mm/slob.c: In function '__kmalloc_node_track_caller':
    mm/slob.c:488: error: 'gfp' undeclared (first use in this function)
    mm/slob.c:488: error: (Each undeclared identifier is reported only once
    mm/slob.c:488: error: for each function it appears in.)

    mm, slob: fix build breakage in __kmalloc_node_track_caller

    "mm, slob: Add support for kmalloc_track_caller()" breaks the build
    because gfp is undeclared.  Fix it.

    Acked-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit e1b0280a37231116dc842c518423e2f69e2dd0f8
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Sat Sep 8 17:47:53 2012 -0300

    mm, slob: Add support for kmalloc_track_caller()

    Currently slob falls back to regular kmalloc for this case.
    With this patch kmalloc_track_caller() is correctly implemented,
    thus tracing the specified caller.

    This is important to trace accurately allocations performed by
    krealloc, kstrdup, kmemdup, etc.

    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 02afe17fca994df167f8382a2f553a13e8216fc6
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Sat Sep 8 17:47:51 2012 -0300

    mm, slob: Use NUMA_NO_NODE instead of -1

    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: Paul Reioux <reioux@gmail.com>

    Conflicts:
    	mm/slob.c

commit bed6c47176e9a37292184adc108b381c543fd48d
Author: Glauber Costa <glommer@parallels.com>
Date:   Mon Oct 22 18:05:36 2012 +0400

    slub: Commonize slab_cache field in struct page

    Right now, slab and slub have fields in struct page to derive which
    cache a page belongs to, but they do it slightly differently.

    slab uses a field called slab_cache, that lives in the third double
    word. slub, uses a field called "slab", living outside of the
    doublewords area.

    Ideally, we could use the same field for this. Since slub heavily makes
    use of the doubleword region, there isn't really much room to move
    slub's slab_cache field around. Since slab does not have such strict
    placement restrictions, we can move it outside the doubleword area.

    The naming used by slab, "slab_cache", is less confusing, and it is
    preferred over slub's generic "slab".

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    CC: David Rientjes <rientjes@google.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 8e2caf465a834b8f1320d8019fb3e1e58ab9f626
Author: Glauber Costa <glommer@parallels.com>
Date:   Fri Oct 19 18:20:27 2012 +0400

    sl[au]b: Process slabinfo_show in common code

    With all the infrastructure in place, we can now have slabinfo_show
    done from slab_common.c. A cache-specific function is called to grab
    information about the cache itself, since that is still heavily
    dependent on the implementation. But with the values produced by it, all
    the printing and handling is done from common code.

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    CC: Christoph Lameter <cl@linux.com>
    CC: David Rientjes <rientjes@google.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit d9216138f96c8dcfca84878af50657da8c6b6a89
Author: Glauber Costa <glommer@parallels.com>
Date:   Fri Oct 19 18:20:26 2012 +0400

    mm/sl[au]b: Move print_slabinfo_header to slab_common.c

    The header format is highly similar between slab and slub. The main
    difference lays in the fact that slab may optionally have statistics
    added here in case of CONFIG_SLAB_DEBUG, while the slub will stick them
    somewhere else.

    By making sure that information conditionally lives inside a
    globally-visible CONFIG_DEBUG_SLAB switch, we can move the header
    printing to a common location.

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    CC: David Rientjes <rientjes@google.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 91edc846f4fe0629464ce2c07f1eaa7d75723fd2
Author: Glauber Costa <glommer@parallels.com>
Date:   Fri Oct 19 18:20:25 2012 +0400

    mm/sl[au]b: Move slabinfo processing to slab_common.c

    This patch moves all the common machinery to slabinfo processing
    to slab_common.c. We can do better by noticing that the output is
    heavily common, and having the allocators to just provide finished
    information about this. But after this first step, this can be done
    easier.

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    CC: David Rientjes <rientjes@google.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 24ee46ce49a57de604496735079f65226a40776d
Author: Jiri Kosina <jkosina@suse.cz>
Date:   Mon Oct 8 09:26:01 2012 +0200

    mm, slab: release slab_mutex earlier in kmem_cache_destroy()

    Commit 1331e7a1bbe1 ("rcu: Remove _rcu_barrier() dependency on
    __stop_machine()") introduced slab_mutex -> cpu_hotplug.lock dependency
    through kmem_cache_destroy() -> rcu_barrier() -> _rcu_barrier() ->
    get_online_cpus().

    Lockdep thinks that this might actually result in ABBA deadlock,
    and reports it as below:

    === [ cut here ] ===
     ======================================================
     [ INFO: possible circular locking dependency detected ]
     3.6.0-rc5-00004-g0d8ee37 #143 Not tainted
     -------------------------------------------------------
     kworker/u:2/40 is trying to acquire lock:
      (rcu_sched_state.barrier_mutex){+.+...}, at: [<ffffffff810f2126>] _rcu_barrier+0x26/0x1e0

     but task is already holding lock:
      (slab_mutex){+.+.+.}, at: [<ffffffff81176e15>] kmem_cache_destroy+0x45/0xe0

     which lock already depends on the new lock.

     the existing dependency chain (in reverse order) is:

     -> #2 (slab_mutex){+.+.+.}:
            [<ffffffff810ae1e2>] validate_chain+0x632/0x720
            [<ffffffff810ae5d9>] __lock_acquire+0x309/0x530
            [<ffffffff810ae921>] lock_acquire+0x121/0x190
            [<ffffffff8155d4cc>] __mutex_lock_common+0x5c/0x450
            [<ffffffff8155d9ee>] mutex_lock_nested+0x3e/0x50
            [<ffffffff81558cb5>] cpuup_callback+0x2f/0xbe
            [<ffffffff81564b83>] notifier_call_chain+0x93/0x140
            [<ffffffff81076f89>] __raw_notifier_call_chain+0x9/0x10
            [<ffffffff8155719d>] _cpu_up+0xba/0x14e
            [<ffffffff815572ed>] cpu_up+0xbc/0x117
            [<ffffffff81ae05e3>] smp_init+0x6b/0x9f
            [<ffffffff81ac47d6>] kernel_init+0x147/0x1dc
            [<ffffffff8156ab44>] kernel_thread_helper+0x4/0x10

     -> #1 (cpu_hotplug.lock){+.+.+.}:
            [<ffffffff810ae1e2>] validate_chain+0x632/0x720
            [<ffffffff810ae5d9>] __lock_acquire+0x309/0x530
            [<ffffffff810ae921>] lock_acquire+0x121/0x190
            [<ffffffff8155d4cc>] __mutex_lock_common+0x5c/0x450
            [<ffffffff8155d9ee>] mutex_lock_nested+0x3e/0x50
            [<ffffffff81049197>] get_online_cpus+0x37/0x50
            [<ffffffff810f21bb>] _rcu_barrier+0xbb/0x1e0
            [<ffffffff810f22f0>] rcu_barrier_sched+0x10/0x20
            [<ffffffff810f2309>] rcu_barrier+0x9/0x10
            [<ffffffff8118c129>] deactivate_locked_super+0x49/0x90
            [<ffffffff8118cc01>] deactivate_super+0x61/0x70
            [<ffffffff811aaaa7>] mntput_no_expire+0x127/0x180
            [<ffffffff811ab49e>] sys_umount+0x6e/0xd0
            [<ffffffff81569979>] system_call_fastpath+0x16/0x1b

     -> #0 (rcu_sched_state.barrier_mutex){+.+...}:
            [<ffffffff810adb4e>] check_prev_add+0x3de/0x440
            [<ffffffff810ae1e2>] validate_chain+0x632/0x720
            [<ffffffff810ae5d9>] __lock_acquire+0x309/0x530
            [<ffffffff810ae921>] lock_acquire+0x121/0x190
            [<ffffffff8155d4cc>] __mutex_lock_common+0x5c/0x450
            [<ffffffff8155d9ee>] mutex_lock_nested+0x3e/0x50
            [<ffffffff810f2126>] _rcu_barrier+0x26/0x1e0
            [<ffffffff810f22f0>] rcu_barrier_sched+0x10/0x20
            [<ffffffff810f2309>] rcu_barrier+0x9/0x10
            [<ffffffff81176ea1>] kmem_cache_destroy+0xd1/0xe0
            [<ffffffffa04c3154>] nf_conntrack_cleanup_net+0xe4/0x110 [nf_conntrack]
            [<ffffffffa04c31aa>] nf_conntrack_cleanup+0x2a/0x70 [nf_conntrack]
            [<ffffffffa04c42ce>] nf_conntrack_net_exit+0x5e/0x80 [nf_conntrack]
            [<ffffffff81454b79>] ops_exit_list+0x39/0x60
            [<ffffffff814551ab>] cleanup_net+0xfb/0x1b0
            [<ffffffff8106917b>] process_one_work+0x26b/0x4c0
            [<ffffffff81069f3e>] worker_thread+0x12e/0x320
            [<ffffffff8106f73e>] kthread+0x9e/0xb0
            [<ffffffff8156ab44>] kernel_thread_helper+0x4/0x10

     other info that might help us debug this:

     Chain exists of:
       rcu_sched_state.barrier_mutex --> cpu_hotplug.lock --> slab_mutex

      Possible unsafe locking scenario:

            CPU0                    CPU1
            ----                    ----
       lock(slab_mutex);
                                    lock(cpu_hotplug.lock);
                                    lock(slab_mutex);
       lock(rcu_sched_state.barrier_mutex);

      *** DEADLOCK ***
    === [ cut here ] ===

    This is actually a false positive. Lockdep has no way of knowing the fact
    that the ABBA can actually never happen, because of special semantics of
    cpu_hotplug.refcount and its handling in cpu_hotplug_begin(); the mutual
    exclusion there is not achieved through mutex, but through
    cpu_hotplug.refcount.

    The "neither cpu_up() nor cpu_down() will proceed past cpu_hotplug_begin()
    until everyone who called get_online_cpus() will call put_online_cpus()"
    semantics is totally invisible to lockdep.

    This patch therefore moves the unlock of slab_mutex so that rcu_barrier()
    is being called with it unlocked. It has two advantages:

    - it slightly reduces hold time of slab_mutex; as it's used to protect
      the cachep list, it's not necessary to hold it over kmem_cache_free()
      call any more
    - it silences the lockdep false positive warning, as it avoids lockdep ever
      learning about slab_mutex -> cpu_hotplug.lock dependency

    Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Reviewed-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Jiri Kosina <jkosina@suse.cz>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit b722f8e34e0cc01aead1a42211d2fbccae968dca
Author: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Date:   Sun Sep 30 17:28:25 2012 +0900

    slab: Fix build failure in __kmem_cache_create()

    Fix build failure with CONFIG_DEBUG_SLAB=y && CONFIG_DEBUG_PAGEALLOC=y caused
    by commit 8a13a4cc "mm/sl[aou]b: Shrink __kmem_cache_create() parameter lists".

    mm/slab.c: In function '__kmem_cache_create':
    mm/slab.c:2474: error: 'align' undeclared (first use in this function)
    mm/slab.c:2474: error: (Each undeclared identifier is reported only once
    mm/slab.c:2474: error: for each function it appears in.)
    make[1]: *** [mm/slab.o] Error 1
    make: *** [mm] Error 2

    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 291d8bda9e26d4ad8526c092ae1a3dedee63dd3e
Author: Pekka Enberg <penberg@kernel.org>
Date:   Sat Sep 29 10:00:59 2012 +0300

    Revert "mm/slab: Fix kmem_cache_alloc_node_trace() declaration"

    This reverts commit 1e5965bf1f018cc30a4659fa3f1a40146e4276f6. Ezequiel
    Garcia has a better fix.

commit bde83ff142fe8ca49b39e4254f5231fc13cd0c64
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Tue Sep 25 08:07:09 2012 -0300

    mm/slab: Fix kmem_cache_alloc_node_trace() declaration

    The bug was introduced in commit 4052147c0afa ("mm, slab: Match SLAB
    and SLUB kmem_cache_alloc_xxx_trace() prototype").

    Reported-by: Fengguang Wu <fengguang.wu@intel.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 4a0576b73ac5c00fc58b11816669b1f7dbaae26c
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Tue Sep 25 08:07:08 2012 -0300

    mm/slab: Fix typo _RET_IP -> _RET_IP_

    The bug was introduced by commit 7c0cb9c64f83 ("mm, slab: Replace
    'caller' type, void* -> unsigned long").

    Reported-by: Fengguang Wu <fengguang.wu@intel.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit ed8f5dbf68f64dbaf4a92c0b7b2f42e4e8437dfa
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Sat Sep 8 17:47:57 2012 -0300

    mm, slab: Rename __cache_alloc() -> slab_alloc()

    This patch does not fix anything and its only goal is to
    produce common code between SLAB and SLUB.

    Cc: Christoph Lameter <cl@linux.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit e5b920a8905c4e02435faee51aa7d2de150899de
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Sat Sep 8 17:47:56 2012 -0300

    mm, slab: Match SLAB and SLUB kmem_cache_alloc_xxx_trace() prototype

    This long (seemingly unnecessary) patch does not fix anything and
    its only goal is to produce common code between SLAB and SLUB.

    Cc: Christoph Lameter <cl@linux.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 920cd114076a9bfbe0eee4c5dab21dae95f5a826
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Sat Sep 8 17:47:55 2012 -0300

    mm, slab: Replace 'caller' type, void* -> unsigned long

    This allows to use _RET_IP_ instead of builtin_address(0), thus
    achiveing implementation consistency in all three allocators.
    Though maybe a nitpick, the real goal behind this patch is
    to be able to obtain common code between SLAB and SLUB.

    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 888a3be007559652f17dd47c619ecf96694e59f2
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Sat Sep 8 17:47:52 2012 -0300

    mm, slab: Remove silly function slab_buffer_size()

    This function is seldom used, and can be simply replaced with cachep->size.

    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 9dd36aa16c32cccb007dbcddfd0d9b6156d7c53f
Author: Joonsoo Kim <js1304@gmail.com>
Date:   Thu Aug 16 00:02:40 2012 +0900

    slub: remove one code path and reduce lock contention in __slab_free()

    When we try to free object, there is some of case that we need
    to take a node lock. This is the necessary step for preventing a race.
    After taking a lock, then we try to cmpxchg_double_slab().
    But, there is a possible scenario that cmpxchg_double_slab() is failed
    with taking a lock. Following example explains it.

    CPU A               CPU B
    need lock
    ...                 need lock
    ...                 lock!!
    lock..but spin      free success
    spin...             unlock
    lock!!
    free fail

    In this case, retry with taking a lock is occured in CPU A.
    I think that in this case for CPU A,
    "release a lock first, and re-take a lock if necessary" is preferable way.

    There are two reasons for this.

    First, this makes __slab_free()'s logic somehow simple.
    With this patch, 'was_frozen = 1' is "always" handled without taking a lock.
    So we can remove one code path.

    Second, it may reduce lock contention.
    When we do retrying, status of slab is already changed,
    so we don't need a lock anymore in almost every case.
    "release a lock first, and re-take a lock if necessary" policy is
    helpful to this.

    Signed-off-by: Joonsoo Kim <js1304@gmail.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 1d86bd9b36c470d02342e69b76b378cd44cd840a
Author: Fengguang Wu <fengguang.wu@intel.com>
Date:   Fri Sep 28 16:34:05 2012 +0800

    slub: init_kmem_cache_cpus() and put_cpu_partial() can be static

    Acked-by: Glauber Costa <glommer@parallels.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Fengguang Wu <fengguang.wu@intel.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit af7d7279dc03e6289eb1d8cd7dad8559c1e472a7
Author: Ezequiel Garcia <elezegarcia@gmail.com>
Date:   Sat Sep 8 17:47:58 2012 -0300

    mm, slub: Rename slab_alloc() -> slab_alloc_node() to match SLAB

    This patch does not fix anything, and its only goal is to enable us
    to obtain some common code between SLAB and SLUB.
    Neither behavior nor produced code is affected.

    Cc: Christoph Lameter <cl@linux.com>
    Signed-off-by: Ezequiel Garcia <elezegarcia@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit f3208b16bab2b7d422bc81b6e8057b76a961b558
Author: Dave Jones <davej@redhat.com>
Date:   Tue Sep 18 15:54:12 2012 -0400

    mm, sl[au]b: Taint kernel when we detect a corrupted slab

    It doesn't seem worth adding a new taint flag for this, so just re-use
    the one from 'bad page'

    Acked-by: Christoph Lameter <cl@linux.com> # SLUB
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Dave Jones <davej@redhat.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 24f1ecd52f3e849499383918bd4af936d674628a
Author: Christoph Lameter <cl@linux.com>
Date:   Tue Sep 11 19:49:38 2012 +0000

    slab: Only define slab_error for DEBUG

    On Tue, 11 Sep 2012, Stephen Rothwell wrote:
    > After merging the final tree, today's linux-next build (sparc64 defconfig)
    > produced this warning:
    >
    > mm/slab.c:808:13: warning: '__slab_error' defined but not used [-Wunused-function]
    >
    > Introduced by commit 945cf2b6199b ("mm/sl[aou]b: Extract a common
    > function for kmem_cache_destroy").  All uses of slab_error() are now
    > guarded by DEBUG.

    There is no use case left for slab builds without DEBUG.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit ddbc4fb34ab323bd63bb676c3894ea90df376054
Author: Joonsoo Kim <js1304@gmail.com>
Date:   Mon Sep 17 14:09:06 2012 -0700

    slab: fix starting index for finding another object

    In array cache, there is a object at index 0, check it.

    Signed-off-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Mel Gorman <mgorman@suse.de>
    Cc: David Miller <davem@davemloft.net>
    Cc: Chuck Lever <chuck.lever@oracle.com>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit f14feeca18e72e3dabdd26be7a5014a1b5efa8c6
Author: Mel Gorman <mgorman@suse.de>
Date:   Mon Sep 17 14:09:03 2012 -0700

    slab: do ClearSlabPfmemalloc() for all pages of slab

    Right now, we call ClearSlabPfmemalloc() for first page of slab when we
    clear SlabPfmemalloc flag.  This is fine for most swap-over-network use
    cases as it is expected that order-0 pages are in use.  Unfortunately it
    is possible that that __ac_put_obj() checks SlabPfmemalloc on a tail
    page and while this is harmless, it is sloppy.  This patch ensures that
    the head page is always used.

    This problem was originally identified by Joonsoo Kim.

    [js1304@gmail.com: Original implementation and problem identification]
    Signed-off-by: Mel Gorman <mgorman@suse.de>
    Cc: David Miller <davem@davemloft.net>
    Cc: Chuck Lever <chuck.lever@oracle.com>
    Cc: Joonsoo Kim <js1304@gmail.com>
    Cc: David Rientjes <rientjes@google.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 6fe84d096e38947dbb8df97fa5ea5777336bc6b8
Author: Michael Wang <wangyun@linux.vnet.ibm.com>
Date:   Wed Sep 5 10:33:18 2012 +0800

    slab: fix the DEADLOCK issue on l3 alien lock

    DEADLOCK will be report while running a kernel with NUMA and LOCKDEP enabled,
    the process of this fake report is:

    	   kmem_cache_free()	//free obj in cachep
    	-> cache_free_alien()	//acquire cachep's l3 alien lock
    	-> __drain_alien_cache()
    	-> free_block()
    	-> slab_destroy()
    	-> kmem_cache_free()	//free slab in cachep->slabp_cache
    	-> cache_free_alien()	//acquire cachep->slabp_cache's l3 alien lock

    Since the cachep and cachep->slabp_cache's l3 alien are in the same lock class,
    fake report generated.

    This should not happen since we already have init_lock_keys() which will
    reassign the lock class for both l3 list and l3 alien.

    However, init_lock_keys() was invoked at a wrong position which is before we
    invoke enable_cpucache() on each cache.

    Since until set slab_state to be FULL, we won't invoke enable_cpucache()
    on caches to build their l3 alien while creating them, so although we invoked
    init_lock_keys(), the l3 alien lock class won't change since we don't have
    them until invoked enable_cpucache() later.

    This patch will invoke init_lock_keys() after we done enable_cpucache()
    instead of before to avoid the fake DEADLOCK report.

    Michael traced the problem back to a commit in release 3.0.0:

    commit 30765b92ada267c5395fc788623cb15233276f5c
    Author: Peter Zijlstra <peterz@infradead.org>
    Date:   Thu Jul 28 23:22:56 2011 +0200

        slab, lockdep: Annotate the locks before using them

        Fernando found we hit the regular OFF_SLAB 'recursion' before we
        annotate the locks, cure this.

        The relevant portion of the stack-trace:

        > [    0.000000]  [<c085e24f>] rt_spin_lock+0x50/0x56
        > [    0.000000]  [<c04fb406>] __cache_free+0x43/0xc3
        > [    0.000000]  [<c04fb23f>] kmem_cache_free+0x6c/0xdc
        > [    0.000000]  [<c04fb2fe>] slab_destroy+0x4f/0x53
        > [    0.000000]  [<c04fb396>] free_block+0x94/0xc1
        > [    0.000000]  [<c04fc551>] do_tune_cpucache+0x10b/0x2bb
        > [    0.000000]  [<c04fc8dc>] enable_cpucache+0x7b/0xa7
        > [    0.000000]  [<c0bd9d3c>] kmem_cache_init_late+0x1f/0x61
        > [    0.000000]  [<c0bba687>] start_kernel+0x24c/0x363
        > [    0.000000]  [<c0bba0ba>] i386_start_kernel+0xa9/0xaf

        Reported-by: Fernando Lopez-Lezcano <nando@ccrma.Stanford.EDU>
        Acked-by: Pekka Enberg <penberg@kernel.org>
        Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
        Link: http://lkml.kernel.org/r/1311888176.2617.379.camel@laptop
        Signed-off-by: Ingo Molnar <mingo@elte.hu>

    The commit moved init_lock_keys() before we build up the alien, so we
    failed to reclass it.

    Cc: <stable@vger.kernel.org> # 3.0+
    Acked-by: Christoph Lameter <cl@linux.com>
    Tested-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
    Signed-off-by: Michael Wang <wangyun@linux.vnet.ibm.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit e12f9375cecb264138038ae30e1bc9ccdedc551c
Author: Joonsoo Kim <js1304@gmail.com>
Date:   Mon Sep 17 14:09:09 2012 -0700

    slub: consider pfmemalloc_match() in get_partial_node()

    get_partial() is currently not checking pfmemalloc_match() meaning that
    it is possible for pfmemalloc pages to leak to non-pfmemalloc users.
    This is a problem in the following situation.  Assume that there is a
    request from normal allocation and there are no objects in the per-cpu
    cache and no node-partial slab.

    In this case, slab_alloc enters the slow path and new_slab_objects() is
    called which may return a PFMEMALLOC page.  As the current user is not
    allowed to access PFMEMALLOC page, deactivate_slab() is called
    ([5091b74a: mm: slub: optimise the SLUB fast path to avoid pfmemalloc
    checks]) and returns an object from PFMEMALLOC page.

    Next time, when we get another request from normal allocation,
    slab_alloc() enters the slow-path and calls new_slab_objects().  In
    new_slab_objects(), we call get_partial() and get a partial slab which
    was just deactivated but is a pfmemalloc page.  We extract one object
    from it and re-deactivate.

      "deactivate -> re-get in get_partial -> re-deactivate" occures repeatedly.

    As a result, access to PFMEMALLOC page is not properly restricted and it
    can cause a performance degradation due to frequent deactivation.
    deactivation frequently.

    This patch changes get_partial_node() to take pfmemalloc_match() into
    account and prevents the "deactivate -> re-get in get_partial()
    scenario.  Instead, new_slab() is called.

    Signed-off-by: Joonsoo Kim <js1304@gmail.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Mel Gorman <mgorman@suse.de>
    Cc: David Miller <davem@davemloft.net>
    Cc: Chuck Lever <chuck.lever@oracle.com>
    Cc: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 9221b0174a036a98c084cd13d7d8049c332e6e39
Author: Christoph Lameter <cl@linux.com>
Date:   Sat Sep 8 18:27:10 2012 +0000

    slub: Zero initial memory segment for kmem_cache and kmem_cache_node

    Tony Luck reported the following problem on IA-64:

      Worked fine yesterday on next-20120905, crashes today. First sign of
      trouble was an unaligned access, then a NULL dereference. SL*B related
      bits of my config:

      CONFIG_SLUB_DEBUG=y
      # CONFIG_SLAB is not set
      CONFIG_SLUB=y
      CONFIG_SLABINFO=y
      # CONFIG_SLUB_DEBUG_ON is not set
      # CONFIG_SLUB_STATS is not set

      And he console log.

      PID hash table entries: 4096 (order: 1, 32768 bytes)
      Dentry cache hash table entries: 262144 (order: 7, 2097152 bytes)
      Inode-cache hash table entries: 131072 (order: 6, 1048576 bytes)
      Memory: 2047920k/2086064k available (13992k code, 38144k reserved,
      6012k data, 880k init)
      kernel unaligned access to 0xca2ffc55fb373e95, ip=0xa0000001001be550
      swapper[0]: error during unaligned kernel access
       -1 [1]
      Modules linked in:

      Pid: 0, CPU 0, comm:              swapper
      psr : 00001010084a2018 ifs : 800000000000060f ip  :
      [<a0000001001be550>]    Not tainted (3.6.0-rc4-zx1-smp-next-20120906)
      ip is at new_slab+0x90/0x680
      unat: 0000000000000000 pfs : 000000000000060f rsc : 0000000000000003
      rnat: 9666960159966a59 bsps: a0000001001441c0 pr  : 9666960159965a59
      ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c8a70433f
      csd : 0000000000000000 ssd : 0000000000000000
      b0  : a0000001001be500 b6  : a00000010112cb20 b7  : a0000001011660a0
      f6  : 0fff7f0f0f0f0e54f0000 f7  : 0ffe8c5c1000000000000
      f8  : 1000d8000000000000000 f9  : 100068800000000000000
      f10 : 10005f0f0f0f0e54f0000 f11 : 1003e0000000000000078
      r1  : a00000010155eef0 r2  : 0000000000000000 r3  : fffffffffffc1638
      r8  : e0000040600081b8 r9  : ca2ffc55fb373e95 r10 : 0000000000000000
      r11 : e000004040001646 r12 : a000000101287e20 r13 : a000000101280000
      r14 : 0000000000004000 r15 : 0000000000000078 r16 : ca2ffc55fb373e75
      r17 : e000004040040000 r18 : fffffffffffc1646 r19 : e000004040001646
      r20 : fffffffffffc15f8 r21 : 000000000000004d r22 : a00000010132fa68
      r23 : 00000000000000ed r24 : 0000000000000000 r25 : 0000000000000000
      r26 : 0000000000000001 r27 : a0000001012b8500 r28 : a00000010135f4a0
      r29 : 0000000000000000 r30 : 0000000000000000 r31 : 0000000000000001
      Unable to handle kernel NULL pointer dereference (address
      0000000000000018)
      swapper[0]: Oops 11003706212352 [2]
      Modules linked in:

      Pid: 0, CPU 0, comm:              swapper
      psr : 0000121008022018 ifs : 800000000000cc18 ip  :
      [<a0000001004dc8f1>]    Not tainted (3.6.0-rc4-zx1-smp-next-20120906)
      ip is at __copy_user+0x891/0x960
      unat: 0000000000000000 pfs : 0000000000000813 rsc : 0000000000000003
      rnat: 0000000000000000 bsps: 0000000000000000 pr  : 9666960159961765
      ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c0270033f
      csd : 0000000000000000 ssd : 0000000000000000
      b0  : a00000010004b550 b6  : a00000010004b740 b7  : a00000010000c750
      f6  : 000000000000000000000 f7  : 1003e9e3779b97f4a7c16
      f8  : 1003e0a00000010001550 f9  : 100068800000000000000
      f10 : 10005f0f0f0f0e54f0000 f11 : 1003e0000000000000078
      r1  : a00000010155eef0 r2  : a0000001012870b0 r3  : a0000001012870b8
      r8  : 0000000000000298 r9  : 0000000000000013 r10 : 0000000000000000
      r11 : 9666960159961a65 r12 : a000000101287010 r13 : a000000101280000
      r14 : a000000101287068 r15 : a000000101287080 r16 : 0000000000000298
      r17 : 0000000000000010 r18 : 0000000000000018 r19 : a000000101287310
      r20 : 0000000000000290 r21 : 0000000000000000 r22 : 0000000000000000
      r23 : a000000101386f58 r24 : 0000000000000000 r25 : 000000007fffffff
      r26 : a000000101287078 r27 : a0000001013c69b0 r28 : 0000000000000000
      r29 : 0000000000000014 r30 : 0000000000000000 r31 : 0000000000000813

    Sedat Dilek and Hugh Dickins reported similar problems as well.

    Earlier patches in the common set moved the zeroing of the kmem_cache
    structure into common code. See "Move allocation of kmem_cache into
    common code".

    The allocation for the two special structures is still done from SLUB
    specific code but no zeroing is done since the cache creation functions
    used to zero. This now needs to be updated so that the structures are
    zeroed during allocation in kmem_cache_init().  Otherwise random pointer
    values may be followed.

    Reported-by: Tony Luck <tony.luck@intel.com>
    Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
    Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
    Reported-by: Hugh Dickins <hughd@google.com>
    Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit b1d7c58c527dff90cb970fbbb864327dc58d80ed
Author: Pekka Enberg <penberg@kernel.org>
Date:   Wed Sep 5 12:07:44 2012 +0300

    Revert "mm/sl[aou]b: Move sysfs_slab_add to common"

    This reverts commit 96d17b7be0a9849d381442030886211dbb2a7061 which
    caused the following errors at boot:

      [    1.114885] kobject (ffff88001a802578): tried to init an initialized object, something is seriously wrong.
      [    1.114885] Pid: 1, comm: swapper/0 Tainted: G        W    3.6.0-rc1+ #6
      [    1.114885] Call Trace:
      [    1.114885]  [<ffffffff81273f37>] kobject_init+0x87/0xa0
      [    1.115555]  [<ffffffff8127426a>] kobject_init_and_add+0x2a/0x90
      [    1.115555]  [<ffffffff8127c870>] ? sprintf+0x40/0x50
      [    1.115555]  [<ffffffff81124c60>] sysfs_slab_add+0x80/0x210
      [    1.115555]  [<ffffffff81100175>] kmem_cache_create+0xa5/0x250
      [    1.115555]  [<ffffffff81cf24cd>] ? md_init+0x144/0x144
      [    1.115555]  [<ffffffff81cf25b6>] local_init+0xa4/0x11b
      [    1.115555]  [<ffffffff81cf24e1>] dm_init+0x14/0x45
      [    1.115836]  [<ffffffff810001ba>] do_one_initcall+0x3a/0x160
      [    1.116834]  [<ffffffff81cc2c90>] kernel_init+0x133/0x1b7
      [    1.117835]  [<ffffffff81cc25c4>] ? do_early_param+0x86/0x86
      [    1.117835]  [<ffffffff8171aff4>] kernel_thread_helper+0x4/0x10
      [    1.118401]  [<ffffffff81cc2b5d>] ? start_kernel+0x33f/0x33f
      [    1.119832]  [<ffffffff8171aff0>] ? gs_change+0xb/0xb
      [    1.120325] ------------[ cut here ]------------
      [    1.120835] WARNING: at fs/sysfs/dir.c:536 sysfs_add_one+0xc1/0xf0()
      [    1.121437] sysfs: cannot create duplicate filename '/kernel/slab/:t-0000016'
      [    1.121831] Modules linked in:
      [    1.122138] Pid: 1, comm: swapper/0 Tainted: G        W    3.6.0-rc1+ #6
      [    1.122831] Call Trace:
      [    1.123074]  [<ffffffff81195ce1>] ? sysfs_add_one+0xc1/0xf0
      [    1.123833]  [<ffffffff8103adfa>] warn_slowpath_common+0x7a/0xb0
      [    1.124405]  [<ffffffff8103aed1>] warn_slowpath_fmt+0x41/0x50
      [    1.124832]  [<ffffffff81195ce1>] sysfs_add_one+0xc1/0xf0
      [    1.125337]  [<ffffffff81195eb3>] create_dir+0x73/0xd0
      [    1.125832]  [<ffffffff81196221>] sysfs_create_dir+0x81/0xe0
      [    1.126363]  [<ffffffff81273d3d>] kobject_add_internal+0x9d/0x210
      [    1.126832]  [<ffffffff812742a3>] kobject_init_and_add+0x63/0x90
      [    1.127406]  [<ffffffff81124c60>] sysfs_slab_add+0x80/0x210
      [    1.127832]  [<ffffffff81100175>] kmem_cache_create+0xa5/0x250
      [    1.128384]  [<ffffffff81cf24cd>] ? md_init+0x144/0x144
      [    1.128833]  [<ffffffff81cf25b6>] local_init+0xa4/0x11b
      [    1.129831]  [<ffffffff81cf24e1>] dm_init+0x14/0x45
      [    1.130305]  [<ffffffff810001ba>] do_one_initcall+0x3a/0x160
      [    1.130831]  [<ffffffff81cc2c90>] kernel_init+0x133/0x1b7
      [    1.131351]  [<ffffffff81cc25c4>] ? do_early_param+0x86/0x86
      [    1.131830]  [<ffffffff8171aff4>] kernel_thread_helper+0x4/0x10
      [    1.132392]  [<ffffffff81cc2b5d>] ? start_kernel+0x33f/0x33f
      [    1.132830]  [<ffffffff8171aff0>] ? gs_change+0xb/0xb
      [    1.133315] ---[ end trace 2703540871c8fab7 ]---
      [    1.133830] ------------[ cut here ]------------
      [    1.134274] WARNING: at lib/kobject.c:196 kobject_add_internal+0x1f5/0x210()
      [    1.134829] kobject_add_internal failed for :t-0000016 with -EEXIST, don't try to register things with the same name in the same directory.
      [    1.135829] Modules linked in:
      [    1.136135] Pid: 1, comm: swapper/0 Tainted: G        W    3.6.0-rc1+ #6
      [    1.136828] Call Trace:
      [    1.137071]  [<ffffffff81273e95>] ? kobject_add_internal+0x1f5/0x210
      [    1.137830]  [<ffffffff8103adfa>] warn_slowpath_common+0x7a/0xb0
      [    1.138402]  [<ffffffff8103aed1>] warn_slowpath_fmt+0x41/0x50
      [    1.138830]  [<ffffffff811955a3>] ? release_sysfs_dirent+0x73/0xf0
      [    1.139419]  [<ffffffff81273e95>] kobject_add_internal+0x1f5/0x210
      [    1.139830]  [<ffffffff812742a3>] kobject_init_and_add+0x63/0x90
      [    1.140429]  [<ffffffff81124c60>] sysfs_slab_add+0x80/0x210
      [    1.140830]  [<ffffffff81100175>] kmem_cache_create+0xa5/0x250
      [    1.141829]  [<ffffffff81cf24cd>] ? md_init+0x144/0x144
      [    1.142307]  [<ffffffff81cf25b6>] local_init+0xa4/0x11b
      [    1.142829]  [<ffffffff81cf24e1>] dm_init+0x14/0x45
      [    1.143307]  [<ffffffff810001ba>] do_one_initcall+0x3a/0x160
      [    1.143829]  [<ffffffff81cc2c90>] kernel_init+0x133/0x1b7
      [    1.144352]  [<ffffffff81cc25c4>] ? do_early_param+0x86/0x86
      [    1.144829]  [<ffffffff8171aff4>] kernel_thread_helper+0x4/0x10
      [    1.145405]  [<ffffffff81cc2b5d>] ? start_kernel+0x33f/0x33f
      [    1.145828]  [<ffffffff8171aff0>] ? gs_change+0xb/0xb
      [    1.146313] ---[ end trace 2703540871c8fab8 ]---

    Conflicts:

    	mm/slub.c

    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 59f8b8cadb6ed338504fcb0df13f17c480f2db87
Author: Christoph Lameter <cl@linux.com>
Date:   Tue Sep 4 23:38:33 2012 +0000

    mm/sl[aou]b: Move kmem_cache refcounting to common code

    Get rid of the refcount stuff in the allocators and do that part of
    kmem_cache management in the common code.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 760008f785e4b7982f442baf1f22c51eb624f851
Author: Christoph Lameter <cl@linux.com>
Date:   Tue Sep 4 23:18:33 2012 +0000

    mm/sl[aou]b: Shrink __kmem_cache_create() parameter lists

    Do the initial settings of the fields in common code. This will allow us
    to push more processing into common code later and improve readability.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit d2e8be2a8509dd6ecd32ffe55495415e1cdeb1b3
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Sep 5 00:20:34 2012 +0000

    mm/sl[aou]b: Move kmem_cache allocations into common code

    Shift the allocations to common code. That way the allocation and
    freeing of the kmem_cache structures is handled by common code.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 47735be4c74cb04824ce5c4a538e27a889be5918
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Sep 5 00:18:32 2012 +0000

    mm/sl[aou]b: Move sysfs_slab_add to common

    Simplify locking by moving the slab_add_sysfs after all locks have been
    dropped. Eases the upcoming move to provide sysfs support for all
    allocators.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit d1287a8f0bca2f5e017cf8f5503740e7c2bc30a2
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Sep 5 00:18:32 2012 +0000

    mm/sl[aou]b: Do slab aliasing call from common code

    The slab aliasing logic causes some strange contortions in slub. So add
    a call to deal with aliases to slab_common.c but disable it for other
    slab allocators by providng stubs that fail to create aliases.

    Full general support for aliases will require additional cleanup passes
    and more standardization of fields in kmem_cache.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit fdf8919dbaf47f0c49a222ac222cd216b8beed15
Author: Christoph Lameter <cl@linux.com>
Date:   Sun Sep 8 17:45:41 2013 -0500

    mm/sl[aou]b: Move duping of slab name to slab_common.c

    Duping of the slabname has to be done by each slab. Moving this code to
    slab_common avoids duplicate implementations.

    With this patch we have common string handling for all slab allocators.
    Strings passed to kmem_cache_create() are copied internally. Subsystems
    can create temporary strings to create slab caches.

    Slabs allocated in early states of bootstrap will never be freed (and
    those can never be freed since they are essential to slab allocator
    operations).  During bootstrap we therefore do not have to worry about
    duping names.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit d2ff2b323f7f419e2d996adc37a1939d5eeb6c77
Author: Christoph Lameter <cl@linux.com>
Date:   Tue Sep 4 23:38:33 2012 +0000

    mm/sl[aou]b: Get rid of __kmem_cache_destroy

    What is done there can be done in __kmem_cache_shutdown.

    This affects RCU handling somewhat. On rcu free all slab allocators do
    not refer to other management structures than the kmem_cache structure.
    Therefore these other structures can be freed before the rcu deferred
    free to the page allocator occurs.

    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit f54e5dac644cea00239d4d0dfe0f6fb2ead7723c
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Sep 5 00:18:32 2012 +0000

    mm/sl[aou]b: Move freeing of kmem_cache structure to common code

    The freeing action is basically the same in all slab allocators.
    Move to the common kmem_cache_destroy() function.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 12464f4e808062f8924ca0d794f42c74da5ff177
Author: Christoph Lameter <cl@linux.com>
Date:   Tue Sep 4 23:18:33 2012 +0000

    mm/sl[aou]b: Extract a common function for kmem_cache_destroy

    kmem_cache_destroy does basically the same in all allocators.

    Extract common code which is easy since we already have common mutex
    handling.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 2f8e28a763e9ccb4d51a15523d8b6104c835db4a
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Sep 5 00:20:33 2012 +0000

    mm/sl[aou]b: Use "kmem_cache" name for slab cache with kmem_cache struct

    Make all allocators use the "kmem_cache" slabname for the "kmem_cache"
    structure.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 7748e82161947557d1e388ebafcf21c9ef7ca1a1
Author: Christoph Lameter <cl@linux.com>
Date:   Sun Sep 8 17:43:07 2013 -0500

    mm/sl[aou]b: Move list_add() to slab_common.c

    Move the code to append the new kmem_cache to the list of slab caches to
    the kmem_cache_create code in the shared code.

    This is possible now since the acquisition of the mutex was moved into
    kmem_cache_create().

    Acked-by: David Rientjes <rientjes@google.com>
    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit d983d8c2f1676a1b5d47fda1b0e76e42b6ebf13d
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Sep 5 00:20:33 2012 +0000

    mm/slab_common: Improve error handling in kmem_cache_create

    Instead of using s == NULL use an errorcode. This allows much more
    detailed diagnostics as to what went wrong. As we add more functionality
    from the slab allocators to the common kmem_cache_create() function we will
    also add more error conditions.

    Print the error code during the panic as well as in a warning if the module
    can handle failure. The API for kmem_cache_create() currently does not allow
    the returning of an error code. Return NULL but log the cause of the problem
    in the syslog.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit b39bdc55bb0f1641a70af583b8faa1003d1206ab
Author: Shuah Khan <shuah.khan@hp.com>
Date:   Thu Aug 16 00:09:46 2012 -0700

    mm/slab: restructure kmem_cache_create() debug checks

    kmem_cache_create() does cache integrity checks when CONFIG_DEBUG_VM is
    defined.  These checks interspersed with the regular code path has lead
    to compile time warnings when compiled without CONFIG_DEBUG_VM defined.
    Restructuring the code to move the integrity checks in to a new function
    would eliminate the current compile warning problem and also will allow
    for future changes to the debug only code to evolve without introducing
    new warnings in the regular path.

    This restructuring work is based on the discussion in the following
    thread:

    https://lkml.org/lkml/2012/7/13/424

    [akpm@linux-foundation.org: fix build, cleanup]
    Signed-off-by: Shuah Khan <shuah.khan@hp.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit a22bf4a3c89fb93ca7dbd3be9b0f4f1d95304870
Author: Pekka Enberg <penberg@kernel.org>
Date:   Thu Aug 16 10:12:18 2012 +0300

    Revert "mm/slab_common.c: cleanup"

    This reverts commit 455ce9eb1cfa083da0def023094190aeb133855a. Andrew
    sent a better version.

    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 2a74503d75fe9a34221e530a3fd8257a070f21b5
Author: Andrew Morton <akpm@linux-foundation.org>
Date:   Tue Aug 14 14:53:22 2012 -0700

    mm/slab_common.c: cleanup

    Eliminate an ifdef and a label by moving all the CONFIG_DEBUG_VM checking
    inside the locked region.

    Cc: Pekka Enberg <penberg@cs.helsinki.fi>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit d09d15031bb2028c3aa1ece68cf2af90617a7165
Author: Shuah Khan <shuah.khan@hp.com>
Date:   Fri Jul 13 17:12:05 2012 -0600

    mm: Fix build warning in kmem_cache_create()

    The label oops is used in CONFIG_DEBUG_VM ifdef block and is defined
    outside ifdef CONFIG_DEBUG_VM block. This results in the following
    build warning when built with CONFIG_DEBUG_VM disabled. Fix to move
    label oops definition to inside a CONFIG_DEBUG_VM block.

    mm/slab_common.c: In function ‘kmem_cache_create’:
    mm/slab_common.c:101:1: warning: label ‘oops’ defined but not used
    [-Wunused-label]

    Signed-off-by: Shuah Khan <shuah.khan@hp.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: Paul Reioux <reioux@gmail.com>

    Conflicts:
    	mm/slab_common.c

commit d7ab7ead64b5c8ff2995b43b9ad42ae4810c37e0
Author: David Rientjes <rientjes@google.com>
Date:   Tue Aug 28 19:57:21 2012 -0700

    mm, slab: lock the correct nodelist after reenabling irqs

    cache_grow() can reenable irqs so the cpu (and node) can change, so ensure
    that we take list_lock on the correct nodelist.

    This fixes an issue with commit 072bb0aa5e06 ("mm: sl[au]b: add
    knowledge of PFMEMALLOC reserve pages") where list_lock for the wrong
    node was taken after growing the cache.

    Reported-and-tested-by: Haggai Eran <haggaie@mellanox.com>
    Signed-off-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 7b2a3234b27b3dbfe536d49ac28d5a74f2c5deec
Author: David Rientjes <rientjes@google.com>
Date:   Thu Aug 16 12:25:31 2012 -0700

    mm, slab: remove page_get_cache

    page_get_cache() isn't called from anything, so remove it.

    Signed-off-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit c4c5011c67fe6b0af54c4aa539609a00604b1369
Author: Michel Lespinasse <walken@google.com>
Date:   Tue Aug 14 14:53:20 2012 -0700

    slab: do not call compound_head() in page_get_cache()

    page_get_cache() does not need to call compound_head(), as its unique
    caller virt_to_slab() already makes sure to return a head page.

    Additionally, removing the compound_head() call makes page_get_cache()
    consistent with page_get_slab().

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Cc: Christoph Lameter <cl@linux-foundation.org>
    Cc: Pekka Enberg <penberg@kernel.org>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 5eb2c51530f5a0336d1d032e0b71c116b2bf57fc
Author: Mel Gorman <mgorman@suse.de>
Date:   Tue Jul 31 16:44:30 2012 -0700

    mm: micro-optimise slab to avoid a function call

    Getting and putting objects in SLAB currently requires a function call but
    the bulk of the work is related to PFMEMALLOC reserves which are only
    consumed when network-backed storage is critical.  Use an inline function
    to determine if the function call is required.

    Signed-off-by: Mel Gorman <mgorman@suse.de>
    Cc: David Miller <davem@davemloft.net>
    Cc: Neil Brown <neilb@suse.de>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Mike Christie <michaelc@cs.wisc.edu>
    Cc: Eric B Munson <emunson@mgebm.net>
    Cc: Eric Dumazet <eric.dumazet@gmail.com>
    Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Christoph Lameter <cl@linux.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 8f04e07de24d1c870691e1618e086d30ebce3da2
Author: Paul Reioux <reioux@gmail.com>
Date:   Sat Nov 2 19:00:22 2013 -0500

    mm: introduce __GFP_MEMALLOC to allow access to
     emergency reserves

    __GFP_MEMALLOC will allow the allocation to disregard the watermarks, much
    like PF_MEMALLOC.  It allows one to pass along the memalloc state in
    object related allocation flags as opposed to task related flags, such as
    sk->sk_allocation.  This removes the need for ALLOC_PFMEMALLOC as callers
    using __GFP_MEMALLOC can get the ALLOC_NO_WATERMARK flag which is now
    enough to identify allocations related to page reclaim.

    Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Signed-off-by: Mel Gorman <mgorman@suse.de>
    Cc: David Miller <davem@davemloft.net>
    Cc: Neil Brown <neilb@suse.de>
    Cc: Mike Christie <michaelc@cs.wisc.edu>
    Cc: Eric B Munson <emunson@mgebm.net>
    Cc: Eric Dumazet <eric.dumazet@gmail.com>
    Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Christoph Lameter <cl@linux.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

    backported to Android Linux 3.4 by faux123

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit 447e7e1c4a2424247c2da22ce620958cbab41227
Author: Paul Reioux <reioux@gmail.com>
Date:   Sun Sep 8 17:41:26 2013 -0500

    mm/slub: Use kmem_cache for the kmem_cache structure

    Do not use kmalloc() but kmem_cache_alloc() for the allocation
    of the kmem_cache structures in slub.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>
    backported for Linux 3.4

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit 3812c28085fb648251c60df672623e8d69c53a8d
Author: Christoph Lameter <cl@linux.com>
Date:   Tue Sep 4 23:06:14 2012 +0000

    mm/slub: Add debugging to verify correct cache use on kmem_cache_free()

    Add additional debugging to check that the objects is actually from the cache
    the caller claims. Doing so currently trips up some other debugging code. It
    takes a lot to infer from that what was happening.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    [ penberg@kernel.org: Use pr_err() ]
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit f8b1f6c6149632cc5f205dde77e5ac1fd241c63b
Author: Joonsoo Kim <js1304@gmail.com>
Date:   Sat Jun 23 03:22:38 2012 +0900

    slub: reduce failure of this_cpu_cmpxchg in put_cpu_partial() after unfreezing

    In current implementation, after unfreezing, we doesn't touch oldpage,
    so it remain 'NOT NULL'. When we call this_cpu_cmpxchg()
    with this old oldpage, this_cpu_cmpxchg() is mostly be failed.

    We can change value of oldpage to NULL after unfreezing,
    because unfreeze_partial() ensure that all the cpu partial slabs is removed
    from cpu partial list. In this time, we could expect that
    this_cpu_cmpxchg is mostly succeed.

    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit d19cff1e3581fdc20c12a3deed52b92fea7f30a5
Author: Christoph Lameter <cl@linux.com>
Date:   Wed May 30 12:54:46 2012 -0500

    slub: Take node lock during object free checks

    Only applies to scenarios where debugging is on:

    Validation of slabs can currently occur while debugging
    information is updated from the fast paths of the allocator.
    This results in various races where we get false reports about
    slab metadata not being in order.

    This patch makes the fast paths take the node lock so that
    serialization with slab validation will occur. Causes additional
    slowdown in debug scenarios.

    Reported-by: Waiman Long <Waiman.Long@hp.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 895cd750c539332dcd3e7f0d39955b058e24f728
Author: Glauber Costa <glommer@parallels.com>
Date:   Fri Aug 3 22:51:37 2012 +0400

    slub: use free_page instead of put_page for freeing kmalloc allocation

    When freeing objects, the slub allocator will most of the time free
    empty pages by calling __free_pages(). But high-order kmalloc will be
    diposed by means of put_page() instead. It makes no sense to call
    put_page() in kernel pages that are provided by the object allocators,
    so we shouldn't be doing this ourselves. Aside from the consistency
    change, we don't change the flow too much. put_page()'s would call its
    dtor function, which is __free_pages. We also already do all of the
    Compound page tests ourselves, and the Mlock test we lose don't really
    matter.

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    CC: David Rientjes <rientjes@google.com>
    CC: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit ea9f5d36e4498e56b2e89eea13a5b00687a89dc5
Author: Christoph Lameter <cl@linux.com>
Date:   Tue Jul 31 16:44:00 2012 -0700

    mm: slub: optimise the SLUB fast path to avoid pfmemalloc checks

    This patch removes the check for pfmemalloc from the alloc hotpath and
    puts the logic after the election of a new per cpu slab.  For a pfmemalloc
    page we do not use the fast path but force the use of the slow path which
    is also used for the debug case.

    This has the side-effect of weakening pfmemalloc processing in the
    following way;

    1. A process that is allocating for network swap calls __slab_alloc.
       pfmemalloc_match is true so the freelist is loaded and c->freelist is
       now pointing to a pfmemalloc page.

    2. A process that is attempting normal allocations calls slab_alloc,
       finds the pfmemalloc page on the freelist and uses it because it did
       not check pfmemalloc_match()

    The patch allows non-pfmemalloc allocations to use pfmemalloc pages with
    the kmalloc slabs being the most vunerable caches on the grounds they
    are most likely to have a mix of pfmemalloc and !pfmemalloc requests. A
    later patch will still protect the system as processes will get throttled
    if the pfmemalloc reserves get depleted but performance will not degrade
    as smoothly.

    [mgorman@suse.de: Expanded changelog]
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Mel Gorman <mgorman@suse.de>
    Cc: David Miller <davem@davemloft.net>
    Cc: Neil Brown <neilb@suse.de>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Mike Christie <michaelc@cs.wisc.edu>
    Cc: Eric B Munson <emunson@mgebm.net>
    Cc: Eric Dumazet <eric.dumazet@gmail.com>
    Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
    Cc: Mel Gorman <mgorman@suse.de>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 35ee70c1791c5ca05622647030cb3047198fff61
Author: Paul Reioux <reioux@gmail.com>
Date:   Sat Nov 2 18:52:34 2013 -0500

    mm: sl[au]b: add knowledge of PFMEMALLOC reserve pages

    When a user or administrator requires swap for their application, they
    create a swap partition and file, format it with mkswap and activate it
    with swapon.  Swap over the network is considered as an option in diskless
    systems.  The two likely scenarios are when blade servers are used as part
    of a cluster where the form factor or maintenance costs do not allow the
    use of disks and thin clients.

    The Linux Terminal Server Project recommends the use of the Network Block
    Device (NBD) for swap according to the manual at
    https://sourceforge.net/projects/ltsp/files/Docs-Admin-Guide/LTSPManual.pdf/download
    There is also documentation and tutorials on how to setup swap over NBD at
    places like https://help.ubuntu.com/community/UbuntuLTSP/EnableNBDSWAP The
    nbd-client also documents the use of NBD as swap.  Despite this, the fact
    is that a machine using NBD for swap can deadlock within minutes if swap
    is used intensively.  This patch series addresses the problem.
    The core issue is that network block devices do not use mempools like
    normal block devices do.  As the host cannot control where they receive
    packets from, they cannot reliably work out in advance how much memory
    they might need.  Some years ago, Peter Zijlstra developed a series of
    patches that supported swap over an NFS that at least one distribution is
    carrying within their kernels.  This patch series borrows very heavily
    from Peter's work to support swapping over NBD as a pre-requisite to
    supporting swap-over-NFS.  The bulk of the complexity is concerned with
    preserving memory that is allocated from the PFMEMALLOC reserves for use
    by the network layer which is needed for both NBD and NFS.

    Patch 1 adds knowledge of the PFMEMALLOC reserves to SLAB and SLUB to
            preserve access to pages allocated under low memory situations
            to callers that are freeing memory.

    Patch 2 optimises the SLUB fast path to avoid pfmemalloc checks

    Patch 3 introduces __GFP_MEMALLOC to allow access to the PFMEMALLOC
            reserves without setting PFMEMALLOC.

    Patch 4 opens the possibility for softirqs to use PFMEMALLOC reserves        for later use by network packet processing.

    Patch 5 only sets page->pfmemalloc when ALLOC_NO_WATERMARKS was required

    Patch 6 ignores memory policies when ALLOC_NO_WATERMARKS is set.

    Patches 7-12 allows network processing to use PFMEMALLOC reserves when
            the socket has been marked as being used by the VM to clean pages. If
            packets are received and stored in pages that were allocated under
            low-memory situations and are unrelated to the VM, the packets
            are dropped.

            Patch 11 reintroduces __skb_alloc_page which the networking
            folk may object to but is needed in some cases to propogate
            pfmemalloc from a newly allocated page to an skb. If there is a
            strong objection, this patch can be dropped with the impact being
            that swap-over-network will be slower in some cases but it should
            not fail.

    Patch 13 is a micro-optimisation to avoid a function call in the
            common case.
    Patch 14 tags NBD sockets as being SOCK_MEMALLOC so they can use
            PFMEMALLOC if necessary.

    Patch 16 adds a statistic to track how often processes get throttled

    Some basic performance testing was run using kernel builds, netperf on
    loopback for UDP and TCP, hackbench (pipes and sockets), iozone and
    sysbench.  Each of them were expected to use the sl*b allocators
    reasonably heavily but there did not appear to be significant performance
    variances.

    For testing swap-over-NBD, a machine was booted with 2G of RAM with a
    swapfile backed by NBD.  8*NUM_CPU processes were started that create
    anonymous memory mappings and read them linearly in a loop.  The total
    size of the mappings were 4*PHYSICAL_MEMORY to use swap heavily under
    memory pressure.

    Without the patches and using SLUB, the machine locks up within minutes
    and runs to completion with them applied.  With SLAB, the story is
    different as an unpatched kernel run to completion.  However, the patched
    kernel completed the test 45% faster.

    MICRO
                                             3.5.0-rc2 3.5.0-rc2
                                             vanilla     swapnbd
    Unrecognised test vmscan-anon-mmap-write
    MMTests Statistics: duration
    Sys Time Running Test (seconds)             197.80    173.07
    User+Sys Time Running Test (seconds)        206.96    182.03
    Total Elapsed Time (seconds)               3240.70   1762.09

    This patch: mm: sl[au]b: add knowledge of PFMEMALLOC reserve pages

    Allocations of pages below the min watermark run a risk of the machine
    hanging due to a lack of memory.  To prevent this, only callers who have
    PF_MEMALLOC or TIF_MEMDIE set and are not processing an interrupt are
    allowed to allocate with ALLOC_NO_WATERMARKS.  Once they are allocated to
    a slab though, nothing prevents other callers consuming free objects
    within those slabs.  This patch limits access to slab pages that were
    alloced from the PFMEMALLOC reserves.

    When this patch is applied, pages allocated from below the low watermark
    are returned with page->pfmemalloc set and it is up to the caller to
    determine how the page should be protected.  SLAB restricts access to any
    page with page->pfmemalloc set to callers which are known to able to
    access the PFMEMALLOC reserve.  If one is not available, an attempt is
    made to allocate a new page rather than use a reserve.  SLUB is a bit more
    relaxed in that it only records if the current per-CPU page was allocated
    from PFMEMALLOC reserve and uses another partial slab if the caller does
    not have the necessary GFP or process flags.  This was found to be
    sufficient in tests to avoid hangs due to SLUB generally maintaining
    smaller lists than SLAB.

    In low-memory conditions it does mean that !PFMEMALLOC allocators can fail
    a slab allocation even though free objects are available because they are
    being preserved for callers that are freeing pages.

    [a.p.zijlstra@chello.nl: Original implementation]
    [sebastian@breakpoint.cc: Correct order of page flag clearing]
    Signed-off-by: Mel Gorman <mgorman@suse.de>
    Cc: David Miller <davem@davemloft.net>
    Cc: Neil Brown <neilb@suse.de>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Mike Christie <michaelc@cs.wisc.edu>
    Cc: Eric B Munson <emunson@mgebm.net>
    Cc: Eric Dumazet <eric.dumazet@gmail.com>
    Cc: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Christoph Lameter <cl@linux.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

    backported for Android Linux 3.4 by faux123

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit 5b2f6f6dbf2c0f4a33a92a923568f6ca05b19f25
Author: David Rientjes <rientjes@google.com>
Date:   Mon Jul 9 14:00:38 2012 -0700

    mm, slub: ensure irqs are enabled for kmemcheck

    kmemcheck_alloc_shadow() requires irqs to be enabled, so wait to disable
    them until after its called for __GFP_WAIT allocations.

    This fixes a warning for such allocations:

    	WARNING: at kernel/lockdep.c:2739 lockdep_trace_alloc+0x14e/0x1c0()

    Acked-by: Fengguang Wu <fengguang.wu@intel.com>
    Acked-by: Steven Rostedt <rostedt@goodmis.org>
    Tested-by: Fengguang Wu <fengguang.wu@intel.com>
    Signed-off-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 1d3d2fbb51d9cd1aed5e01a55f1ae11512823a48
Author: Christoph Lameter <cl@linux.com>
Date:   Sun Sep 8 17:35:51 2013 -0500

    mm, sl[aou]b: Move kmem_cache_create mutex handling to common code

    Move the mutex handling into the common kmem_cache_create()
    function.

    Then we can also move more checks out of SLAB's kmem_cache_create()
    into the common code.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit a066abc2185c7f060fcf6e7dc4659175247a43c6
Author: Christoph Lameter <cl@linux.com>
Date:   Sun Sep 8 17:31:53 2013 -0500

    mm, sl[aou]b: Use a common mutex definition

    Use the mutex definition from SLAB and make it the common way to take a sleeping lock.

    This has the effect of using a mutex instead of a rw semaphore for SLUB.

    SLOB gains the use of a mutex for kmem_cache_create serialization.
    Not needed now but SLOB may acquire some more features later (like slabinfo
    / sysfs support) through the expansion of the common code that will
    need this.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit c38e410e0fe3483e0858a8a178e11a0f640da6d7
Author: Christoph Lameter <cl@linux.com>
Date:   Fri Jul 6 15:25:11 2012 -0500

    mm, sl[aou]b: Common definition for boot state of the slab allocators

    All allocators have some sort of support for the bootstrap status.

    Setup a common definition for the boot states and make all slab
    allocators use that definition.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit da76a447d02839d688457fcaf17cfb6b78337d59
Author: Julia Lawall <Julia.Lawall@lip6.fr>
Date:   Sun Jul 8 13:37:40 2012 +0200

    slub: remove invalid reference to list iterator variable

    If list_for_each_entry, etc complete a traversal of the list, the iterator
    variable ends up pointing to an address at an offset from the list head,
    and not a meaningful structure.  Thus this value should not be used after
    the end of the iterator.  The patch replaces s->name by al->name, which is
    referenced nearby.

    This problem was found using Coccinelle (http://coccinelle.lip6.fr/).

    Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 3a1f170de0a04387560b06f413907d7a0a840a50
Author: Joonsoo Kim <js1304@gmail.com>
Date:   Sat Jun 9 02:23:16 2012 +0900

    slub: refactoring unfreeze_partials()

    Current implementation of unfreeze_partials() is so complicated,
    but benefit from it is insignificant. In addition many code in
    do {} while loop have a bad influence to a fail rate of cmpxchg_double_slab.
    Under current implementation which test status of cpu partial slab
    and acquire list_lock in do {} while loop,
    we don't need to acquire a list_lock and gain a little benefit
    when front of the cpu partial slab is to be discarded, but this is a rare case.
    In case that add_partial is performed and cmpxchg_double_slab is failed,
    remove_partial should be called case by case.

    I think that these are disadvantages of current implementation,
    so I do refactoring unfreeze_partials().

    Minimizing code in do {} while loop introduce a reduced fail rate
    of cmpxchg_double_slab. Below is output of 'slabinfo -r kmalloc-256'
    when './perf stat -r 33 hackbench 50 process 4000 > /dev/null' is done.

    ** before **
    Cmpxchg_double Looping
    ------------------------
    Locked Cmpxchg Double redos   182685
    Unlocked Cmpxchg Double redos 0

    ** after **
    Cmpxchg_double Looping
    ------------------------
    Locked Cmpxchg Double redos   177995
    Unlocked Cmpxchg Double redos 1

    We can see cmpxchg_double_slab fail rate is improved slightly.

    Bolow is output of './perf stat -r 30 hackbench 50 process 4000 > /dev/null'.

    ** before **
     Performance counter stats for './hackbench 50 process 4000' (30 runs):

         108517.190463 task-clock                #    7.926 CPUs utilized            ( +-  0.24% )
             2,919,550 context-switches          #    0.027 M/sec                    ( +-  3.07% )
               100,774 CPU-migrations            #    0.929 K/sec                    ( +-  4.72% )
               124,201 page-faults               #    0.001 M/sec                    ( +-  0.15% )
       401,500,234,387 cycles                    #    3.700 GHz                      ( +-  0.24% )
       <not supported> stalled-cycles-frontend
       <not supported> stalled-cycles-backend
       250,576,913,354 instructions              #    0.62  insns per cycle          ( +-  0.13% )
        45,934,956,860 branches                  #  423.297 M/sec                    ( +-  0.14% )
           188,219,787 branch-misses             #    0.41% of all branches          ( +-  0.56% )

          13.691837307 seconds time elapsed                                          ( +-  0.24% )

    ** after **
     Performance counter stats for './hackbench 50 process 4000' (30 runs):

         107784.479767 task-clock                #    7.928 CPUs utilized            ( +-  0.22% )
             2,834,781 context-switches          #    0.026 M/sec                    ( +-  2.33% )
                93,083 CPU-migrations            #    0.864 K/sec                    ( +-  3.45% )
               123,967 page-faults               #    0.001 M/sec                    ( +-  0.15% )
       398,781,421,836 cycles                    #    3.700 GHz                      ( +-  0.22% )
       <not supported> stalled-cycles-frontend
       <not supported> stalled-cycles-backend
       250,189,160,419 instructions              #    0.63  insns per cycle          ( +-  0.09% )
        45,855,370,128 branches                  #  425.436 M/sec                    ( +-  0.10% )
           169,881,248 branch-misses             #    0.37% of all branches          ( +-  0.43% )

          13.596272341 seconds time elapsed                                          ( +-  0.22% )

    No regression is found, but rather we can see slightly better result.

    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 5a67fef21e22e941adfe40bd4b044785833aedc1
Author: Joonsoo Kim <js1304@gmail.com>
Date:   Fri May 18 22:01:17 2012 +0900

    slub: use __cmpxchg_double_slab() at interrupt disabled place

    get_freelist(), unfreeze_partials() are only called with interrupt disabled,
    so __cmpxchg_double_slab() is suitable.

    Acked-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 471d61964f8b9e75277737495e7fecd7ef5febb3
Author: Glauber Costa <glommer@parallels.com>
Date:   Thu Jun 21 00:59:18 2012 +0400

    slab: move FULL state transition to an initcall

    During kmem_cache_init_late(), we transition to the LATE state,
    and after some more work, to the FULL state, its last state

    This is quite different from slub, that will only transition to
    its last state (previously SYSFS), in a (late)initcall, after a lot
    more of the kernel is ready.

    This means that in slab, we have no way to taking actions dependent
    on the initialization of other pieces of the kernel that are supposed
    to start way after kmem_init_late(), such as cgroups initialization.

    To achieve more consistency in this behavior, that patch only
    transitions to the UP state in kmem_init_late. In my analysis,
    setup_cpu_cache() should be happy to test for >= UP, instead of
    == FULL. It also has passed some tests I've made.

    We then only mark FULL state after the reap timers are in place,
    meaning that no further setup is expected.

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 4c047cb36c138e3ff1a5439195a466cb610c302d
Author: Feng Tang <feng.tang@intel.com>
Date:   Mon Jul 2 14:29:10 2012 +0800

    slab: Fix a typo in commit 8c138b "slab: Get rid of obj_size macro"

    Commit  8c138b only sits in Pekka's and linux-next tree now, which tries
    to replace obj_size(cachep) with cachep->object_size, but has a typo in
    kmem_cache_free() by using "size" instead of "object_size", which casues
    some regressions.

    Reported-and-tested-by: Fengguang Wu <wfg@linux.intel.com>
    Signed-off-by: Feng Tang <feng.tang@intel.com>
    Cc: Christoph Lameter <cl@linux.com>
    Acked-by: Glauber Costa <glommer@parallels.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit ac35837691194906ffcaaca8d8b0d10806b6214a
Author: Thierry Reding <thierry.reding@avionic-design.de>
Date:   Fri Jun 22 19:42:49 2012 +0200

    mm, slab: Build fix for recent kmem_cache changes

    Commit 3b0efdf ("mm, sl[aou]b: Extract common fields from struct
    kmem_cache") renamed the kmem_cache structure's "next" field to "list"
    but forgot to update one instance in leaks_show().

    Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 53d481436f4c3bba39e1aabe05cf613a362c6a93
Author: Glauber Costa <glommer@parallels.com>
Date:   Thu Jun 14 16:17:21 2012 +0400

    slab: rename gfpflags to allocflags

    A consistent name with slub saves us an acessor function.
    In both caches, this field represents the same thing. We would
    like to use it from the mem_cgroup code.

    Signed-off-by: Glauber Costa <glommer@parallels.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    CC: Pekka Enberg <penberg@cs.helsinki.fi>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 52a97b9175ec3bb82de12c04e3d88ae0d58819a3
Author: Andi Kleen <ak@linux.intel.com>
Date:   Sat Jun 9 02:40:03 2012 -0700

    slab/mempolicy: always use local policy from interrupt context

    slab_node() could access current->mempolicy from interrupt context.
    However there's a race condition during exit where the mempolicy
    is first freed and then the pointer zeroed.

    Using this from interrupts seems bogus anyways. The interrupt
    will interrupt a random process and therefore get a random
    mempolicy. Many times, this will be idle's, which noone can change.

    Just disable this here and always use local for slab
    from interrupts. I also cleaned up the callers of slab_node a bit
    which always passed the same argument.

    I believe the original mempolicy code did that in fact,
    so it's likely a regression.

    v2: send version with correct logic
    v3: simplify. fix typo.
    Reported-by: Arun Sharma <asharma@fb.com>
    Cc: penberg@kernel.org
    Cc: cl@linux.com
    Signed-off-by: Andi Kleen <ak@linux.intel.com>
    [tdmackey@twitter.com: Rework control flow based on feedback from
    cl@linux.com, fix logic, and cleanup current task_struct reference]
    Acked-by: David Rientjes <rientjes@google.com>
    Acked-by: Christoph Lameter <cl@linux.com>
    Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
    Signed-off-by: David Mackey <tdmackey@twitter.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 04afd0a713eae5039922d6afb3704ca52d10ae6e
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Jun 13 10:24:58 2012 -0500

    slab: Get rid of obj_size macro

    The size of the slab object is frequently needed. Since we now
    have a size field directly in the kmem_cache structure there is no
    need anymore of the obj_size macro/function.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit a216554828d75cb101e93c35f2816fb812f2633b
Author: Christoph Lameter <cl@linux.com>
Date:   Tue Jul 10 18:31:05 2012 -0500

    slob: Fix early boot kernel crash

    Commit fd3142a59af2012a7c5dc72ec97a4935ff1c5fc6 broke
    slob since a piece of a change for a later patch slipped into
    it.

    Fengguang Wu writes:

      The commit crashes the kernel w/o any dmesg output (the attached one is
      created by the script as a summary for that run). This is very
      reproducible in kvm for the attached config.

    Reported-by: Fengguang Wu <fengguang.wu@intel.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 1385c0580d385aa601eb8f42c879c23b76064ea1
Author: Christoph Lameter <cl@linux.com>
Date:   Sun Sep 8 17:30:44 2013 -0500

    mm, sl[aou]b: Extract common code for kmem_cache_create()

    Kmem_cache_create() does a variety of sanity checks but those
    vary depending on the allocator. Use the strictest tests and put them into
    a slab_common file. Make the tests conditional on CONFIG_DEBUG_VM.

    This patch has the effect of adding sanity checks for SLUB and SLOB
    under CONFIG_DEBUG_VM and removes the checks in SLAB for !CONFIG_DEBUG_VM.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit ac9919a4c3d9dfccb9992d14514b09bb6a9e2913
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Jun 13 10:24:57 2012 -0500

    mm, sl[aou]b: Extract common fields from struct kmem_cache

    Define a struct that describes common fields used in all slab allocators.
    A slab allocator either uses the common definition (like SLOB) or is
    required to provide members of kmem_cache with the definition given.

    After that it will be possible to share code that
    only operates on those fields of kmem_cache.

    The patch basically takes the slob definition of kmem cache and
    uses the field namees for the other allocators.

    It also standardizes the names used for basic object lengths in
    allocators:

    object_size	Struct size specified at kmem_cache_create. Basically
    		the payload expected to be used by the subsystem.

    size		The size of memory allocator for each object. This size
    		is larger than object_size and includes padding, alignment
    		and extra metadata for each object (f.e. for debugging
    		and rcu).

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 26d134f429b567c07d3f0ee50ef7c139224fb57a
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Jun 13 10:24:54 2012 -0500

    slob: Remove various small accessors

    Those have become so simple that they are no longer needed.

    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Acked-by: David Rientjes <rientjes@google.com>
    signed-off-by: Christoph Lameter <cl@linux.com>

    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 9c41465cabdb0e389b4f36a5e049a7c6075217c9
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Jun 13 10:24:53 2012 -0500

    slob: No need to zero mapping since it is no longer in use

    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 9f910e4b6445beb988caa80b68b7d8dd18c6002f
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Jun 13 10:24:56 2012 -0500

    slab: Remove some accessors

    Those are rather trivial now and its better to see inline what is
    really going on.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit f574eec4d911be45fd14b9595818681da9a84504
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Jun 13 10:24:55 2012 -0500

    slab: Use page struct fields instead of casting

    Add fields to the page struct so that it is properly documented that
    slab overlays the lru fields.

    This cleans up some casts in slab.

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 271bdd23e4e9dca3fe65e5e9625260ed190b1475
Author: Christoph Lameter <cl@linux.com>
Date:   Wed Jun 13 10:24:52 2012 -0500

    slob: Define page struct fields used in mm_types.h

    Define the fields used by slob in mm_types.h and use struct page instead
    of struct slob_page in slob. This cleans up numerous of typecasts in slob.c and
    makes readers aware of slob's use of page struct fields.

    [Also cleans up some bitrot in slob.c. The page struct field layout
    in slob.c is an old layout and does not match the one in mm_types.h]

    Reviewed-by: Glauber Costa <glommer@parallels.com>
    Acked-by: David Rientjes <rientjes@google.com>
    Reviewed-by: Joonsoo Kim <js1304@gmail.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 36d4de5c1bdde06ef272474374c252349733ca9c
Author: Christoph Lameter <cl@linux.com>
Date:   Sat Nov 2 18:49:05 2013 -0500

    slub: pass page to node_match() instead of kmem_cache_cpu structure

    Avoid passing the kmem_cache_cpu pointer to node_match. This makes the
    node_match function more generic and easier to understand.

    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit c703c7b45ab155facf26b9cff3cfae1df9ef1bec
Author: Christoph Lameter <cl@linux.com>
Date:   Wed May 9 10:09:58 2012 -0500

    slub: Use page variable instead of c->page.

    Store the value of c->page to avoid additional fetches
    from per cpu data.

    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 6da175927dcbfcddd1726d2be3f357853151cd1f
Author: Christoph Lameter <cl@linux.com>
Date:   Wed May 9 10:09:57 2012 -0500

    slub: Separate out kmem_cache_cpu processing from deactivate_slab

    Processing on fields of kmem_cache_cpu is cleaner if code working on fields
    of this struct is taken out of deactivate_slab().

    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 2812a78abc062c11ab2c91ed0f5e5630bd30a58f
Author: Christoph Lameter <cl@linux.com>
Date:   Wed May 9 10:09:56 2012 -0500

    slub: Get rid of the node field

    The node field is always page_to_nid(c->page). So its rather easy to
    replace. Note that there maybe slightly more overhead in various hot paths
    due to the need to shift the bits from page->flags. However, that is mostly
    compensated for by a smaller footprint of the kmem_cache_cpu structure (this
    patch reduces that to 3 words per cache) which allows better caching.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 7b5f7029ba8eb6cf4cbea28f4f50658806418ee4
Author: Christoph Lameter <cl@linux.com>
Date:   Wed May 9 10:09:55 2012 -0500

    slub: new_slab_objects() can also get objects from partial list

    Moving the attempt to get a slab page from the partial lists simplifies
    __slab_alloc which is rather complicated.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit c9951151964252e668ce36c61ab78146fa94510a
Author: Christoph Lameter <cl@linux.com>
Date:   Wed May 9 10:09:54 2012 -0500

    slub: Simplify control flow in __slab_alloc()

    Simplify control flow a bit avoiding nesting.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 0e85e32ca6bc07adce529517a610b074da3949c3
Author: Christoph Lameter <cl@linux.com>
Date:   Sun Sep 8 17:29:21 2013 -0500

    slub: Acquire_slab() avoid loop

    Avoid the loop in acquire slab and simply fail if there is a conflict.

    This will cause the next page on the list to be considered.

    Acked-by: David Rientjes <rientjes@google.com>
    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>
    Signed-off-by: Paul Reioux <reioux@gmail.com>

commit afaa5690425a84b9e1c384ebb1ad75b07febd201
Author: Christoph Lameter <cl@linux.com>
Date:   Wed May 9 10:09:52 2012 -0500

    slub: Add frozen check in __slab_alloc

    Verify that objects returned from __slab_alloc come from slab pages
    in the correct state.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

commit 06a9e7bd34b49b92f26a3bc2a386b323c753dfc9
Author: Christoph Lameter <cl@linux.com>
Date:   Wed May 9 10:09:51 2012 -0500

    slub: Use freelist instead of "object" in __slab_alloc

    The variable "object" really refers to a list of objects that we
    are handling.

    Signed-off-by: Christoph Lameter <cl@linux.com>
    Signed-off-by: Pekka Enberg <penberg@kernel.org>

Conflicts:
	mm/slub.c
block/partitions: optimize memory allocation in check_partition()

Currently, sizeof(struct parsed_partitions) may be 64KB in 32bit arch, so
it is easy to trigger page allocation failure by check_partition,
especially in hotplug block device situation(such as, USB mass storage,
MMC card, ...), and Felipe Balbi has observed the failure.

This patch does below optimizations on the allocation of struct
parsed_partitions to try to address the issue:

- make parsed_partitions.parts as pointer so that the pointed memory can
  fit in 32KB buffer, then approximate 32KB memory can be saved

- vmalloc the buffer pointed by parsed_partitions.parts because 32KB is
  still a bit big for kmalloc

- given that many devices have the partition count limit, so only
  allocate disk_max_parts() partitions instead of 256 partitions always

Signed-off-by: Ming Lei <ming.lei@canonical.com>
Reported-by: Felipe Balbi <balbi@ti.com>
Cc: Jens Axboe <axboe@kernel.dk>
Reviewed-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
LMK: Optimize lowmem_shrink.

By comparing with selected_oom_score_adj instead of min_score_adj,
we may do less calculation.

Signed-off-by: Leon Ma <xindong.ma@intel.com>

Signed-off-by: Paul Reioux <reioux@gmail.com>
lib: Update RWSEM to Linux 3.10

Squashed commit of the following:

commit 28f9a942e4e1578c0e26e8211ba49bdf520de95c
Author: Davidlohr Bueso <davidlohr.bueso@hp.com>
Date:   Tue May 7 06:46:02 2013 -0700

    rwsem: no need for explicit signed longs

    Change explicit "signed long" declarations into plain "long" as suggested
    by Peter Hurley.

    Signed-off-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Reviewed-by: Michel Lespinasse <walken@google.com>
    Signed-off-by: Michel Lespinasse <walken@google.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 353ad88b6af95bc7bb127b3b5b1cfa36c72a7b67
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:46:00 2013 -0700

    rwsem: do not block readers at head of queue if other readers are active

    This change fixes a race condition where a reader might determine it
    needs to block, but by the time it acquires the wait_lock the rwsem has
    active readers and no queued waiters.

    In this situation the reader can run in parallel with the existing
    active readers; it does not need to block until the active readers
    complete.

    Thanks to Peter Hurley for noticing this possible race.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit e9149aa93e4e6919da4fe9e77a27f9c6a0159463
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:59 2013 -0700

    rwsem: implement support for write lock stealing on the fastpath

    When we decide to wake up readers, we must first grant them as many read
    locks as necessary, and then actually wake up all these readers.  But in
    order to know how many read shares to grant, we must first count the
    readers at the head of the queue.  This might take a while if there are
    many readers, and we want to be protected against a writer stealing the
    lock while we're counting.  To that end, we grant the first reader lock
    before counting how many more readers are queued.

    We also require some adjustments to the wake_type semantics.

    RWSEM_WAKE_NO_ACTIVE used to mean that we had found the count to be
    RWSEM_WAITING_BIAS, in which case the rwsem was known to be free as
    nobody could steal it while we hold the wait_lock.  This doesn't make
    sense once we implement fastpath write lock stealing, so we now use
    RWSEM_WAKE_ANY in that case.

    Similarly, when rwsem_down_write_failed found that a read lock was
    active, it would use RWSEM_WAKE_READ_OWNED which signalled that new
    readers could be woken without checking first that the rwsem was
    available.  We can't do that anymore since the existing readers might
    release their read locks, and a writer could steal the lock before we
    wake up additional readers.  So, we have to use a new RWSEM_WAKE_READERS
    value to indicate we only want to wake readers, but we don't currently
    hold any read lock.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 20addabcb540eb4b4e11d7e8af037543767e081d
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:58 2013 -0700

    rwsem: simplify __rwsem_do_wake

    This is mostly for cleanup value:

    - We don't need several gotos to handle the case where the first
      waiter is a writer. Two simple tests will do (and generate very
      similar code).

    - In the remainder of the function, we know the first waiter is a reader,
      so we don't have to double check that. We can use do..while loops
      to iterate over the readers to wake (generates slightly better code).

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit e56c8eea2628841a06a2420e2c6ca8b8c6bd4434
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:57 2013 -0700

    rwsem: skip initial trylock in rwsem_down_write_failed

    We can skip the initial trylock in rwsem_down_write_failed() if there
    are known active lockers already, thus saving one likely-to-fail
    cmpxchg.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Acked-by: Rik van Riel <riel@redhat.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 286040cc550df96bef6e97a5bc0231ded9a0187b
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:56 2013 -0700

    rwsem: avoid taking wait_lock in rwsem_down_write_failed

    In rwsem_down_write_failed(), if there are active locks after we wake up
    (i.e.  the lock got stolen from us), skip taking the wait_lock and go
    back to sleep immediately.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Acked-by: Rik van Riel <riel@redhat.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 6c7229b31a4c099157242c58f5d33ce728daff4b
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:55 2013 -0700

    rwsem: use cmpxchg for trying to steal write lock

    Using rwsem_atomic_update to try stealing the write lock forced us to
    undo the adjustment in the failure path.  We can have simpler and faster
    code by using cmpxchg instead.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Acked-by: Rik van Riel <riel@redhat.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 85dcaeae08b5738f8ea0b08734ab97e57bb9e8e2
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:54 2013 -0700

    rwsem: more agressive lock stealing in rwsem_down_write_failed

    Some small code simplifications can be achieved by doing more agressive
    lock stealing:

    - When rwsem_down_write_failed() notices that there are no active locks
      (and thus no thread to wake us if we decided to sleep), it used to wake
      the first queued process. However, stealing the lock is also sufficient
      to deal with this case, so we don't need this check anymore.

    - In try_get_writer_sem(), we can steal the lock even when the first waiter
      is a reader. This is correct because the code path that wakes readers is
      protected by the wait_lock. As to the performance effects of this change,
      they are expected to be minimal: readers are still granted the lock
      (rather than having to acquire it themselves) when they reach the front
      of the wait queue, so we have essentially the same behavior as in
      rwsem-spinlock.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Rik van Riel <riel@redhat.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 082a2e9c40e01d70c59ca4e3a9b552b81d573332
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:53 2013 -0700

    rwsem: simplify rwsem_down_write_failed

    When waking writers, we never grant them the lock - instead, they have
    to acquire it themselves when they run, and remove themselves from the
    wait_list when they succeed.

    As a result, we can do a few simplifications in rwsem_down_write_failed():

    - We don't need to check for !waiter.task since __rwsem_do_wake() doesn't
      remove writers from the wait_list

    - There is no point releaseing the wait_lock before entering the wait loop,
      as we will need to reacquire it immediately. We can change the loop so
      that the lock is always held at the start of each loop iteration.

    - We don't need to get a reference on the task structure, since the task
      is responsible for removing itself from the wait_list. There is no risk,
      like in the rwsem_down_read_failed() case, that a task would wake up and
      exit (thus destroying its task structure) while __rwsem_do_wake() is
      still running - wait_lock protects against that.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Rik van Riel <riel@redhat.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 6b3306dccd59d5dcf22e55a27827bccf51a796e1
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:52 2013 -0700

    rwsem: simplify rwsem_down_read_failed

    When trying to acquire a read lock, the RWSEM_ACTIVE_READ_BIAS
    adjustment doesn't cause other readers to block, so we never have to
    worry about waking them back after canceling this adjustment in
    rwsem_down_read_failed().

    We also never want to steal the lock in rwsem_down_read_failed(), so we
    don't have to grab the wait_lock either.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Rik van Riel <riel@redhat.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit c6b315c32d1ffbb33c75ab5e0f8ee27c89d206c4
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:51 2013 -0700

    rwsem: move rwsem_down_failed_common code into rwsem_down_{read,write}_failed

    Remove the rwsem_down_failed_common function and replace it with two
    identical copies of its code in rwsem_down_{read,write}_failed.

    This is because we want to make different optimizations in
    rwsem_down_{read,write}_failed; we are adding this pure-duplication
    step as a separate commit in order to make it easier to check the
    following steps.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Rik van Riel <riel@redhat.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 7d2869248ccd7971fc1b79e4bf22cb4be446faf4
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:50 2013 -0700

    rwsem: shorter spinlocked section in rwsem_down_failed_common()

    This change reduces the size of the spinlocked and TASK_UNINTERRUPTIBLE
    sections in rwsem_down_failed_common():

    - We only need the sem->wait_lock to insert ourselves on the wait_list;
      the waiter node can be prepared outside of the wait_lock.

    - The task state only needs to be set to TASK_UNINTERRUPTIBLE immediately
      before checking if we actually need to sleep; it doesn't need to protect
      the entire function.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Rik van Riel <riel@redhat.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 8c163f8ddf5c0bd018706dbdd44fb96a073401d0
Author: Michel Lespinasse <walken@google.com>
Date:   Tue May 7 06:45:49 2013 -0700

    rwsem: make the waiter type an enumeration rather than a bitmask

    We are not planning to add some new waiter flags, so we can convert the
    waiter type into an enumeration.

    Background: David Howells suggested I do this back when I tried adding
    a new waiter type for unfair readers. However, I believe the cleanup
    applies regardless of that use case.

    Signed-off-by: Michel Lespinasse <walken@google.com>
    Reviewed-by: Rik van Riel <riel@redhat.com>
    Reviewed-by: Peter Hurley <peter@hurleysoftware.com>
    Acked-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

commit 9a126edbed8ccfedda04d5c3fe256cd139d4bcca
Author: Alex Shi <alex.shi@intel.com>
Date:   Tue Feb 5 21:11:55 2013 +0800

    rwsem: Implement writer lock-stealing for better scalability

    Commit 5a505085f043 ("mm/rmap: Convert the struct anon_vma::mutex
    to an rwsem") changed struct anon_vma::mutex to an rwsem, which
    caused aim7 fork_test performance to drop by 50%.

    Yuanhan Liu did the following excellent analysis:

        https://lkml.org/lkml/2013/1/29/84

    and found that the regression is caused by strict, serialized,
    FIFO sequential write-ownership of rwsems. Ingo suggested
    implementing opportunistic lock-stealing for the front writer
    task in the waitqueue.

    Yuanhan Liu implemented lock-stealing for spinlock-rwsems,
    which indeed recovered much of the regression - confirming
    the analysis that the main factor in the regression was the
    FIFO writer-fairness of rwsems.

    In this patch we allow lock-stealing to happen when the first
    waiter is also writer. With that change in place the
    aim7 fork_test performance is fully recovered on my
    Intel NHM EP, NHM EX, SNB EP 2S and 4S test-machines.

    Reported-by: lkp@linux.intel.com
    Reported-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
    Signed-off-by: Alex Shi <alex.shi@intel.com>
    Cc: David Howells <dhowells@redhat.com>
    Cc: Michel Lespinasse <walken@google.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Anton Blanchard <anton@samba.org>
    Cc: Arjan van de Ven <arjan@linux.intel.com>
    Cc: paul.gortmaker@windriver.com
    Link: https://lkml.org/lkml/2013/1/29/84
    Link: http://lkml.kernel.org/r/1360069915-31619-1-git-send-email-alex.shi@intel.com
    [ Small stylistic fixes, updated changelog. ]
    Signed-off-by: Ingo Molnar <mingo@kernel.org>

commit 77302ff69a93cfba32aaee98d2bb864ba8886e40
Author: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Date:   Fri Feb 1 18:59:16 2013 +0800

    rwsem-spinlock: Implement writer lock-stealing for better scalability

    We (Linux Kernel Performance project) found a regression
    introduced by commit:

      5a505085f043 mm/rmap: Convert the struct anon_vma::mutex to an rwsem

    which converted all anon_vma::mutex locks rwsem write locks.

    The semantics are the same, but the behavioral difference is
    quite huge in some cases. After investigating it we found the
    root cause: mutexes support lock stealing while rwsems don't.

    Here is the link for the detailed regression report:

      https://lkml.org/lkml/2013/1/29/84

    Ingo suggested adding write lock stealing to rwsems:

        "I think we should allow lock-steal between rwsem writers - that
         will not hurt fairness as most rwsem fairness concerns relate to
         reader vs. writer fairness"

    And here is the rwsem-spinlock version.

    With this patch, we got a double performance increase in one
    test box with following aim7 workfile:

        FILESIZE: 1M
        POOLSIZE: 10M
        10 fork_test

     /usr/bin/time output w/o patch                       /usr/bin/time_output with patch
     -- Percent of CPU this job got: 369%                 Percent of CPU this job got: 537%
     Voluntary context switches: 640595016                Voluntary context switches: 157915561

    We got a 45% increase in CPU usage and saved about 3/4 voluntary context switches.

    Reported-by: LKP project <lkp@linux.intel.com>
    Suggested-by: Ingo Molnar <mingo@kernel.org>
    Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
    Cc: Alex Shi <alex.shi@intel.com>
    Cc: David Howells <dhowells@redhat.com>
    Cc: Michel Lespinasse <walken@google.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Andrew Morton <akpm@linux-foundation.org>
    Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Cc: Anton Blanchard <anton@samba.org>
    Cc: Arjan van de Ven <arjan@linux.intel.com>
    Cc: paul.gortmaker@windriver.com
    Link: http://lkml.kernel.org/r/1359716356-23865-1-git-send-email-yuanhan.liu@linux.intel.com
    Signed-off-by: Ingo Molnar <mingo@kernel.org>

commit 95f8b6683b3f83e40a8ab0d93a9d155954b64396
Author: Andrei F <luxneb@gmail.com>
Date:   Wed Aug 21 21:49:39 2013 +0400

    ARM: rwsem: Update header from initial implementation

    Cleanup due to:

    c16a87ce063f79e0ec7d25ce2950e1bc6db03c72 rwsem: Cleanup includes
    1c8ed640d918290ddc1de5ada02ef6686a733c9f rwsem: Move duplicate struct rwsem declaration to linux/rwsem.h
    12249b34414dba7f386aadcf6be7ca36c6878300 rwsem: Move duplicate init macros and functions to linux/rwsem.h
    41e5887fa39ab272d9266a09cbefdef270e28b93 rwsem: Unify the duplicate rwsem_is_locked() inlines
    aac72277fda6ef788bb8d5deaa502ce9b9b6e472 rwsem: Move duplicate function prototypes to linux/rwsem.h
Disable ondemand
ARM: sched_clock: Load cycle count after epoch stabilizes

There is a small race between when the cycle count is read from
the hardware and when the epoch stabilizes. Consider this
scenario:

 CPU0                           CPU1
 ----                           ----
 cyc = read_sched_clock()
 cyc_to_sched_clock()
                                 update_sched_clock()
                                  ...
                                  cd.epoch_cyc = cyc;
  epoch_cyc = cd.epoch_cyc;
  ...
  epoch_ns + cyc_to_ns((cyc - epoch_cyc)

The cyc on cpu0 was read before the epoch changed. But we
calculate the nanoseconds based on the new epoch by subtracting
the new epoch from the old cycle count. Since epoch is most likely
larger than the old cycle count we calculate a large number that
will be converted to nanoseconds and added to epoch_ns, causing
time to jump forward too much.

Fix this problem by reading the hardware after the epoch has
stabilized.

Change-Id: I995133b229b2c2fedd5091406d1dc366d8bfff7b
Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Git-commit: 336ae1180df5f69b9e0fb6561bec01c5f64361cf
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[sboyd: reworked for file movement kernel/time -> arm/kernel]
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>

Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
cpufreq: interactive: merge mako changes & in-kernel touchboost implementation. If you use this you gotta ditch the power.msm8974.so file inside /system/lib/hw or it will most likely cause some weird conflicts.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>

Conflicts:
	drivers/cpufreq/cpufreq_interactive.c
cpufreq: interactive: remove cpu_online() check, cpufreq takes care of that by itself. Also update the input_boost_freq to a valid freq from the cpufreq table for msm8974 and increase hispeed_freq for some extra performance juice when needed.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: increase timer_rate to diminish unwanted CPU spikes. Increase input_boost_freq to compensate.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: decrease go_hispeed_load to 85.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: remove get_cpu_idle_time because we're now using the generic version from cpufreq.c. Rework boostpulse to boost directly from the cpufreq target function instead of going through the speedtask thread. Add Thread Migration notifier to boost the target cpu that receives a thread migration to ensure no loss of perceived performance.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: I guess queuing everything up to the governor kthread is what should be done to avoid panics.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: instead of bumping the target_cpu freq to the source_cpu current freq make it boost to SYNC_FREQ instead at 960MHz (maybe configurable in the future?) and that should be enough and help with the performance/battery ratio.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: fix input_boost_freq not being always choosen for target_freq when boosted is true. Also do not boost the target_cpu to SYNC_FREQ if the source_cpu current frequency is not higher than target_cpu current freq to avoid overusage of SYNC_FREQ.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: instead of using choose_freq() to calculate the new_freq lets try a more ondemand approach by using the actual load scaling to make it more consistent, specially when gaming.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: fix a bug where the thread_migration notifier was wrongly pushing the cpu to use higher freqs a lot more than it should.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: fallback to previous new_freq calculation as it is much more conservative. Also fix the new_freq logic, specially if boosted is true.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: interactive: Add a sampling_down_factor for max frequencies

Change min_sample_time to 100ms when running at max

Change-Id: Ia7e35b0625bedf20e7ef3a1f52e5828ffbfed93e
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
cpufreq: interactive: qqueue input boost to a high priority workqueue instead of having it go through the kthread speedchange path. This saves us spinlocks/unlocks everytime there's a touch and it saves us a couple ms to wait until the kthread is woken, then going through its code path, then the calculations, then finally calling the cpufreq target function. I measured a couple ms improvements in terms of responsiveness after touching the display and having cpufreq boosting the online cpus.

Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
cpufreq: interactive: Attempt to fix a Kernel Panic introduced on r18 which was caused by a cpu core going offline while a queued work was being processed in that cpu. Fix is easy, the work is now being queued to cpu0 and its bound to that cpu until its finished.

Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>