staging: zcache: support multiple clients, prep for KVM and RAMster

This is version 3 of an update to zcache, incorporating feedback from the list.
This patch adds support to the in-kernel transcendent memory ("tmem") code
and the zcache driver for multiple clients, which will be needed for both
RAMster and KVM support.  It also adds additional tmem callbacks to support
RAMster and corresponding no-op stubs in the zcache driver.  In v2, I've
also taken the liberty of adding some additional sysfs variables to
both surface information and allow policy control.  Those experimenting
with zcache should find them useful.  V3 clarifies some code walking
and declaring arrays.

Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>

[v3: error27@gmail.com: fix array bounds/walking]
[v2: konrad.wilk@oracle.com: fix bools, add check for NULL, fix a comment]
[v2: sjenning@linux.vnet.ibm.com: add info/tunables for poor compression]
[v2: marcusklemm@googlemail.com: add tunable for max persistent pages]
Acked-by: Dan Carpenter <error27@gmail.com>
Cc: Nitin Gupta <ngupta@vflare.org>
Cc: linux-mm@kvack.org
Cc: kvm@vger.kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: fix zcache building

zcache is only building tmem.c and not building zcache.c. To keep the
module name, zcache.c must be renamed if symbols from tmem.c are to
remain unexported.

Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@holoscopio.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: module is GPL

This avoids tainting the kernel as if a proprietary module was loaded.
The kernel will still be tainted because this is a staging driver.

Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@holoscopio.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: include module.h for MODULE_LICENSE

The oncoming cleanup of module.h usage requires the explicit inclusion
of module.h when it was otherwise being included indirectly. Otherwise,
building zcache will fail.

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@holoscopio.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

zcache: Use div_u64 for 64-bit division

xv_get_total_size_bytes returns a u64 value and it's used in a division.
This causes build failures in 32-bit architectures, as reported by Randy
Dunlap.

Reported-by: Randy Dunlap <rdunlap@xenotime.net>
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@holoscopio.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Dan Magenheimer <dan.magenheimer@oracle.com>
Cc: Nitin Gupta <ngupta@vflare.org>
Acked-by: Randy Dunlap <rdunlap@xenotime.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

zcache: Fix build error when sysfs is not defined

Signed-off-by: Nitin Gupta <ngupta@vflare.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: fix possible sleep under lock

zcache_new_pool() calls kmalloc() with GFP_KERNEL which has
__GFP_WAIT set.  However, zcache_new_pool() gets called on
a stack that holds the swap_lock spinlock, leading to a
possible sleep-with-lock situation. The lock is obtained
in enable_swap_info().

The patch replaces GFP_KERNEL with GFP_ATOMIC.

v2: replace with GFP_ATOMIC, not GFP_IOFS

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: fix typos

The patch fixes two typos in zcache-main.c

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: fix crash on high memory swap

zcache_put_page() was modified to pass page_address(page) instead of the
actual page structure. In combination with the function signature changes
to tmem_put() and zcache_pampd_create(), zcache_pampd_create() tries to
(re)derive the page structure from the virtual address.  However, if the
original page is a high memory page (or any unmapped page), this
virt_to_page() fails because the page_address() in zcache_put_page()
returned NULL.

This patch changes zcache_put_page() and zcache_get_page() to pass
the page structure instead of the page's virtual address, which
may or may not exist.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

Staging: zcache: signedness bug in tmem_get()

"ret" needs to be signed for the error handling to work properly.

Signed-off-by: Dan Carpenter <error27@gmail.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: fix cleancache crash

After commit c5f5c4db3938 ("staging: zcache: fix crash on high memory
swap") cleancache crashes on the first successful get.  This was caused
by a remaining virt_to_page() call in zcache_pampd_get_data_and_free()
that only gets run in the cleancache path.

The patch converts the virt_to_page() to struct page casting like was
done for other instances in c5f5c4db3938.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Tested-By: Valdis Kletnieks <valdis.kletnieks@vt.edu>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

staging: zcache: fix crash on cpu remove

In the case that a cpu is taken offline before zcache_do_preload() is
ever called on the cpu, the per-cpu zcache_preloads structure will
be uninitialized.  In the CPU_DEAD case for zcache_cpu_notifier(),
kp->obj is not checked before calling kmem_cache_free() on it.
If it is NULL, a crash results.

This patch ensures that both kp->obj and kp->page are not NULL before
calling the respective free functions. In practice, just checking
one or the other should be sufficient since they are assigned together
in zcache_do_preload(), but I check both for safety.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Acked-by: Dave Hansen <dave@linux.vnet.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: reduce tmem bucket lock contention

tmem uses hash buckets each with their own rbtree and lock to
quickly lookup tmem objects.  tmem has TMEM_HASH_BUCKETS (256)
buckets per pool.  However, because of the way the tmem_oid is
generated for frontswap pages, only 16 unique tmem_oids are being
generated, resulting in only 16 of the 256 buckets being used.
This cause high lock contention for the per bucket locks.

This patch changes SWIZ_BITS to include more bits of the offset.
The result is that all 256 hash buckets are potentially used resulting in a
95% drop in hash bucket lock contention.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: remove zcache_direct_reclaim_lock

zcache_do_preload() currently does a spin_trylock() on the
zcache_direct_reclaim_lock. Holding this lock intends to prevent
shrink_zcache_memory() from evicting zbud pages as a result
of a preload.

However, it also prevents two threads from
executing zcache_do_preload() at the same time.  The first
thread will obtain the lock and the second thread's spin_trylock()
will fail (an aborted preload) causing the page to be either lost
(cleancache) or pushed out to the swap device (frontswap). It
also doesn't ensure that the call to shrink_zcache_memory() is
on the same thread as the call to zcache_do_preload().

Additional, there is no need for this mechanism because all
zcache_do_preload() calls that come down from cleancache already
have PF_MEMALLOC set in the process flags which prevents
direct reclaim in the memory manager. If the zcache_do_preload()
call is done from the frontswap path, we _want_ reclaim to be
done (which it isn't right now).

This patch removes the zcache_direct_reclaim_lock and related
statistics in zcache.

Based on v3.1-rc8

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Reviewed-by: Dave Hansen <dave@linux.vnet.ibm.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

Staging: zcache: Fix calls to obsolete function

Function "strict_strtol" replaced by "kstrtol" as suggested by the checkpatch script

Signed-off-by: Bernhard Heinloth <bernhard@heinloth.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

zcache: fix deadlock condition

I discovered this deadlock condition awhile ago working on RAMster
but it affects zcache as well.  The list spinlock must be
locked prior to the page spinlock and released after.  As
a result, the page copy must also be done while the locks are held.

Applies to 3.2.  Konrad, please push (via GregKH?)...
this is definitely a bug fix so need not be pushed during
a -rc0 window.

Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

zcache: Set SWIZ_BITS to 8 to reduce tmem bucket lock contention.

SWIZ_BITS > 8 results in a much larger number of "tmem_obj"
allocations, likely one per page-placed-in-frontswap.  The
tmem_obj is not huge (roughly 100 bytes), but it is large
enough to add a not-insignificant memory overhead to zcache.

The SWIZ_BITS=8  will get roughly the same lock contention
without the space wastage.

The effect of SWIZ_BITS can be thought of as "2^SWIZ_BITS is
the number of unique oids that be generated" (This concept is
limited to frontswap's use of tmem).

Acked-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

staging: zcache: fix serialization bug in zv stats

In a multithreaded workload, the zv_curr_dist_counts
and zv_cumul_dist_counts statistics are being corrupted
because the increments and decrements in zv_create
and zv_free are not atomic.

This patch converts these statistics and their corresponding
increments/decrements/reads to atomic operations.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

staging: zcache: crypto API support

This patch allow zcache to use the crypto API for page compression.
It replaces the direct LZO compress/decompress calls with calls
into the crypto compression API. The compressor to be used is
specified in the kernel boot line with the zcache parameter like:
zcache=lzo or zcache=deflate.  If the specified compressor can't
be loaded, zcache uses lzo as the default compressor.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

staging: zcache: avoid AB-BA deadlock condition

Commit 9256a47 fixed a deadlock condition, being sure that the buddy
list spinlock is always taken before the page spinlock.

However in zbud_free_and_delist() locking order is the opposite
(page lock -> list lock).

Possible unsafe locking scenario (reported by lockdep):

        CPU0                    CPU1
        ----                    ----
   lock(&(&zbpg->lock)->rlock);
                                lock(zbud_budlists_spinlock);
                                lock(&(&zbpg->lock)->rlock);
   lock(zbud_budlists_spinlock);

Fix by grabbing the locks in opposite order in zbud_free_and_delist().

Signed-off-by: Andrea Righi <andrea@betterlinux.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
drivers: qcache: Port various fixes from zcache

 * Fix various lockups to improve stability

Change-Id: Ifeec71a934aa5f08b1b0b57bb05e22d0d7bf84e6
msm_fb: display: add wait after overlay kickoff

MDP writeback overlay kick off needs to be a blocking call to avoid
WFD tearing issues.

Change-Id: I03ce16b1e11c458bc8602880f22f1aed536a2e5a
Signed-off-by: Huaibin Yang <huaibiny@codeaurora.org>
(cherry picked from commit 7c31ee7933fff437acfaa69cf443c81638a495de)
msm: Remove idle stats and cpuidle hooks

Calling RCU functions from within the RCU quiescent state is
illegal and probably a bug. The idlestats code uses an atomic
notifier called from msm_cpuidle_enter() and that atomic notifier
takes an rcu read lock. We could fix this with a raw notifier,
but this code is unused so let's just remove the code entirely.

Change-Id: I319f0eabb56a8721c19a3b2b4bc9a7f416c88384
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>

Conflicts:
	arch/arm/mach-msm/cpuidle.c
msm: cpuidle: Don't disable irqs again

cpu_idle() disables irqs before calling down into the cpuidle
driver via cpuidle_idle_call(). Disabling irqs again in
msm_cpuidle_enter() is wasteful and unnecessary. It also inhibits
our ability to determine the largest irq latency when the cpu
idles for extended periods of time because the irqsoff trace is
reenabled when the irqs are disabled in msm_cpuidle_enter().

Note that we still reenable irqs before returning, as is required
by the ARM idle loop.

Change-Id: I925478d8fef88ab490959622bc68303f59cc6e79
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
msm: cpuidle: Update C state time stats in the cpu idle device.

msm_pm_idle_enter function returns time spent by cpu in particular
C state. Update the cpu idle device last residency with the return
value which is required for cpuidle stats.

CRs-fixed: 391456
Change-Id: If9f1b59e05c233e53ba37ae0fe9399f12275f3d8
Signed-off-by: Anji Jonnala <anjir@codeaurora.org>
ion: Fix build with qcache disabled

 * Qcache is still unstable and buggy, but currently crashes if fully
   disabled on 8960.
 * Allow disabling of Qcache for 8960 and fix the dependency in msm_ion.

Change-Id: I3b2fc40ecce757530b64ddabb03081aaa425b603
d2: Disable Qcache, enable Zcache

 * Qcache is a great concept (use empty carveout space as backing for
   cleancache), but it's unstable and results in a performance
   degradation.
 * Zcache improves performance and does not show the same type of
   degradation over time.
 * While Qcache *should* be a better technique, let's disable it
   entirely and use Zcache for now until the issues have been fixed.

Change-Id: I6dc827b8a89b59dd80dfdc862720d525447d413b
d2: Only enable FMEM if QCACHE is enabled

 * Currently not recommended for use.

Change-Id: Ifcb56cf06e247f16442e3f70d6e5422c19db317c
mutex: Place lock in contended state after fastpath_lock failure

ARM recently moved to asm-generic/mutex-xchg.h for its mutex
implementation after the previous implementation was found to be missing
some crucial memory barriers. However, this has revealed some problems
running hackbench on SMP platforms due to the way in which the
MUTEX_SPIN_ON_OWNER code operates.

The symptoms are that a bunch of hackbench tasks are left waiting on an
unlocked mutex and therefore never get woken up to claim it. This boils
down to the following sequence of events:

        Task A        Task B        Task C        Lock value
0                                                     1
1       lock()                                        0
2                     lock()                          0
3                     spin(A)                         0
4       unlock()                                      1
5                                   lock()            0
6                     cmpxchg(1,0)                    0
7                     contended()                    -1
8       lock()                                        0
9       spin(C)                                       0
10                                  unlock()          1
11      cmpxchg(1,0)                                  0
12      unlock()                                      1

At this point, the lock is unlocked, but Task B is in an uninterruptible
sleep with nobody to wake it up.

This patch fixes the problem by ensuring we put the lock into the
contended state if we fail to acquire it on the fastpath, ensuring that
any blocked waiters are woken up when the mutex is released.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: <stable@vger.kernel.org>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-6e9lrw2avczr0617fzl5vqb8@git.kernel.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
ARM: 7467/1: mutex: use generic xchg-based implementation for ARMv6+

The open-coded mutex implementation for ARMv6+ cores suffers from a
severe lack of barriers, so in the uncontended case we don't actually
protect any accesses performed during the critical section.

Furthermore, the code is largely a duplication of the ARMv6+ atomic_dec
code but optimised to remove a branch instruction, as the mutex fastpath
was previously inlined. Now that this is executed out-of-line, we can
reuse the atomic access code for the locking (in fact, we use the xchg
code as this produces shorter critical sections).

This patch uses the generic xchg based implementation for mutexes on
ARMv6+, which introduces barriers to the lock/unlock operations and also
has the benefit of removing a fair amount of inline assembly code.

Change-Id: I6ec1a724cfc6dcf228c0bae7f0b867c6435a92f1
Cc: <stable@vger.kernel.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Nicolas Pitre <nico@linaro.org>
Reported-by: Shan Kang <kangshan0910@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Revert "msm: kgsl: Allow the wait_timeout to be disabled"

This reverts commit 80572858ea4daccb8567d3679ec126fc19bea8ee.
Revert "msm: kgsl: Don't use 'wait_timeout' in the idle functions"

This reverts commit ec3fb19476c1b88ae4057314e3a1593ab4fceecb.
Revert "msm: pm: clean msm_pm_pc_pgd in the cache after writing"

This reverts commit e6141d285de04aefa9bc35913b8d554cd99a652c.
Revert "msm: idle-v7: Remove redundant cache clean"

This reverts commit bb3fb8e3c56edc84dcece8d3579a66e67f8386a2.
Revert "msm: idle-v7: Power collapse Krait with caches enabled"

This reverts commit 18711d3aec24f0a63254c453410c3b249f667ad2.
Revert "msm: memory: Calculate DDR size early at boot time"

This reverts commit cb042524dd9accfc3e0213139c84168a18b6e2e8.
Revert "GOV: Fix touch booster for powerHAL"

This reverts commit 5493ba5a606f1ec188e0cb42fc7ec623dbfab108.
CONFIG: Version 0.6
CONFIG: CM10 update