Makefile: *updating GCC flags*
defconfig: *update*
*squashing some fixes and reverts*

msm: avs: Remove vestigial software based AVS

      Older targets like 8x50 had the first version of Apdaptive Voltage
      Scaling done in software. This feature was never enabled and supported.
      Remove the code and the associated config items.

      Change-Id: If3bdab7d0bdb5d51e099f7c37f56592826edc102
      Signed-off-by: Praveen Chidambaram <pchidamb@codeaurora.org>

drivers: msm_nand: *silence GCC warnings*

      Signed-off-by: moddingg33k <moddingg33k@t-online.de>

fs: ext4: reverting huawei specific changes

      Signed-off-by: moddingg33k <moddingg33k@t-online.de>

msm: acpuclock: Add a null check for acpuclk_data

      If acpuclock fails to probe before a client calls acpuclk_get_rate,
      a null dereference will occur. Fix this by adding the appropriate
      checks.

      Change-Id: If873a64711e9cc807b1734996f7c9358bb08161a
      Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>

Revert "ARM: armv7 no need update SCTLR for alignment"

      This reverts commit 6dfaacc2dc708f11863d144ac41882af4315d4df.

PM / tracing: remove deprecated power trace API

      The text in Documentation said it would be removed in 2.6.41;
      the text in the Kconfig said removal in the 3.1 release.  Either
      way you look at it, we are well past both, so push it off a cliff.

      Note that the POWER_CSTATE and the POWER_PSTATE are part of the
      legacy tracing API.  Remove all tracepoints which use these flags.
      As can be seen from context, most already have a trace entry via
      trace_cpu_idle anyways.

      Also, the cpufreq/cpufreq.c PSTATE one is actually unpaired, as
      compared to the CSTATE ones which all have a clear start/stop.
      As part of this, the trace_power_frequency also becomes orphaned,
      so it too is deleted.

      Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
      Acked-by: Steven Rostedt <rostedt@goodmis.org>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

cpufreq: ondemand: reverted huawei changes
[ARM: CRYPTO]: optimized AES and SHA1 routines & NEON accelerated XOR

arm/crypto: Add optimized AES and SHA1 routines

      Add assembler versions of AES and SHA1 for ARM platforms.  This has provided
      up to a 50% improvement in IPsec/TCP throughout for tunnels using AES128/SHA1.

      Platform   CPU SPeed    Endian   Before (bps)   After (bps)   Improvement

      IXP425      533 MHz      big     11217042        15566294        ~38%
      KS8695      166 MHz     little    3828549         5795373        ~51%

      Signed-off-by: David McCullough <ucdevel@gmail.com>
      Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

ARM: 7626/1: arm/crypto: Make asm SHA-1 and AES code Thumb-2 compatible

      This patch fixes aes-armv4.S and sha1-armv4-large.S to work
      natively in Thumb.  This allows ARM/Thumb interworking workarounds
      to be removed.

      I also take the opportunity to convert some explicit assembler
      directives for exported functions to the standard
      ENTRY()/ENDPROC().

      For the code itself:

	* In sha1_block_data_order, use of TEQ with sp is deprecated in
	  ARMv7 and not supported in Thumb.  For the branches back to
	  .L_00_15 and .L_40_59, the TEQ is converted to a CMP, under the
	  assumption that clobbering the C flag here will not cause
	  incorrect behaviour.

	  For the first branch back to .L_20_39_or_60_79 the C flag is
	  important, so sp is moved temporarily into another register so
	  that TEQ can be used for the comparison.

	* In the AES code, most forms of register-indexed addressing with
	  shifts and rotates are not permitted for loads and stores in
	  Thumb, so the address calculation is done using a separate
	  instruction for the Thumb case.

      The resulting code is unlikely to be optimally scheduled, but it
      should not have a large impact given the overall size of the code.
      I haven't run any benchmarks.

      Signed-off-by: Dave Martin <dave.martin@linaro.org>
      Tested-by: David McCullough <ucdevel@gmail.com> (ARM only)
      Acked-by: David McCullough <ucdevel@gmail.com>
      Acked-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 7723/1: crypto: sha1-armv4-large.S: fix SP handling

      Make the SHA1 asm code ABI conformant by making sure all stack
      accesses occur above the stack pointer.

      Origin:
      http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=1a9d60d2

      Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
      Acked-by: Nicolas Pitre <nico@linaro.org>
      Cc: stable@vger.kernel.org
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 7837/3: fix Thumb-2 bug in AES assembler code

      Patch 638591c enabled building the AES assembler code in Thumb2 mode.
      However, this code used arithmetic involving PC rather than adr{l}
      instructions to generate PC-relative references to the lookup tables,
      and this needs to take into account the different PC offset when
      running in Thumb mode.

      Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
      Acked-by: Nicolas Pitre <nico@linaro.org>
      Cc: stable@vger.kernel.org
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

      ARM: move AES typedefs and function prototypes to separate header

      Put the struct definitions for AES keys and the asm function prototypes in a
      separate header and export the asm functions from the module.
      This allows other drivers to use them directly.

      Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

      ARM: add support for bit sliced AES using NEON instructions

      Bit sliced AES gives around 45% speedup on Cortex-A15 for encryption
      and around 25% for decryption. This implementation of the AES algorithm
      does not rely on any lookup tables so it is believed to be invulnerable
      to cache timing attacks.

      This algorithm processes up to 8 blocks in parallel in constant time. This
      means that it is not usable by chaining modes that are strictly sequential
      in nature, such as CBC encryption. CBC decryption, however, can benefit from
      this implementation and runs about 25% faster. The other chaining modes
      implemented in this module, XTS and CTR, can execute fully in parallel in
      both directions.

      The core code has been adopted from the OpenSSL project (in collaboration
      with the original author, on cc). For ease of maintenance, this version is
      identical to the upstream OpenSSL code, i.e., all modifications that were
      required to make it suitable for inclusion into the kernel have been made
      upstream. The original can be found here:

	  http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=6f6a6130

      Note to integrators:
      While this implementation is significantly faster than the existing table
      based ones (generic or ARM asm), especially in CTR mode, the effects on
      power efficiency are unclear as of yet. This code does fundamentally more
      work, by calculating values that the table based code obtains by a simple
      lookup; only by doing all of that work in a SIMD fashion, it manages to
      perform better.

      Cc: Andy Polyakov <appro@openssl.org>
      Acked-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

ARM: add .gitignore entry for aesbs-core.S

      This avoids this file being incorrectly added to git.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

CRYPTO: Fix more AES build errors

      Building a multi-arch kernel results in:

      arch/arm/crypto/built-in.o: In function `aesbs_xts_decrypt':
      sha1_glue.c:(.text+0x15c8): undefined reference to `bsaes_xts_decrypt'
      arch/arm/crypto/built-in.o: In function `aesbs_xts_encrypt':
      sha1_glue.c:(.text+0x1664): undefined reference to `bsaes_xts_encrypt'
      arch/arm/crypto/built-in.o: In function `aesbs_ctr_encrypt':
      sha1_glue.c:(.text+0x184c): undefined reference to `bsaes_ctr32_encrypt_blocks'
      arch/arm/crypto/built-in.o: In function `aesbs_cbc_decrypt':
      sha1_glue.c:(.text+0x19b4): undefined reference to `bsaes_cbc_encrypt'

      This code is already runtime-conditional on NEON being supported, so
      there's no point compiling it out depending on the minimum build
      architecture.

      Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

crypto: create generic version of ablk_helper

      Create a generic version of ablk_helper so it can be reused
      by other architectures.

      Acked-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
      Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
      Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

---------------------------------------------------------------------------------------------------

ARM: crypto: add NEON accelerated XOR implementation

      Add a source file xor-neon.c (which is really just the reference
      C implementation passed through the GCC vectorizer) and hook it
      up to the XOR framework.

      Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
      Acked-by: Nicolas Pitre <nico@linaro.org>

ARM: 7835/2: fix modular build of xor_blocks() with NEON enabled

      Commit 0195659 introduced a NEON accelerated version of the xor_blocks()
      function, but it needs the changes in this patch to allow it to be built
      as a module rather than statically into the kernel.

      This patch creates a separate module xor-neon.ko which exports the NEON
      inner xor_blocks() functions depended upon by the regular xor.ko if it
      is built with CONFIG_KERNEL_MODE_NEON=y

      Reported-by: Josh Boyer <jwboyer@fedoraproject.org>
      Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

Optimized ARM RWSEM algorithm

      RWSEM implementation for ARM using atomic functions.
      Heavily based on arch/sh/include/asm/rwsem.h

      Signed-off-by: Ashwin Chaugule <ashwinc@codeaurora.org>
[ARM: MM] implement LoUIS API for cache maintenance ops

ARM: mm: implement LoUIS API for cache maintenance ops

      ARM v7 architecture introduced the concept of cache levels and related
      control registers. New processors like A7 and A15 embed an L2 unified cache
      controller that becomes part of the cache level hierarchy. Some operations in
      the kernel like cpu_suspend and __cpu_disable do not require a flush of the
      entire cache hierarchy to DRAM but just the cache levels belonging to the
      Level of Unification Inner Shareable (LoUIS), which in most of ARM v7 systems
      correspond to L1.

      The current cache flushing API used in cpu_suspend and __cpu_disable,
      flush_cache_all(), ends up flushing the whole cache hierarchy since for
      v7 it cleans and invalidates all cache levels up to Level of Coherency
      (LoC) which cripples system performance when used in hot paths like hotplug
      and cpuidle.

      Therefore a new kernel cache maintenance API must be added to cope with
      latest ARM system requirements.

      This patch adds flush_cache_louis() to the ARM kernel cache maintenance API.

      This function cleans and invalidates all data cache levels up to the
      Level of Unification Inner Shareable (LoUIS) and invalidates the instruction
      cache for processors that support it (> v7).

      This patch also creates an alias of the cache LoUIS function to flush_kern_all
      for all processor versions prior to v7, so that the current cache flushing
      behaviour is unchanged for those processors.

      v7 cache maintenance code implements a cache LoUIS function that cleans and
      invalidates the D-cache up to LoUIS and invalidates the I-cache, according
      to the new API.

      Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
      Reviewed-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Tested-by: Shawn Guo <shawn.guo@linaro.org>

ARM: mm: rename jump labels in v7_flush_dcache_all function

      This patch renames jump labels in v7_flush_dcache_all in order to define
      a specific flush cache levels entry point.

      Acked-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Tested-by: Shawn Guo <shawn.guo@linaro.org>

ARM: kernel: update cpu_suspend code to use cache LoUIS operations

      In processors like A15/A7 L2 cache is unified and integrated within the
      processor cache hierarchy, so that it is not considered an outer cache
      anymore. For processors like A15/A7 flush_cache_all() ends up cleaning
      all cache levels up to Level of Coherency (LoC) that includes
      the L2 unified cache.

      When a single CPU is suspended (CPU idle) a complete L2 clean is not
      required, so generic cpu_suspend code must clean the data cache using the
      newly introduced cache LoUIS function.

      The context and stack pointer (context pointer) are cleaned to main memory
      using cache area functions that operate on MVA and guarantee that the data
      is written back to main memory (perform cache cleaning up to the Point of
      Coherency - PoC) so that the processor can fetch the context when the MMU
      is off in the cpu_resume code path.

      outer_cache management remains unchanged.

      Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
      Reviewed-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Tested-by: Shawn Guo <shawn.guo@linaro.org>

ARM: kernel: update __cpu_disable to use cache LoUIS maintenance API

      When a CPU is hotplugged out caches that reside in its power domain
      lose their contents and so must be cleaned to the next memory level.

      Currently, __cpu_disable calls flush_cache_all() that for new generation
      processor like A15/A7 ends up cleaning and invalidating all cache levels
      up to Level of Coherency, which includes the unified L2.

      This ends up being a waste of cycles since the L2 cache contents are not
      lost on power down.

      This patch updates __cpu_disable to use the new LoUIS API cache operations.

      Acked-by: Nicolas Pitre <nico@linaro.org>
      Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
      Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Tested-by: Shawn Guo <shawn.guo@linaro.org>
[ARM: GENERIC] *squashing updates*

ARM: user generic termios.h

ARM: 7493/1: use generic unaligned.h

      This moves ARM over to the asm-generic/unaligned.h header. This has the
      benefit of better code generated especially for ARMv7 on gcc 4.7+
      compilers.

      As Arnd Bergmann, points out: The asm-generic version uses the "struct"
      version for native-endian unaligned access and the "byteshift" version
      for the opposite endianess. The current ARM version however uses the
      "byteshift" implementation for both.

      Thanks to Nicolas Pitre for the excellent analysis:

      Test case:

      int foo (int *x) { return get_unaligned(x); }
      long long bar (long long *x) { return get_unaligned(x); }

      With the current ARM version:

      foo:
	      ldrb	r3, [r0, #2]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 2B], MEM[(const u8 *)x_1(D) + 2B]
	      ldrb	r1, [r0, #1]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 1B], MEM[(const u8 *)x_1(D) + 1B]
	      ldrb	r2, [r0, #0]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D)], MEM[(const u8 *)x_1(D)]
	      mov	r3, r3, asl #16	@ tmp154, MEM[(const u8 *)x_1(D) + 2B],
	      ldrb	r0, [r0, #3]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 3B], MEM[(const u8 *)x_1(D) + 3B]
	      orr	r3, r3, r1, asl #8	@, tmp155, tmp154, MEM[(const u8 *)x_1(D) + 1B],
	      orr	r3, r3, r2	@ tmp157, tmp155, MEM[(const u8 *)x_1(D)]
	      orr	r0, r3, r0, asl #24	@,, tmp157, MEM[(const u8 *)x_1(D) + 3B],
	      bx	lr	@

      bar:
	      stmfd	sp!, {r4, r5, r6, r7}	@,
	      mov	r2, #0	@ tmp184,
	      ldrb	r5, [r0, #6]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 6B], MEM[(const u8 *)x_1(D) + 6B]
	      ldrb	r4, [r0, #5]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 5B], MEM[(const u8 *)x_1(D) + 5B]
	      ldrb	ip, [r0, #2]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 2B], MEM[(const u8 *)x_1(D) + 2B]
	      ldrb	r1, [r0, #4]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 4B], MEM[(const u8 *)x_1(D) + 4B]
	      mov	r5, r5, asl #16	@ tmp175, MEM[(const u8 *)x_1(D) + 6B],
	      ldrb	r7, [r0, #1]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 1B], MEM[(const u8 *)x_1(D) + 1B]
	      orr	r5, r5, r4, asl #8	@, tmp176, tmp175, MEM[(const u8 *)x_1(D) + 5B],
	      ldrb	r6, [r0, #7]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 7B], MEM[(const u8 *)x_1(D) + 7B]
	      orr	r5, r5, r1	@ tmp178, tmp176, MEM[(const u8 *)x_1(D) + 4B]
	      ldrb	r4, [r0, #0]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D)], MEM[(const u8 *)x_1(D)]
	      mov	ip, ip, asl #16	@ tmp188, MEM[(const u8 *)x_1(D) + 2B],
	      ldrb	r1, [r0, #3]	@ zero_extendqisi2	@ MEM[(const u8 *)x_1(D) + 3B], MEM[(const u8 *)x_1(D) + 3B]
	      orr	ip, ip, r7, asl #8	@, tmp189, tmp188, MEM[(const u8 *)x_1(D) + 1B],
	      orr	r3, r5, r6, asl #24	@,, tmp178, MEM[(const u8 *)x_1(D) + 7B],
	      orr	ip, ip, r4	@ tmp191, tmp189, MEM[(const u8 *)x_1(D)]
	      orr	ip, ip, r1, asl #24	@, tmp194, tmp191, MEM[(const u8 *)x_1(D) + 3B],
	      mov	r1, r3	@,
	      orr	r0, r2, ip	@ tmp171, tmp184, tmp194
	      ldmfd	sp!, {r4, r5, r6, r7}
	      bx	lr

      In both cases the code is slightly suboptimal.  One may wonder why
      wasting r2 with the constant 0 in the second case for example.  And all
      the mov's could be folded in subsequent orr's, etc.

      Now with the asm-generic version:

      foo:
	      ldr	r0, [r0, #0]	@ unaligned	@,* x
	      bx	lr	@

      bar:
	      mov	r3, r0	@ x, x
	      ldr	r0, [r0, #0]	@ unaligned	@,* x
	      ldr	r1, [r3, #4]	@ unaligned	@,
	      bx	lr	@

      This is way better of course, but only because this was compiled for
      ARMv7. In this case the compiler knows that the hardware can do
      unaligned word access.  This isn't that obvious for foo(), but if we
      remove the get_unaligned() from bar as follows:

      long long bar (long long *x) {return *x; }

      then the resulting code is:

      bar:
	      ldmia	r0, {r0, r1}	@ x,,
	      bx	lr	@

      So this proves that the presumed aligned vs unaligned cases does have
      influence on the instructions the compiler may use and that the above
      unaligned code results are not just an accident.

      Still... this isn't fully conclusive without at least looking at the
      resulting assembly fron a pre ARMv6 compilation.  Let's see with an
      ARMv5 target:

      foo:
	      ldrb	r3, [r0, #0]	@ zero_extendqisi2	@ tmp139,* x
	      ldrb	r1, [r0, #1]	@ zero_extendqisi2	@ tmp140,
	      ldrb	r2, [r0, #2]	@ zero_extendqisi2	@ tmp143,
	      ldrb	r0, [r0, #3]	@ zero_extendqisi2	@ tmp146,
	      orr	r3, r3, r1, asl #8	@, tmp142, tmp139, tmp140,
	      orr	r3, r3, r2, asl #16	@, tmp145, tmp142, tmp143,
	      orr	r0, r3, r0, asl #24	@,, tmp145, tmp146,
	      bx	lr	@

      bar:
	      stmfd	sp!, {r4, r5, r6, r7}	@,
	      ldrb	r2, [r0, #0]	@ zero_extendqisi2	@ tmp139,* x
	      ldrb	r7, [r0, #1]	@ zero_extendqisi2	@ tmp140,
	      ldrb	r3, [r0, #4]	@ zero_extendqisi2	@ tmp149,
	      ldrb	r6, [r0, #5]	@ zero_extendqisi2	@ tmp150,
	      ldrb	r5, [r0, #2]	@ zero_extendqisi2	@ tmp143,
	      ldrb	r4, [r0, #6]	@ zero_extendqisi2	@ tmp153,
	      ldrb	r1, [r0, #7]	@ zero_extendqisi2	@ tmp156,
	      ldrb	ip, [r0, #3]	@ zero_extendqisi2	@ tmp146,
	      orr	r2, r2, r7, asl #8	@, tmp142, tmp139, tmp140,
	      orr	r3, r3, r6, asl #8	@, tmp152, tmp149, tmp150,
	      orr	r2, r2, r5, asl #16	@, tmp145, tmp142, tmp143,
	      orr	r3, r3, r4, asl #16	@, tmp155, tmp152, tmp153,
	      orr	r0, r2, ip, asl #24	@,, tmp145, tmp146,
	      orr	r1, r3, r1, asl #24	@,, tmp155, tmp156,
	      ldmfd	sp!, {r4, r5, r6, r7}
	      bx	lr

      Compared to the initial results, this is really nicely optimized and I
      couldn't do much better if I were to hand code it myself.

      Signed-off-by: Rob Herring <rob.herring@calxeda.com>
      Reviewed-by: Nicolas Pitre <nico@linaro.org>
      Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
      Reviewed-by: Arnd Bergmann <arnd@arndb.de>
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

arm: Add XN bit to pgrprot_stronglyordered

      pgprot_stronglyordered sets page attributes to the strongly
      ordered memory type. This type is often used for regions of memory which
      cannot be speculated into. Strongly ordered memory by itself prevents
      speculative data accesses but it will not prevent speculative instruction
      accesses. Since strongly ordered memory will never be used for
      instructions, set the XN bit as well to prevent speculative instruction
      fethces.

      Bug: 10750953
      Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
      Signed-off-by: Mekala Natarajan <mekalan@codeaurora.org>

ARM: 7495/1: mutex: use generic atomic_dec-based implementation for ARMv6+

      Commit a76d7bd96d65 ("ARM: 7467/1: mutex: use generic xchg-based
      implementation for ARMv6+") removed the barrier-less, ARM-specific
      mutex implementation in favour of the generic xchg-based code.

      Since then, a bug was uncovered in the xchg code when running on SMP
      platforms, due to interactions between the locking paths and the
      MUTEX_SPIN_ON_OWNER code. This was fixed in 0bce9c46bf3b ("mutex: place
      lock in contended state after fastpath_lock failure"), however, the
      atomic_dec-based mutex algorithm is now marginally more efficient for
      ARM (~0.5% improvement in hackbench scores on dual A15).

      This patch moves ARMv6+ platforms to the atomic_dec-based mutex code.

      Acked-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Will Deacon <will.deacon@arm.com>
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 7587/1: implement optimized percpu variable access

      Use the previously unused TPIDRPRW register to store percpu offsets.
      TPIDRPRW is only accessible in PL1, so it can only be used in the kernel.

      This replaces 2 loads with a mrc instruction for each percpu variable
      access. With hackbench, the performance improvement is 1.4% on Cortex-A9
      (highbank). Taking an average of 30 runs of "hackbench -l 1000" yields:

      Before: 6.2191
      After: 6.1348

      Will Deacon reported similar delta on v6 with 11MPCore.

      The asm "memory clobber" are needed here to ensure the percpu offset
      gets reloaded. Testing by Will found that this would not happen in
      __schedule() which is a bit of a special case as preemption is disabled
      but the execution can move cores.

      Signed-off-by: Rob Herring <rob.herring@calxeda.com>
      Acked-by: Will Deacon <will.deacon@arm.com>
      Acked-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 7747/1: pcpu: ensure __my_cpu_offset cannot be re-ordered across barrier()

      __my_cpu_offset is non-volatile, since we want its value to be cached
      when we access several per-cpu variables in a row with preemption
      disabled. This means that we rely on preempt_{en,dis}able to hazard
      with the operation via the barrier() macro, so that we can't end up
      migrating CPUs without reloading the per-cpu offset.

      Unfortunately, GCC doesn't treat a "memory" clobber on a non-volatile
      asm block as a side-effect, and will happily re-order it before other
      memory clobbers (including those in prempt_disable()) and cache the
      value. This has been observed to break the cmpxchg logic in the slub
      allocator, leading to livelock in kmem_cache_alloc in mainline kernels.

      This patch adds a dummy memory input operand to __my_cpu_offset,
      forcing it to be ordered with respect to the barrier() macro.

      Cc: <stable@vger.kernel.org>
      Cc: Rob Herring <rob.herring@calxeda.com>
      Reviewed-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Will Deacon <will.deacon@arm.com>
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: smp: flush L1 cache in cpu_die()

      Flush the L1 cache for the CPU which is going down in cpu_die() so
      that we don't end up with all platforms doing this.  This ensures
      that any cache lines we own are pushed out before the cache becomes
      inaccessible.

      We may end up subsequently creating some dirty cache lines - for
      example, with the complete() call, but this update must become
      visible to other CPUs before __cpu_die() can proceed.  Subsequent
      accesses from the platforms cpu_die() function should _not_ matter.

      Also place a mb() after the complete() call to ensure that this is
      visible to other CPUs.

      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 7590/1: /proc/interrupts: limit the display of IPIs to online CPUs only

      This is what is done for the regular interrupts in kernel/irqs/proc.c
      already, before calling arch_show_interrupts().  Not doing so for the
      IPIs causes the column headers not to match with the content whenever
      some CPUs are offline.

      Signed-off-by: Nicolas Pitre <nico@linaro.org>
      Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
byteorder: allow arch to opt to use GCC intrinsics for byteswapping

Since GCC 4.4, there have been __builtin_bswap32() and __builtin_bswap16()
intrinsics. A __builtin_bswap16() came a little later (4.6 for PowerPC,
48 for other platforms).

By using these instead of the inline assembler that most architectures
have in their __arch_swabXX() macros, we let the compiler see what's
actually happening. The resulting code should be at least as good, and
much *better* in the cases where it can be combined with a nearby load
or store, using a load-and-byteswap or store-and-byteswap instruction
(e.g. lwbrx/stwbrx on PowerPC, movbe on Atom).

When GCC is sufficiently recent *and* the architecture opts in to using
the intrinsics by setting CONFIG_ARCH_USE_BUILTIN_BSWAP, they will be
used in preference to the __arch_swabXX() macros. An architecture which
does not set ARCH_USE_BUILTIN_BSWAP will continue to use its own
hand-crafted macros.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Acked-by: H. Peter Anvin <hpa@linux.intel.com>
binfmt_elf.c: use get_random_int() to fix entropy depleting.

Usually the entropy pool depletes very fast and stays in the 100-200 range.
After this patch, the entropy stays in the 600-1000 range without the risk of
depleting in normal workloads.

Change-Id: Ice0d0f14691046249d7b4a4d31af98ab80c6306b
introduce for_each_thread() to replace the buggy while_each_thread()

while_each_thread() and next_thread() should die, almost every lockless
usage is wrong.

1. Unless g == current, the lockless while_each_thread() is not safe.

   while_each_thread(g, t) can loop forever if g exits, next_thread()
   can't reach the unhashed thread in this case. Note that this can
   happen even if g is the group leader, it can exec.

2. Even if while_each_thread() itself was correct, people often use
   it wrongly.

   It was never safe to just take rcu_read_lock() and loop unless
   you verify that pid_alive(g) == T, even the first next_thread()
   can point to the already freed/reused memory.

This patch adds signal_struct->thread_head and task->thread_node to
create the normal rcu-safe list with the stable head.  The new
for_each_thread(g, t) helper is always safe under rcu_read_lock() as
long as this task_struct can't go away.

Note: of course it is ugly to have both task_struct->thread_node and the
old task_struct->thread_group, we will kill it later, after we change
the users of while_each_thread() to use for_each_thread().

Perhaps we can kill it even before we convert all users, we can
reimplement next_thread(t) using the new thread_head/thread_node.  But
we can't do this right now because this will lead to subtle behavioural
changes.  For example, do/while_each_thread() always sees at least one
task, while for_each_thread() can do nothing if the whole thread group
has died.  Or thread_group_empty(), currently its semantics is not clear
unless thread_group_leader(p) and we need to audit the callers before we
can change it.

So this patch adds the new interface which has to coexist with the old
one for some time, hopefully the next changes will be more or less
straightforward and the old one will go away soon.

Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Sergey Dyasly <dserrg@gmail.com>
Tested-by: Sergey Dyasly <dserrg@gmail.com>
Reviewed-by: Sameer Nanda <snanda@chromium.org>
Acked-by: David Rientjes <rientjes@google.com>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mandeep Singh Baines <msb@chromium.org>
Cc: "Ma, Xindong" <xindong.ma@intel.com>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: "Tu, Xiaobing" <xiaobing.tu@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
suspend: enable freeze timeout configuration through sys

At present, the value of timeout for freezing is 20s, which is
meaningless in case that one thread is frozen with mutex locked
and another thread is trying to lock the mutex, as this time of
freezing will fail unavoidably.
And if there is no new wakeup event registered, the system will
waste at most 20s for such meaningless trying of freezing.

With this patch, the value of timeout can be configured to smaller
value, so such meaningless trying of freezing will be aborted in
earlier time, and later freezing can be also triggered in earlier
time. And more power will be saved.
In normal case on mobile phone, it costs real little time to freeze
processes. On some platform, it only costs about 20ms to freeze
user space processes and 10ms to freeze kernel freezable threads.

Signed-off-by: Liu Chuansheng <chuansheng.liu@intel.com>
Signed-off-by: Li Fei <fei.li@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
freezer: *squashing updates by Colin Cross (05/2013)*

freezer: add unsafe versions of freezable helpers for NFS

      NFS calls the freezable helpers with locks held, which is unsafe
      and will cause lockdep warnings when 6aa9707 "lockdep: check
      that no locks held at freeze time" is reapplied (it was reverted
      in dbf520a).  NFS shouldn't be doing this, but it has
      long-running syscalls that must hold a lock but also shouldn't
      block suspend.  Until NFS freeze handling is rewritten to use a
      signal to exit out of the critical section, add new *_unsafe
      versions of the helpers that will not run the lockdep test when
      6aa9707 is reapplied, and call them from NFS.

      In practice the likley result of holding the lock while freezing
      is that a second task blocked on the lock will never freeze,
      aborting suspend, but it is possible to manufacture a case using
      the cgroup freezer, the lock, and the suspend freezer to create
      a deadlock.  Silencing the lockdep warning here will allow
      problems to be found in other drivers that may have a more
      serious deadlock risk, and prevent new problems from being added.

      Signed-off-by: Colin Cross <ccross@android.com>
      Acked-by: Pavel Machek <pavel@ucw.cz>
      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

freezer: add unsafe versions of freezable helpers for CIFS

      CIFS calls wait_event_freezekillable_unsafe with a VFS lock held,
      which is unsafe and will cause lockdep warnings when 6aa9707
      "lockdep: check that no locks held at freeze time" is reapplied
      (it was reverted in dbf520a).  CIFS shouldn't be doing this, but
      it has long-running syscalls that must hold a lock but also
      shouldn't block suspend.  Until CIFS freeze handling is rewritten
      to use a signal to exit out of the critical section, add a new
      wait_event_freezekillable_unsafe helper that will not run the
      lockdep test when 6aa9707 is reapplied, and call it from CIFS.

      In practice the likley result of holding the lock while freezing
      is that a second task blocked on the lock will never freeze,
      aborting suspend, but it is possible to manufacture a case using
      the cgroup freezer, the lock, and the suspend freezer to create
      a deadlock.  Silencing the lockdep warning here will allow
      problems to be found in other drivers that may have a more
      serious deadlock risk, and prevent new problems from being added.

      Acked-by: Pavel Machek <pavel@ucw.cz>
      Acked-by: Tejun Heo <tj@kernel.org>
      Reviewed-by: Jeff Layton <jlayton@redhat.com>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

lockdep: remove task argument from debug_check_no_locks_held

      The only existing caller to debug_check_no_locks_held calls it
      with 'current' as the task, and the freezer needs to call
      debug_check_no_locks_held but doesn't already have a current
      task pointer, so remove the argument.  It is already assuming
      that the current task is relevant by dumping the current stack
      trace as part of the warning.

      This was originally part of 6aa9707099c (lockdep: check that
      no locks held at freeze time) which was reverted in
      dbf520a9d7d4.

      Original-author: Mandeep Singh Baines <msb@chromium.org>
      Acked-by: Pavel Machek <pavel@ucw.cz>
      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

freezer: shorten freezer sleep time using exponential backoff

      All tasks can easily be frozen in under 10 ms, switch to using
      an initial 1 ms sleep followed by exponential backoff until
      8 ms.  Also convert the printed time to ms instead of centiseconds.

      Acked-by: Pavel Machek <pavel@ucw.cz>
      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

freezer: skip waking up tasks with PF_FREEZER_SKIP set

      Android goes through suspend/resume very often (every few seconds when
      on a busy wifi network with the screen off), and a significant portion
      of the energy used to go in and out of suspend is spent in the
      freezer.  If a task has called freezer_do_not_count(), don't bother
      waking it up.  If it happens to wake up later it will call
      freezer_count() and immediately enter the refrigerator.

      Combined with patches to convert freezable helpers to use
      freezer_do_not_count() and convert common sites where idle userspace
      tasks are blocked to use the freezable helpers, this reduces the
      time and energy required to suspend and resume.

      Acked-by: Tejun Heo <tj@kernel.org>
      Acked-by: Pavel Machek <pavel@ucw.cz>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

freezer: convert freezable helpers to freezer_do_not_count()

      Freezing tasks will wake up almost every userspace task from
      where it is blocking and force it to run until it hits a
      call to try_to_sleep(), generally on the exit path from the syscall
      it is blocking in.  On resume each task will run again, usually
      restarting the syscall and running until it hits the same
      blocking call as it was originally blocked in.

      Convert the existing wait_event_freezable* wrappers to use
      freezer_do_not_count().  Combined with a previous patch,
      these tasks will not run during suspend or resume unless they wake
      up for another reason, in which case they will run until they hit
      the try_to_freeze() in freezer_count(), and then continue processing
      the wakeup after tasks are thawed.

      This results in a small change in behavior, previously a race
      between freezing and a normal wakeup would be won by the wakeup,
      now the task will freeze and then handle the wakeup after thawing.

      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

freezer: convert freezable helpers to static inline where possible

      Some of the freezable helpers have to be macros because their
      condition argument needs to get evaluated every time through
      the wait loop.  Convert the others to static inline to make
      future changes easier.

      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

freezer: add new freezable helpers using freezer_do_not_count()

      Freezing tasks will wake up almost every userspace task from
      where it is blocking and force it to run until it hits a
      call to try_to_sleep(), generally on the exit path from the syscall
      it is blocking in.  On resume each task will run again, usually
      restarting the syscall and running until it hits the same
      blocking call as it was originally blocked in.

      To allow tasks to avoid running on every suspend/resume cycle,
      this patch adds additional freezable wrappers around blocking calls
      that call freezer_do_not_count().  Combined with the previous patch,
      these tasks will not run during suspend or resume unless they wake
      up for another reason, in which case they will run until they hit
      the try_to_freeze() in freezer_count(), and then continue processing
      the wakeup after tasks are thawed.

      Additional patches will convert the most common locations that
      userspace blocks in to use freezable helpers.

      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

binder: use freezable blocking calls

      Avoid waking up every thread sleeping in a binder call during
      suspend and resume by calling a freezable blocking call.  Previous
      patches modified the freezer to avoid sending wakeups to threads
      that are blocked in freezable blocking calls.

      This call was selected to be converted to a freezable call because
      it doesn't hold any locks or release any resources when interrupted
      that might be needed by another freezing task or a kernel driver
      during suspend, and is a common site where idle userspace tasks are
      blocked.

      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

epoll: use freezable blocking call

      Avoid waking up every thread sleeping in an epoll_wait call during
      suspend and resume by calling a freezable blocking call.  Previous
      patches modified the freezer to avoid sending wakeups to threads
      that are blocked in freezable blocking calls.

      This call was selected to be converted to a freezable call because
      it doesn't hold any locks or release any resources when interrupted
      that might be needed by another freezing task or a kernel driver
      during suspend, and is a common site where idle userspace tasks are
      blocked.

      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

select: use freezable blocking call

      Avoid waking up every thread sleeping in a select call during
      suspend and resume by calling a freezable blocking call.  Previous
      patches modified the freezer to avoid sending wakeups to threads
      that are blocked in freezable blocking calls.

      This call was selected to be converted to a freezable call because
      it doesn't hold any locks or release any resources when interrupted
      that might be needed by another freezing task or a kernel driver
      during suspend, and is a common site where idle userspace tasks are
      blocked.

      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

futex: use freezable blocking call

      Avoid waking up every thread sleeping in a futex_wait call during
      suspend and resume by calling a freezable blocking call.  Previous
      patches modified the freezer to avoid sending wakeups to threads
      that are blocked in freezable blocking calls.

      This call was selected to be converted to a freezable call because
      it doesn't hold any locks or release any resources when interrupted
      that might be needed by another freezing task or a kernel driver
      during suspend, and is a common site where idle userspace tasks are
      blocked.

      Acked-by: Tejun Heo <tj@kernel.org>
      Acked-by: Thomas Gleixner <tglx@linutronix.de>
      Acked-by: Darren Hart <dvhart@linux.intel.com>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

nanosleep: use freezable blocking call

      Avoid waking up every thread sleeping in a nanosleep call during
      suspend and resume by calling a freezable blocking call.  Previous
      patches modified the freezer to avoid sending wakeups to threads
      that are blocked in freezable blocking calls.

      This call was selected to be converted to a freezable call because
      it doesn't hold any locks or release any resources when interrupted
      that might be needed by another freezing task or a kernel driver
      during suspend, and is a common site where idle userspace tasks are
      blocked.

      Acked-by: Tejun Heo <tj@kernel.org>
      Acked-by: Thomas Gleixner <tglx@linutronix.de>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

sigtimedwait: use freezable blocking call

      Avoid waking up every thread sleeping in a sigtimedwait call during
      suspend and resume by calling a freezable blocking call.  Previous
      patches modified the freezer to avoid sending wakeups to threads
      that are blocked in freezable blocking calls.

      This call was selected to be converted to a freezable call because
      it doesn't hold any locks or release any resources when interrupted
      that might be needed by another freezing task or a kernel driver
      during suspend, and is a common site where idle userspace tasks are
      blocked.

      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

af_unix: use freezable blocking calls in read

      Avoid waking up every thread sleeping in read call on an AF_UNIX
      socket during suspend and resume by calling a freezable blocking
      call.  Previous patches modified the freezer to avoid sending
      wakeups to threads that are blocked in freezable blocking calls.

      This call was selected to be converted to a freezable call because
      it doesn't hold any locks or release any resources when interrupted
      that might be needed by another freezing task or a kernel driver
      during suspend, and is a common site where idle userspace tasks are
      blocked.

      Acked-by: Tejun Heo <tj@kernel.org>
      Signed-off-by: Colin Cross <ccross@android.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

HACK: ARM: disable sleeping while atomic warning in do_signal

      ARM disables interrupts in do_signal, which triggers a warning in
      try_to_freeze, see details at https://lkml.org/lkml/2011/8/23/221.
      To prevent the warnings, add try_to_freeze_nowarn and call it from
      do_signal.

      Change-Id: If7482de21c386adc705fa1ac4ecb8c7ece5bb356
      Signed-off-by: Colin Cross <ccross@android.com>
sched: Fix hotplug vs. set_cpus_allowed_ptr()

Lai found that:

  WARNING: CPU: 1 PID: 13 at arch/x86/kernel/smp.c:124 native_smp_send_reschedule+0x2d/0x4b()
  ...
  migration_cpu_stop+0x1d/0x22

was caused by set_cpus_allowed_ptr() assuming that cpu_active_mask is
always a sub-set of cpu_online_mask.

This isn't true since 5fbd036b552f ("sched: Cleanup cpu_active madness").

So set active and online at the same time to avoid this particular
problem.

Fixes: 5fbd036b552f ("sched: Cleanup cpu_active madness")
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Gautham R. Shenoy <ego@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michael wang <wangyun@linux.vnet.ibm.com>
Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Toshi Kani <toshi.kani@hp.com>
Link: http://lkml.kernel.org/r/53758B12.8060609@cn.fujitsu.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
sched: Improve scalability via 'CPU buddies', which withstand random perturbations

Traversing an entire package is not only expensive, it also leads to tasks
bouncing all over a partially idle and possible quite large package.  Fix
that up by assigning a 'buddy' CPU to try to motivate.  Each buddy may try
to motivate that one other CPU, if it's busy, tough, it may then try its
SMT sibling, but that's all this optimization is allowed to cost.

Sibling cache buddies are cross-wired to prevent bouncing.

4 socket 40 core + SMT Westmere box, single 30 sec tbench runs, higher is better:

 clients     1       2       4        8       16       32       64      128
 ..........................................................................
 pre        30      41     118      645     3769     6214    12233    14312
 post      299     603    1211     2418     4697     6847    11606    14557

A nice increase in performance.

Signed-off-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1339471112.7352.32.camel@marge.simpson.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
sched: Reorder 'struct lb_env' members to reduce its size

Members of 'struct lb_env' are not in appropriate order to reuse compiler
added padding on 64bit architectures. In this patch we reorder those struct
members and help reduce the size of the structure from 96 bytes to 80
bytes on 64 bit architectures.

Suggested-by: Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>
Signed-off-by: Prashanth Nageshappa <prashanth@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/4FE06DDE.7000403@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
sched: Reset loop counters if all tasks are pinned and we need to redo load balance

While load balancing, if all tasks on the source runqueue are pinned,
we retry after excluding the corresponding source cpu. However, loop counters
env.loop and env.loop_break are not reset before retrying, which can lead
to failure in moving the tasks. In this patch we reset env.loop and
env.loop_break to their inital values before we retry.

Signed-off-by: Prashanth Nageshappa <prashanth@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/4FE06EEF.2090709@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
[KERNEL LOCKS] *big update*

rwsem: Implement writer lock-stealing for better scalability
rwsem: shorter spinlocked section in rwsem_down_failed_common()
rwsem: move rwsem_down_failed_common code into rwsem_down_{read,write}_failed
rwsem: simplify rwsem_down_read_failed
rwsem: simplify rwsem_down_write_failed
rwsem: more agressive lock stealing in rwsem_down_write_failed
rwsem: use cmpxchg for trying to steal write lock
rwsem: avoid taking wait_lock in rwsem_down_write_failed
rwsem: skip initial trylock in rwsem_down_write_failed
rwsem: simplify __rwsem_do_wake
rwsem: implement support for write lock stealing on the fastpath
rwsem: do not block readers at head of queue if other readers are active
rwsem: no need for explicit signed longs
rwsem: check counter to avoid cmpxchg calls

mutex: Move mutex spinning code from sched/core.c back to mutex.c
mutex: Make more scalable by doing less atomic operations
mutex: Queue mutex spinners with MCS lock to reduce cacheline contention
mutex: Back out architecture specific check for negative mutex count
arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not
mutex: Add support for wound/wait style locks
mutex: Fix/document access-once assumption in mutex_can_spin_on_owner()
mutex: Do not unnecessarily deal with waiters
mutex: Avoid label warning when !CONFIG_MUTEX_SPIN_ON_OWNER
locking/mutexes: Unlock the mutex without the wait_lock
locking/mutexes: Add extra reschedule point

lockdep: Check if nested lock is actually held
lockdep: Rename print_unlock_inbalance_bug() to print_unlock_imbalance_bug()
lockdep: Print more info when MAX_LOCK_DEPTH is exceeded
Export __lockdep_no_validate__
lockdep: Remove unnecessary 'hlock_next' variable
lockdep, rwsem: provide down_write_nest_lock()

semaphore: use unlikely() for down's timeout
semaphore: use `bool' type for semaphore_waiter's up

kernel/spinlock.c: add default arch_*_relax definitions for GENERIC_LOCKBREAK

Revert "rw_semaphore: remove up/down_read_non_owner"
[MSM HOTPLUG] *squashing updates*

msm: hotplug/platsmp: Fix cpu invaldation pointer arithmetic

      These functions take two void pointers indicating where to start
      and end the invalidation and cache flush. The pointer arithmetic
      is considering the pen_release pointer as an int pointer so
      adding sizeof(pen_release) is adding 16 bytes to the pointer and
      specifying that as the end. This inadvertently invalidates data
      next to the pen_release variable which could be bad. Fix the
      pointer math to only invalidate or clean the region for the
      pen_release variable.

      Change-Id: If0cf0e5de7c9f4b8c70662b29957a1f347809124
      Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>

msm: hotplug: Change the pen variable in one place

      Other WFI based hotplug implementations set the pen variable to
      -1 in C code and do the appropriate flushing operations in one
      location. They also don't try to invalidate the cache on a
      spurious wakeup and report how many spurious wakeups there were.
      Let's do the same in our code so that we can remove our
      dependence on the dmac_{inv,flush}_range() functions (which are
      no longer upstream) and easily report spurious wakeups to the
      kernel logs.

      This does delay the time we set the pen back to its default
      value, but that doesn't actually matter because generic ARM
      hotplug code already waits for a completion before continuing.

      Change-Id: Ic36fbf744fe8abeda20ad9859a207b5e37c5e63c
      Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>

msm: hotplug: remove code related to cpu_killed

      Earlier the completion event cpu_killed was used to synchronize
      notify to hot plug framework that the CPU is dead. But now it is
      replaced with another completion event cpu_died and it is placed
      in the hotplug framework itself. So there is no need for a
      platform to implement a completion event.

      But in msm platform driver the completion event related code
      was not removed with the changes in hotplug framework. This
      patch removes the unused code.

      Change-Id: I4418e5e397821e39a7b6a6efa997c4a866e9b46b
      Signed-off-by: Venkat Devarasetty <vdevaras@codeaurora.org>

msm: hotplug: Block hotplug removal of CPU0

      Hotplug removal of CPU0 is not currently supported. Attempts to
      remove it may result in undefined behaviour. Explicitly block
      any such attempts before other hotplug notifiers get the chance
      to run by setting the notifier priority to INT_MAX.

      Change-Id: I9e1f5b4a0db554295a097220417f2bdf4c690f5f
      Signed-off-by: Matt Wagantall <mattw@codeaurora.org>

msm: platsmp-8625: modify pen_release variable to volatile

      Since pen_release is a shared variable between cpus. By marking it
      as volatile, compiler won't cache that variable into registers and
      won't optimize the code which access pen_release variable.

      Change-Id: Ib5b7aa5932b35d64d05f9e4a4672df2bb9207c59
      Signed-off-by: tirupathireddy <tirupath@codeaurora.org>

msm: platsmp-8625: Add cold_boot_done as per_cpu variable

      cold_boot_done is declared as an ordinary variable, and useful for
      single secondary cpu, right now. It will not work in the scenarios
      where we have more than two cores in the system. Usage of per-cpu
      variable will make the code more generic for multi-core environment.

      Change-Id: I4b2480243b2adaadf5c5a2a3534f66faebd87553
      Signed-off-by: tirupathireddy <tirupath@codeaurora.org>

msm: pm2: Raise SPI interrupt instread of SGI to non boot CPUs

      Once APPS comes out of PC and mpdecision want to bring the non-boot
      cpus to online, first CORE0 raises the SPI interrupt(MPCORE) to bring
      the cores(1/2/3) online. While bringing the cores online, CORE0 raises
      the SPI interrupt instead of SGI. Because once APPS does a PC QGIC
      looses its state and none of the non-boot cpus are not capable of
      receiving the SGI interrupt until cores configure the QGIC PPIs&SGIs
      by themselves.

      Change-Id: Ic4aedeffbcd3fa2f0aa15e2e8ed7e25a03468f87
      Signed-off-by: Murali Nalajala <mnalajal@codeaurora.org>
mm, mempolicy: remove per-process flag

PF_MEMPOLICY is an unnecessary optimization for CONFIG_SLAB users.
There's no significant performance degradation to checking
current->mempolicy rather than current->flags & PF_MEMPOLICY in the
allocation path, especially since this is considered unlikely().

Running TCP_RR with netperf-2.4.5 through localhost on 16 cpu machine with
64GB of memory and without a mempolicy:

	threads		before		after
	16		1249409		1244487
	32		1281786		1246783
	48		1239175		1239138
	64		1244642		1241841
	80		1244346		1248918
	96		1266436		1254316
	112		1307398		1312135
	128		1327607		1326502

Per-process flags are a scarce resource so we should free them up whenever
possible and make them available.  We'll be using it shortly for memcg oom
reserves.

Signed-off-by: David Rientjes <rientjes@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Jianguo Wu <wujianguo@huawei.com>
Cc: Tim Hockin <thockin@google.com>
Cc: Christoph Lameter <cl@linux.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
mm: per-thread vma caching

This patch is a continuation of efforts trying to optimize find_vma(),
avoiding potentially expensive rbtree walks to locate a vma upon faults.
The original approach (https://lkml.org/lkml/2013/11/1/410), where the
largest vma was also cached, ended up being too specific and random,
thus further comparison with other approaches were needed.  There are
two things to consider when dealing with this, the cache hit rate and
the latency of find_vma().  Improving the hit-rate does not necessarily
translate in finding the vma any faster, as the overhead of any fancy
caching schemes can be too high to consider.

We currently cache the last used vma for the whole address space, which
provides a nice optimization, reducing the total cycles in find_vma() by
up to 250%, for workloads with good locality.  On the other hand, this
simple scheme is pretty much useless for workloads with poor locality.
Analyzing ebizzy runs shows that, no matter how many threads are
running, the mmap_cache hit rate is less than 2%, and in many situations
below 1%.

The proposed approach is to replace this scheme with a small per-thread
cache, maximizing hit rates at a very low maintenance cost.
Invalidations are performed by simply bumping up a 32-bit sequence
number.  The only expensive operation is in the rare case of a seq
number overflow, where all caches that share the same address space are
flushed.  Upon a miss, the proposed replacement policy is based on the
page number that contains the virtual address in question.  Concretely,
the following results are seen on an 80 core, 8 socket x86-64 box:

1) System bootup: Most programs are single threaded, so the per-thread
   scheme does improve ~50% hit rate by just adding a few more slots to
   the cache.

+----------------+----------+------------------+
| caching scheme | hit-rate | cycles (billion) |
+----------------+----------+------------------+
| baseline       | 50.61%   | 19.90            |
| patched        | 73.45%   | 13.58            |
+----------------+----------+------------------+

2) Kernel build: This one is already pretty good with the current
   approach as we're dealing with good locality.

+----------------+----------+------------------+
| caching scheme | hit-rate | cycles (billion) |
+----------------+----------+------------------+
| baseline       | 75.28%   | 11.03            |
| patched        | 88.09%   | 9.31             |
+----------------+----------+------------------+

3) Oracle 11g Data Mining (4k pages): Similar to the kernel build workload.

+----------------+----------+------------------+
| caching scheme | hit-rate | cycles (billion) |
+----------------+----------+------------------+
| baseline       | 70.66%   | 17.14            |
| patched        | 91.15%   | 12.57            |
+----------------+----------+------------------+

4) Ebizzy: There's a fair amount of variation from run to run, but this
   approach always shows nearly perfect hit rates, while baseline is just
   about non-existent.  The amounts of cycles can fluctuate between
   anywhere from ~60 to ~116 for the baseline scheme, but this approach
   reduces it considerably.  For instance, with 80 threads:

+----------------+----------+------------------+
| caching scheme | hit-rate | cycles (billion) |
+----------------+----------+------------------+
| baseline       | 1.06%    | 91.54            |
| patched        | 99.97%   | 14.18            |
+----------------+----------+------------------+

[akpm@linux-foundation.org: fix nommu build, per Davidlohr]
[akpm@linux-foundation.org: document vmacache_valid() logic]
[akpm@linux-foundation.org: attempt to untangle header files]
[akpm@linux-foundation.org: add vmacache_find() BUG_ON]
[hughd@google.com: add vmacache_valid_mm() (from Oleg)]
[akpm@linux-foundation.org: coding-style fixes]
[akpm@linux-foundation.org: adjust and enhance comments]
Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
Reviewed-by: Rik van Riel <riel@redhat.com>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Michel Lespinasse <walken@google.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Tested-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
[CPU HotPlug] Provide lockless versions of callback registration functions

CPU hotplug: Add lockdep annotations to get/put_online_cpus()

      Add lockdep annotations for get/put_online_cpus() and
      cpu_hotplug_begin()/cpu_hotplug_end().

      Cc: Ingo Molnar <mingo@kernel.org>
      Reviewed-by: Oleg Nesterov <oleg@redhat.com>
      Signed-off-by: Gautham R. Shenoy <ego@linux.vnet.ibm.com>
      Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

CPU hotplug: Provide lockless versions of callback registration functions

      The following method of CPU hotplug callback registration is not safe
      due to the possibility of an ABBA deadlock involving the cpu_add_remove_lock
      and the cpu_hotplug.lock.

	      get_online_cpus();

	      for_each_online_cpu(cpu)
		      init_cpu(cpu);

	      register_cpu_notifier(&foobar_cpu_notifier);

	      put_online_cpus();

      The deadlock is shown below:

		CPU 0                                         CPU 1
		-----                                         -----

	Acquire cpu_hotplug.lock
	[via get_online_cpus()]

						    CPU online/offline operation
						    takes cpu_add_remove_lock
						    [via cpu_maps_update_begin()]

	Try to acquire
	cpu_add_remove_lock
	[via register_cpu_notifier()]

						    CPU online/offline operation
						    tries to acquire cpu_hotplug.lock
						    [via cpu_hotplug_begin()]

				  *** DEADLOCK! ***

      The problem here is that callback registration takes the locks in one order
      whereas the CPU hotplug operations take the same locks in the opposite order.
      To avoid this issue and to provide a race-free method to register CPU hotplug
      callbacks (along with initialization of already online CPUs), introduce new
      variants of the callback registration APIs that simply register the callbacks
      without holding the cpu_add_remove_lock during the registration. That way,
      we can avoid the ABBA scenario. However, we will need to hold the
      cpu_add_remove_lock throughout the entire critical section, to protect updates
      to the callback/notifier chain.

      This can be achieved by writing the callback registration code as follows:

	      cpu_maps_update_begin(); [ or cpu_notifier_register_begin(); see below ]

	      for_each_online_cpu(cpu)
		      init_cpu(cpu);

	      /* This doesn't take the cpu_add_remove_lock */
	      __register_cpu_notifier(&foobar_cpu_notifier);

	      cpu_maps_update_done();  [ or cpu_notifier_register_done(); see below ]

      Note that we can't use get_online_cpus() here instead of cpu_maps_update_begin()
      because the cpu_hotplug.lock is dropped during the invocation of CPU_POST_DEAD
      notifiers, and hence get_online_cpus() cannot provide the necessary
      synchronization to protect the callback/notifier chains against concurrent
      reads and writes. On the other hand, since the cpu_add_remove_lock protects
      the entire hotplug operation (including CPU_POST_DEAD), we can use
      cpu_maps_update_begin/done() to guarantee proper synchronization.

      Also, since cpu_maps_update_begin/done() is like a super-set of
      get/put_online_cpus(), the former naturally protects the critical sections
      from concurrent hotplug operations.

      Since the names cpu_maps_update_begin/done() don't make much sense in CPU
      hotplug callback registration scenarios, we'll introduce new APIs named
      cpu_notifier_register_begin/done() and map them to cpu_maps_update_begin/done().

      In summary, introduce the lockless variants of un/register_cpu_notifier() and
      also export the cpu_notifier_register_begin/done() APIs for use by modules.
      This way, we provide a race-free way to register hotplug callbacks as well as
      perform initialization for the CPUs that are already online.

      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Ingo Molnar <mingo@kernel.org>
      Acked-by: Oleg Nesterov <oleg@redhat.com>
      Acked-by: Toshi Kani <toshi.kani@hp.com>
      Reviewed-by: Gautham R. Shenoy <ego@linux.vnet.ibm.com>
      Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Doc/cpu-hotplug: Specify race-free way to register CPU hotplug callbacks

      Recommend the usage of the new CPU hotplug callback registration APIs
      (__register_cpu_notifier() etc), when subsystems need to also perform
      initialization for already online CPUs. Provide examples of correct
      and race-free ways of achieving this, and point out the kinds of code
      that are error-prone.

      Cc: Rob Landley <rob@landley.net>
      Cc: Ingo Molnar <mingo@kernel.org>
      Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
      Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

---------------------------------------------------------------------------------------------------

CPU hotplug, perf: Fix CPU hotplug callback registration

arm, hw-breakpoint: Fix CPU hotplug callback registration

zsmalloc: Fix CPU hotplug callback registration

drivers/base/topology.c: Fix CPU hotplug callback registration

oprofile, nmi-timer: Fix CPU hotplug callback registration

profile: Fix CPU hotplug callback registration

mm, vmstat: Fix CPU hotplug callback registration

net/core/flow.c: Fix CPU hotplug callback registration

net/iucv/iucv.c: Fix CPU hotplug callback registration