vfp.h: restore original value for FPSID_ARCH_MASK

This value got changed by: https://www.codeaurora.org/cgit/quic/la/kernel/msm/commit/arch/arm/include/asm/vfp.h?h=msm-3.4&id=3f2bc4d6eb5a4fada842462ba22bb6bbb41d00c7

Since there is no history for this change, restore it back to the
original value chosen by ARM.

(FPSID_ARCH_MASK is currently only being used by 'arch/arm/vfp/vfpmodule.c')

Signed-off-by: moddingg33k <moddingg33k@t-online.de>
Revert "arm: Implement ticket spin-locks"

This reverts: https://www.codeaurora.org/cgit/quic/la/kernel/msm/commit/arch/arm/include/asm/spinlock.h?h=msm-3.4&id=8e7b85734e72e3f9e8383017097a9c2faacfac14

Signed-off-by: moddingg33k <moddingg33k@t-online.de>
ARM: 7446/1: spinlock: use ticket algorithm for ARMv6+ locking implementation

Ticket spinlocks ensure locking fairness by introducing a FIFO-like
nature to the granting of lock acquisitions and also reducing the
thundering herd effect when spinning on a lock by allowing the cacheline
to remain in a shared state amongst the waiting CPUs. This is especially
important on systems where memory-access times are not necessarily
uniform when accessing the lock structure (for example, on a
multi-cluster platform where the lock is allocated into L1 when a CPU
releases it).

This patch implements the ticket spinlock algorithm for ARM, replacing
the simpler implementation for ARMv6+ processors.

	Reviewed-by: Nicolas Pitre <nico@linaro.org>
	Signed-off-by: Will Deacon <will.deacon@arm.com>
	Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

ARM: 7484/1: Don't enable GENERIC_LOCKBREAK with ticket spinlocks

Now that ARM has implemented its spinlocks with tickets we don't
need to use the generic lockbreak algorithm. Remove the Kconfig
from ARM so that we use the arch_spin_is_contended() definition
from the asm header. This also saves a word in each lock because
we don't need the break_lock member anymore.

	Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
	Acked-by: Will Deacon <will.deacon@arm.com>
	Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7447/1: rwlocks: remove unused branch labels from trylock routines

The ARM arch_{read,write}_trylock implementations include unused
backwards branch labels, since we don't retry the locking operation
if the exclusive store fails.

This patch removes the labels.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7632/1: spinlock: avoid exclusive accesses on unlock() path

When unlocking a spinlock, all we need to do is increment the owner
field of the lock. Since only one CPU can be performing an unlock()
operation for a given lock, this doesn't need to be exclusive.

This patch simplifies arch_spin_unlock to use non-exclusive accesses
when updating the owner field of the lock.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7749/1: spinlock: retry trylock operation if strex fails on free lock

An exclusive store instruction may fail for reasons other than lock
contention (e.g. a cache eviction during the critical section) so, in
line with other architectures using similar exclusive instructions
(alpha, mips, powerpc), retry the trylock operation if the lock appears
to be free but the strex reported failure.

Reported-by: Tony Thompson <anthony.thompson@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7811/1: locks: use early clobber in arch_spin_trylock

The res variable is written before we've finished with the input
operands (namely the lock address), so ensure that we mark it as `early
clobber' to avoid unintended register sharing.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7812/1: rwlocks: retry trylock operation if strex fails on free lock

Commit 15e7e5c1ebf5 ("ARM: 7749/1: spinlock: retry trylock operation if
strex fails on free lock") modifying our arch_spin_trylock to retry the
acquisition if the lock appeared uncontended, but the strex failed.

This patch does the same for rwlocks, which were missed by the original
patch.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7955/1: spinlock: ensure we have a compiler barrier before sev

When unlocking a spinlock, we require the following, strictly ordered
sequence of events:

	<barrier>	/* dmb */
	<unlock>
	<barrier>	/* dsb */
	<sev>

Whilst the code does indeed reflect this in terms of the architecture,
the final <barrier> + <sev> have been contracted into a single inline
asm without a "memory" clobber, therefore the compiler is at liberty to
reorder the unlock to the end of the above sequence. In such a case,
a waiting CPU may be woken up before the lock has been unlocked, leading
to extremely poor performance.

This patch reworks the dsb_sev() function to make use of the dsb()
macro and ensure ordering against the unlock.

Cc: <stable@vger.kernel.org>
Reported-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7404/1: cmpxchg64: use atomic64 and local64 routines for cmpxchg64

The cmpxchg64 routines for ARMv6+ CPUs replicate inline assembly that
already exists for atomic64 operations. Furthermore, the cmpxchg64 code
uses the "memory" constraint in the clobber list rather than identifying
the region of memory that is actually modified.

This patch replaces the ARMv6+ cmpxchg64 code with macros that expand to
the atomic64_ and local64_ variants, casting the pointer parameter to
the appropriate container type.

Cc: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7720/1: ARM v6/v7 cmpxchg64 shouldn't clear upper 32 bits of the old/new value

The implementation of cmpxchg64() for the ARM v6 and v7 architecture
casts parameter 2 and 3 (the old and new 64bit values) to an unsigned
long before calling the atomic_cmpxchg64() function. This clears
the top 32 bits of the old and new values, resulting in the wrong
values being compare-exchanged. Luckily, this only appears to be used
for 64-bit sched_clock, which we don't (yet) have on ARM.

This bug was introduced by commit 3e0f5a15f500 ("ARM: 7404/1: cmpxchg64:
use atomic64 and local64 routines for cmpxchg64").

Cc: <stable@vger.kernel.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Jaccon Bastiaansen <jaccon.bastiaansen@gmail.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7852/1: cmpxchg: implement barrier-less cmpxchg64_local

Our cmpxchg64 macros are wrappers around atomic64_cmpxchg. Whilst this is
great for code re-use, there is a case for barrier-less cmpxchg where it
is known to be safe (for example cmpxchg64_local and cmpxchg-based
lockrefs).

This patch introduces a 64-bit cmpxchg implementation specifically
for the cmpxchg64_* macros, so that it can be later used by the lockref
code.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7853/1: cmpxchg: implement cmpxchg64_relaxed

This patch introduces cmpxchg64_relaxed for arm, which performs a 64-bit
cmpxchg operation without barrier semantics. cmpxchg64_local is updated
to use the new operation.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: smp_on_up: move inline asm ALT_SMP patching macro out of spinlock.h

Patching UP/SMP alternatives inside inline assembly blocks is useful
outside of the spinlock implementation, where it is used for sev and wfe.

This patch lifts the macro into processor.h and gives it a scarier name
to (a) avoid conflicts in the global namespace and (b) to try and deter
its usage unless you "know what you're doing". The W macro for generating
wide instructions when targetting Thumb-2 is also made available under
the name WASM, to reduce the potential for conflicts with other headers.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
ARM: 7854/1: lockref: add support for lockless lockrefs using cmpxchg64

Our spinlocks are only 32-bit (2x16-bit tickets) and, on processors
with 64-bit atomic instructions, cmpxchg64 makes use of the double-word
exclusive accessors.

This patch wires up the cmpxchg-based lockless lockref implementation
for ARM.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Revert "arm: VFP: Report bounce statistics using procfs"

This reverts: https://www.codeaurora.org/cgit/quic/la/kernel/msm/commit/arch/arm/vfp/vfpmodule.c?h=msm-3.4&id=a754e6a13ce47f90bdacae8879e35268052d4d5d

Signed-off-by: moddingg33k <moddingg33k@t-online.de>
ARM: Align with upstream

Undo our diff from upstream where upstreamed patches were
slightly different or merges introduced duplicate symbols, etc.

Change-Id: Iab263aa3d85b7b8b414b43afd6d70be358e6b7c9
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
ARM: sched_clock: update epoch_cyc on resume

Many clocks that are used to provide sched_clock will reset during
suspend.  If read_sched_clock returns 0 after suspend, sched_clock will
appear to jump forward.  This patch resets cd.epoch_cyc to the current
value of read_sched_clock during resume, which causes sched_clock() just
after suspend to return the same value as sched_clock() just before
suspend.

In addition, during the window where epoch_ns has been updated before
suspend, but epoch_cyc has not been updated after suspend, it is unknown
whether the clock has reset or not, and sched_clock() could return a
bogus value.  Add a suspended flag, and return the pre-suspend epoch_ns
value during this period.

The new behavior is triggered by calling setup_sched_clock_needs_suspend
instead of setup_sched_clock.

Change-Id: Ieb02d180d17c8f3405f0888f4896eb9424d0cdfd
Signed-off-by: Colin Cross <ccross@android.com>
ARM: 7169/1: topdown mmap support

Similar to other architectures, this adds topdown mmap support in user
process address space allocation policy. This allows mmap sizes greater
than 2GB. This support is largely copied from MIPS and the generic
implementations.

The address space randomization is moved into arch_pick_mmap_layout.

Tested on V-Express with ubuntu and a mmap test from here:
https://bugs.launchpad.net/bugs/861296

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
wakeup: Use irqsave/irqrestore for events_lock

Jon Medhurst (Tixy) recently noticed a problem with the
events_lock usage. One of the Android patches that uses
wakeup_sources calls wakeup_source_add() with irqs disabled.
However, the event_lock usage in wakeup_source_add() uses
spin_lock_irq()/spin_unlock_irq(), which reenables interrupts.
This results in lockdep warnings.

The fix is to use spin_lock_irqsave()/spin_lock_irqrestore()
instead for the events_lock.

Full bug report here:
https://bugs.launchpad.net/linaro-landing-team-arm/+bug/1037565

Cc: Rafael J. Wysocki <rjw@sisk.pl>
Cc: Android Team <kernel-team@android.com>
Cc: Arve Hjønnevåg <arve@android.com>
Cc: Jon Medhurst (Tixy) <tixy@linaro.org>
Reported-and-debugged-by: Jon Medhurst (Tixy) <tixy@linaro.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Change-Id: Id455188931229f04e353969af389e49b069cb34c