rwsem-spinlock: Implement writer lock-stealing for better scalability

We (Linux Kernel Performance project) found a regression
introduced by commit:

  5a505085f043 mm/rmap: Convert the struct anon_vma::mutex to an rwsem

which converted all anon_vma::mutex locks rwsem write locks.

The semantics are the same, but the behavioral difference is
quite huge in some cases. After investigating it we found the
root cause: mutexes support lock stealing while rwsems don't.

Here is the link for the detailed regression report:

  https://lkml.org/lkml/2013/1/29/84

Ingo suggested adding write lock stealing to rwsems:

    "I think we should allow lock-steal between rwsem writers - that
     will not hurt fairness as most rwsem fairness concerns relate to
     reader vs. writer fairness"

And here is the rwsem-spinlock version.

With this patch, we got a double performance increase in one
test box with following aim7 workfile:

    FILESIZE: 1M
    POOLSIZE: 10M
    10 fork_test

 /usr/bin/time output w/o patch                       /usr/bin/time_output with patch
 -- Percent of CPU this job got: 369%                 Percent of CPU this job got: 537%
 Voluntary context switches: 640595016                Voluntary context switches: 157915561

We got a 45% increase in CPU usage and saved about 3/4 voluntary context switches.

Reported-by: LKP project <lkp@linux.intel.com>
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Cc: Alex Shi <alex.shi@intel.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Michel Lespinasse <walken@google.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Anton Blanchard <anton@samba.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: paul.gortmaker@windriver.com
Link: http://lkml.kernel.org/r/1359716356-23865-1-git-send-email-yuanhan.liu@linux.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Revert qualcomm's contributions on mutex.h

Related commit is 3f2bc4d6eb5.

Change-Id: Ia92b90b44bf92b7e40ec984e428379ee1a6c047b
Signed-off-by: Iliyan Malchev <malchev@google.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
ARM: 7467/1: mutex: use generic xchg-based implementation for ARMv6+

The open-coded mutex implementation for ARMv6+ cores suffers from a
severe lack of barriers, so in the uncontended case we don't actually
protect any accesses performed during the critical section.

Furthermore, the code is largely a duplication of the ARMv6+ atomic_dec
code but optimised to remove a branch instruction, as the mutex fastpath
was previously inlined. Now that this is executed out-of-line, we can
reuse the atomic access code for the locking (in fact, we use the xchg
code as this produces shorter critical sections).

This patch uses the generic xchg based implementation for mutexes on
ARMv6+, which introduces barriers to the lock/unlock operations and also
has the benefit of removing a fair amount of inline assembly code.

Cc: <stable@vger.kernel.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Nicolas Pitre <nico@linaro.org>
Reported-by: Shan Kang <kangshan0910@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
ARM: 7495/1: mutex: use generic atomic_dec-based implementation for ARMv6+

Commit a76d7bd96d65 ("ARM: 7467/1: mutex: use generic xchg-based
implementation for ARMv6+") removed the barrier-less, ARM-specific
mutex implementation in favour of the generic xchg-based code.

Since then, a bug was uncovered in the xchg code when running on SMP
platforms, due to interactions between the locking paths and the
MUTEX_SPIN_ON_OWNER code. This was fixed in 0bce9c46bf3b ("mutex: place
lock in contended state after fastpath_lock failure"), however, the
atomic_dec-based mutex algorithm is now marginally more efficient for
ARM (~0.5% improvement in hackbench scores on dual A15).

This patch moves ARMv6+ platforms to the atomic_dec-based mutex code.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Revert "Revert "suspend: remove disable/enable cpus every time suspend is called (which is called every damn 5 seconds -.-).""

This reverts commit 33dac936dd839354359e2f0002136749d1e0339e.
mako_hotplug: queue the screen on/off works to a dedicated workqueue instead of pushing them to the global system queue. Also increase the minimum online time of a CPU from 1 second to 2 second.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
load_stats: small optimization to save 2 calls and inline both functions.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
ASoc: Reduce number of periods in low latency driver.

- Reduce number of periods to 2 from 4.

Signed-off-by: SathishKumar Mani <smani@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
ASoc: msm: Increase max buffer size to 4k for lowlatency playback

- Small buffersize is resulting in scheduling issues.
- Increase max buffer size so that user space can
  configure large buffer size to avoid scheduling
  issues.

bug-id: 6865729
Signed-off-by: SathishKumar Mani <smani@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
ASoC: msm: Reduce min buffer size for low latency

The current low latency driver has 512 bytes as
the min buffer size threshold. With this reducing
the playback time to lower values is not possible.
Setting it to 128 bytes gives us more room
to try out lower buffer sizes from the user space

Signed-off-by: Alexy Joseph <alexyj@codeaurora.org>
Signed-off-by: Mekala Natarajan <mekalan@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
ASoC: msm: Change QOS value for low latency path

There were variations in the time taken to
return from each write. This was due
to the delay in switching in and out of
power collapse. To manage this set QOS
value to 1ms so that the core has enough
time to wake up from power collapse

Signed-off-by: Alexy Joseph <alexyj@codeaurora.org>
Signed-off-by: Mekala Natarajan <mekalan@codeaurora.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
userns: Add kuid_t and kgid_t and associated infrastructure in uidgid.h

Start distinguishing between internal kernel uids and gids and
values that userspace can use.  This is done by introducing two
new types: kuid_t and kgid_t.  These types and their associated
functions are infrastructure are declared in the new header
uidgid.h.

Ultimately there will be a different implementation of the mapping
functions for use with user namespaces.  But to keep it simple
we introduce the mapping functions first to separate the meat
from the mechanical code conversions.

Export overflowuid and overflowgid so we can use from_kuid_munged
and from_kgid_munged in modular code.

Acked-by: Serge Hallyn <serge.hallyn@canonical.com>
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
userns: Add a Kconfig option to enforce strict kuid and kgid type checks

Make it possible to easily switch between strong mandatory
type checks and relaxed type checks so that the code can
easily be tested with the type checks and then built
with the strong type checks disabled so the resulting
code can be used.

Require strong mandatory type checks when enabling the user namespace.
It is very simple to make a typo and use the wrong type allowing
conversions to/from userspace values to be bypassed by accident,
the strong type checks prevent this.

Acked-by: Serge Hallyn <serge.hallyn@canonical.com>
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Added BITS_PER_LONG_LONG definition avoiding lib/vsprintf.c errors.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Squash vsprintf from upstream.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
lib: vsprintf: remove warning.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
cpufreq: add generic get_cpuidle_time to the cpufreq headers so that we don't need to implement this everytime we need it.

Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>