[PATCH 3/3] Optimize ARM memset and memzero functions

The ARM memset and memzero functions are optimized with lower overhead
for small requests, generation of more 16-bit Thumb2 instructions when
compiled in Thumb2 mode, and configurable destination alignment before
the main block-copying loop.

The new compile-time constant MEMSET_WRITE_ALIGN_BYTES is introduced
in assembler.h, to augment the CALGN macro that previously regulated
32-byte write alignment but was only used on the Feroceon platform.
MEMSET_WRITE_ALIGN_BYTES can have values of 0 (no write alignment),
8, or 32. Apart from Feroceon, memset write alignment of 32 bytes
appears to benefit the armv6 platform, while the armv7 platform
seems benefit from alignment to 8 bytes for memset/memzero.

The CALGN macro is renamed to MEMSET_CALGN for memset and memzero; the
original CALGN macro is reserved for the memcpy family of functions
(memcpy, copy_from_user, copy_to_user) currently implemented in
copy_template.S, and the associated compile time constant
WRITE_ALIGN_BYTES defines the write alignment for memcpy-related
functions that will be utilized in subsequent contributions. Because
the current CALGN implementation in copy_template.S only implements
32-byte write alignment and is broken on Thumb2, it is only enabled
when WRITE_ALIGN_BYTES is equal to 32 and Thumb2 mode is not enabled.

Finally, memset and memzero now include a directive to enable unified
ARM assembler syntax.

Signed-off-by: Harm Hanemaaijer <fgenfb@yahoo.com>
ARM: 7626/1: arm/crypto: Make asm SHA-1 and AES code Thumb-2 compatible

This patch fixes aes-armv4.S and sha1-armv4-large.S to work
natively in Thumb.  This allows ARM/Thumb interworking workarounds
to be removed.

I also take the opportunity to convert some explicit assembler
directives for exported functions to the standard
ENTRY()/ENDPROC().

For the code itself:

  * In sha1_block_data_order, use of TEQ with sp is deprecated in
    ARMv7 and not supported in Thumb.  For the branches back to
    .L_00_15 and .L_40_59, the TEQ is converted to a CMP, under the
    assumption that clobbering the C flag here will not cause
    incorrect behaviour.

    For the first branch back to .L_20_39_or_60_79 the C flag is
    important, so sp is moved temporarily into another register so
    that TEQ can be used for the comparison.

  * In the AES code, most forms of register-indexed addressing with
    shifts and rotates are not permitted for loads and stores in
    Thumb, so the address calculation is done using a separate
    instruction for the Thumb case.

The resulting code is unlikely to be optimally scheduled, but it
should not have a large impact given the overall size of the code.
I haven't run any benchmarks.

Signed-off-by: Dave Martin <dave.martin@linaro.org>
Tested-by: David McCullough <ucdevel@gmail.com> (ARM only)
Acked-by: David McCullough <ucdevel@gmail.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7723/1: crypto: sha1-armv4-large.S: fix SP handling

Make the SHA1 asm code ABI conformant by making sure all stack
accesses occur above the stack pointer.

Origin:
http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=1a9d60d2

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Cc: stable@vger.kernel.org
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
[PATCH 2/3] Optimize copy_page for modern ARM platforms

The existing implementation of copy_page for ARM appears to be
optimized for older platforms. Benchmark testing in a sandbox
environment shows suboptimal performance on modern platforms
like armv6 and armv7, with speed-ups ranging from 10% (Cortex A8)
to 80% (armv6 used in Raspberry Pi) being achievable.

This commit optimizes copy_page and introduces the new compile-time
constant PREFETCH_DISTANCE, defined in cache.h, which when
multiplied by L1_CACHE_BYTES is equal to the offset used for
prefetches performed with the PLD instruction. For platforms where
L1_CACHE_BYTES is 32 (armv5 and armv6), copy_page processes 32 bytes
at a time while doing one prefetch per iteration, while for armv7
(with L1_CACHE_BYTES equal to 64), 64 bytes are processed at at time
with one prefetch per iteration. When no preload instruction is
available (platforms earlier than armv5), no preload instructions
are generated and 32 bytes are processed at at time.

To facilitate specifying instructions for architectures with no
preload instruction, the NO_PLD macro is added to assembler.h,
augmenting the PLD macro.

Signed-off-by: Harm Hanemaaijer <fgenfb@yahoo.com>
fs/sync: Make sync() satisfy many requests with one invocation

Dave Jones reported RCU stalls, overly long hrtimer interrupts, and
amazingly long NMI handlers from a trinity-induced workload involving
lots of concurrent sync() calls (https://lkml.org/lkml/2013/7/23/369).
There are any number of things that one might do to make sync() behave
better under high levels of contention, but it is also the case that
multiple concurrent sync() system calls can be satisfied by a single
sys_sync() invocation.

Given that this situation is reminiscent of rcu_barrier(), this commit
applies the rcu_barrier() approach to sys_sync().  This approach uses
a global mutex and a sequence counter.  The mutex is held across the
sync() operation, which eliminates contention between concurrent sync()
operations.  The counter is incremented at the beginning and end of
each sync() operation, so that it is odd while a sync() operation is in
progress and even otherwise, just like sequence locks.

The code that used to be in sys_sync() is now in do_sync(), and sys_sync()
now handles the concurrency.  The sys_sync() function first takes a
snapshot of the counter, then acquires the mutex, and then takes another
snapshot of the counter.  If the values of the two snapshots indicate that
a full do_sync() executed during the mutex acquisition, the sys_sync()
function releases the mutex and returns ("Our work is done!").  Otherwise,
sys_sync() increments the counter, invokes do_sync(), and increments
the counter again.

This approach allows a single call to do_sync() to satisfy an arbitrarily
large number of sync() system calls, which should eliminate issues due
to large numbers of concurrent invocations of the sync() system call.

Changes since v1 (https://lkml.org/lkml/2013/7/24/683):

o	Add a pair of memory barriers to keep the increments from
	bleeding into the do_sync() code.  (The failure probability
	is insanely low, but when you have several hundred million
	devices running Linux, you can expect several hundred instances
	of one-in-a-million failures.)

o	Actually CC some people who have experience in this area.

Reported-by: Dave Jones <davej@redhat.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jan Kara <jack@suse.cz>
Cc: Curt Wohlgemuth <curtw@google.com>
Cc: Jens Axboe <jaxboe@fusionio.com>
Cc: linux-fsdevel@vger.kernel.org

Signed-off-by: Paul Reioux <reioux@gmail.com>
netfilter: xt_qtaguid: remove AID_* dependency for access control

qtaguid limits what can be done with /ctrl and /stats based on group
membership.
This changes removes AID_NET_BW_STATS and AID_NET_BW_ACCT, and picks
up the groups from the gid of the matching proc entry files.

Signed-off-by: JP Abgrall <jpa@google.com>
Change-Id: I42e477adde78a12ed5eb58fbc0b277cdaadb6f94
gpu: ion: Modify ION_NUM_HEAPS to reflect actual max

ION_NUM_HEAPS is meant to represent the actual max number
of heap types in the system, but currently doesn't reflect
any custom heaps.  This patch changes it to 16.

Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
fs/proc/base.c (affects lowmemorykiller): oom_adj -17 triggers oom_score_adj of -1000

Prior to this patch, oom_score_adj was automatically calculated based
on the oom_adj value.
When setting an oom_adj value of the minium value (-17), automatically
set oom_score_adj to its minimum value (-1000) as well.
fs/dcache.c: adaptive vfs_cache_pressure on suspend / resume

low vfs_cache pressure values save battery while idle, while
a low value also causes stutter with longer up times.
mutex: Make more scalable by doing less atomic operations

Date	Thu, 4 Apr 2013 10:54:16 -0400

In the __mutex_lock_common() function, an initial entry into
the lock slow path will cause two atomic_xchg instructions to be
issued. Together with the atomic decrement in the fast path, a total
of three atomic read-modify-write instructions will be issued in
rapid succession. This can cause a lot of cache bouncing when many
tasks are trying to acquire the mutex at the same time.

This patch will reduce the number of atomic_xchg instructions used by
checking the counter value first before issuing the instruction. The
atomic_read() function is just a simple memory read. The atomic_xchg()
function, on the other hand, can be up to 2 order of magnitude or even
more in cost when compared with atomic_read(). By using atomic_read()
to check the value first before calling atomic_xchg(), we can avoid a
lot of unnecessary cache coherency traffic. The only downside with this
change is that a task on the slow path will have a tiny bit
less chance of getting the mutex when competing with another task
in the fast path.

The same is true for the atomic_cmpxchg() function in the
mutex-spin-on-owner loop. So an atomic_read() is also performed before
calling atomic_cmpxchg().

The mutex locking and unlocking code for the x86 architecture can allow
any negative number to be used in the mutex count to indicate that some
tasks are waiting for the mutex. I am not so sure if that is the case
for the other architectures. So the default is to avoid atomic_xchg()
if the count has already been set to -1. For x86, the check is modified
to include all negative numbers to cover a larger case.

The following table shows the scalability data on an 8-node 80-core
Westmere box with a 3.7.10 kernel. The numactl command is used to
restrict the running of the high_systime workloads to 1/2/4/8 nodes
with hyperthreading on and off.

+-----------------+------------------+------------------+----------+
|  Configuration  | Mean Transaction | Mean Transaction | % Change |
|		  |  Rate w/o patch  | Rate with patch  |	   |
+-----------------+------------------------------------------------+
|		  |              User Range 1100 - 2000		   |
+-----------------+------------------------------------------------+
| 8 nodes, HT on  |      36980       |      148590      | +301.8%  |
| 8 nodes, HT off |      42799       |      145011      | +238.8%  |
| 4 nodes, HT on  |	 61318       |      118445      |  +51.1%  |
| 4 nodes, HT off |     158481       |      158592      |   +0.1%  |
| 2 nodes, HT on  |     180602       |      173967      |   -3.7%  |
| 2 nodes, HT off |     198409       |      198073      |   -0.2%  |
| 1 node , HT on  |     149042       |      147671      |   -0.9%  |
| 1 node , HT off |     126036       |      126533      |   +0.4%  |
+-----------------+------------------------------------------------+
|		  |              User Range 200 - 1000		   |
+-----------------+------------------------------------------------+
| 8 nodes, HT on  |      41525       |      122349      | +194.6%  |
| 8 nodes, HT off |      49866       |      124032      | +148.7%  |
| 4 nodes, HT on  |	 66409       |      106984      |  +61.1%  |
| 4 nodes, HT off |     119880       |      130508      |   +8.9%  |
| 2 nodes, HT on  |     138003       |      133948      |   -2.9%  |
| 2 nodes, HT off |     132792       |      131997      |   -0.6%  |
| 1 node , HT on  |     116593       |      115859      |   -0.6%  |
| 1 node , HT off |     104499       |      104597      |   +0.1%  |
+-----------------+------------------+------------------+----------+
AIM7 benchmark run has a pretty large run-to-run variance due to random
nature of the subtests executed. So a difference of less than +-5%
may not be really significant.

This patch improves high_systime workload performance at 4 nodes
and up by maintaining transaction rates without significant drop-off
at high node count.  The patch has practically no impact on 1 and 2
nodes system.

The table below shows the percentage time (as reported by perf
record -a -s -g) spent on the __mutex_lock_slowpath() function by
the high_systime workload at 1500 users for 2/4/8-node configurations
with hyperthreading off.

+---------------+-----------------+------------------+---------+
| Configuration | %Time w/o patch | %Time with patch | %Change |
+---------------+-----------------+------------------+---------+
|    8 nodes    |      65.34%     |      0.69%       |  -99%   |
|    4 nodes    |       8.70%	  |      1.02%	     |  -88%   |
|    2 nodes    |       0.41%     |      0.32%       |  -22%   |
+---------------+-----------------+------------------+---------+
It is obvious that the dramatic performance improvement at 8
nodes was due to the drastic cut in the time spent within the
__mutex_lock_slowpath() function.

The table below show the improvements in other AIM7 workloads (at 8
nodes, hyperthreading off).

+--------------+---------------+----------------+-----------------+
|   Workload   | mean % change | mean % change  | mean % change   |
|              | 10-100 users  | 200-1000 users | 1100-2000 users |
+--------------+---------------+----------------+-----------------+
| alltests     |     +0.6%     |   +104.2%      |   +185.9%       |
| five_sec     |     +1.9%     |     +0.9%      |     +0.9%       |
| fserver      |     +1.4%     |     -7.7%      |     +5.1%       |
| new_fserver  |     -0.5%     |     +3.2%      |     +3.1%       |
| shared       |    +13.1%     |   +146.1%      |   +181.5%       |
| short        |     +7.4%     |     +5.0%      |     +4.2%       |
+--------------+---------------+----------------+-----------------+
Signed-off-by: Waiman Long <Waiman.Long@hp.com>
Reviewed-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
mutex: restrict mutex spinning to only one task per mutex

Date	Thu, 4 Apr 2013 10:54:17 -0400

The current mutex spinning code allow multiple tasks to spin on a
single mutex concurrently. There are two major problems with this
approach:

 1. This is not very energy efficient as the spinning tasks are not
    doing useful work. The spinning tasks may also block other more
    important or useful tasks from running as preemption is disabled.
    Only one of the spinners will get the mutex at any time. The
    other spinners will have to wait for much longer to get it.

 2. The mutex data structure on x86-64 should be 32 bytes. The spinning
    code spin on lock->owner which, in most cases, should be in the same
    64-byte cache line as the lock->wait_lock spinlock. As a result,
    the mutex spinners are contending the same cacheline with other
    CPUs trying to get the spinlock leading to increased time spent
    on the spinlock as well as on the mutex spinning.

These problems are worse on system with large number of CPUs. One way
to reduce the effect of these two problems is to allow only one task
to be spinning on a mutex at any time.

This patch adds a new spinner field in the mutex.h to limit the
number of spinner to only one task. That will increase the size of
the mutex by 8 bytes in a 64-bit environment (4 bytes in a 32-bit
environment).

The AIM7 benchmarks were run on 3.7.10 derived kernels to show the
performance changes with this patch on a 8-socket 80-core system
with hyperthreading off.  The table below shows the mean % change
in performance over a range of users for some AIM7 workloads with
just the less atomic operation patch (patch 1) vs the less atomic
operation patch plus this one (patches 1+2).

+--------------+-----------------+-----------------+-----------------+
|   Workload   | mean % change   | mean % change   | mean % change   |
|              | 10-100 users    | 200-1000 users  | 1100-2000 users |
+--------------+-----------------+-----------------+-----------------+
| alltests     |     -0.2%       |     -3.8%       |    -4.2%        |
| five_sec     |     -0.6%       |     -2.0%       |    -2.4%        |
| fserver      |     +2.2%       |    +16.2%       |    +2.2%        |
| high_systime |     -0.3%       |     -4.3%       |    -3.0%        |
| new_fserver  |     +3.9%       |    +16.0%       |    +9.5%        |
| shared       |     -1.7%       |     -5.0%       |    -4.0%        |
| short        |     -7.7%       |     +0.2%       |    +1.3%        |
+--------------+-----------------+-----------------+-----------------+
It can be seen that this patch improves performance for the fserver and
new_fserver workloads while suffering some slight drop in performance
for the other workloads.

Signed-off-by: Waiman Long <Waiman.Long@hp.com>
Reviewed-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
usb: host: ehci: allow ehci_* symbols to be unused

not all platforms will use all of those ehci_*
symbols on their hc_driver structure. Sometimes
we might need to provide a modified version of
a certain method or not provide it at all, as is
the case with OMAPs which don't support port handoff
feature.

Whenever we compile a kernel for an OMAP board with
EHCI enabled, we get compile warnings:

drivers/usb/host/ehci-hub.c:1079: warning: 'ehci_relinquish_port' \
	defined but not used
drivers/usb/host/ehci-hub.c:1088: warning: 'ehci_port_handed_over' \
	defined but not used

In order to cleanup those warnings, we're adding
__maybe_unused annotation to those functions.

Signed-off-by: Felipe Balbi <balbi@ti.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
usb: musb: gadget: use pm autosuspend in musb_gadget_pullup

If musb_gadget_pullup doesn't use pm autosuspend, next may happen:

time |   pseudocode
------------------------------------------------------------
t1   | pm_runtime_put_autosuspend() scedules suspend at 't4'
     |     /* some code running */
t2   | musb_gadget_pullup: pm_runtime_get_sync() <-- cancels requested autosuspend
     |     /* some code running */
t3   | musb_gadget_pullup: pm_runtime_put() <-- suspends device (may happen before t4!)
     |
t4   | <- suspend is expected here, but device may be already suspended between t3 and t4

So use pm autosuspend in musb_gadget_pullup and prevent such situation

Change-Id: I8b9618c6d24bea3810c736f8fc0195802ce79c55
Signed-off-by: Ruslan Bilovol <ruslan.bilovol@ti.com>
(cherry picked from commit fd5404d9033defa72dbbdaace600a67fcf9763bb)
USB: musb: do not proceed with interrupt completions if qh is not ready

It's not safe to modify hw_ep if qh is not ready.
This may corrupt data in musb_qh. In RX/TX completions this
may happen on SMP system with current musb host implementation
The main issue is with musb_giveback function. It unlocks and
and after that locks musb->lock so during this time
musb_qh data may be corrupted. This results in null-pointers
dereferences / programming of busy DMA channel / etc in
different places.

This often happens during unlinking URB for ISO and looks like:
CPU0                                  CPU1
...                                   musb_urb_dequeue
...                                      spin_lock_irqsave(&musb->lock)
...                                      (work with musb_qh)
...                                      musb_giveback
...                                         spin_unlock(&musb->lock)
generic_interrupt()                         ...
   spin_lock_irqsave(&musb->lock)           ...
   (work with musb_qh)                      ...
   spin_unlock_irqrestore(&musb->lock)      ...
...                                         spin_lock(&musb->lock)
...                                      (continues work with musb_qh)
...                                      spin_lock_irqrestore(&musb->lock)

Go out from completion if qh is not ready.

Change-Id: I9c9280a7ab7f8e89b2e8bf745800217f903108fc
Signed-off-by: Ruslan Bilovol <ruslan.bilovol@ti.com>
(cherry picked from commit d0d4f29c2ac05a66a8b26c00ddd436e71519a1d5)
random: remove warning
change to max os x

Signed-off-by: Willi Ye <williye97@gmail.com>

Conflicts:
	Makefile