Revert "Apply arch: arm: lib: optimize memcpy for cortex-A15 to memutils"

This reverts commit 5f266ebd51938e2d37272bc6772168e666ab450f.
Revert "hells_defconfig: Enable memcpy optimizations for cortex A-15"

This reverts commit cd77f077ec980ab5695072cc72684efc128f294a.
Revert "arch: arm: lib: optimize memcpy for cortex-A15"

This reverts commit 2514854cf2320f2921c967d7070949716dcf1788.
Revert "msm: memutils: memcpy, memmove, copy_page optimization"

c17187cf1c653c87c1ae893a327e2b74165722fc
msm: memutils: memcpy, memmove, copy_page optimization

Preload farther to take advantage of the memory bus, and assume
64-byte cache lines.  Unroll some pairs of ldm/stm as well, for
unexplainable reasons.

Future enhancements should include,

- #define for how far to preload, possibly defined separately for
  memcpy, copy_*_user
- Tuning for misaligned buffers
- Tuning for memmove
- Tuning for small buffers
- Understanding mechanism behind ldm/stm unroll causing some gains
  in copy_to_user

BASELINE (msm8960pro):
======================================================================
memcpy 1000MB at 5MB       : took 808850 usec, bandwidth 1236.236 MB/s
copy_to_user 1000MB at 5MB : took 810071 usec, bandwidth 1234.234 MB/s
copy_from_user 1000MB at 5M: took 942926 usec, bandwidth 1060.060 MB/s
memmove 1000GB at 5MB      : took 848588 usec, bandwidth 1178.178 MB/s
copy_to_user 1000GB at 4kB : took 847916 usec, bandwidth 1179.179 MB/s
copy_from_user 1000GB at 4k: took 935113 usec, bandwidth 1069.069 MB/s
copy_page 1000GB at 4kB    : took 779459 usec, bandwidth 1282.282 MB/s

THIS PATCH:
======================================================================
memcpy 1000MB at 5MB       : took 346223 usec, bandwidth 2888.888 MB/s
copy_to_user 1000MB at 5MB : took 348084 usec, bandwidth 2872.872 MB/s
copy_from_user 1000MB at 5M: took 348176 usec, bandwidth 2872.872 MB/s
memmove 1000GB at 5MB      : took 348267 usec, bandwidth 2871.871 MB/s
copy_to_user 1000GB at 4kB : took 377018 usec, bandwidth 2652.652 MB/s
copy_from_user 1000GB at 4k: took 371829 usec, bandwidth 2689.689 MB/s
copy_page 1000GB at 4kB    : took 383763 usec, bandwidth 2605.605 MB/s

Signed-off-by: Chris Fries <C.Fries@motorola.com>

arm: lib: Fix makefile bug

When HAS_MACH_MEMUTILS is defined, two version of memcpy.S would
be built at the same time, fix the issue here.

Signed-off-by: Hong-Mei Li <a21834@motorola.com>
Reviewed-on: http://gerrit.mot.com/629464
SLTApproved: Slta Waiver <sltawvr@motorola.com>
Tested-by: Jira Key <jirakey@motorola.com>
Reviewed-by: Yi-Wei Zhao <gbjc64@motorola.com>
Submit-Approved: Jira Key <jirakey@motorola.com>

msm: memutils: memcpy, memmove optimization

1. To fit 8x26 64-bytes cache line size, we change preload steps as 64Bytes.
2. According to tune result, change preload distance as 5 cache lines.
3. According to tune result, re-arrange the ld/str order as back-to-back
   for copy_from_user.

The markable improvement:
	memcpy : 5%
	copy_to_user : 9%
	copy_from_user : 13%
	memmove : 37%

Raw data is as below:

BASELINE:
	memcpy 1000MB at 5MB       : took 1547098 usec, bandwidth 646.646 MB/s
	copy_to_user 1000MB at 5MB : took 1704308 usec, bandwidth 586.586 MB/s
	copy_from_user 1000MB at 5M: took 1777090 usec, bandwidth 562.562 MB/s
	memmove 1000GB at 5MB      : took 1066205 usec, bandwidth 937.937 MB/s
	copy_to_user 1000GB at 4kB : took 1774866 usec, bandwidth 563.563 MB/s
	copy_from_user 1000GB at 4k: took 1797654 usec, bandwidth 556.556 MB/s
	copy_page 1000GB at 4kB    : took 1644606 usec, bandwidth 608.608 MB/s
	memmove 1000GB at 4kB      : took 1236227 usec, bandwidth 808.808 MB/s

THIS PATCH:
	memcpy 1000MB at 5MB       : took 1475835 usec, bandwidth 677.677 MB/s
	copy_to_user 1000MB at 5MB : took 1559060 usec, bandwidth 641.641 MB/s
	copy_from_user 1000MB at 5M: took 1561603 usec, bandwidth 640.640 MB/s
	memmove 1000GB at 5MB      : took 861664 usec, bandwidth 1160.160 MB/s
	copy_to_user 1000GB at 4kB : took 1673501 usec, bandwidth 597.597 MB/s
	copy_from_user 1000GB at 4k: took 1674006 usec, bandwidth 597.597 MB/s
	copy_page 1000GB at 4kB    : took 1691358 usec, bandwidth 591.591 MB/s
	memmove 1000GB at 4kB      : took 882985 usec, bandwidth 1132.132 MB/s

Signed-off-by: Hong-Mei Li <a21834@motorola.com>
Reviewed-on: http://gerrit.pcs.mot.com/550383
SLT-Approved: Slta Waiver <sltawvr@motorola.com>
Tested-by: Jira Key <jirakey@motorola.com>
Reviewed-by: Klocwork kwcheck <klocwork-kwcheck@sourceforge.mot.com>
Reviewed-by: Check Patch <CHEKPACH@motorola.com>
Reviewed-by: Yi-Wei Zhao <gbjc64@motorola.com>
Submit-Approved: Jira Key <jirakey@motorola.com>

msm: memutils: 8974: memcpy, memmove optimization

According to tune result, change preload distance to 10
cache lines for 8974.
And considering bkk-b is sharing code for 8974, 8226, 8x16,
we seperate the code for 8974 specially.

The markable improvement:
    memcpy : 67%
    copy_to_user : 37%
    copy_from_user : 37%
    memmove : 70%

Raw data is as below:

BEFORE:
	memcpy 1000MB at 5MB       : took 659744 usec, bandwidth 1515.515 MB/s
	copy_to_user 1000MB at 5MB : took 681202 usec, bandwidth 1467.467 MB/s
	copy_from_user 1000MB at 5M: took 682052 usec, bandwidth 1466.466 MB/s
	memmove 1000GB at 5MB      : took 763133 usec, bandwidth 1310.310 MB/s
	copy_to_user 1000GB at 4kB : took 714487 usec, bandwidth 1399.399 MB/s
	copy_from_user 1000GB at 4k: took 699861 usec, bandwidth 1428.428 MB/s
	copy_page 1000GB at 4kB    : took 634857 usec, bandwidth 1575.575 MB/s
	memmove 1000GB at 4kB      : took 791389 usec, bandwidth 1263.263 MB/s
	memcpy 1000MB at 5MB       : took 659744 usec, bandwidth 1515.515 MB/s

AFTER:
	memcpy 1000MB at 5MB       : took 395178 usec, bandwidth 2530.530 MB/s
	copy_to_user 1000MB at 5MB : took 497667 usec, bandwidth 2009.009 MB/s
	copy_from_user 1000MB at 5M: took 496832 usec, bandwidth 2012.012 MB/s
	memmove 1000GB at 5MB      : took 449217 usec, bandwidth 2226.226 MB/s
	copy_to_user 1000GB at 4kB : took 489006 usec, bandwidth 2044.044 MB/s
	copy_from_user 1000GB at 4k: took 483982 usec, bandwidth 2066.066 MB/s
	copy_page 1000GB at 4kB    : took 377170 usec, bandwidth 2651.651 MB/s
	memmove 1000GB at 4kB      : took 483015 usec, bandwidth 2070.070 MB/s

Signed-off-by: Hong-Mei Li <a21834@motorola.com>
Reviewed-on: http://gerrit.mot.com/634422
Tested-by: Jira Key <jirakey@motorola.com>
Reviewed-by: Yi-Wei Zhao <gbjc64@motorola.com>
SLTApproved: Christopher Fries <c.fries@motorola.com>
Reviewed-by: John Zafiris <johnz@motorola.com>
Submit-Approved: Jira Key <jirakey@motorola.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
hells_defconfig: Enable MEMUTILS
hells_defconfig: hC-MIUI-b7.2
Add build-script

credits to @anarkia1976
hells_defconfig: Remove Localversion
asm-generic: io: don't perform swab during {in,out} string functions

The {in,out}s{b,w,l} functions are designed to operate on a stream of
bytes and therefore should not perform any byte-swapping, regardless of
the CPU byte order.

This patch fixes the generic IO header so that {in,out}s{b,w,l} call the
__raw_{read,write} functions directly rather than going via the
endian-correcting accessors.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Cc: Mike Frysinger <vapier@gentoo.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Ben Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
ARM: nommu: provide dummy cpu_switch_mm implementation

cpu_switch_mm is a logical nop on nommu systems, so define it as such
when !CONFIG_MMU.

Signed-off-by: Will Deacon <will.deacon@arm.com>
ARM: mpu: add PMSA related registers and bitfields to existing headers

This patch adds the following definitions relevant to the PMSA:

Add SCTLR bit 17, (CR_BR - Background Region bit) to the list of CR_*
bitfields. This bit determines whether to use the architecturally defined
memory map

Add the MPUIR to the available registers when using read_cpuid macro. The
MPUIR is the MPU type register.

Signed-off-by: Jonathan Austin <jonathan.austin@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
CC:"Uwe Kleine-König" <u.kleine-koenig@pengutronix.de>
Signed-off-by: Jake Weinstein <xboxlover360@gmail.com>
ARM: mpu: add header for MPU register layouts and region data

This commit adds definitions relevant to the ARM v7 PMSA compliant MPU.

The register layouts and region configuration data is made accessible to asm
as well as C-code so that it can be used in early bring-up of the MPU.

The mpu region information structs assume that the properties for the I/D side
are the same, though the implementation could be trivially extended for future
platforms where this is no-longer true.

The MPU_*_REGION defines are used for the basic, static MPU region setup.

Signed-off-by: Jonathan Austin <jonathan.austin@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Jake Weinstein <xboxlover360@gmail.com>
ARM: mpu: add early bring-up code for the ARMv7 PMSA-compliant MPU

This patch adds initial support for using the MPU, which is necessary for
SMP operation on PMSAv7 processors because it is the only way to ensure
memory is shared. This is an initial patch and full SMP support is added
later in this series.

The setup of the MPU is performed in a way analagous to that for the MMU:
Very early initialisation before the C environment is brought up, followed
by a sanity check and more complete initialisation in C.

This patch provides the simplest possible memory region configuration:
MPU_PROBE_REGION: Reserved for probing MPU details, not enabled
MPU_BG_REGION: A 'background' region that specifies all memory strongly ordered
MPU_RAM_REGION: A single shared, cacheable, normal region for the valid RAM.

In this early initialisation code we simply map the whole of the address
space with the BG_REGION and (at least) the kernel with the RAM_REGION. The
MPU has region alignment constraints that require us to round past the end
of the kernel.

As region 2 has a higher priority than region 1, it overrides the strongly-
ordered behaviour for RAM only.

Subsequent patches will add more complete initialisation from the C-world
and support for bringing up secondary CPUs.

Signed-off-by: Jonathan Austin <jonathan.austin@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
CC: Hyok S. Choi <hyok.choi@samsung.com>
Signed-off-by: Jake Weinstein <xboxlover360@gmail.com>

Conflicts:
	arch/arm/kernel/head-nommu.S
ARM: mpu: add MPU initialisation for secondary cores

The MPU initialisation on the primary core is performed in two stages, one
minimal stage to ensure the CPU can boot and a second one after
sanity_check_meminfo. As the memory configuration is known by the time we
boot secondary cores only a single step is necessary, provided the values
for DRSR are passed to secondaries.

This patch implements this arrangement. The configuration generated for the
MPU regions is made available to the secondary core, which can then use the
asm MPU intialisation code to program a complete region configuration.

This is necessary for SMP configurations without an MMU, as the MPU
initialisation is the only way to ensure that memory is specified as
'shared'.

Signed-off-by: Jonathan Austin <jonathan.austin@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
CC: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Jake Weinstein <xboxlover360@gmail.com>
ARM: prefetch: remove redundant "cc" clobber

The pld instruction does not affect the condition flags, so don't bother
clobbering them.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
ARM: atomics: prefetch the destination word for write prior to strex

The cost of changing a cacheline from shared to exclusive state can be
significant, especially when this is triggered by an exclusive store,
since it may result in having to retry the transaction.

This patch prefixes our atomic access implementations with pldw
instructions (on CPUs which support them) to try and grab the line in
exclusive state from the start. Only the barrier-less functions are
updated, since memory barriers can limit the usefulness of prefetching
data.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
ARM: atomic64: fix endian-ness in atomic.h

Fix inline asm for atomic64_xxx functions in arm atomic.h. Instead of
%H operand specifiers code should use %Q for least significant part
of the value, and %R for the most significant part of the value. %H
always returns the higher of the two register numbers, and therefore
it is not endian neutral. %H should be used with ldrexd and strexd
instructions.

Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk>
ARM: 7983/1: atomics: implement a better __atomic_add_unless for v6+

Looking at perf profiles of multi-threaded hackbench runs, a significant
performance hit appears to manifest from the cmpxchg loop used to
implement the 32-bit atomic_add_unless function. This can be mitigated
by writing a direct implementation of __atomic_add_unless which doesn't
require iteration outside of the atomic operation.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7447/1: rwlocks: remove unused branch labels from trylock routines

The ARM arch_{read,write}_trylock implementations include unused
backwards branch labels, since we don't retry the locking operation
if the exclusive store fails.

This patch removes the labels.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>