arch: arm: lib: optimize memcpy for cortex-A15

The LDRD/STRD instruction is faster than LDM/STM on Cortex-A15.
Also optimized preload cache size for Cortex-A15.

Added USE_LDRDSTRD_OVER_LDMSTM to turn on LDRD/STRD optimization.
Added ARM_PLD_SIZE, default 32. Should set to 64 for Cortex-A15.

Bug 1185248

Signed-off-by: Chandler Zhang <chazhang@nvidia.com>
Reviewed-by: Krishna Reddy <vdumpa@nvidia.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
Apply arch: arm: lib: optimize memcpy for cortex-A15 to memutils

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>

Conflicts:
	arch/arm/mach-msm/memutils/copy_from_user.S
	arch/arm/mach-msm/memutils/copy_template.S
	arch/arm/mach-msm/memutils/copy_to_user.S
	arch/arm/mach-msm/memutils/memcpy.S
ksm: Provide support to use deferred timers for scanner thread

KSM thread to scan pages is getting schedule on definite timeout.
That wakes up CPU from idle state and hence may affect the power
consumption. Provide an optional support to use deferred timer
which suites low-power use-cases.

To enable deferred timers,
$ echo 1 > /sys/kernel/mm/ksm/deferred_timer

Signed-off-by: Chintan Pandya <cpandya@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
arm: mach-msm: Remove the unused rmt_storage code

The rmt_storage module is no longer used.
Remove the unused module and cleanup the makefile and kconfig.

CRs-Fixed: 549951
Change-Id: Ie9ebaa2209ea9fa0df05dc892504fcb882f00f29
Signed-off-by: Nikhilesh Reddy <reddyn@codeaurora.org>
Signed-off-by: engstk <eng.stk@sapo.pt>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>

Conflicts:

	arch/arm/mach-msm/Kconfig
	arch/arm/mach-msm/Makefile

Signed-off-by: engstk <eng.stk@sapo.pt>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
msm: Allow lowmem to be non contiguous and mixed.

Any image that is expected to have a lifetime of
the entire system can give the virtual address
space back for use in vmalloc.

Change-Id: I81ce848cd37e8573d706fa5d1aa52147b3c8da12
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
mm: Update is_vmalloc_addr to account for vmalloc savings

is_vmalloc_addr current assumes that all vmalloc addresses
exist between VMALLOC_START and VMALLOC_END. This may not be
the case when interleaving vmalloc and lowmem. Update the
is_vmalloc_addr to properly check for this.

Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
msm: Adjust the low memory boundary

The unused virtual address space in low memory is given to
vmalloc for use. This reduces the low memory space and
increases vmalloc space. Adjust the vmalloc_min in order
to increase the low memory.

Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
msm: Update the kernel log messages

After reclaiming vmalloc and adjusting lowmem, the two regions
are no longer contiguous. Update the messages printed during
bootup to list all regions corresponding to low memory and vmalloc
respectively

Change-Id: I2aeb25026c068e27860c3df1496fc6f5c420b818
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
ARM: 7629/1: mm: Fix missing XN flag for for MT_MEMORY_SO

Commit {ARM: mm: Add strongly ordered descriptor support}
added XN flag at section level but missed it at PTE level.

Fix it by adding the L_PTE_XN to MT_MEMORY_SO PTE descriptor.

Reported-by: Richard Woodruff <r-woodruff2@ti.com>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
mm: make is_vmalloc_addr lockless

is_vmalloc_addr currently takes the vmap_area_lock and walks
the list of vmalloc areas to determine if an address is actually
a vmalloc address. Unfortunately, the current locking structure
with vmap_area_lock does not disable irqs which means that it may
be possible to recusively take the vmap_area_lock if an irq occurs
while walking the tree list. Considering the list of possible
vmalloc vs. not vmalloc ranges is not going to change after bootup,
skip the tree walking and just keep a bitmap of which virtual
addresses are set aside for vmalloc and which are not.

Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
ARM: 7541/1: Add ARM ERRATA 775420 workaround

arm: Add ARM ERRATA 775420 workaround

Workaround for the 775420 Cortex-A9 (r2p2, r2p6,r2p8,r2p10,r3p0) erratum.
In case a date cache maintenance operation aborts with MMU exception, it
might cause the processor to deadlock. This workaround puts DSB before
executing ISB if an abort may occur on cache maintenance.

Based on work by Kouei Abe and feedback from Catalin Marinas.

Signed-off-by: Kouei Abe <kouei.abe.cp@rms.renesas.com>
[ horms@verge.net.au: Changed to implementation
  suggested by catalin.marinas@arm.com ]
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
ARM: 7953/1: mm: ensure TLB invalidation is complete before enabling MMU

During __v{6,7}_setup, we invalidate the TLBs since we are about to
enable the MMU on return to head.S. Unfortunately, without a subsequent
dsb instruction, the invalidation is not guaranteed to have completed by
the time we write to the sctlr, potentially exposing us to junk/stale
translations cached in the TLB.

This patch reworks the init functions so that the dsb used to ensure
completion of cache/predictor maintenance is also used to ensure
completion of the TLB invalidation.

Reported-by: Albin Tonnerre <Albin.Tonnerre@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
ARM: 8115/1: LPAE: reduce damage caused by idmap to virtual memory layout

commit 811a2407a3cf7bbd027fbe92d73416f17485a3d8 upstream.

On LPAE, each level 1 (pgd) page table entry maps 1GiB, and the level 2
(pmd) entries map 2MiB.

When the identity mapping is created on LPAE, the pgd pointers are copied
from the swapper_pg_dir.  If we find that we need to modify the contents
of a pmd, we allocate a new empty pmd table and insert it into the
appropriate 1GB slot, before then filling it with the identity mapping.

However, if the 1GB slot covers the kernel lowmem mappings, we obliterate
those mappings.

When replacing a PMD, first copy the old PMD contents to the new PMD, so
that we preserve the existing mappings, particularly the mappings of the
kernel itself.

[rewrote commit message and added code comment -- rmk]

Fixes: ("ARM: LPAE: Add identity mapping support for the 3-level page table format")
Signed-off-by: Konstantin Khlebnikov <k.khlebnikov@samsung.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
ARM: 8128/1: abort: don't clear the exclusive monitors

The ARMv6 and ARMv7 early abort handlers clear the exclusive monitors
upon entry to the kernel, but this is redundant:

  - We clear the monitors on every exception return since commit
    200b812d0084 ("Clear the exclusive monitor when returning from an
    exception"), so this is not necessary to ensure the monitors are
    cleared before returning from a fault handler.

  - Any dummy STREX will target a temporary scratch area in memory, and
    may succeed or fail without corrupting useful data. Its status value
    will not be used.

  - Any other STREX in the kernel must be preceded by an LDREX, which
    will initialise the monitors consistently and will not depend on the
    earlier state of the monitors.

Therefore we have no reason to care about the initial state of the
exclusive monitors when a data abort is taken, and clearing the monitors
prior to exception return (as we already do) is sufficient.

This patch removes the redundant clearing of the exclusive monitors from
the early abort handlers.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Zefan Li <lizefan@huawei.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
ARM: 8129/1: errata: work around Cortex-A15 erratum 830321 using dummy strex

On revisions of Cortex-A15 prior to r3p3, a CLREX instruction at PL1 may
falsely trigger a watchpoint exception, leading to potential data aborts
during exception return and/or livelock.

This patch resolves the issue in the following ways:

  - Replacing our uses of CLREX with a dummy STREX sequence instead (as
    we did for v6 CPUs).

  - Removing the clrex code from v7_exit_coherency_flush and derivatives,
    since this only exists as a minor performance improvement when
    non-cached exclusives are in use (Linux doesn't use these).

Benchmarking on a variety of ARM cores revealed no measurable
performance difference with this change applied, so the change is
performed unconditionally and no new Kconfig entry is added.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
[lizf: Backported to 3.4:
 - Drop changes to arch/arm/include/asm/cacheflush.h and
   arch/arm/mach-exynos/mcpm-exynos.c]
Signed-off-by: Zefan Li <lizefan@huawei.com>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
ARM: 7693/1: mm: clean-up in order to reduce to call kmap_high_get()

In kmap_atomic(), kmap_high_get() is invoked for checking already
mapped area. In __flush_dcache_page() and dma_cache_maint_page(),
we explicitly call kmap_high_get() before kmap_atomic()
when cache_is_vipt(), so kmap_high_get() can be invoked twice.
This is useless operation, so remove one.

v2: change cache_is_vipt() to cache_is_vipt_nonaliasing() in order to
be self-documented

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Chintan Pandya <cpandya@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
msm: Fix compiler error due to CONFIG_HIGHMEM being disabled

If CONFIG_HIGHMEM is disabled, a compile error is seen as
the printk is part of the conditional define. Move the printk
to be independent of the conditional config define.

Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
msm: Increase the kernel virtual area to include lowmem

Even though lowmem is accounted for in vmalloc space, allocation
comes only from the region bounded by VMALLOC_START and VMALLOC_END.
The kernel virtual area can now allocate from any unmapped region
starting from PAGE_OFFSET.

Change-Id: I291b9eb443d3f7445fd979bd7b09e9241ff22ba3
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
mm: Update is_vmalloc_addr to account for vmalloc savings

is_vmalloc_addr current assumes that all vmalloc addresses
exist between VMALLOC_START and VMALLOC_END. This may not be
the case when interleaving vmalloc and lowmem. Update the
is_vmalloc_addr to properly check for this.

Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
mm: make is_vmalloc_addr lockless

is_vmalloc_addr currently takes the vmap_area_lock and walks
the list of vmalloc areas to determine if an address is actually
a vmalloc address. Unfortunately, the current locking structure
with vmap_area_lock does not disable irqs which means that it may
be possible to recusively take the vmap_area_lock if an irq occurs
while walking the tree list. Considering the list of possible
vmalloc vs. not vmalloc ranges is not going to change after bootup,
skip the tree walking and just keep a bitmap of which virtual
addresses are set aside for vmalloc and which are not.

Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>