ARM: mm: implement LoUIS API for cache maintenance ops

ARM v7 architecture introduced the concept of cache levels and related
control registers. New processors like A7 and A15 embed an L2 unified cache
controller that becomes part of the cache level hierarchy. Some operations in
the kernel like cpu_suspend and __cpu_disable do not require a flush of the
entire cache hierarchy to DRAM but just the cache levels belonging to the
Level of Unification Inner Shareable (LoUIS), which in most of ARM v7 systems
correspond to L1.

The current cache flushing API used in cpu_suspend and __cpu_disable,
flush_cache_all(), ends up flushing the whole cache hierarchy since for
v7 it cleans and invalidates all cache levels up to Level of Coherency
(LoC) which cripples system performance when used in hot paths like hotplug
and cpuidle.

Therefore a new kernel cache maintenance API must be added to cope with
latest ARM system requirements.

This patch adds flush_cache_louis() to the ARM kernel cache maintenance API.

This function cleans and invalidates all data cache levels up to the
Level of Unification Inner Shareable (LoUIS) and invalidates the instruction
cache for processors that support it (> v7).

This patch also creates an alias of the cache LoUIS function to flush_kern_all
for all processor versions prior to v7, so that the current cache flushing
behaviour is unchanged for those processors.

v7 cache maintenance code implements a cache LoUIS function that cleans and
invalidates the D-cache up to LoUIS and invalidates the I-cache, according
to the new API.

Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
ARM: mm: rename jump labels in v7_flush_dcache_all function

This patch renames jump labels in v7_flush_dcache_all in order to define
a specific flush cache levels entry point.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
ARM: kernel: update cpu_suspend code to use cache LoUIS operations

In processors like A15/A7 L2 cache is unified and integrated within the
processor cache hierarchy, so that it is not considered an outer cache
anymore. For processors like A15/A7 flush_cache_all() ends up cleaning
all cache levels up to Level of Coherency (LoC) that includes
the L2 unified cache.

When a single CPU is suspended (CPU idle) a complete L2 clean is not
required, so generic cpu_suspend code must clean the data cache using the
newly introduced cache LoUIS function.

The context and stack pointer (context pointer) are cleaned to main memory
using cache area functions that operate on MVA and guarantee that the data
is written back to main memory (perform cache cleaning up to the Point of
Coherency - PoC) so that the processor can fetch the context when the MMU
is off in the cpu_resume code path.

outer_cache management remains unchanged.

Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
ARM: kernel: update __cpu_disable to use cache LoUIS maintenance API

When a CPU is hotplugged out caches that reside in its power domain
lose their contents and so must be cleaned to the next memory level.

Currently, __cpu_disable calls flush_cache_all() that for new generation
processor like A15/A7 ends up cleaning and invalidating all cache levels
up to Level of Coherency, which includes the unified L2.

This ends up being a waste of cycles since the L2 cache contents are not
lost on power down.

This patch updates __cpu_disable to use the new LoUIS API cache operations.

Acked-by: Nicolas Pitre <nico@linaro.org>
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
ARM: mm: update __v7_setup() to the new LoUIS cache maintenance API

The ARMv7 processor setup function __v7_setup() cleans and invalidates the
CPU cache before enabling MMU to start the CPU with a clean CPU local cache.

But on ARMv7 architectures like Cortex-[A15/A8], this code will end
up flushing the L2 caches(up to level of Coherency) which is undesirable
and expensive. The setup functions are used in the CPU hotplug scenario too
and hence flushing all cache levels should be avoided.

This patch replaces the cache flushing call with the newly introduced
v7 dcache LoUIS API where only cache levels up to LoUIS are cleaned and
invalidated when a processors executes __v7_setup which is the expected
behavior.

For processors like A9 and A5 where the L2 cache is an outer one the
behavior should be unchanged.

Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
block: do not notify urgent request, when flush with data in flight

MMC device driver implements URGENT request execution with priority
(using stop flow), as a result currently running (and prepared) request
may be reinserted back into I/O scheduler. This will break block layer
logic of flushes (flush request should not be inserted into I/O scheduler).

Block layer flush machinery keep q->flush_data_in_flight list updated with
started but not completed flush requests with data (REQ_FUA).

This change will not notify underling block device driver about pending
urgent request during flushes in flight.

Change-Id: I8b654925a3c989250fcb8f4f7c998795fb203923
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
mmc: queue: do not clean current request when urgent in progress

As a result of following fix: "mmc: core: do not reinsert prepeared
FUA and FLUSH requests in stop flow" REQ_FUA and REQ_FLUSH requests
are not reinserted back into i/o scheduler, but instead started request
execution.

This change will prevent cleaning current request pointer for such
requests.

Change-Id: I25f8706954fb538be62182c87d5fb20354696b7a
CRs-fixed: 600127
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
msm: msm_bus: Fix the type error affecting sum of abs

On legacy chipsets, uint16 was being used to write bandwidth
values to the Fabric registers. However, NoCs support 64-bit integers.
The issue surfaces only when sum of active ab values exceeds the max
ib value in the system, as then the sum of ab values don't impact the
clock anymore. This is exactly what was observed in H-flip video usecase.

This patch fixex the bandwidth sum type due to which clocks were not
updated in those cases where requested ab values are higher than the ib
values.

Change-Id: I701ddb7e7b2d66dd14ef74bc38f3748ac40b76ec
CRs-Fixed: 480365
Signed-off-by: Gagan Mac <gmac@codeaurora.org>
msm: msm_bus: Account for multiple channels in clock frequency

Currently, for multiple channels on a bus or memory, the aggregated
ab value is divided by the number of channels while calculating
bandwidth values for NoCs. However, the channels are not accounted
for while calculating the clock frequency. This factors
into the calculations only when sum of all ab values exceeds the
max of ib values. In that case, we risk running the clocks higher
than required. This patch fixes the issue.

Change-Id: Ia2ddb5f3bfa6049bcffc909c3b79e6382525129c
Signed-off-by: Gagan Mac <gmac@codeaurora.org>
msm: msm_bus: Set clock rate before enabling clocks

This will prevent the clocks from being set to max
when the bus driver enables clocks to set the NoC
registers

Change-Id: I9f29e5545fd5519af3e66122492a3c0de5c6df7d
Signed-off-by: Gagan Mac <gmac@codeaurora.org>
msm: msm_bus: send correct value to bw division function

While calculating bw for existing client request
sometimes it can go to negative if client unregisters
or send request with lesser ab/ib values. Bus driver
will divide bandwidth if interleaving enabled.

Added a macro wrapper to handle negative bw values
since the division utility function can't handle
negative values

CRs-Fixed: 521940
Change-Id: Ic3a216b8efb535593fe02ad3228d71808d2e0f98
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
msm: msm_bus: Fix the type error causing bandwidth overflow

On legacy chipsets, long int was being used to store
return value after calculating interleaved bw. However,
NoCs support 64-bit integers ab/ib values. problme occurs
if client request for higher bw and if difference of ab
value exceeds the range of 32 bit integer, the Value
overflows and turns negative, which leads to wrong bw calculation.

This patch fixes this integer overflow by correcting argument
type to store bw.

CRs-Fixed: 537213
Change-Id: I8c6c79ba245a988c2c54ccaca3f3eaf5cb857ce5
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
msm: msm_bus: Fix division by Zero error in kernel

Avoid the error by checking for zero slave ports before
dividing bandwidth.

CRs-Fixed: 495496
Change-Id: I8e9b1e61ba3b47ed270d2f8cf60ed975b5c49889
Signed-off-by: Gagan Mac <gmac@codeaurora.org>
msm: msm_bus: Add checks during bus client unregistration

During unregistration clients should withdraw all pending
requests. This was earlier done in update-request function.
However, that caused acpuclock client requests to be
incorrectly zeroed out.

It seems more appropriate to let the client ensure that
they correctly clean up their requests than to have bus
driver forcefully remove non-zero requests. Hence, a
warning has been added in the unregistration path.

Change-Id: Icfd4f62f46fdd3219dc685007029be21b983c2d0
CRs-Fixed: 510389
Signed-off-by: Gagan Mac <gmac@codeaurora.org>
msm: idle-v7: Separate PC termination for hotplug

When a core is power collapsed as a part of hotplug the state of the CPU
isn't restored and hence there isn't a need to save the registers. This
change is needed by a subsequent patch that will reserve a DMA memory to
save the register contents. The DMA memory region isn't available until
arch initcall and the kernel thermal module could hotplug a core for
thermal considerations.

CRs-fixed: 489391
Change-Id: Idf6ec5212482ccff91500edc4988516389189627
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
msm: pm-8x60: Use dma_alloc_coherent for the saved region

dma_alloc_coherent is the linux standard way of allocating coherent
(i.e. uncached) memory. This API provides more guarantess than the
custom allocate_contiguous_ebi function, including the ability to
specify allocations from the lower 4G of memory. Switch to this API
for allocation of uncached memory. Because using dma_alloc_coherent
may have other setup involved, switch the setup function to be a
later initcall as well (from core_initcall to arch_initcall).

Change-Id: I96ea4575e5ee82ccbf7e691ba75ea1414dc6d81a
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
msm: clock-debug: Add support to show all the enabled clocks

Add a new debugfs file <debugfs>/clk/enabled_clocks to show all the
currently enabled clock, their rate and their dependency chain.

Example output:
cat /sys/kernel/debug/clk/enabled_clocks
Enabled clocks:
        gcc_lpass_q6_axi_clk:1:1 [0]
        gcc_mss_q6_bimc_axi_clk:1:1 [0]
        gcc_mss_cfg_ahb_clk:1:1 [0]
        gcc_boot_rom_ahb_clk:1:1 [0]
        xo_a_clk:2:2 [19200000]
        gpll0_ao:2:2 [600000000]
        kpss_ahb_clk_src:1:1 [19200000] ->
xo_a_clk:2:2 [19200000]
        bimc_msmbus_a_clk:1:1 [400000000] ->
bimc_a_clk:1:1 [400000000]
        qdss_clk:4:4 [1]
        qdss_clk:4:4 [1]
        qdss_clk:4:4 [1]
        qdss_clk:4:4 [1]
        qdss_clk:4:4 [1]
           ....
Enabled clock count: 42

Change-Id: I6eda59c7e61b77518676e3aa93895ccb792176ac
Signed-off-by: Xiaogang Cui <xiaogang@codeaurora.org>
n1: Update defconfig

Change-Id: I0ccf4fae008c3e5b247025c5b7287521041e84fb