Perf: Check perf activity on correct CPU

The CPU hotplug notifier is not always preempt safe.
So relying on smp_processor_id() is not correct.

Use the cpu id from the notifier as the target
CPU to check for perf activity before hotplug or power
collapse.

CRs-Fixed: 454688
Change-Id: I1a96b236904a2c8ae2f82bbceab4ca8a195395d1
Signed-off-by: Ashwin Chaugule <ashwinc@codeaurora.org>
Perf: Fix counts across power collapse

After entry and exit from power collapse, the perf pm_notifier was
calling perf_pmu_disable/enable functions from perf core. Perf core
maintains a counter that it uses to match disable and enable calls but
since pm_notifier is invoked asynchronously, the counter was getting a
non-zero value which prevented the pmu from being enabled. As a result of
this, we were seeing zero counts for some cores across power collapse.

Make the architecture specific calls directly from pm_notifier.

Change-Id: I744e190906a20e70e1e4798235215c385018b121
Signed-off-by: Sheetal Sahasrabudhe <sheetals@codeaurora.org>
msm: remove perf_debug
ARM: Fix negative idle stats for offline cpu

We see negative idle stats because of cpu dying without
cleaning up idle entry statistics. When a cpu is offline,
the most immediate thing you'd want to do is just call
into cpu_die() and not waste time calling notifier(IDLE_START).

CRs-Fixed: 414554
Change-Id: Iadc6a3ca39997e0ccf65d2a29b004e24b1b211a1
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
Signed-off-by: Taniya Das <tdas@codeaurora.org>
arm: Remove no-longer-required RCU_NONIDLE wrapper

Commit 21111be8 "ARM: Fix negative idle stats for offline cpu" moved call
to cpu_die() to occur outside of rcu_idle_enter()/rcu_idle_exit() section.
As a result, the RCU_NONIDLE() wrapper to complete() call in
arch/arm/kernel/process.c:cpu_die() is no longer required (and is
technically incorrect to have). Removing RCU_NONIDLE() wrapper also removes
this warning seen during CPU offline:

[Note: Below message has been edited to fit 75-char per line limit.
Insignificant portions of warning message has been removed in each line.]

------------[ cut here ]------------
WARNING: at kernel/rcutree.c:456 rcu_idle_exit_common+0x4c/0xe0()
Modules linked in:
(unwind_backtrace+0x0/0x120) from (warn_slowpath_common+0x4c/0x64)
(warn_slowpath_common+0x4c/0x64) from (warn_slowpath_null+0x18/0x1c)
(warn_slowpath_null+0x18/0x1c) from (rcu_idle_exit_common+0x4c/0xe0)
(rcu_idle_exit_common+0x4c/0xe0) from (rcu_idle_exit+0xa8/0xc0)
(rcu_idle_exit+0xa8/0xc0) from (cpu_die+0x24/0x5c)
(cpu_die+0x24/0x5c) from (cpu_idle+0xdc/0xf0)
(cpu_idle+0xdc/0xf0) from (0x8160)
---[ end trace 61bf21937a496a37 ]---

Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
Change-Id: I721e6b20651674e6f6f584cf8d814af00b688c91
msm: remove smd_debug
cpufreq: ondemand: fix timer-related list corruption in store_powersave_bias()

The dbs_timer_init() call in store_powersave_bias() re-initializes the
dbs_info workqueue, call on dbs_timer_exit() to ensure
outstanding work is cleared prior to making this call. Also, grab the
percpu timer_mutex lock to avoid race conditions with respect to the
dbs timer.

Change-Id: I79f3d43eeb51d2d8e21edd0fe043d6583333951f
Signed-off-by: Osvaldo Banuelos <osvaldob@codeaurora.org>
ondemand: add sysfs entry for down_diff_multi_core

This change adds sysfs entry for the ondemand governor's attribute
down_differential_multi_core. This is required to be able to program
this knob to attain better perf/power for use cases like camcorder.

CRs-fixed: 561456
Change-Id: Ib7c1223215c6b714c286c231ed37f3abfae741c6
Signed-off-by: Krishna Vanka <kvanka@codeaurora.org>
ARM: mm: implement LoUIS API for cache maintenance ops

ARM v7 architecture introduced the concept of cache levels and related
control registers. New processors like A7 and A15 embed an L2 unified cache
controller that becomes part of the cache level hierarchy. Some operations in
the kernel like cpu_suspend and __cpu_disable do not require a flush of the
entire cache hierarchy to DRAM but just the cache levels belonging to the
Level of Unification Inner Shareable (LoUIS), which in most of ARM v7 systems
correspond to L1.

The current cache flushing API used in cpu_suspend and __cpu_disable,
flush_cache_all(), ends up flushing the whole cache hierarchy since for
v7 it cleans and invalidates all cache levels up to Level of Coherency
(LoC) which cripples system performance when used in hot paths like hotplug
and cpuidle.

Therefore a new kernel cache maintenance API must be added to cope with
latest ARM system requirements.

This patch adds flush_cache_louis() to the ARM kernel cache maintenance API.

This function cleans and invalidates all data cache levels up to the
Level of Unification Inner Shareable (LoUIS) and invalidates the instruction
cache for processors that support it (> v7).

This patch also creates an alias of the cache LoUIS function to flush_kern_all
for all processor versions prior to v7, so that the current cache flushing
behaviour is unchanged for those processors.

v7 cache maintenance code implements a cache LoUIS function that cleans and
invalidates the D-cache up to LoUIS and invalidates the I-cache, according
to the new API.

Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>

Change-Id: I561c3b187b5b87b25de07ea82a4ef73a42761e27
Signed-off-by: Karthik Parsha <kparsha@codeaurora.org>
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
ARM: kernel: update cpu_suspend code to use cache LoUIS operations

In processors like A15/A7 L2 cache is unified and integrated within the
processor cache hierarchy, so that it is not considered an outer cache
anymore. For processors like A15/A7 flush_cache_all() ends up cleaning
all cache levels up to Level of Coherency (LoC) that includes
the L2 unified cache.

When a single CPU is suspended (CPU idle) a complete L2 clean is not
required, so generic cpu_suspend code must clean the data cache using the
newly introduced cache LoUIS function.

The context and stack pointer (context pointer) are cleaned to main memory
using cache area functions that operate on MVA and guarantee that the data
is written back to main memory (perform cache cleaning up to the Point of
Coherency - PoC) so that the processor can fetch the context when the MMU
is off in the cpu_resume code path.

outer_cache management remains unchanged.

Change-Id: Id5a6d34d55d1b0a5ef112cafc3ad21c0da4b0b84
Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
autosmp: remove unnecessary options in workqueue allocation
sched: Micro-optimize the smart wake-affine logic

Smart wake-affine is using node-size as the factor currently, but the overhead
of the mask operation is high.

Thus, this patch introduce the 'sd_llc_size' percpu variable, which will record
the highest cache-share domain size, and make it to be the new factor, in order
to reduce the overhead and make it more reasonable.

Tested-by: Davidlohr Bueso <davidlohr.bueso@hp.com>
Tested-by: Michael Wang <wangyun@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Acked-by: Michael Wang <wangyun@linux.vnet.ibm.com>
Cc: Mike Galbraith <efault@gmx.de>
Link: http://lkml.kernel.org/r/51D5008E.6030102@linux.vnet.ibm.com
[ Tidied up the changelog. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
msm: boards: Enable Krait Retention.
Add SPM sequences to do Krait retention for 8960,8930
and 8064. Also added a new rpmrs level containing the
latencies and power measurements for this new low power
state.

CRs-Fixed: 369864
Change-Id: Ib30a9a978cf38b35037810a4001c9277837acd4c
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
____________________

msm: pm: Update spm retention sequence to support per core retention
On krait revision < 3, power driver is not using full spm sequence and
calling into trustzone as a part of executing krait retention.
Also krait retention is not supported on secondary cores.

krait pass3 is upgraded to use full spm sequence, hence add support
for per core retention and avoid calling into trustzone
while executing retention.

Change-Id: I6a2b3d7c4d0fd5b9a340957b16b940b7a09bbd91
Signed-off-by: Anji Jonnala <anjir@codeaurora.org>
msm: acpuclock-krait: Work-around retention corner-case

The CPU clock source MUXes cannot switch while a CPU is in
retention, but callers of acpuclock APIs may not known that
the CPU is in retention.

Work around this problem be always performing the CPU MUX
switch on the affected CPU using smp_call_function_single()
in cases where cross-calling might otherwise be possible.
The one exception to this is during hotplug code paths,
where cross-calling is expected by design (since the target
CPU is not online and cannot be in retention).

Change-Id: I76b6de436e71da7979f30bdce841f414c26bfe44
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
msm: pm: Add API to enable/disable retention mode
On 8974 targets, retention mode voltage is controlled by the LDO
regulator, which in turn is powered through a ganged PMIC regulators. At
certain low voltage levels, the ganged PMIC will not be able to power
the LDO retention voltage. The regulator framework would invoke this
API to enable/disable retention mode in these scenarios.

When disabling retention modes, send a IPI to wakeup all online cores
out of idle, to ensure that no core is in a retention low power mode.
Also, disabled support for retention during suspend/hotplug operations
to ensure that a CPU isn't already in retention when the voltage is
lowered. This shouldn't have an effect on suspend, as the driver always
chooses the deepest sleep mode for suspend/hotplug.

Change-Id: I0af18168968c2aaf05ca99188d30762612c87de6
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
msm: pm-8x60: Disable premption in msm_pm_enable_retention

msm_pm_retention_enable() makes smp_call_function_many to ensure that
when a retention is disabled, the cores come out of its low power mode.
Per the smp_call_function_many documentations, preemption should be
disabled before invoking the API.

CRs-fixed: 463188
Change-Id: I4ae9137e8f8e821af729becd0e2d6e6886d5e787
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
msm: pm-8x60: Add early return to msm_pm_enable_retention()

If retention is already disabled, it is not necessary to send IPIs to
the running CPUs to wake them up. Insert an early return for this
scenario to reduce overhead during Krait rail voltage scaling, which
calls this API on every voltage change.

Change-Id: I5db0bafae187ff8a68488e86d4787df9c45787e4
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
msm: pm-8x60: Fix bug where CPU doesn't enter PC
When Krait regulator disables retention mode, an incorrect ordering of
the switch case disables power collapse.

Change-Id: Ic58b0356530e9ecb10d379e2f99d14016bc251e4
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
msm: pm: support retention on CPU0 at suspend