msm: pm: Support for Krait Retention

Introduce a new low power state for Krait, retention.
This state will lower the voltage to the Krait core to
retention voltage in addition to clock gating. This state
will be in between WFI and Standalone power collapse in
terms of time overhead and power savings.

Change-Id: I6b1f0a0d2e4954a705a047fd4a07f4dd1fb3518a
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
(cherry picked from commit d27ca4a7526a890f9d2979acefda3561e49d2e66)

Signed-off-by: Sudhir Sharma <sudsha@codeaurora.org>
(cherry picked from commit 3af7249c36ab53923fdf216df3b3ba2c7f847851)

Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: avs: Add API to restore the AVSCSR register

During power collapse, the AVSCSR register is always enabled and never
restored. This could result in AVS getting enabled on resume, when the
core is running at a frequency that doesn't support AVS.

Add new API, avs_enable, which restores the AVSCSR register based on the
saved AVSCSR register. Update the avs_disable API, to return the
AVSCSR register value prior to disabling the AVS.

(cherry picked from commit ad138da17fe08d46412d56b84309c5d17f7d5931)

Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>

Change-Id: I2bd2e1d6aea058e0f44c474da6c3121654f1236e
Signed-off-by: Neha Pandey <nehap@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: pm: Adjust for events with hard wakeups

Get the next event and setup a new timer to expire
earlier than the next event after accounting for the
latency of the low power state being entered. This
will ensure that the core is active to handle the event
when it arrives.

(cherry picked from commit dc318fd0c3d04f7af9720af50c0eb0c6ed2653e9)
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>

Change-Id: I6609a886df227c68ce78e270ef7d235e07725d44
Signed-off-by: Neha Pandey <nehap@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: pm-8x60: save and restore CP15 registers

To prevent the Krait core configuration from being indvertently changed
when doing L2 power collapse, save a few CP15 registers and restore them
back after coming out of power collapse. This needs to be done after
switching the Krait from active freq to QSB clock source.

Before doing power collapse, the core0 clock switches over to QSB. However,
QSB can run at any frequency and the voltage required for that
frequency, needs to be ensured when coming out power collapse. If the
QSB clock at resume, is a higher frequency at than the QSB clock was at
collapse, then we may run the risk of running at higher clock with a
lower voltage.

Hence, switch to 1.0 V (safe vdd) when going into PC (after switching to
QSB clock source) and the SPM would resume at 1.0V. After the core
starts running, restore the CP15 registers and switch to the previously
saved active voltage, before switching clocks to the active frequency.

This is applicable only for APQ8064.

Change-Id: I75a70f5e07bd581fc8028bed79de7ee796f4c411
Signed-off-by: Praveen Chidambaram <pchidamb@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: pm-8x60: Fix bug where device enter low power mode

An incorrect ordering in the check of allowed modes, results in
standalone power collapse not getting selected when the secondary
cores are online.

Change-Id: Idc7177b78bd0ce7563cc27afbf6c12d3bc1f9f9a
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
Revert "msm: pm-8x60: save and restore CP15 registers"

This reverts commit a591ed4375d4ec98ee775886c20981631ca75c68.
sched: Change rq->nr_running to unsigned int

Since there's a PID space limit of 30bits (see
futex.h:FUTEX_TID_MASK) and allocating that many tasks (assuming a
lower bound of 2 pages per task) would still take 8T of memory it
seems reasonable to say that unsigned int is sufficient for
rq->nr_running.

When we do get anywhere near that amount of tasks I suspect other
things would go funny, load-balancer load computations would really
need to be hoisted to 128bit etc.

So save a few bytes and convert rq->nr_running and friends to
unsigned int.

Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-y3tvyszjdmbibade5bw8zl81@git.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
sched/nohz: Fix rq->cpu_load[] calculations

While investigating why the load-balancer did funny I found that the
rq->cpu_load[] tables were completely screwy.. a bit more digging
revealed that the updates that got through were missing ticks followed
by a catchup of 2 ticks.

The catchup assumes the cpu was idle during that time (since only nohz
can cause missed ticks and the machine is idle etc..) this means that
esp. the higher indices were significantly lower than they ought to
be.

The reason for this is that its not correct to compare against jiffies
on every jiffy on any other cpu than the cpu that updates jiffies.

This patch cludges around it by only doing the catch-up stuff from
nohz_idle_balance() and doing the regular stuff unconditionally from
the tick.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: pjt@google.com
Cc: Venkatesh Pallipadi <venki@google.com>
Link: http://lkml.kernel.org/n/tip-tp4kj18xdd5aj4vvj0qg55s2@git.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
sched/nohz: Fix rq->cpu_load calculations some more

Follow up on commit 556061b00 ("sched/nohz: Fix rq->cpu_load[]
calculations") since while that fixed the busy case it regressed the
mostly idle case.

Add a callback from the nohz exit to also age the rq->cpu_load[]
array. This closes the hole where either there was no nohz load
balance pass during the nohz, or there was a 'significant' amount of
idle time between the last nohz balance and the nohz exit.

So we'll update unconditionally from the tick to not insert any
accidental 0 load periods while busy, and we try and catch up from
nohz idle balance and nohz exit. Both these are still prone to missing
a jiffy, but that has always been the case.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: pjt@google.com
Cc: Venkatesh Pallipadi <venki@google.com>
Link: http://lkml.kernel.org/n/tip-kt0trz0apodbf84ucjfdbr1a@git.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
sched/nohz: Rewrite and fix load-avg computation -- again

Thanks to Charles Wang for spotting the defects in the current code:

 - If we go idle during the sample window -- after sampling, we get a
   negative bias because we can negate our own sample.

 - If we wake up during the sample window we get a positive bias
   because we push the sample to a known active period.

So rewrite the entire nohz load-avg muck once again, now adding
copious documentation to the code.

Reported-and-tested-by: Doug Smythies <dsmythies@telus.net>
Reported-and-tested-by: Charles Wang <muming.wq@gmail.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: stable@kernel.org
Link: http://lkml.kernel.org/r/1340373782.18025.74.camel@twins
[ minor edits ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>

Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
sched: Improve scalability via 'CPU buddies', which withstand random perturbations

Traversing an entire package is not only expensive, it also leads to tasks
bouncing all over a partially idle and possible quite large package.  Fix
that up by assigning a 'buddy' CPU to try to motivate.  Each buddy may try
to motivate that one other CPU, if it's busy, tough, it may then try its
SMT sibling, but that's all this optimization is allowed to cost.

Sibling cache buddies are cross-wired to prevent bouncing.

4 socket 40 core + SMT Westmere box, single 30 sec tbench runs, higher is better:

 clients     1       2       4        8       16       32       64      128
 ..........................................................................
 pre        30      41     118      645     3769     6214    12233    14312
 post      299     603    1211     2418     4697     6847    11606    14557

A nice increase in performance.

Signed-off-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1339471112.7352.32.camel@marge.simpson.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
sched: Fix race in task_group()

Stefan reported a crash on a kernel before a3e5d1091c1 ("sched:
Don't call task_group() too many times in set_task_rq()"), he
found the reason to be that the multiple task_group()
invocations in set_task_rq() returned different values.

Looking at all that I found a lack of serialization and plain
wrong comments.

The below tries to fix it using an extra pointer which is
updated under the appropriate scheduler locks. Its not pretty,
but I can't really see another way given how all the cgroup
stuff works.

Reported-and-tested-by: Stefan Bader <stefan.bader@canonical.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1340364965.18025.71.camel@twins
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
sched: Fix nohz_idle_balance()

On tickless systems, one CPU runs load balance for all idle CPUs.

The cpu_load of this CPU is updated before starting the load balance
of each other idle CPUs. We should instead update the cpu_load of
the balance_cpu.

Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Venkatesh Pallipadi <venki@google.com>
Cc: Suresh Siddha <suresh.b.siddha@intel.com>
Link: http://lkml.kernel.org/r/1347509486-8688-1-git-send-email-vincent.guittot@linaro.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
sched: Fix load avg vs cpu-hotplug

Rabik and Paul reported two different issues related to the same few
lines of code.

Rabik's issue is that the nr_uninterruptible migration code is wrong in
that he sees artifacts due to this (Rabik please do expand in more
detail).

Paul's issue is that this code as it stands relies on us using
stop_machine() for unplug, we all would like to remove this assumption
so that eventually we can remove this stop_machine() usage altogether.

The only reason we'd have to migrate nr_uninterruptible is so that we
could use for_each_online_cpu() loops in favour of
for_each_possible_cpu() loops, however since nr_uninterruptible() is the
only such loop and its using possible lets not bother at all.

The problem Rabik sees is (probably) caused by the fact that by
migrating nr_uninterruptible we screw rq->calc_load_active for both rqs
involved.

So don't bother with fancy migration schemes (meaning we now have to
keep using for_each_possible_cpu()) and instead fold any nr_active delta
after we migrate all tasks away to make sure we don't have any skewed
nr_active accounting.

[ paulmck: Move call to calc_load_migration to CPU_DEAD to avoid
miscounting noted by Rakib. ]

Reported-by: Rakib Mullick <rakib.mullick@gmail.com>
Reported-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: pm: Add support for retention on each core

Allow each core on 8974 to enter retention and retain
the state of the CPU. On 8960 as hardware restriction does
not permit all cores to enter retention at the same time,
allow only core 0 to enter retention when all other cores
are offline to maintain backward compatibility.

Change-Id: I6690e901afc9fa3a27010c264fa3acc9a0722c1a
Signed-off-by: Priyanka Mathur <pmathur@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: pm-8x60: Do not default to WFI when not enabled

Return MSM_PM_SLEEP_MODE_NOT_SELECTED in msm_pm_idle_prepare as the
default mode instead of WFI. Skip enter low power mode when this
mode is selected.

Change-Id: Id4a07ba1353b358726c81ed441e0327a9154de8a
Signed-off-by: Priyanka Mathur <pmathur@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: pm: Add API to enable/disable retention mode

On 8974 targets, retention mode voltage is controlled by the LDO
regulator, which in turn is powered through a ganged PMIC regulators. At
certain low voltage levels, the ganged PMIC will not be able to power
the LDO retention voltage. The regulator framework would invoke this
API to enable/disable retention mode in these scenarios.

When disabling retention modes, send a IPI to wakeup all online cores
out of idle, to ensure that no core is in a retention low power mode.
Also, disabled support for retention during suspend/hotplug operations
to ensure that a CPU isn't already in retention when the voltage is
lowered. This shouldn't have an effect on suspend, as the driver always
chooses the deepest sleep mode for suspend/hotplug.

Change-Id: I0af18168968c2aaf05ca99188d30762612c87de6
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: tracing: Add ftrace support for LPM driver

Add trace event support to capture various parameters
of low power modes like power collapse, standalone power
collapse, retention and wfi during its entry and exit.

Change-Id: I0f8b918702ae47b354aa193aa7ef20b48110fb0f
Signed-off-by: Priyanka Mathur <pmathur@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: tracing: Add ftrace support for LPM resources

Add trace event support to capture sleep value for
each of the lpm resources l2, pxo, vdd mem and vdd
dig during power collapse.

Change-Id: Ie0d7528ecd5a87631ff584953bfe0bd77043add9
Signed-off-by: Priyanka Mathur <pmathur@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
msm: pm-8x60: Merge msm_pm_idle_prepare into msm_pm_idle_enter

Modify LPM drivers to call msm_pm_idle_enter when idle. enter_sleep will
look at available time to sleep and the latencies of different low power
modes and decide the low power mode to enter. It returns the low power
mode enumeration back to cpuidle.

Change-Id: Icbdaa6604824ea7d10c27f2d932d47d354da425e
Signed-off-by: Priyanka Mathur <pmathur@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>