cpufreq: ondemand: change freq sync code to use per-CPU kthreads

The migration notifier may run in a context where it is not safe to
enqueue a work item. For example:

(__queue_work+0x1f4/0x40c)
(queue_work_on+0x34/0x44)
(dbs_migration_notify+0x2c/0x40)
(notifier_call_chain+0x38/0x68)
(__atomic_notifier_call_chain+0x34/0x44)
(atomic_notifier_call_chain+0x14/0x18)
(try_to_wake_up+0x350/0x36c)
(autoremove_wake_function+0xc/0x34)
(__wake_up_common+0x48/0x7c)
(__wake_up+0x3c/0x50)
(trustee_thread+0x1d0/0x528)
(kthread+0x80/0x8c)

The trustee code already takes a lock internal to the workqueue
implementation and is not expecting a wake up to come around and
enter the workqueue code again, where the same lock will be needed.
Instead of relying on a workqueue, use a per-CPU kthread to do
the frequency syncing.

Change-Id: I7555ee40867792fa8ec4ea8f9a6309323775e797
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
Signed-off-by: Ajay Dudani <adudani@codeaurora.org>
cpufreq: ondemand: Fix update_sampling_rate race with hotplug

update_sampling_rate has a for loop which goes through each
online cpu and possibly queue up the ondemand work for them.
But while doing this it doesnt take any hotplug lock which
could potentially cause a race condition where ondemand work
is queued after the hotplug code (which sets the policy to NULL)
in the governor has cancelled any pending work. This could cause
a crash while trying to access the NULL policy in dbs_check_cpu.

Protecting the for_each_online_cpu loop with get_online_cpus()
and put_online_cpus().

Change-Id: Ia3f43ca7e4bed542834ab03ca1191d728f13311c
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Revert "msm: cpufreq: Remove cross-calling limitation"

This reverts commit be01a17afb6b6e85456217caa1ac11eb0b6674e1.

Re-introduce cross calling in cpufreq to prevent race conditions in the
hardware when changing PLLs.

A core could be in retention, while another core, could try and set the
frequency of the core in retention, causing a change in PLL and when the
core exits retention, the PLL mismatch could hang the SPM state machine.

Change-Id: I99b454a0f146652f2d8a369f79f6d35748160dcb
Signed-off-by: Praveen Chidambaram <pchidamb@codeaurora.org>
msm: cpufreq: Configure WQ for higer priority

When the workqueue runs at a lower priority, it gets starved by higher
priority threads. Bump the WQ priority to high to ensure cpufreq work
queue gets a fair scheduling chance

Change-Id: I9e994da94a347dceb884e72ec3dd3da468a4471d
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
msm: cpufreq: Relax constraints on "msm-cpufreq" workqueue

This workqueue is not used in memory reclaim paths, so the
WQ_MEM_RECLAIM flag is not needed. Additionally, there is no
need to restrict the queue to having one in-flight work item.
Remove these constraints.

Change-Id: I9edde40917d3ec885ce061133de20680634321d0
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
msm: cpufreq: Ensure cpufreq change happens on corresponding CPU

Checking the cpus_allowed mask of the current thread before changing the
frequency doesn't guarantee that the rest of the execution will continue on
the same CPU. The only way to guarantee this is to schedule a work on the
specific CPU and also prevent hotplug  of that CPU (already done by
existing code).

Change-Id: I51a02fcc777a47d3c16f2d83c47e96f2c59f7ae6
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
msm: cpufreq: Add support for CPU clocks and msm-cpufreq device

Add support for using clock APIs to do CPU/L2 frequency scaling. When clock
APIs are used to control CPU/L2 frequencies, the cpufreq driver must handle
the CPU -> L2 -> DDR bandwidth voting. Use a msm-cpufreq device to provide
the per SoC list of CPU frequencies and their L2/DDR bandwidth mapping
information.

If CPU/L2 clocks are not available, fallback to using acpuclock APIs. We
eventually want to remove the use of non-standard acpuclock APIs.

Change-Id: I2c4e2c3967d73e8cdbd9833f3cb36f3d75e27b4a
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

Conflicts:
	arch/arm/mach-msm/cpufreq.c
msm: cpufreq: Update frequency index

A deprecated api can be used to limit the minimum and maximum cpu speeds.
Another driver temporarily needs to switch back to this api, instead of
using the new version. Make this api functional again by ensuring a stale
value is updated properly.

Change-Id: Ia63772e43ec534cf2bb58e7627a87315dae5d891
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
sched: remove migration notification from RT class

Commit 88a7e37d265 (sched: provide per cpu-cgroup option to
notify on migrations) added a notifier call when a task is moved
to a different CPU. Unfortunately the two call sites in the RT
sched class where this occurs happens with a runqueue lock held.
This can result in a deadlock if the notifier call attempts to do
something like wake up a task.

Fortunately the benefit of 88a7e37d265 comes mainly from notifying
on migration of non-RT tasks, so we can simply ignore the movements
of RT tasks.

CRs-Fixed: 491370
Change-Id: I8849d826bf1eeaf85a6f6ad872acb475247c5926
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
Signed-off-by: Mekala Natarajan <mekalan@codeaurora.org>
lib: spinlock_debug: Avoid livelock in do_raw_spin_lock

The logic in do_raw_spin_lock attempts to acquire a spinlock
by invoking arch_spin_trylock in a loop with a delay between
each attempt. Now consider the following situation in a 2
CPU system:

1. Cpu 0 continually acquires and releases a spinlock in a
   tight loop; it stays in this loop until some condition X
   is satisfied. X can only be satisfied by another CPU.

2. Cpu 1 tries to acquire the same spinlock, in an attempt
   to satisfy the aforementioned condition X. However, it
   never sees the unlocked value of the lock because the
   debug spinlock code uses trylock instead of just lock;
   it checks at all the wrong moments - whenever Cpu 0 has
   locked the lock.

Now in the absence of debug spinlocks, the architecture specific
spinlock code can correctly allow Cpu1 to wait in a "queue"
(e.g., ticket spinlocks), ensuring that it acquires the lock at
some point. However, with the debug spinlock code, livelock
can easily occur due to the use of try_lock, which obviously
cannot put the CPU in that "queue".

Address this by actually attempting arch_spin_lock once it is
suspected that there is a spinlock lockup. If we're in a
situation that is described above, the arch_spin_lock should
succeed; otherwise other timeout mechanisms (e.g., watchdog)
should alert the system of a lockup.

Additionally, the delay between each trylock attempt adds up
to a total delay of 1 second. On some embedded architectures,
this is an unacceptably large value. Reduce this to a ~62 ms
delay.

CRs-Fixed: 393351
Change-Id: I56f0d8f5e483cd7e8167fc7a363b9de7777a0830
Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
lib: spinlock_debug: increase spin dump timeout

On certain targets the worst case console buffer draining time can
exceed the aggressive spin dump timeout value. Relax the timeout value
to cater to these scenarios.

Change-Id: I7312ffe06bafd2dc7adf0f4e8b73a53fc5839235
Signed-off-by: Syed Rameez Mustafa <rameezmustafa@codeaurora.org>
msm: acpuclock-krait: Enable HFPLL for init only if switching to it

The existing code left the HFPLL enabled in hfpll_init(), even if
the CPU was detected to be at a non-HFPLL rate. Correct this for
power savings between when hfpll_init() and the first runtime CPU
frequency switch happen. This also ensure votes for HFPLL
regulators are not left unnecessarily asserted.

Change-Id: Iaca5dc7e4769bdbd494d669726ba9b500256f793
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
trace: power: Adding trace events for cpufreq

Added two events to help debugging cpufreq:
1) cpufreq_sampling_event: To capture governor sampling instants
   when cpu load is evaluated and an appropriate frequency is
   selected.
2) cpufreq_freq_synced: to capture frequency synchronization of
   destination cpu with the source cpu triggered by foreground
   thread migrations.

Change-Id: I75bc7c3104b6e0750d9ede95602098d335f4f533
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
tracing/sched: Add trace events to track cpu hotplug.

Add ftrace event trace_sched_cpu_hotplug to track cpu
hot-add and hot-remove events.

This is useful in a variety of power, performance and
debug analysis scenarios.

Change-Id: I5d202c7a229ffacc3aafb7cf9afee0b0ee7b0931
Signed-off-by: Arun Bharadwaj <abharadw@codeaurora.org>

Conflicts:
	kernel/cpu.c
arm: Remove no-longer-required RCU_NONIDLE wrapper

Commit 21111be8 "ARM: Fix negative idle stats for offline cpu" moved call
to cpu_die() to occur outside of rcu_idle_enter()/rcu_idle_exit() section.
As a result, the RCU_NONIDLE() wrapper to complete() call in
arch/arm/kernel/process.c:cpu_die() is no longer required (and is
technically incorrect to have). Removing RCU_NONIDLE() wrapper also removes
this warning seen during CPU offline:

[Note: Below message has been edited to fit 75-char per line limit.
Insignificant portions of warning message has been removed in each line.]

------------[ cut here ]------------
WARNING: at kernel/rcutree.c:456 rcu_idle_exit_common+0x4c/0xe0()
Modules linked in:
(unwind_backtrace+0x0/0x120) from (warn_slowpath_common+0x4c/0x64)
(warn_slowpath_common+0x4c/0x64) from (warn_slowpath_null+0x18/0x1c)
(warn_slowpath_null+0x18/0x1c) from (rcu_idle_exit_common+0x4c/0xe0)
(rcu_idle_exit_common+0x4c/0xe0) from (rcu_idle_exit+0xa8/0xc0)
(rcu_idle_exit+0xa8/0xc0) from (cpu_die+0x24/0x5c)
(cpu_die+0x24/0x5c) from (cpu_idle+0xdc/0xf0)
(cpu_idle+0xdc/0xf0) from (0x8160)
---[ end trace 61bf21937a496a37 ]---

Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
Change-Id: I721e6b20651674e6f6f584cf8d814af00b688c91
Revert "msm: move printk out of spin lock low_water_lock"

This reverts commit 749c4210f338496abf2e16ef51308ff756b34f7b.

This patch is a hack intended to solve a particular instance of a more
generic problem. The cause of the spindump timeout has been fixed with
the commit f37a3dc536e4ecd72961a2d9c77f5587c5c55d8f. Furthermore this
change also introduces a race condition since the printk is actually
intended to be protected with the spinlock.

Change-Id: I8ca057f47760c7bb5b826cceeac92ae5e124e904
Signed-off-by: Syed Rameez Mustafa <rameezmustafa@codeaurora.org>
PM / QoS: Fix deadlock during PM QoS vote to default value

pm_qos_work_fn() calls pm_qos_update_request() which does
cancel_delayed_work_sync() on the same pm_qos_work_fn()
causing deadlock. Fix this by updating the target with
default timeout value directly instead of calling
pm_qos_update_request().

CRs-Fixed: 526216
Change-Id: I7de2fb1c89f87b0ebf7427116b8920aec50d5b2b
Signed-off-by: Sujit Reddy Thumma <sthumma@codeaurora.org>
cpufreq: fix msm_cpu_early_suspend_handler define but not used