cpufreq: Retain only online cpus in managed_policy->cpus

cpufreq's handling of cpu online event can incorrectly initialize
managed_policy->cpus to include offline cpus. This causes cpufreq
governors to include offline cpus in their idle time computation, which
causes frequency to shoot up beyond normal levels (on systems where 2 or
more cpus need to be synchronized wrt frequency change).

This frequency shootup was not seen prior to commit 21111be8 which
changed idle time accounting for offline cpus. Prior to commit 21111be8,
idle time statistics for offline cpus would include their offline time
as well. Post commit 21111be8, offline time is not included in a cpu's
idle time statistics.

As a result, ondemand (or other cpufreq) governors would see unchanged
idle time for offline cpus, causing them to interpret those (offline)
cpus as 100% busy, which is wrong.

This patch fixes the issue by modifying cpufreq's hotplug handling to
retain only online cpus in managed_policy->cpus field. cpufreq governors
would then skip such offline cpus from idle time calculations.

Change-Id: Id2eba8c0ec3367609369b2e42a4e1386ff2e7ab5
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
cpufreq: Save user policy min/max instead of policy min/max during hotplug

When a CPU is hotplugged off, various fields of the policy are saved so
that they can be restored when the CPU is hotplugged back in.

The existing code saves the policy min/max field during hotplug remove and
restores it into user policy min/max during hotplug add. This results in
the loss of the user policy min/max values across a hotplug remove/add.

Fix it by saving the user policy min/max instead of policy min/max during
hotplug remove. During a hotplug add, after user policy min/max is
restored, the policy min/max is already recomputed. So, there's no risk of
going beyound any limits imposed by the CPUFREQ_ADJUST/INCOMPATIBLE
notifiers.

Change-Id: Ib8e09fa324c1f80f095c5754b5dcf2685e8e4a66
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Signed-off-by: Paul Reioux <reioux@gmail.com>
msm: cpufreq: Only apply driver limits for scaling_min/max_freq writes

When new values are written to scaling_min/max_freq sysfs files, the
current code applies all the limits imposed by various ADJUST and
INCOMPATIBLE notifiers handlers before storing them.

When the ADJUST and/or INCOMPATIBLE notifiers change their limits
frequently, this behavior makes it almost impossible for a
user/userspace process to store the scaling_min/max_freq limits
that go beyond the instantaneous limits imposed by the notifiers.
Since these sysfs nodes are typically meant to set limits that need
to be enforced for the foreseeable future, this is not a very user
friendly behavior.

So, change the behavior to only apply limits that are enforced by
the cpufreq driver. Typically, these are just the absolute limits
of the HW and don't change very often.

Change-Id: I1ccfaa2d1ee4ea595f882485d359dbdb407a0176
Signed-off-by: Deepak Katragadda <dkatraga@codeaurora.org>
cpufreq: Fix policy getting stuck when user & kernel min/max don't overlap

Every __cpufreq_set_policy starts with checking the new policy min/max has
some overlap with the current policy min/max. This works out fine until we
end up with the policy min/max being set to a range that doesn't overlap
with the user policy min/max. Once we get into this situation, the check at
the start of __cpufreq_set_policy fails and prevents us from getting out of
this state.

This only happens when one of the CPUFREQ_ADJUST/CPUFREQ_INCOMPATIBLE
notifiers called inside __cpufreq_set_policy pick a min/max outside the
range of user policy min/max.

The real intent of the check at the start of __cpufreq_set_policy is to
make sure userspace can't set user policy min > user policy max. Since
__cpufreq_set_policy always gets called only with current user policy
min/max except when the actual user space policy min/max is changed, we can
fix the issue by simply checking the new policy min/max against current
user policy min/max.

Change-Id: Iaac805825e64d7985c41fb9052bd96baacdf3d6f
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
cpufreq: update the input even handler to handle sync cores correctly.

Input even handler is scheduling work items to update frequency on all
online cores. This is not required in case of sync cores. Hence this
handler is updated to change frequency only on one core per a group
of synchronous CPUs.

CRs-Fixed: 389200
Change-Id: I9c0af9c05c06517ed44f77253e2ac365b4894b5b
Signed-off-by: Krishna Vanka <kvanka@codeaurora.org>
Revert "cpufreq: update the input even handler to handle sync cores correctly."

This reverts commit 749884de4f5ce5788658260b46d15373a1d3b547.

Hotplug stability issues have been observed with the original change
when booting will maxcpus set to be less than the total number
of available CPUs. Revert the change until the issue is resolved.

Change-Id: Id9a26693e23bff83dce638918525411ca02dfc3e
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
cpufreq: Avoid using smp_processor_id() in preemptible context

Even though this work item runs on only one cpu at a time (due to
queue_work_on()) it is possible for the work item to be preempted
and so use of smp_processor_id() is illegal.

BUG: using smp_processor_id() in preemptible [00000000]
code: kworker/3:1/4162 caller is dbs_refresh_callback+0xc/0x188
[<c00151b0>] (unwind_backtrace+0x0/0x120) from [<c0279058>]
(debug_smp_processor_id+0xbc/0xf0)
[<c0279058>] (debug_smp_processor_id+0xbc/0xf0) from [<c0454b54>]
(dbs_refresh_callback+0xc/0x188)
[<c0454b54>] (dbs_refresh_callback+0xc/0x188) from [<c0087290>]
(process_one_work+0x354/0x648)
[<c0087290>] (process_one_work+0x354/0x648) from [<c0089754>]
(worker_thread+0x1a8/0x2a8)
[<c0089754>] (worker_thread+0x1a8/0x2a8) from [<c008e480>]
(kthread+0x90/0xa0)
[<c008e480>] (kthread+0x90/0xa0) from [<c000f438>]
(kernel_thread_exit+0x0/0x8)

The intent of the code is to determine which CPU this work item
is running on, which we can easily do by passing that information
in a wrapper struct around the work struct. Do this so we avoid
this problem.

Change-Id: I05ca0ff2b3cbaa239930463ea0760e3e9d75145f
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
ondemand: Add support to consider other cpu load when scaling frequencies

Currently only current cpu load is considered when frequency scaling.
As a side effect of this when scheduler migrates a thread to a slower
core, the execution time increases unexpectedly.

The new algorithm not only considers current cpu load for frequency
scaling, but also takes into account load of other online cpus. The
treshold for this can be controlled using up_threshold_any_cpu_load.
When the up_threshold_any_cpu_load is crossed in any cpu then the
current cpu is set to sync_freq.

Also, when more than one cpu is online a two step ramp-up mechanism
is used. When the up_threshold_multi_core is crossed the cpu is ramped
up to optimal_freq which is less than policy->max and set to a power
friendly value.

Change-Id: I64bed1b142cde1f239e73aa1f78981c78aaf59ce
Signed-off-by: Narayanan Gopalakrishnan <nargop@codeaurora.org>
cpufreq: ondemand: Fix kernel warning in cpufreq_governor_dbs

GOV_STOP event destroys the timer_mutex and is not initialized back
upon GOV_START event.
The Kernel complains at mutex_lock(during GOV_LIMITS event) after
mutex_destroy.

[<c0014918>] (unwind_backtrace+0x0/0x11c)
[<c007c004>] (warn_slowpath_common+0x4c/0x64)
[<c007c034>] (warn_slowpath_null+0x18/0x1c)
[<c079eef8>] (__mutex_lock_slowpath+0x130/0x3c4)
[<c079f1ac>] (mutex_lock+0x20/0x3c) from [<c0540900>]
[<c0540900>] (cpufreq_governor_dbs+0x3c8/0x468)
[<c053d500>] (__cpufreq_governor+0x90/0xe0)
[<c053d774>] (__cpufreq_set_policy+0x224/0x258)
[<c053da58>] (cpufreq_add_dev_interface+0x2b0/0x308)
[<c053e784>] (cpufreq_add_dev+0x500/0x56c)
[<c079247c>] (cpufreq_cpu_callback+0x88/0x9c)
[<c07a2ca8>] (notifier_call_chain+0x38/0x68)
[<c007ded0>] (__cpu_notify+0x28/0x40)
[<c0790024>] (_cpu_up+0xe4/0x118)
[<c0790024>] (cpu_up+0x64/0x80)
[<c0765840>] (store_online+0x48/0x78)
[<c038d0c8>] (dev_attr_store+0x18/0x24)
[<c017be40>] (sysfs_write_file+0x108/0x13c)
[<c012cdfc>] (vfs_write+0xb0/0x128)
[<c012d034>](sys_write+0x38/0x64)

Change-Id: Ie2a796b17209243de731ccf2ab04b7f9e7048df8
Signed-off-by: Anji Jonnala <anjir@codeaurora.org>
msm: cpufreq: Implement suspend/resume for cpufreq driver

Cpufreq suspend variable get's updated in kernel pm notifier
callback which gets called in process context. This will create
window where cpufreq governor may try to change the frequency
before suspend variable is updated while resume.

Define suspend/resume for cpufreq driver to update suspend variable
in atomic context.

Don't update the policy->cur upon cpufreq driver failure.

CRs-fixed: 423791
Change-Id: Id78705f5c3b8bf801c6253e9362299280903769f
Signed-off-by: Anji Jonnala <anjir@codeaurora.org>

Conflicts:
	arch/arm/mach-msm/cpufreq.c
msm: cpufreq: make the gov stop part fully atomic

In the function cpufreq_governor_dbs, the case CPUFREQ_GOV_STOP
is not completely atomic. The removal of sysfs is outside the
critical path. If a START/STOP done frequently from user space
then there is chance of adding the sysfs interface before removal.
This creates a warning from sysfs saying creating an already
existing entry.

Also once it fails to create, cpufreq_governor_dbs returns with
error code. The dbs_enable value is incremented which is not
rolled back and cpufreq core tries to set the old
governor. This might create further problems.

Hence the removal of sysfs entry should happen in the critical
path.

CRs-fixed: 440969
Change-Id: I98891756caf2ce0b85e39e9b4f58034af1ed97ce
Signed-off-by: Venkat Devarasetty <vdevaras@codeaurora.org>
cpufreq: Fix optimal_freq logic on ramp down

When ondemand governor decides to ramp down the frequency, the missing
check for frequency being lower than optimal_freq was causing frequency
to be ramped down to optimal_freq even if the frequency decided by ondemand
governor was higher.

Change-Id: I00f9ae4c61ccda4039f3338dee2c29b8cb790be4
Signed-off-by: Veena Sambasivan <veenas@codeaurora.org>
msm: Synchronize CPU frequency on thread migration

Sometimes when foreground threads, crucial for UI, migrate
from a core at higher frequency to the one running at a lower
frequency, it could badly affect performance for cases like
scrolling.
This change tries to raise the frequency of the destination core
to the same value as the source core, if it is not already running
at the same or higher frequency.  In order for the ondemand
governor to have a fair idea of load in the next sample, it also
reschedules the next sample one sampling interval from the
frequency transition after which the ondemand governor again
takes over.

Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Change-Id: I9d2fbb166b0cc042d3fd57cea75a5066df4c794b
cpufreq: Use dedicated high-priority workqueues

In the process of scaling CPU frequencies, cpufreq and the APIS is
calls may hold resources that will prevent threads critical for
system stability from running.

Specifically, thermal-mitigation software may need to respond
quickly to high-temperature conditions and throttle the CPU speed
in an effort to cool down. If the system is already in the middle
of a CPU frequency switch being executed on behalf of a cpufreq
governor, thermal mitigation will be blocked until the scaling
action has completed. If the system is under heave load of medium
or high-priority threads, this may take a long while and the system
may potentially surpass thermal limits in the meantime.

Resolve this by using high priority workqueues for all ondemand
and conservative governor scaling operations.  This will allow them
to complete promptly and release their hold on resources necessary
for maintaining system stability.

Change-Id: I2f56052c131442838036cf4cdd8059f7c09bb805
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Signed-off-by: Ajay Dudani <adudani@codeaurora.org>
cpufreq: ondemand: change freq sync code to use per-CPU kthreads

The migration notifier may run in a context where it is not safe to
enqueue a work item. For example:

(__queue_work+0x1f4/0x40c)
(queue_work_on+0x34/0x44)
(dbs_migration_notify+0x2c/0x40)
(notifier_call_chain+0x38/0x68)
(__atomic_notifier_call_chain+0x34/0x44)
(atomic_notifier_call_chain+0x14/0x18)
(try_to_wake_up+0x350/0x36c)
(autoremove_wake_function+0xc/0x34)
(__wake_up_common+0x48/0x7c)
(__wake_up+0x3c/0x50)
(trustee_thread+0x1d0/0x528)
(kthread+0x80/0x8c)

The trustee code already takes a lock internal to the workqueue
implementation and is not expecting a wake up to come around and
enter the workqueue code again, where the same lock will be needed.

Instead of relying on a workqueue, use a per-CPU kthread to do
the frequency syncing.

Change-Id: I7555ee40867792fa8ec4ea8f9a6309323775e797
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
Signed-off-by: Ajay Dudani <adudani@codeaurora.org>
cpufreq: ondemand: Fix store_powersave_bias() race with hotplug

store_powersave_bias() assumes that CPUs will not be taken offline or
brought online while executing its for_each_online_cpu() loops. Fix
this race by blocking hotplug while executing in these loop.

CRs-Fixed: 494407
Change-Id: I0d5520444ec21a8d9b12fb73c06ac8442449fa85
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
cpufreq: ondemand: Fix locking in store_powersave_bias

store_powersave_bias() assumes that dbs_info->cur_policy will
not change while it is executing. If it does, memory corruption
can potentially result. Fix this by acquiring the dbs_lock.

Most noticeably, this resolves timer-related list corruptions
encountered when queue_delayed_work_on() is called from within
store_powersave_bias() while a CPU is being hotplugged in and
out.

CRs-Fixed: 494407
Change-Id: I0fd5cbef6834545274cad9481e574ec50b37b871
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
cpufreq: ondemand: Fix locking issue in store_powersave_bias

store_powersave_bias takes timer_mutex before calling dbs_timer_exit
which tries to cancel the delayed work do_dbs_timer which in turn
tries to take the same lock. This can cause a lock recursion under
the race condition where can cancel_delayed_work_sync is called
when the work has already started executing.

This can be avoided by taking that lock after calling
dbs_timer_exit.

Change-Id: I7f862286e66f1ddc1e13e4eeee369dd188fc10d5
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
cpufreq: ondemand: Boost CPU frequency only for touch input

Originally CPU frequency were boosted for any input event.
In some case, sensor sends a lot of input event, which
keep CPU in high frequency.  CPU frequency only need to
be boosted when real user interaction happens.

Change-Id: Ia3ad755b98d8363a17729926610b5dd6f0075288
CR-Fixed: 507519
Signed-off-by: Tingwei Zhang <tingwei@codeaurora.org>
cpufreq: ondemand: Fix update_sampling_rate race with hotplug

update_sampling_rate has a for loop which goes through each
online cpu and possibly queue up the ondemand work for them.
But while doing this it doesnt take any hotplug lock which
could potentially cause a race condition where ondemand work
is queued after the hotplug code (which sets the policy to NULL)
in the governor has cancelled any pending work. This could cause
a crash while trying to access the NULL policy in dbs_check_cpu.

Protecting the for_each_online_cpu loop with get_online_cpus()
and put_online_cpus().

Change-Id: Ia3f43ca7e4bed542834ab03ca1191d728f13311c
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>