cpufreq: interactive: Modifying sync_freq implementation

1. Check for up_threshold_any_cpu_freq instead of sync_freq to
   boost the frequency to sync_freq.
2. Change the load threshold name from sync_freq_load_threshold to
   up_threshold_any_cpu_load
3. Do not consider CPUs with load less than up_threshold_any_cpu_load
   while evaluating max load across CPUs

Change-Id: Ia0e537edbf38a5006c1a22f5c472daa0d086ffc9
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
cpufreq: interactive: Use default min_sample_time if SDF is zero

Use min_sample_time to step down from max frequencies if
sampling_down_factor is set to zero.

Change-Id: I2e89bef6220557cb95efd20d658e0c05753b4c3c
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
cpufreq: interactive: Fix null pointer dereference in interactive governor

For the sync_freq feature currently we check pcpu->policy->cur frequency
for each online cpu.  But for a CPU that isn't using interactive governor
or for an offline CPU, pcpu->policy can be null or an invalid value.
This patch tries to avoid that scenario by using pcpu->target_freq
instead of policy->cur to get the frequency of an online CPU.

Kernel crash without this patch:
[   20.132373] Unable to handle kernel NULL pointer dereference at virtual address 00000028
[   20.132375] pgd = c34f34c0
[   20.132377] pgd = ef6f2440
[   20.132383] [00000028] *pgd=00000000
[   20.132385]
[   20.132388] [00000028] *pgd=2e98f003, *pmd=00000000
[   20.132390] Internal error: Oops: 205 [#1] PREEMPT SMP ARM
[   20.132394] Modules linked in:
[   20.132398] CPU: 0 PID: 1560 Comm: chown Tainted: G        W    3.10.0-perf-gb12057b-00001-ga2c6c16-dirty #7
[   20.132401] task: ef9af300 ti: ee49c000 task.ti: ee49c000
[   20.132411] PC is at cpufreq_interactive_timer+0x10c/0x650
[   20.132415] LR is at cpufreq_interactive_timer+0x128/0x650
<snip>
[   20.133002] [<c07eb204>] (cpufreq_interactive_timer+0x10c/0x650) from [<c02804d8>] (call_timer_fn+0x80/0x198)
[   20.133012] [<c02804d8>] (call_timer_fn+0x80/0x198) from [<c0280acc>] (run_timer_softirq+0x1f8/0x270)
[   20.133019] [<c0280acc>] (run_timer_softirq+0x1f8/0x270) from [<c0279e20>] (__do_softirq+0x12c/0x2d4)
[   20.133025] [<c0279e20>] (__do_softirq+0x12c/0x2d4) from [<c027a2d4>] (irq_exit+0x74/0xc8)
[   20.133034] [<c027a2d4>] (irq_exit+0x74/0xc8) from [<c0206a00>] (handle_IRQ+0x68/0x8c)
[   20.133041] [<c0206a00>] (handle_IRQ+0x68/0x8c) from [<c02004b8>] (gic_handle_irq+0x3c/0x60)
[   20.133051] [<c02004b8>] (gic_handle_irq+0x3c/0x60) from [<c0ac6900>] (__irq_svc+0x40/0x70)
<snip>

Change-Id: Ie834f5d383de4d41e0fe6fbd40c8b0a0c05d82f5
Signed-off-by: Vijay Ganti <viganti@codeaurora.org>
cpufreq: interactive: delete timers for GOV_START

Make sure that timers cpu_timer and cpu_slack_timer
deactivated before addition of new.

Change-Id: If31c4049606871df6f00efdc24b1d713c86a6f69
Signed-off-by: Shridhar Rasal <srasal@nvidia.com>
Signed-off-by: Bharat Nihalani <bnihalani@nvidia.com>
Git-commit: b430123367e78a6557bac3cf1558bcb85193fb12
Git-repo: https://android.googlesource.com/kernel/common/
[mattw@codeaurora.org: resolved trivial context conflict]
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
cpufreq: Sync on thread migration optimizations

When threads migrate between CPUs the destination CPU will temporarily
have a boosted frequency and the source CPU will have a residual higher
frequency even though the load could have just been reduced. To avoid
leaving CPUs at a high frequencies notify the source CPU after a thread
migration and make the interactive governor's timer reschedule
conditionally.

Change-Id: Id55055f2a116ca7ffd2a34dc3d0ec519c888a111
Signed-off-by: Patrick Cain <pcain@codeaurora.org>
cpufreq: interactive: Don't reset sampling down factor at max freq

A previous patch sets the min_sample_time to zero if the cpufreq
min/max limits have changed which would bring down the frequency
in the next sample even if min_sample_time doesnt allow it to do so.
This could severly affect benchmarks if this is applied to max
frequencies as min_sample_time for max frequencies is set to
sampling_down_factor.
This patch resets the limits_changed flag for cases where timer fires
while running at max frequencies so that sampling_down_factor is
still applicable as before.

Change-Id: I397133cb15eea4ab999d0f69617b4922438c6e11
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
cpufreq: interactive: Remove unnecessary cpu_online() check

Cpufreq no longer calls governor callback for offlined cpus. i.e. All
policy->cpus are guaranteed to be online. Hence we don't need explicit check to
see if cpu is online or not.

Change-Id: I9ad85ea4addd5b4a40952e59ed730dd15e328690
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
cpufreq: interactive: Move definition of cpufreq_gov_interactive downwards

This moves definition of cpufreq_gov_interactive towards the bottom of file, so
that we don't have to add prototype of cpufreq_governor_interactive() in the
beginning of file.

Change-Id: I04bd1004954eb36502c5cd7e35d3d7274cddaf95
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Git-commit: e50d640cacfe61e69f1b4fa92fa0174c688e919c
Git-Repo: https://android.googlesource.com/kernel/common.git
[junjiew@codeaurora.org: Resolve merge conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
cpufreq: Move get_cpu_idle_time() to cpufreq.c

Governors other than ondemand and conservative can also use
get_cpu_idle_time() and they aren't required to compile
cpufreq_governor.c. So, move these independent routines to
cpufreq.c instead.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Git-commit: 72a4ce340a7ebf39e1c6fdc8f5feb4f974d6c635
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[junjiew@codeaurora.org: update non-upstream files]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

Change-Id: I3227454599250011a19a752c962e3cacb1fb0301
cpufreq: interactive: Squashed cleanup patches

Revert "cpufreq: interactive: Remove unnecessary cpu_online() check"

This reverts commit 7b46e2e6d8a3c3c32706ee3c3c3b507d050bedfb.

Revert "cpufreq: interactive: Allow 1 ms error in above_hispeed_delay comparisons"

This reverts commit ff6af80775231f1dfab8353d0b665718222218bf.

Commit ff6af807 tries to avoid a corner case where frequency is stuck in
hispeed_freq for one additional window. For example, if timer_rate is
20ms, and go_hispeed_delay is 40ms, frequency might be stuck at
hispeed_freq for 60ms due to imprecision in jiffies. Same problem can be
easily solved by making go_hispeed_delay 1ms smaller instead of changing
the code.

Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

cpufreq: interactive: Revert timer start modification

commit f8b276565ce95bd788af0fc8d060f92632590a40
(cpufreq: Sync on thread migration optimizations)
introduced a change to cpufreq_interactive_timer_start() in order
to reschedule the timer differently based on whether min or max
is changed. A better way is to reschedule the timer only when
necessary.

Revert timer start modification in preparation for the final fix.

Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

Revert "cpufreq: interactive: Optimize sync_freq implementation"

This reverts commit 4da7f27d4dbd5bcfb8479467444f2cf198217933.

cpufreq: interactive: Revert sync on thread migration optimizations

Commit f8b276565ce95bd788af0fc8d060f92632590a40
(cpufreq: Sync on thread migration optimizations) is no longer needed
for targets with synchronous CPUs.

Part of that commit has already been reverted in
a913b3afcaa55356f5f793ae895312c8ae7e1f71
(cpufreq: interactive: Revert timer start modification)

This commit reverts the remaining changes.

Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

cpufreq: interactive: Revert sync freq feature

Sync freq feature is not valid for a HMP system with clusters.

This commit reverts commit f3d1980b4db4c20f21acaed59cc511c2df84cec2
(cpufreq: interactive: sync freq feature for interactive governor)

Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

cpufreq: interactive: Remove sampling_down_factor

Remove sampling_down_factor feature.

This commit revert d094d23694edf41fee50ebc544795d6116451643
(cpufreq: interactive: Add a sampling_down_factor for max frequencies)
and subsequent modifications related to sampling down factor.

Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

Conflicts:
	drivers/cpufreq/cpufreq_interactive.c

Change-Id: I9e5fd451e7eaefe8ef7775365cc5a4527d37188f
cpufreq: interactive: Align timer windows for all CPUs

It's more advantageous to evaluate all CPUs at same time so that
interactive governor gets a complete picture of the load on
each CPU at a specific time. It could also reduce number of speed
changes made if there are many CPUs controlled by same policy. In
addition, waking up all CPUs at same time would allow the cluster
to go into a deeper sleep state when it's idle.

Change-Id: I6915050c5339ef1af106eb906ebe4b7c618061e2
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
cpufreq: interactive: Reset last evaluated jiffy in idle callback

When CPU has been busy for a long time, last evaluated jiffy will be
quite behind because the timer would have been canceled. We don't want
to schedule a timer to fire in the past as load will always be 100%.

Reset last evaluated jiffy so that timer will be scheduled for the
next window.

Change-Id: Ie25e65eab1f16acdeda267987ca605d653f1f32a
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
cpufreq: interactive: Round up timer_rate to match jiffy

Timers are scheduled in unit of jiffies. Round up timer_rate so that
it matches the actual sampling period.

Change-Id: I47e666f835752528331f50b1e76784e6d67f8bcf
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
cpufreq: interactive: Use del_timer/add_timer_on to rearm timers

Replace mod_timer_pinned() with del_timer(), add_timer_on().
mod_timer_pinned() always adds timer onto current CPU. Interactive
governor expects each CPU's timers to be running on the same CPU.
If cpufreq_interactive_timer_resched() is called from another CPU,
the timer will be armed on the wrong CPU.

Replacing mod_timer_pinned() with del_timer() and add_timer_on()
guarantees timers are still run on the right CPU even if another
CPU reschedules the timer. This would provide more flexibility
for future changes.

Change-Id: I3a10be37632afc0ea4e0cc9c86323b9783b216b1
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
cpufreq: cpu-boost: Fix queue_delayed_work_on() race with hotplug

Calling queue_delayed_work_on() on a CPU that's in the process of getting
hotplugged out can result in that CPU infinitely looping in
msm_pm_wait_cpu_shutdown(). If queue_delayed_work_on() is called after the
CPU is hotplugged out, it could wake up the CPU without going through the
hotplug path and cause instability. To avoid this, make sure the CPU is and
stays online while queuing a work on it.

Change-Id: I1b4aae3db803e476b1a7676d08f495c1f38bb154
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
cpufreq: cpu-boost: Fix deadlock in wake_up of sync threads

If wake_up() is called on the current task on a CPU, the call will wait
until the current task is switched out before it wakes it up again and
returns.

The sync notifier for a CPU always runs on that CPU.

These two together can result in a deadlock if the sync notifier on CPU A
tries to wake up the sync thread of CPU A as it goes to sleep (is the
current task). A previous commit fixed this by adding a check to the sync
notifier to not wake up the sync thread of CPU A if it's the current task.

But this is still not sufficient to prevent deadlocks.

Sync thread of CPU A could be the current task on CPU B and sync thread of
CPU B could be the current task on CPU A.  At this point, if sync notifier
of CPU A and B try to wake up the sync threads of CPU A and B, it will
result in CPU A waiting for the current task in CPU B to get switched out
and CPU B waiting for the current task in CPU A to get switched out.  This
will result in a deadlock.

Prevent this scenario from happening by pinning the sync threads of each
CPU to run on that CPU. By doing this, we guarantee that sync notifiers
will only try to wake up sync threads running on that CPU. The fix added by
"cpufreq: cpu-boost: Resolve deadlock when waking up sync thread" ensures a
deadlock doesn't happen when a sync notifier tries to wake up a sync thread
running on that CPU.

Change-Id: I864e545529722a23886dd5a82f66089155d2d193
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
cpufreq: cpu-boost: Don't register for cpufreq notifiers too early

The cpufreq notifiers should be registered only after all the data
structures used in the notifier callbacks have been initialized. So, move
the cpufreq notifier registration to a later point in the init function.

Change-Id: I043ab5bc0ebb98164c40549fe151a8d801c8c186
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
cpufreq: cpu-boost: Resolve deadlock when waking up sync thread

CPU boost driver receives notification from scheduler when threads
migrate towards a cpu and in turn wakes up a sync thread associated
with that cpu to handle any required frequency transitions. The wakeup
call however can lead to a deadlock inside scheduler under some
circumstance. The deadlock is seen when sync thread is the only thread
running on a cpu and goes to sleep (say by calling wait_event() ->
schedule()). Midway through this sleep (schedule()) call, while cpu is
still running in context of sync thread, scheduler attempts a load
balance (realizing that cpu is about to become idle) which can result
in tasks being migrated towards the cpu going idle. This will cause
migration notification to be issued and in turn a wakeup on sync
thread. The wakeup call however gets stuck in below while() loop
inside scheduler:

try_to_wake_up(struct task_struct *p, ...)
{

        /*
         * If the owning (remote) cpu is still in the middle of
	 * schedule() with this task as prev, wait until its done
	 * referencing the task.
         */
	while (p->on_cpu)
		cpu_relax();

}

A possible fix could be to teach try_to_wake_up() about this
special case. Another fix, implemented in this patch and that helps
minimize scheduler changes, is to have cpu boost driver not issue a
wakeup under this special circumstance, which was found to occur very
infrequently.

Change-Id: I92bc68a22d51595a208673fe2a1eedfa97004f9e
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
cpufreq: cpu-boost: Re-issue boosts above minimum frequency

Frequency boosts where the source CPU frequency is greater than CPU's
minimum frequency should always go through regardless of the destination
CPU's current frequency. This fixes a performance issue where the governor
lowers the CPU frequency shortly after a thread is migrated to it because
the boost wasn't re-issued.

Change-Id: I449545a688d84b0a6e834f5a51dcb499caa84d29
Signed-off-by: Patrick Cain <pcain@codeaurora.org>
cpufreq: Sync on thread migration optimizations

When threads migrate between CPUs the destination CPU will temporarily
have a boosted frequency and the source CPU will have a residual higher
frequency even though the load could have just been reduced. To avoid
leaving CPUs at a high frequencies notify the source CPU after a thread
migration and make the interactive governor's timer reschedule
conditionally.

Change-Id: Id55055f2a116ca7ffd2a34dc3d0ec519c888a111
Signed-off-by: Patrick Cain <pcain@codeaurora.org>