cpufreq: Add cpu-boost driver

When certain bursty and important events take place, it might take a while
for the current cpufreq governor to notice the new load and react to it.
That would result in poor user experience. To alleviate this, the cpu-boost
driver boosts the frequency of a CPU for a short duration to maintain good
user experience while the governor catches up.

Specifically, this commit deals with ensuring that when "important" tasks
migrate from a fast CPU to a slow CPU, the frequency of the slow CPU is
boosted to be at least as high as the fast CPU for a short duration.

Since this driver enforces the boost by hooking into standard cpufreq
ADJUST notifiers, it has several advantages:
- More portable across kernel versions where the cpufreq internals might
  have been rewritten.
- Governor agnostic and hence works with multiple governors like
  conservative, ondemand, interactive, etc.
- Does not affect the sampling period/logic of existing governors.
- Can have the boost period adjusted independent of governor sampling
  period.

Change-Id: Ibd814a20043d0aba64ee7637a4a79b9ffa1b0991
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: Add a sync limit to cpu-boost

Perform frequency synchronization only when source CPU's frequency
is less than sync_threshold, else sync to the sync_threshold.

Change-Id: I544c414568d4e015b80ce5891dd215275bac95da
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Make boost thread start/stoppable

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: Add Input Boost feature to the cpu-boost driver

On incoming input events boost the frequency of all online cpus
for at least input_boost_ms ms. This is accomplished by changing
the policy->min of all the online cpus to input_boost_freq.

Change-Id: Idb0ab75d68ae4ceff259cbbaaec1a9bb3bc871d3
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: Reset cpu-boost to upstream

This reverts 6f42fce2531, make boost thread start/stoppable.

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Use hotplug thread infrastructure

Before this patch, thread migration ran in uninterruptible threads.
Use the relatively new hotplug thread infrastructure with parked
threads to reduce cpu load (x4).

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Fix queue_delayed_work_on() race with hotplug

Calling queue_delayed_work_on() on a CPU that's in the process of getting
hotplugged out can result in that CPU infinitely looping in
msm_pm_wait_cpu_shutdown(). If queue_delayed_work_on() is called after the
CPU is hotplugged out, it could wake up the CPU without going through the
hotplug path and cause instability. To avoid this, make sure the CPU is and
stays online while queuing a work on it.

Change-Id: I1b4aae3db803e476b1a7676d08f495c1f38bb154
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>

Conflicts:
	drivers/cpufreq/cpu-boost.c

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Remove unused waitqueue

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Don't register for cpufreq notifiers too early

The cpufreq notifiers should be registered only after all the data
structures used in the notifier callbacks have been initialized. So, move
the cpufreq notifier registration to a later point in the init function.

Change-Id: I043ab5bc0ebb98164c40549fe151a8d801c8c186
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: Sync on thread migration optimizations

When threads migrate between CPUs the destination CPU will temporarily
have a boosted frequency and the source CPU will have a residual higher
frequency even though the load could have just been reduced. To avoid
leaving CPUs at a high frequencies notify the source CPU after a thread
migration and make the interactive governor's timer reschedule
conditionally.

Change-Id: Id55055f2a116ca7ffd2a34dc3d0ec519c888a111
Signed-off-by: Patrick Cain <pcain@codeaurora.org>

Conflicts:

	drivers/cpufreq/cpufreq_interactive.c

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>

Conflicts:
	drivers/cpufreq/cpufreq_interactive.c
cpufreq: cpu-boost: Re-issue boosts above minimum frequency

Frequency boosts where the source CPU frequency is greater than CPU's
minimum frequency should always go through regardless of the destination
CPU's current frequency. This fixes a performance issue where the governor
lowers the CPU frequency shortly after a thread is migrated to it because
the boost wasn't re-issued.

Change-Id: I449545a688d84b0a6e834f5a51dcb499caa84d29
Signed-off-by: Patrick Cain <pcain@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Introduce scheduler assisted load based syncs

Previously, on getting a migration notification cpu-boost changed
the scaling min of the destination frequency to match that of the
source frequency or sync_threshold whichever was minimum.

If the scheduler migration notification is extended with task load
(cpu demand) information, the cpu boost driver can use this load to
compute a suitable frequency for the migrating task. The required
frequency for the task is calculated by taking the load percentage
of the max frequency and no sync is performed if the load is less
than a particular value (migration_load_threshold).This change is
beneficial for both perf and power as demand of a task is taken into
consideration while making cpufreq decisions and unnecessary syncs
for lightweight tasks are avoided.

The task load information provided by scheduler comes from a
window-based load collection mechanism which also normalizes the
load collected by the scheduler to the max possible frequency
across all CPUs.

Change-Id: Id2ba91cc4139c90602557f9b3801fb06b3c38992
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Handle wakeup hints received for foreground tasks

A previous change modifies the notification conditions in the scheduler
to call the notifier chain even on foreground_thread wakeups for tasks
having load more than a threshold value.

If load_based_syncs is turned OFF then we do not need to perform
cpu boost for wakeup hints for foreground tasks from scheduler

Change-Id: Ifbdd3eccac5c9892dfc3a3c3edbfc0df766478ed
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Make it less noisy

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Increase sync thread policy and priority

More or less all cpufreq related threads run in a high priority.

Also, CMs patch series was somehow out of sync and incomplete,
readd the unchanged policy update to source cpu bit. And readd
the check for a possible corner case in try_to_wake_up() that
was already missing before.

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Consider only task load to decide on sync frequency

Currently we take the maximum between the source CPU frequency and
the calculated frequency based on the migrating task load to decide on
the frequency to sync the destination CPU to. This was done to handle
short bursts in workloads of tasks which migrated immediately after
causing a ramp up on the source CPU. Since their load history wasn't
high enough, the destination CPU synced to a lower frequency which
wasn't sufficient for the spike in workload.

But as such cases are rare, taking the higher of source and calculated
frequency can lead to destination CPU unnecessarily spending a
considerable amount of time at higher frequencies which in turn can
hurt power.

With this change we make sure only the migrating task load is used
to calculate the sync frequency for destination CPU when load based
syncing is enabled.

Change-Id: Ib1489d256c42ea7712aad2179aebffc87c549836
Signed-off-by: Girish Ghongdemath <girishsg@codeaurora.org>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>

Conflicts:

	drivers/cpufreq/cpu-boost.c

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpuboost: Always consider current policy max

We fixed calculations in the scheduler in 58cd101c69c508a448a03fc480ceb35f0da12386.
Now repeat this in the cpu-boost driver.

Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpuboost: Add hotplug boosting

If we decide to hotplug a core then we are making a strong
statement that we think that the system load is high.  Let's
have faith in our hotplug driver(s) and boost the cpufreq
when we are hotplugging cores.

[myfluxi: shorter boost, minor style changes]

Change-Id: Id35bae6c27c39097c7aca70eebf1498ed0c3dfdc
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpu-boost: Fix cores getting stuck. Finally.
* In exclusive hotplug system, if a core is not taken down
  explicitly, it is observed that it gets stuck at input
  boost frequency.
* This issue maybe particular to only this kernel but this
  fix resolves it completely.
* Root cause seems to be removal work is lost if queued on
  sibling cores. So simply fix this by queuing on boot CPU.

* Finally, I can use min_cpus_online > 1 without any issue.

Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>
cpufreq: cpu-boost: Implement additional checks

* Return early without calculations if last
  input_boost_work is pending.
* Disable input boost if input_boost_freq value is
  less than or equal to min possible frequency.
* Disable hotplug boost if plug_boost_freq value is
  less than or equal to min possible frequency.
* When we check for current frequency greater than
  boost frequency, also check if boost is removed
  properly or not. If not, do so.

Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
Signed-off-by: Stefanescu Cristian <howerkraft@gmail.com>