cpufreq: interactive: New 'interactive' governor

This governor is designed for latency-sensitive workloads, such as
interactive user interfaces.  The interactive governor aims to be
significantly more responsive to ramp CPU quickly up when CPU-intensive
activity begins.

Existing governors sample CPU load at a particular rate, typically
every X ms.  This can lead to under-powering UI threads for the period of
time during which the user begins interacting with a previously-idle system
until the next sample period happens.

The 'interactive' governor uses a different approach. Instead of sampling
the CPU at a specified rate, the governor will check whether to scale the
CPU frequency up soon after coming out of idle.  When the CPU comes out of
idle, a timer is configured to fire within 1-2 ticks.  If the CPU is very
busy from exiting idle to when the timer fires then we assume the CPU is
underpowered and ramp to MAX speed.

If the CPU was not sufficiently busy to immediately ramp to MAX speed, then
the governor evaluates the CPU load since the last speed adjustment,
choosing the highest value between that longer-term load or the short-term
load since idle exit to determine the CPU speed to ramp to.

A realtime thread is used for scaling up, giving the remaining tasks the
CPU performance benefit, unlike existing governors which are more likely to
schedule rampup work to occur after your performance starved tasks have
completed.

The tuneables for this governor are:
/sys/devices/system/cpu/cpufreq/interactive/min_sample_time:
	The minimum amount of time to spend at the current frequency before
	ramping down. This is to ensure that the governor has seen enough
	historic CPU load data to determine the appropriate workload.
	Default is 80000 uS.
/sys/devices/system/cpu/cpufreq/interactive/go_maxspeed_load
	The CPU load at which to ramp to max speed.  Default is 85.

Change-Id: Ib2b362607c62f7c56d35f44a9ef3280f98c17585
Signed-off-by: Mike Chan <mike@android.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Bug: 3152864
cpufreq interactive governor save/restore IRQs around cpumask spinlocks

Need to use irqsave/restore spin locking for cpumasks since these
are accessed in timers and in thread context.

Change-Id: I4a53eaf0ced7e73b445feddba90ec11482de9126
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq interactive governor: fix crash on CPU shutdown

Don't reference the saved copy of the CPU's cpufreq policy
pointer after the governor has been stopped for the CPU.

When the governor is stopped for a CPU:

* Use del_timer_sync() to wait for a currently-running timer
function to stop.

* Delete the timer when the governor is stopped for the
associated CPU, not when the last CPU is stopped.

* Flush any speed down work ongoing.

* Reset the timestamp that is used to tell if the timer function
has had a chance to run since last idle exit.

Check the governor enabled flag for the CPU before re-arming the
timer from within the timer function and at idle exit (in case
stopping the governor at runtime).

Check the governor enabled flag for the CPU in the worker
function and thread before using the policy pointer.  (There is
still a tiny window in the thread that needs more work to close.)

Change-Id: Ifaddf7a495a8dae15a579a57bdc654f7c47f6ada
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq interactive: support shared CPU scaling

Change-Id: Id5267f04067bf023f6b140b4de2e88ef7287e941
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive: remove debug trace code

Remove debug trace code in preparation of upstreaming

Change-Id: I0905885e75031f5e9d7cb06878fb68c1fd06d4fe
Signed-off-by: Allen Martin <amartin@nvidia.com>
cpufreq: interactive: use idle notifier

Convert interactive governor to use idle notifier instead of
hooking pm_idle directly.

Change-Id: I47e007f330468ac559240a0ae8a3cb06a89ccb67
Signed-off-by: Allen Martin <amartin@nvidia.com>
cpufreq: interactive: Add error checking on sysfs interfaces

This adds better error checking on tunable parameters on sysfs
interfaces.  Also fixes return value from these functions, previously
on success they would return 0 which would cause a infinite loop.

Change-Id: Ic05038492166f8673d007202092471f98a2f0dfa
Signed-off-by: Allen Martin <amartin@nvidia.com>
cpufreq: interactive: add sysfs control for timer rate

Add a new sysfs control that tunes the rate of the timer used to
increase cpu frequency

Change-Id: I1aa13ae54bb43aff5b3688984d2955f56aae1658
Signed-off-by: Allen Martin <amartin@nvidia.com>
cpufreq: interactive: cleanup checkpatch warnings

Cleanup some style warnings reported by checkpatch

Change-Id: Ie2e6903d52867fb3347e009d7efa3bc4ca755cea
Signed-off-by: Allen Martin <amartin@nvidia.com>
cpufreq: interactive: fix checkpatch warnings on long lines

Fix up checkpatch warning introduced by long lines in timer_rate
patch.

Change-Id: I22b105dafb1b49390799bb7577464da03f0f8afb
Signed-off-by: Allen Martin <amartin@nvidia.com>
cpufreq: interactive: Fix handling of SMP where CPUs must scale in tandem

The interactive governor relies on quirks of the Tegra 2 cpufreq
implementation for handling SMP systems where the CPUs do not have
separate rate controls.  It needs to determine the maximum rate
for all CPUs covered by the policy and set that speed.

Change-Id: I1ed9fa171e5a9c45a1fa5944e3fa823eb157e81f
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive: fix possible Division by zero.

The following dump was seen sometimes while resuming,
the only division by zero on this function can happen after
delta_time is reassigned, since at the start of the
function, there is jump that protects against values
less than 1000.

After that, If delta_time and delta_idle == 0,
we will hit a div 0

Division by zero in kernel.
Backtrace:
[<c0057184>] (dump_backtrace+0x0/0x110) from [<c05d5ecc>] (dump_stack+0x18/0x1c)
r6:010f3000 r5:c113dfb0 r4:c004afb0 r3:c6ff0000
[<c05d5eb4>] (dump_stack+0x0/0x1c) from [<c00572cc>] (__div0+0x1c/0x20)
[<c00572b0>] (__div0+0x0/0x20) from [<c02195d4>] (Ldiv0+0x8/0x10)
[<c03dfd64>] (cpufreq_interactive_timer+0x0/0x2c0) from [<c00a7efc>] (run_timer_softirq+0x154/0x260)
[<c00a7da8>] (run_timer_softirq+0x0/0x260) from [<c00a0c8c>] (__do_softirq+0xc8/0x194)
[<c00a0bc4>] (__do_softirq+0x0/0x194) from [<c00a1008>] (irq_exit+0xb4/0xb8)
[<c00a0f54>] (irq_exit+0x0/0xb8) from [<c00584b4>] (ipi_timer+0x44/0x48)
r4:c004a040 r3:00000001
[<c0058470>] (ipi_timer+0x0/0x48) from [<c004c3e4>] (do_local_timer+0x68/0x84)
r5:c004ae2c r4:c07991e8
[<c004c37c>] (do_local_timer+0x0/0x84) from [<c0052948>] (__irq_svc+0x48/0xe0)

Change-Id: I639882db67b8d711c5710778ebc212f0f6a998e3

Acked-by: Nishanth Menon <nm@ti.com>
Signed-off-by: Axel Haslam <axelhaslam@ti.com>
cpufreq: interactive governor: default timer 10ms, maxspeed load 95%

Modify default timer from 30ms to 10ms, sampling 2 jiffies after
idle exit on ARM as in Honeycomb.

Modify default go_maxspeed_load from 85% loaded to 95% loaded, for
use in phones where power savings is more important (tablets may be
best served overriding this).

Change-Id: I3361a6279979bfae1df5262666a2e30ea7a44328
Signed-off-by: Todd Poynor <toddpoynor@google.com>
power: cpufreq interactive governor: use default sample time 20ms

Lower the default time at which a higher speed is allowed to run
before lowering based on lower CPU load from 80ms to 20ms.  Most
Android devices should trade power for performance here,
although tablets and non-battery-powered devices may want to
override this default.

Change-Id: I1a4f7faeca12793c51d5b92db30a63cca8d4f1be
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive governor: apply intermediate load on current speed

Calculate intermediate speed by applyng CPU load to current speed, not
max speed.

Change-Id: Idecf598b9a203b07c989c5d9e9c6efc67a1afc2e
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive governor: scale to max only if at min speed

Change-Id: Ieffb2aa56b5290036285c948718be7be0d3af9e8
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive governor: go to intermediate hi speed before max

* Add attribute hispeed_freq, which defaults to max.

* Rename go_maxspeed_load to go_hispeed_load.

* If hit go_hispeed_load and at min speed, go to hispeed_freq;
  if hit go_hispeed_load and already above min speed go to max
  speed.

Change-Id: I1050dec5f013fc1177387352ba787a7e1c68703e
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive governor: default 20ms timer

Change-Id: Ie9952f07b38667f2932474090044195c57976faa
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq interactive governor: event tracing

Change-Id: Ic13614a3da2faa2d4bd215ca3eb7191614f0cf66
Signed-off-by: Todd Poynor <toddpoynor@google.com>
cpufreq: interactive: apply intermediate load to max speed not current

Evaluate spikes in load (below go_hispeed_load) against the maximum
speed supported by the device, not the current speed (which tends to
make it too difficult to raise speed to intermediate levels until
very busy).

Change-Id: Ib937006abf8bedb60891a739acd733e89b732ae0
Signed-off-by: Todd Poynor <toddpoynor@google.com>