cpufreq: interactive: allow arbitrary speed / delay mappings

Accept a string of delays and speeds at which to apply the delay before
raising each step above hispeed. For example, "80000 1300000:200000
1500000:40000" means that the delay at or above 1GHz, until 1.3GHz is 80 msecs,
the delay until 1.5GHz is 200 msecs and the delay at or above 1.5GHz is 40
msecs when hispeed_freq is 1GHz.

[toddpoynor@google.com: add documentation]
Change-Id: Ifeebede8b1acbdd0a53e5c6916bccbf764dc854f
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
Git-commit: e43dde3b6cec1e1d36a477713bcfb1dda070bef2
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: add io_is_busy interface

Previously the idle time returned from get_cpu_idle_time_us included the
iowait time. So the iowait time was always calculated as idle time.

But now the idle time returned from get_cpu_idle_time_us does not include
the iowait time anymore because of below commit which cause the iowait time
always calculated as busy time:
    6beea0c nohz: Fix update_ts_time_stat idle accounting

Add the io_is_busy interface, as does the ondemand governor, and let the user
configure the iowait time as busy or idle through the io_is_busy sysfs
interface.

By default, io_is_busy is disabled.

[toddpoynor@google.com: minor updates]
Change-Id: If7d70ff864c43bc9c8d7fd7cfc66f930d339f9b4
Signed-off-by: Lianwei Wang <lian-wei.wang@motorola.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-commit: fbcad7234e0aef2725d9f33ac5a0c3a9cc61a9ac
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: fix crash on error paths in get_tokenized_data

Use separate variable for error code, free proper pointer.

Change-Id: Ia83cccb195997789ac6afbf5b8761f7b278196d6
Reported-by: Arve Hjønnevåg <arve@android.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-commit: cad1684410541be5363d4af9ae8f3f35683a4dd4
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: base above_hispeed_delay on target freq, not current

Time to wait should be based on the intended target speed, not the
actual speed (which may be held high by another CPU).

Change-Id: Ifc5bb55d06adddb9a02af90af05398a78f282272
Reported-by: Arve Hjønnevåg <arve@android.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-commit: c4483b5550079c46dbc68005920a14f371d3f4eb
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: fix uninitialized spinlock

Add missing spinlock init

Backtrace:
[<c0011ce4>] (dump_backtrace+0x0/0x10c) from [<c0662a68>] (dump_stack+0x18/0x1c)
 r6:00000032 r5:c0bd09ec r4:e6848000 r3:00000000
[<c0662a50>] (dump_stack+0x0/0x1c) from [<c06670b0>] (spin_dump+0x80/0x94)
[<c0667030>] (spin_dump+0x0/0x94) from [<c06670f0>] (spin_bug+0x2c/0x30)
 r5:c08f91fc r4:c0bd09ec
[<c06670c4>] (spin_bug+0x0/0x30) from [<c0245f74>] (do_raw_spin_unlock+0x88/0xcc)
 r5:e547bac0 r4:c0bd09ec
[<c0245eec>] (do_raw_spin_unlock+0x0/0xcc) from [<c066c9cc>] (_raw_spin_unlock_irqrestore+0x14/0x40)
 r5:e547bac0 r4:60000013
[<c066c9b8>] (_raw_spin_unlock_irqrestore+0x0/0x40) from [<c044b884>] (store_above_hispeed_delay+0x6c/0x80)
 r4:c0b4cf78 r3:00000007
[<c044b818>] (store_above_hispeed_delay+0x0/0x80) from [<c0235d24>] (kobj_attr_store+0x1c/0x28)
 r7:e68ff000 r6:00000032 r5:e58137c0 r4:e61cde80
[<c0235d08>] (kobj_attr_store+0x0/0x28) from [<c0156b78>] (sysfs_write_file+0x104/0x184)
[<c0156a74>] (sysfs_write_file+0x0/0x184) from [<c0100680>] (vfs_write+0xb0/0x140)
[<c01005d0>] (vfs_write+0x0/0x140) from [<c0100900>] (sys_write+0x44/0x70)
 r8:00000000 r7:00000004 r6:00000032 r5:bee43c90 r4:e5600300
[<c01008bc>] (sys_write+0x0/0x70) from [<c000e400>] (ret_fast_syscall+0x0/0x30)
 r9:e6842000 r8:c000e584 r6:00000032 r5:bee43c90 r4:00000009

Change-Id: I80a1e0b3fecb24adba501ff44f568479deeff7fa
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
Git-commit: 2e745d359ffaa9f4acc4b00950d14322ff364919
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: handle errors from cpufreq_frequency_table_target

Add checks for error return from cpufreq_frequency_table_target, and be
less noisy on the existing call with an error check.  CPU hotplug and
system shutdown may cause this call to return -EINVAL.

Bug: 8613560
Change-Id: Id78d8829920462c0db1c7e14e717d91740d6cb44
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-commit: 952c6d49444061849e2e254b0d701ebb1664f4b3
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: reduce chance of zero time delta on load eval

Reschedule load sampling timer after timestamp of sample start taken,
hold spinlock across entire sequence to avoid preemption.  Avoid the
WARN for zero time delta in the load sampling timer function.

Change-Id: Idc10a756f09141decb6df92669521a1ebf0dbc10
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-commit: 20075d8e9d42aac6e248c7cbe6a2f8a0a00d4ba4
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: avoid underflow on active time calculation

Check for idle time delta less than elapsed time delta, avoid
underflow computing active time.

Change-Id: I3e4c6ef1ad794eec49ed379c0c50fa727fd6ad28
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
Git-commit: 3022f93a1aff25b3c9174ac659d7e4a143892267
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: fix race on cpufreq TRANSITION notifier

The cpufreq TRANSTION notifier callback does not check the
governor_enabled state on affected CPUS, which will case
kernel panic in update_load because the policy object maybe
NULL or invalid when governor_enabled is false.

Change-Id: Ie0f1718124f61e2f9b5da57abc6981ada5b83908
Signed-off-by: Lianwei Wang <a22439@motorola.com>
Git-commit: 44011f4524e86ca5a5e90bb941499cbe37a21987
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: resched timer if max freq raised

When the policy max freq is raised, and before the timer is
rescheduled in idle callback, the cpu freq may stuck at a
lower freq.

The target_freq shall be updated too, else on a high load
situation, the new_freq is always equal to target_freq and
which will cause freq stuck at a lower freq too.

Reschedule the timer on gov limits callback.

Change-Id: I6c187001ab43e859731429b64f75a74eebc37a24
Signed-off-by: Lianwei Wang <a22439@motorola.com>
Git-commit: 0edc2b4c9c30a695d9500d3204acdf5f3ddfa027
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: fix show_target_loads and show_above_hispeed_delay

Remove a trailing whitespace from target_loads and above_hispeed_delay. Problem
happens when user-space program tried to restore parameters that saved before
changing parameters. In this case was returned error(EINVAL).

Change-Id: I5a74e3824602cd6f2b74651adda5ec1b627e61e9
Signed-off-by: Minsung Kim <ms925.kim@samsung.com>
Git-commit: e116c66cdee9080bf8ecb35c2179d79bd00c3043
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: Add a sampling_down_factor for max frequencies

Change min_sample_time to 100ms when running at max

Change-Id: Ia7e35b0625bedf20e7ef3a1f52e5828ffbfed93e
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: Allow 1 ms error in above_hispeed_delay comparisons

Allow for an error of 1 ms while taking into account
above_hispeed_delay for a frequency

Change-Id: I744e44387152e4efb5978df4f2b9533bf79d4582
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: Reset floor_validate_time if busy at max for 100ms

When the interactive governor selects to run at max frequency it doesn't
re-schedule the timer until it hits an idle. This change checks if the CPU
has been continuously busy for last 100ms on hitting an idle start. If yes,
then floor_validate_time is reset so that the CPU stays at max frequency
for at least another 100 ms before stepping down.
This is an important feature for detecting CPU intensive workloads which
require high frequencies for achieving better performance.

Change-Id: I7d48ffbc3d50a80af9be3bf94667ee3d0120b763
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: Remove trace event from idle_start handler

Removed the trace_cpufreq_interactive_idle_start.
Also fix a crash resulting from accessing NULL policy before taking
the pcpu->enable_sem lock. The policy can be NULL if the core is
hotplugged out before the enable_sem lock is taken.

Change-Id: I7e2809cc016b3b383a44cdf3c697013e2d2b5417
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: sync freq feature for interactive governor

1) Add load info to cpufreq_interactive_cpuinfo
2) If load on any other online cpu exceeds sync_freq_load_threshold,
   do not allow the frequency to drop below sync_freq

Change-Id: I3617e10f87b85178914a18bcf04ac2a31a4f1ec1
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: Modifying sync_freq implementation

1. Check for up_threshold_any_cpu_freq instead of sync_freq to
   boost the frequency to sync_freq.
2. Change the load threshold name from sync_freq_load_threshold to
   up_threshold_any_cpu_load
3. Do not consider CPUs with load less than up_threshold_any_cpu_load
   while evaluating max load across CPUs

Change-Id: Ia0e537edbf38a5006c1a22f5c472daa0d086ffc9
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: Use default min_sample_time if SDF is zero

Use min_sample_time to step down from max frequencies if
sampling_down_factor is set to zero.

Change-Id: I2e89bef6220557cb95efd20d658e0c05753b4c3c
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: Fix null pointer dereference in interactive governor

For the sync_freq feature currently we check pcpu->policy->cur frequency
for each online cpu.  But for a CPU that isn't using interactive governor
or for an offline CPU, pcpu->policy can be null or an invalid value.
This patch tries to avoid that scenario by using pcpu->target_freq
instead of policy->cur to get the frequency of an online CPU.

Kernel crash without this patch:
[   20.132373] Unable to handle kernel NULL pointer dereference at virtual address 00000028
[   20.132375] pgd = c34f34c0
[   20.132377] pgd = ef6f2440
[   20.132383] [00000028] *pgd=00000000
[   20.132385]
[   20.132388] [00000028] *pgd=2e98f003, *pmd=00000000
[   20.132390] Internal error: Oops: 205 [#1] PREEMPT SMP ARM
[   20.132394] Modules linked in:
[   20.132398] CPU: 0 PID: 1560 Comm: chown Tainted: G        W    3.10.0-perf-gb12057b-00001-ga2c6c16-dirty #7
[   20.132401] task: ef9af300 ti: ee49c000 task.ti: ee49c000
[   20.132411] PC is at cpufreq_interactive_timer+0x10c/0x650
[   20.132415] LR is at cpufreq_interactive_timer+0x128/0x650
<snip>
[   20.133002] [<c07eb204>] (cpufreq_interactive_timer+0x10c/0x650) from [<c02804d8>] (call_timer_fn+0x80/0x198)
[   20.133012] [<c02804d8>] (call_timer_fn+0x80/0x198) from [<c0280acc>] (run_timer_softirq+0x1f8/0x270)
[   20.133019] [<c0280acc>] (run_timer_softirq+0x1f8/0x270) from [<c0279e20>] (__do_softirq+0x12c/0x2d4)
[   20.133025] [<c0279e20>] (__do_softirq+0x12c/0x2d4) from [<c027a2d4>] (irq_exit+0x74/0xc8)
[   20.133034] [<c027a2d4>] (irq_exit+0x74/0xc8) from [<c0206a00>] (handle_IRQ+0x68/0x8c)
[   20.133041] [<c0206a00>] (handle_IRQ+0x68/0x8c) from [<c02004b8>] (gic_handle_irq+0x3c/0x60)
[   20.133051] [<c02004b8>] (gic_handle_irq+0x3c/0x60) from [<c0ac6900>] (__irq_svc+0x40/0x70)
<snip>

Change-Id: Ie834f5d383de4d41e0fe6fbd40c8b0a0c05d82f5
Signed-off-by: Vijay Ganti <viganti@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>
cpufreq: interactive: delete timers for GOV_START

Make sure that timers cpu_timer and cpu_slack_timer
deactivated before addition of new.

Change-Id: If31c4049606871df6f00efdc24b1d713c86a6f69
Signed-off-by: Shridhar Rasal <srasal@nvidia.com>
Signed-off-by: Bharat Nihalani <bnihalani@nvidia.com>
Git-commit: b430123367e78a6557bac3cf1558bcb85193fb12
Git-repo: https://android.googlesource.com/kernel/common/
[mattw@codeaurora.org: resolved trivial context conflict]
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Signed-off-by: Aditya Amirullah <adityaamirullah@gmail.com>