msm: cpufreq: increase priority of thread that increases frequencies

When the cpufreq governors request for increase in frequency from kworker
threads, there is chance that the kworker thread is cpu starved due to
other high priority threads, leading to undesirable increase in frequency
ramp-up latency. Add the calling thread to SCHED_FIFO policy when
requesting for increase in frequency and restore back to original policy
after ramp-up is completed.

Change-Id: Ie4fa199b7f9087717cb94dfcc2eddea27cc0012b
Signed-off-by: Narayanan Gopalakrishnan <nargop@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
Revert "Squash faux's Intellidemand for geeb base off mako"

This reverts commit 0b9f4c16ac78f44480b8b7b70171893e79637e7a.
Revert "Add MSM_SLEEPER mod (max screen off frequency)"

This reverts commit 187649d7aeb0c9ad9f5325c2a8ce9960af03d89b.
Revert "msm: cpufreq: Remove cross-calling limitation"

This reverts commit be01a17afb6b6e85456217caa1ac11eb0b6674e1.

Re-introduce cross calling in cpufreq to prevent race conditions in the
hardware when changing PLLs.

A core could be in retention, while another core, could try and set the
frequency of the core in retention, causing a change in PLL and when the
core exits retention, the PLL mismatch could hang the SPM state machine.

Change-Id: I99b454a0f146652f2d8a369f79f6d35748160dcb
Signed-off-by: Praveen Chidambaram <pchidamb@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
msm: cpufreq: Initialize cpufreq driver early at boot

MSM cpufreq driver registering with the CPUFreq framework during
late_initcall() may cause the device to be available a few seconds after
the clock driver is ready for changing frequency. This could be bad for
boot and/or thermal performance, since the cpu will continue to boot at
whatever frequency the bootloaders set it to.

Initialize cpufreq driver as part of device_initcalL() but only after
all the clock drivers have had a chance to initialize their devices.

Change-Id: Iaa4867108f59a368ee5b511d35ea2e8e219d39d3
Signed-off-by: Praveen Chidambaram <pchidamb@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>

Conflicts:
	arch/arm/mach-msm/Makefile
msm: cpufreq: Always update frequency limits when core is online

Remove condition in msm_cpufreq_init() so that when cpu is set to
online, it will call set_cpu_freq() and update the frequency limits.

CRs-Fixed: 462816
Change-Id: I40ac33c9cb4b42590a83f9d61b74f23379f5642e
Signed-off-by: Jennifer Liu <chihning@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
msm: cpufreq: Configure WQ for higer priority

When the workqueue runs at a lower priority, it gets starved by higher
priority threads. Bump the WQ priority to high to ensure cpufreq work
queue gets a fair scheduling chance

Change-Id: I9e994da94a347dceb884e72ec3dd3da468a4471d
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
msm: cpufreq: Relax constraints on "msm-cpufreq" workqueue

This workqueue is not used in memory reclaim paths, so the
WQ_MEM_RECLAIM flag is not needed. Additionally, there is no
need to restrict the queue to having one in-flight work item.
Remove these constraints.

Change-Id: I9edde40917d3ec885ce061133de20680634321d0
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
msm: cpufreq: Ensure cpufreq change happens on corresponding CPU

Checking the cpus_allowed mask of the current thread before changing the
frequency doesn't guarantee that the rest of the execution will continue on
the same CPU. The only way to guarantee this is to schedule a work on the
specific CPU and also prevent hotplug  of that CPU (already done by
existing code).

Change-Id: I51a02fcc777a47d3c16f2d83c47e96f2c59f7ae6
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
msm: cpufreq: Add support for CPU clocks and msm-cpufreq device

Add support for using clock APIs to do CPU/L2 frequency scaling. When clock
APIs are used to control CPU/L2 frequencies, the cpufreq driver must handle
the CPU -> L2 -> DDR bandwidth voting. Use a msm-cpufreq device to provide
the per SoC list of CPU frequencies and their L2/DDR bandwidth mapping
information.

If CPU/L2 clocks are not available, fallback to using acpuclock APIs. We
eventually want to remove the use of non-standard acpuclock APIs.

Change-Id: I2c4e2c3967d73e8cdbd9833f3cb36f3d75e27b4a
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
msm: cpufreq: Update frequency index

A deprecated api can be used to limit the minimum and maximum cpu speeds.
Another driver temporarily needs to switch back to this api, instead of
using the new version. Make this api functional again by ensuring a stale
value is updated properly.

Change-Id: Ia63772e43ec534cf2bb58e7627a87315dae5d891
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
msm: cpufreq: Fix msm_cpufreq_get_freq() for sync CPUs

When CPUs are in sync, only CPU0 has a non-NULL cpu_clk. clk_get_rate()
on other cpu_clk will return 0. msm_cpufreq_get_freq() is used in multiple
places in cpufreq framework to determine current hardware frequency.

Fix msm_cpufreq_get_freq() to return the right hardware frequency for
systems with synchronous CPUs.

Change-Id: I6a880764ad51fa9a257a328798b38d574750686a
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Signed-off-by: houst0nn <houstonn@gmail.com>
Squash faux's Intellidemand for geeb base off mako

cpufreq/cpufreq_intellidemand: intellidemand cpufreq governor (version 4.0)

Intellidemand is inspired by ondemand governor. Its main features are:

 - auto cpu hotplugging based on runnable thread average calculations
 - detection of browsing mode (long screen on activities with moderate run
   queue depth =< 45 )
 - limit max frequencies when in browsing mode to conserve battery for long
   duration tasks

Browsing State Determination:
 1- Browsing state is enabled as power-on default. Browsing state is disabled
    when run queue depth is > 45 (very high instantaneous spikes)

Browsing State Limit Max Freq Logic (sliding window algorithm):
 1- While Screen is ON > 3 min, and average load of the CPU is < 35%, then
    CPU Max frequencies will remain power-on default
 2- While Screen is On, if average CPU load > 35%, both CPUs max frequencies
    will be limited to a lower user configurable max value
 3- While Screen is On, After both CPUs' Max is limited to lowered max, when
    the average CPU load has dropped down to less than < 25%, then both CPUs
    Max freq will be restored to back to power-on default

Runnable Thread Auto-hotplug (fast hotplug / slow unplug algorithm):
 1- When average number of running thread is < threshold, 2nd cpu will be
    unplugged.
 2- When > threshold, 2nd cpu will be hotplugged.
 3- The logic applies a fast hotplug and slow unplug algorithm so 2nd core
    will stay enabled longer to help deal with performance spikes

changes from Ver 2.0 -> Ver 3.0:
  - remove GPU Busy dependency for browsing state detection (this will make
    the governor more portable to other platforms without specific GPU
    architecture knowledge)

dependencies:
  -- Qualcomm's run queue average logic (SOC independent logic)
  -- TI's runnable thread average logic (SOC independent logic)

forward ported to Linux 3.4 for use on Mako

Signed-off-by: Paul Reioux <reioux@gmail.com>

intellidemand: Add boost pulse capability

Boost pulse increases UI performance by increasing CPU freq and
reduce sampling time.

forward ported for Linux 3.4 for use on Mako
Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: add bug fixes

 - Fix crash input event handler on governor switch
 - Fix panic when setting sampling rate

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: add a run queue persistence to determine browsing state

persistence will allow for better browsing state detection rather than just
purely based on run queue depth alone

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: add minimum cpu performance lock

a minimum cpu frequency performance lock is add to ensure smooth
operations when only 1 core is active

Signed-off-by: faux123 <reioux@gmail.com>

Intellidemand: update for quadcore operations. bump version to 4.0

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: code derps clean up!

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: increase boost duration to 2.5 seconds from 1.5

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: add eco mode (dual core operations) bump version to 4.1

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: tweak for not using min performance lock

Signed-off-by: faux123 <reioux@gmail.com>

Intellidemand: adjust the nr_run hysteresis for better UI response

Signed-off-by: faux123 <reioux@gmail.com>

cpufreq: Avoid using smp_processor_id() in preemptible context

Even though this work item runs on only one cpu at a time (due to
queue_work_on()) it is possible for the work item to be preempted
and so use of smp_processor_id() is illegal.

BUG: using smp_processor_id() in preemptible [00000000]
code: kworker/3:1/4162 caller is dbs_refresh_callback+0xc/0x188
[<c00151b0>] (unwind_backtrace+0x0/0x120) from [<c0279058>]
(debug_smp_processor_id+0xbc/0xf0)
[<c0279058>] (debug_smp_processor_id+0xbc/0xf0) from [<c0454b54>]
(dbs_refresh_callback+0xc/0x188)
[<c0454b54>] (dbs_refresh_callback+0xc/0x188) from [<c0087290>]
(process_one_work+0x354/0x648)
[<c0087290>] (process_one_work+0x354/0x648) from [<c0089754>]
(worker_thread+0x1a8/0x2a8)
[<c0089754>] (worker_thread+0x1a8/0x2a8) from [<c008e480>]
(kthread+0x90/0xa0)
[<c008e480>] (kthread+0x90/0xa0) from [<c000f438>]
(kernel_thread_exit+0x0/0x8)

The intent of the code is to determine which CPU this work item
is running on, which we can easily do by passing that information
in a wrapper struct around the work struct. Do this so we avoid
this problem.

Change-Id: I05ca0ff2b3cbaa239930463ea0760e3e9d75145f
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
adapted for Intellidemand from CAF reference

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: Add support to consider other cpu load when scaling frequencies

Currently only current cpu load is considered when frequency scaling.
As a side effect of this when scheduler migrates a thread to a slower
core, the execution time increases unexpectedly.

The new algorithm not only considers current cpu load for frequency
scaling, but also takes into account load of other online cpus. The
treshold for this can be controlled using up_threshold_any_cpu_load.
When the up_threshold_any_cpu_load is crossed in any cpu then the
current cpu is set to sync_freq.

Also, when more than one cpu is online a two step ramp-up mechanism
is used. When the up_threshold_multi_core is crossed the cpu is ramped
up to optimal_freq which is less than policy->max and set to a power
friendly value.

Change-Id: I64bed1b142cde1f239e73aa1f78981c78aaf59ce
Signed-off-by: Narayanan Gopalakrishnan <nargop@codeaurora.org>

intellidemand: Fix kernel warning in cpufreq_governor_dbs

GOV_STOP event destroys the timer_mutex and is not initialized back
upon GOV_START event.
The Kernel complains at mutex_lock(during GOV_LIMITS event) after
mutex_destroy.

[<c0014918>] (unwind_backtrace+0x0/0x11c)
[<c007c004>] (warn_slowpath_common+0x4c/0x64)
[<c007c034>] (warn_slowpath_null+0x18/0x1c)
[<c079eef8>] (__mutex_lock_slowpath+0x130/0x3c4)
[<c079f1ac>] (mutex_lock+0x20/0x3c) from [<c0540900>]
[<c0540900>] (cpufreq_governor_dbs+0x3c8/0x468)
[<c053d500>] (__cpufreq_governor+0x90/0xe0)
[<c053d774>] (__cpufreq_set_policy+0x224/0x258)
[<c053da58>] (cpufreq_add_dev_interface+0x2b0/0x308)
[<c053e784>] (cpufreq_add_dev+0x500/0x56c)
[<c079247c>] (cpufreq_cpu_callback+0x88/0x9c)
[<c07a2ca8>] (notifier_call_chain+0x38/0x68)
[<c007ded0>] (__cpu_notify+0x28/0x40)
[<c0790024>] (_cpu_up+0xe4/0x118)
[<c0790024>] (cpu_up+0x64/0x80)
[<c0765840>] (store_online+0x48/0x78)
[<c038d0c8>] (dev_attr_store+0x18/0x24)
[<c017be40>] (sysfs_write_file+0x108/0x13c)
[<c012cdfc>] (vfs_write+0xb0/0x128)
[<c012d034>](sys_write+0x38/0x64)

Change-Id: Ie2a796b17209243de731ccf2ab04b7f9e7048df8
Signed-off-by: Anji Jonnala <anjir@codeaurora.org>

Adapted for Intellidemand from CAF reference

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: give intellidemand's cpu_dbs_info its own name

Signed-off-by: Paul Reioux <reioux@gmail.com>

intellidemand: add default values for dbs sync and optimal tuneables

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: remove cpu cores autoplug capability (replaced by intelli_plug)

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: remove eco mode option (replaced by intelli_plug)

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: remove unused min performance lock defines

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: Don't update the policy->cur upon cpufreq driver failure

Signed-off-by: faux123 <reioux@gmail.com>

msm: cpufreq: make the gov stop part fully atomic

In the function cpufreq_governor_dbs, the case CPUFREQ_GOV_STOP
is not completely atomic. The removal of sysfs is outside the
critical path. If a START/STOP done frequently from user space
then there is chance of adding the sysfs interface before removal.
This creates a warning from sysfs saying creating an already
existing entry.

Also once it fails to create, cpufreq_governor_dbs returns with
error code. The dbs_enable value is incremented which is not
rolled back and cpufreq core tries to set the old
governor. This might create further problems.

Hence the removal of sysfs entry should happen in the critical
path.

CRs-fixed: 440969
Change-Id: I98891756caf2ce0b85e39e9b4f58034af1ed97ce
Signed-off-by: Venkat Devarasetty <vdevaras@codeaurora.org>
Signed-off-by: faux123 <reioux@gmail.com>

Remove deprecated logic. Based on http://tinyurl.com/kokpohm

Change-Id: I3c70fba9c2452fbdae1e1c14a2240fc73ae403a9
Signed-off-by: Josue Rivera <prbassplayer@gmail.com>

intellidemand: allow clean compilation when LMF is disabled

bumped version to 4.2

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: change boost tunable scope from global to local

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: change permission for boostpulse from root to system

for use with Android PowerHAL

Signed-off-by: faux123 <reioux@gmail.com>

intellidemand: fix permission for sampling_rate as well

to work with Android's PowerHAL

Signed-off-by: faux123 <reioux@gmail.com>

cpufreq: intellidemand: Set MIN_LATENCY_MULTIPLIER to 20

Currently MIN_LATENCY_MULTIPLIER is set defined as 100 and so on a system with
transition latency of 1 ms, the minimum sampling time comes to be around 100 ms.
That is quite big if you want to get better performance for your system.

Redefine MIN_LATENCY_MULTIPLIER to 20 so that we can support 20ms sampling rate
for such platforms.

Signed-off-by: faux123 <reioux@gmail.com>

cpufreq: intellidemand: Fix optimal_freq logic on ramp down

When ondemand governor decides to ramp down the frequency, the missing
check for frequency being lower than optimal_freq was causing frequency
to be ramped down to optimal_freq even if the frequency decided by ondemand
governor was higher.

Change-Id: I00f9ae4c61ccda4039f3338dee2c29b8cb790be4
Signed-off-by: Veena Sambasivan <veenas@codeaurora.org>
Signed-off-by: Paul Reioux <reioux@gmail.com>

intellidemand: re-enable performance lock option

with low freq enabled kernels, minimum performance lock will help music
playback to be smooth and glitch free

Signed-off-by: Paul Reioux <reioux@gmail.com>

Intellidemand: limit min cpu perflock to boot cpu only

Signed-off-by: Paul Reioux <reioux@gmail.com>

Add Intellidemand

Change-Id: I0fe516e656bf18e8d4ee6c34c8499c3cd4949e97
Signed-off-by: Josue Rivera <prbassplayer@gmail.com>

Conflicts:
	arch/arm/mach-msm/cpufreq.c
Add MSM_SLEEPER mod (max screen off frequency)

Conflicts:
	arch/arm/mach-msm/Makefile
	arch/arm/mach-msm/cpufreq.c