PM / devfreq: Check for GPU busy time before sending the sample

Sometimes GPU busy time is very low during sampling. Accumulate
the statistics and send them with the next sample.

Change-Id: I77e568b48afc3a4922c7c341d9478705b7ae3a30
Signed-off-by: Suman Tatiraju <sumant@codeaurora.org>
devfreq: Backport MSM devfreq features from 3.10

PM / devfreq: Rewrite devfreq_update_status() to fix multiple bugs

The current devfreq_update_status() has the following bugs:
- When the new frequency doesn't have a valid level, the time spent in the
  new frequency is counted towards the next valid frequency switch instead
  of being ignored.
- The time spent on the previous frequency is added to the new frequency's
  stats instead of the previous frequency's stats.

This patch fixes all of this.

Change-Id: I7bbdafca78d1565cdbf815356556fe6d6cca9e3f
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Allocate memory using the right data type

Long and int have different sizes on a 64-bit machine. Allocate
memory for the time_in_state table using the right data type.

Change-Id: Iecc1acc12ddabdb436c212f205aaed8dbb49a2b4
Signed-off-by: Suman Tatiraju <sumant@codeaurora.org>

devfreq: Make cpubw_hwmon governor reusable and hardware agnostic

Since various hardware platforms might have different ways of monitoring
the CPU bandwidth, make the cpubw_hwmon governor hardware agnostic so that
it can be reused across these hardware platforms.

Change-Id: If7af01537dbf1df66551bea2bc73232d9e9264f7
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Use freq_table for available_frequencies

Some devices use freq_table instead of OPP. For those devices, the
available_frequencies file shows up empty. Fix that by using freq_table to
generate the available_frequencies data when OPP is not present.

Change-Id: Ibea8b388ee81c55d2eeddd8a1e2c18c91faed8c7
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

msm: devfreq_cpubw: Move devfreq_cpubw to drivers/devfreq

In order to share devfreq_cpubw among all targets, it needs to be
moved outside arch/arm/mach-msm directory. Move the file and
Kconfig to drivers/devfreq.

Change-Id: I4dd476b315b04f84b07855bec3ed5b6061549ea9
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

PM / devfreq: Generic cpufreq governor

This devfreq governor is a generic implementation that can scale any
devfreq device based on the current CPU frequency of all ONLINE CPUs. It
allows for specifying CPU freq to devfreq mapping for specific devices.
When such a mapping is not present, it defaults to scaling the device
frequency in proportion to the CPU frequency.

Change-Id: I8d3b1fea26572defbad2ce8b6cf87930caa25ca6
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Fix warning in governor_cpufreq

The cpu_khz variable would be uninitialized when debug messages are printed
for OFFLINE CPUs. Fix that by initializing it to 0.

Change-Id: Ic707c13b1048625c0ecc0b0057af2f84f17462f6
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Add devfreq driver for simple device

This driver registers a devfreq device that allows devfreq governors to
scale the frequency of a device clock. This single driver can be used to
support multiple devices as long as those devices don't have a need or
mechanism to monitor their load and use clock APIs to control their
device/core clock.

If devices need to support device specific status monitoring, they could
extend this driver to allow registering device specific status monitoring
funtions or write their own specific devfreq device driver.

Change-Id: Ie1797acf7b35cac6dc49428e270c23082634eb1e
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Add cache HW monitor governor

The cache HW monitor devfreq governor uses the hardware counters to
determine the load on the cache and the appropriate frequency needed to
support that load. This governor can be used in conjunction with the cache
devfreq device to dynamically scale the cache frequency based on the
demand/actual usage from the CPU subsystem.

The governor is written to be agnostic of the actual counters used to
determine the load. On Krait based CPUs, the governor uses the Krait L2 PM
counters which can conflict with certain profiling tools.

The Krait L2 performance monitor counters have the capability to count
different types of requests going to the L2 cache. They also have the
capability to raise interrupts when they overflow. This driver uses those
counters to determine the true usage of L2 from the Krait processor
subsystem and then recommends L2 frequency based on the measured values and
the following tunable parameters.

The driver provides various tunables that allow it to be tuned more in
favor of power or performance:

- cycles_per_high_req: The no. of cache clock cycles that are necessary to
  efficiently process a high-work request to the cache. A higher value
  means higher power and potentially higher performance. A lower value
  means lower power and potentially lower performance.

- cycles_per_med_req: The no. of cache clock cycles that are necessary to
  efficiently process a medium-work request to the cache. A higher value
  means higher power and potentially higher performance. A lower value
  means lower power and potentially lower performance.

- polling_ms: The sampling period in milliseconds. This only affects the
  sampling period when cache use is ramping down or is increasing very
  slowly (See tolerance_mrps).

- min_busy: The minimum percentage of time the cache should be busy. This
  is also applied as a lower bound to the measured busy percentage before
  it's used in calculations. This has to be lower than or equal to
  max_busy. Lower values will make the scaling more aggressive.

- max_busy: The maximum percentage of time the cache should be busy. This
  is also applied as an upper bound to the measured busy percentage before
  it's used in calculations. This has to be greater than or equal to
  min_busy. Lower values will make the scaling more aggressive.

- tolerance_mrps: The minimum increase (in millions of requests per second)
  in cache requests, compared to previous sample, that will trigger an IRQ
  to immediately re-evaluate the cache frequency.

- decay_rate: The parameter controls the rate at which the history is
  forgotten when ramping down. This is expressed as a percentage of history
  to be forgotten. So 100% means ignore history, 0% means never forget the
  historical max. The default 90% means forget 90% of history each time.

- guard_band_mhz: This is a margin that's added to the computed cache
  frequency to account for the time it takes between the load increasing
  and the governor/device finishes ramping up the cache frequency.

Change-Id: I918ae178cd3c9d14cb7714d7eb312cbbbb0d977b
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

clock-a7/krait: Change initcall level for early probe

Requirement to probe CPU clock drivers and register msm-cpufreq early
mandates initcall level changes in these drivers. Order among generic
clock driver, CPU clock driver and msm-cpufreq is unchanged, but
CPU clock drivers and msm-cpufreq are moved to earlier level to meet
new requirement.

Change-Id: I91c615795e2b114059d34f41b2f562a3e394e341
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

cpufreq: delete __cpuinit usage from all cpufreq files

The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications.  For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.

After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out.  Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.

This removes all the drivers/cpufreq uses of the __cpuinit macros
from all C files.

[1] https://lkml.org/lkml/2013/5/20/589

[v2: leave 2nd lines of args misaligned as requested by Viresh]
Cc: "Rafael J. Wysocki" <rjw@sisk.pl>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: cpufreq@vger.kernel.org
Cc: linux-pm@vger.kernel.org
Acked-by: Dirk Brandewie <dirk.j.brandewie@intel.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Git-commit: 2760984f6578d5a462155bb4727766d0c8b68387
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[junjiew@codeaurora.org: resolve trivial merge conflicts. Remove __cpuinit
 for arch/arm/mach-msm/cpufreq.c as well]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

Change-Id: I9deb4ad1ffb557e928975f866c39e37f7180da35

msm: thermal: Make core control and PSM independent of cpu frequency

Updated the KTM to perform PMIC SW Mode(PSM) threshold monitoring
and core control independent of the CPU frequency table
initialization.

Change-Id: I45653ef3087830cc96326b8e770d826530b7fed5
Signed-off-by: Ram Chandrasekar <rkumbako@codeaurora.org>

PM / devfreq: Move cpubw device from msm_cpufreq to cpufreq governor

Enable the cpufreq governor for the cpubw device on all targets by:
- Adding the devfreq-cpufreq node for mapping CPU freq to cpubw
- Enabling the cpufreq governor configs when CPUBW config is selected
- Switching the default governor for cpubw device to "cpufreq"

Change-Id: Ic10b645123192ed13c66340f288d6a8553d6f251
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

cpufreq: Add policy create/remove notifiers

Change-Id: I1158f2650b8f6ce141ac1253ed70a3d6c036a559

of: Add support for reading a u32 from a multi-value property.

This patch adds an of_property_read_u32_index() function to allow
reading a single indexed u32 value from a property containing multiple
u32 values.

Change-Id: I741d447bf90d0150939ced92a9a150b7a6de25bc
Signed-off-by: Tony Prisk <linux@prisktech.co.nz>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Rob Herring <robherring2@gmail.com>

ARM: dts: msm: Scale cache with CPUfreq using devfreq governor and device

For SoCs with asynchronous cache:
- Add qcom,cache device to provide the clock and the list of usable
  frequencies to the devfreq clock driver. Also set the default governor
  for this device to "cpufreq".
- Add CPUfreq to cache freq mapping for the cache device.
- Enable the DEVFREQ_SIMPLE_DEV config for these targets.

Change-Id: I5bda30420e79935d8d660306dd25d45042a3945b
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

msm: cpufreq: Remove acpuclock calls from cpufreq

msm-cpufreq has moved to use generic clock APIs for getting and
setting clock frequency. Remove the usage of deprecated acpuclock
APIs.

Change-Id: Ia05a86a58e031a3ec4ea59cb00fba1c33cddc78b
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

msm: cpufreq: Remove use of cpu_is_* API

cpu_is_* API is deprecated. Relevant targets now rely on is_sync for
proper cpufreq mask setup. Remove unnecessary cpu_is_* calls from
cpufreq.c

Change-Id: I326d24c24f37b34fba36dbf9fc08aa5071180b51
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

qcom-cpufreq: Block hotplug until cpufreq is ready

Hotplug before qcom-cpufreq is ready could lead to inconsistent CPU
clock state. Block hotplug by returning NOTIFY_BAD in hotplug callback
until qcom-cpufreq is probed.

Change-Id: I72a2f98c083c9b21b95ecafdb5a5be7a7682e842
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

qcom-cpufreq: Fix hotplug blocking logic

Change 2b63b10 qcom-cpufreq: Block hotplug until cpufreq is ready
intended to block hotplug until qcom-cpufreq driver has got all
clocks. However, its implementation still leaves a small window when
CPU 0 has got its clock but others haven't. If a hotplug happens on
another CPU that doesn't have its clock pointer, refcount of that CPU
clock will again be wrong.

Fix the issue by introducing a ready flag.

Change-Id: I1e05b063b0584088f717b82709a3f1d55bc6561b
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

qcom-cpufreq: Remove use of device_suspended in the hotplug path

The CPU_DOWN_PREPARE notifier in cpufreq.c already takes care
of performing a CPUFREQ_GOV_STOP, which waits for in-progress
frequency switches to complete and prevents any additional
frequency switches from being performed. Remove the redundant
code for accomplishing the same thing from qcom-cpufreq.

Besides being good cleanup, this change should also resolve
a potential deadlock in out-of-memory conditions. The scenario
arises if CPU1, executing in msm_cpufreq_target(), holds the
qcom-cpufreq suspend_mutex while waiting for the completion
at the end of that same function. In an out-of-memory condition,
the work performing that completion may not be able to
immediately run.

At the same time, CPU2 may be executing CPU_DOWN_PREPARE hotplug
notifiers. It will be holding the global hotplug mutex at this
time, and will eventually become blocked in
msm_cpufreq_cpu_callback() while waiting for the CPU1 to release
the device_suspended lock.

If invoked, the out-of-memory killer will attempt to acquire
the hotplug mutex via get_online_cpus() as part of
out_of_memory().  This lock is already held by CPU2. CPU2 will
not release the lock until the device_suspended lock is released
by CPU1. The device_suspended lock will not be release until the
completion in msm_cpufreq_target() is marked as complete, which
won't happen until the out-of-memory killer is able to free
some memory. Nasty deadlock.

Clearly, the out-of-memory condition itself needs to be resolved,
but there is no harm in resolving the deadlock also.

Change-Id: I5f59864aeb12a8c2aa81de16446af09a6d514da4
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>

cpufreq: msm: Clean up suspend hooks

Change-Id: I241ecff962546f175c393283e5635f921c963525

PM / devfreq: Move cpubw device from msm_cpufreq to cpufreq governor

Enable the cpufreq governor for the cpubw device on all targets by:
- Adding the devfreq-cpufreq node for mapping CPU freq to cpubw
- Enabling the cpufreq governor configs when CPUBW config is selected
- Switching the default governor for cpubw device to "cpufreq"

Change-Id: Ic10b645123192ed13c66340f288d6a8553d6f251
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

Conflicts:
	arch/arm/boot/dts/msm8974.dtsi

cpufreq: Delete unused CPUBW and cache scaling code from cpufreq driver

The devfreq cpufreq governor and the devfreq devices for CPUBW and cache
now handle CPUBW and cache frequency scaling based on CPU frequency. So,
delete all the unused code in the cpufreq driver that relates to scaling
CPUBW and cache.

Also update the msm-cpufreq DT format/data accordingly.

Change-Id: I30bf7d8478c964ffa300dd78039cd8c280ec63fc
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

drivers: qcom-cpufreq: Move disabling of clocks to CPU_DYING

Some SoCs contain CPU clock trees the elements of which are
gated off when the CPUs enter power collapse. This includes
sources of glitch free muxes; therefore when the CPUs enter
power collapse, those muxes cannot be switched.

Now in the cpufreq driver, the CPU clocks are disabled in
the CPU_DEAD notifier, which implies that the CPU muxes
are switched to a safe source *after* the CPUs are power
collapsed. However, the source of the GFMUX may already be
turned off, causing the mux to get stuck.

Ideally, the mux should allow a static switch, since the
clock to the CPU is gated. Some implementations do not
allow this.

Change-Id: I37d3426f20250c59756a0b55d1284efad5359a23
Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>

qcom-cpufreq: Allow all CPUs to get cpu_clk on targets with sync CPUs

For targets with synchronous CPU clocks, only CPU0 has a clock pointer in
cpu_clks[]. This forces CPU0 to be the last CPU to be hotplugged out, or go
to certain low power mode that needs to disable CPU clocks.

Remove this requirement by adding clock pointer for all CPUs. Drivers
that ask for CPU clocks will get same clock for CPUs in sync.

Change-Id: Ib97af2c3b0fdbc8dbb78364178225cc64ed02036
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

qcom-cpufreq: Remove per-cpu workqueue

It's no longer a requirement to pin frequency change on the CPU that
is being scaled. Therefore, there is no longer a need for per-cpu
workqueue in qcom-cpufreq. Remove the workqueue.

Change-Id: Ic6fd7f898fa8b1b1226a178b04530c24f0398daa
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>

cpufreq: Manage only online cpus

cpufreq core doesn't manage offline cpus and if driver->init() has returned
mask including offline cpus, it may result in unwanted behavior by cpufreq core
or governors.

We need to get only online cpus in this mask. There are two places to fix this
mask, cpufreq core and cpufreq driver. It makes sense to do this at common place
and hence is done in core.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

cpufreq: Notify governors when cpus are hot-[un]plugged

Because cpufreq core and governors worry only about the online cpus, if a cpu is
hot [un]plugged, we must notify governors about it, otherwise be ready to expect
something unexpected.

We already have notifiers in the form of CPUFREQ_GOV_START/CPUFREQ_GOV_STOP, we
just need to call them now.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

cpufreq: Simplify cpufreq_add_dev()

Currently cpufreq_add_dev() firsts allocates policy, calls
driver->init() and then checks if this CPU is already managed or not.
And if it is already managed, its policy is freed.

We can save all this if we somehow know that CPU is managed or not in
advance.  policy->related_cpus contains the list of all valid sibling
CPUs of policy->cpu. We can check this to see if the current CPU is
already managed.

From now on, platforms don't really need to set related_cpus from
their init() routines, as the same work is done by core too.

If a platform driver needs to set the related_cpus mask with some
additional CPUs, other than CPUs present in policy->cpus, they are
free to do it, though, as we don't override anything.

[rjw: Changelog]
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Conflicts:
	drivers/cpufreq/cpufreq.c
	drivers/cpufreq/spear-cpufreq.c

Change-Id: I9dbe6dc4ee385d2d0dfa1bd2c182cbaaa118cd06

cpufreq: interactive: Remove unnecessary cpu_online() check

Cpufreq no longer calls governor callback for offlined cpus. i.e. All
policy->cpus are guaranteed to be online. Hence we don't need explicit check to
see if cpu is online or not.

Change-Id: I9ad85ea4addd5b4a40952e59ed730dd15e328690
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>

PM / devfreq: Refactor CPUBW HWmon governor to be more generic

The refactored governor will be called bw_hwmon since it's no longer
limited to monitoring the CPU BW in hardware.

The refactor allows the governor to support multiple devfreq devices. This
is done by having different HW monitor instances register their capability
to monitor different devfreq devices and then picking the right HW monitor
based on which devfreq device is using this governor.

The refactor also fixes a very unlikely race condition that can happen when
starting/stopping the governor.

Change-Id: Ifeeeb256115f6ae3075f391d9b4b02666e06f737
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

Conflicts:
	drivers/devfreq/Kconfig
	drivers/devfreq/governor_cpubw_hwmon.c

PM / devfreq: bw_hwmon: Move IRQ handling to device specific drivers

The bandwidth monitoring devices might have more than one IRQ to handle or
might have notifications from other drivers instead of using actual IRQs.
So, refactor the governor to move the IRQ handling to the bandwidth
monitoring device specific drivers and just provide an API that can be used
to request a re-evaluation.

The device specific driver can call this API to request an immediate
re-evaluation whenever the bandwidth has exceeded the previously set limit
instead of waiting for the periodic update.

Change-Id: I8bfd85d0cb59a1fd0116000aac430680e483b422
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Make the CPUBW voting device driver to be more generic

Making the driver more generic would allow it to be reused for voting for
BW requirements of other devices like GPU, etc.

Instead of cpubw and qcom,cpubw, the driver and the device are now called
devbw and qcom,devbw respectively.

Change-Id: I49e5a5ba6d46517da19d013413abd471a730af29
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Add MSM BIMC bwmon support for bw_hwmon governor

The BIMC bwmon device supports monitoring read/write traffic from each BIMC
master port. It also has the capability to raise an IRQ when the traffic
count exceeds a programmable threshold. This allows for it to be used with
the bw_hwmon governor to scale the BW requests from each BIMC master.

Change-Id: Ie8a1471226411e23954ed556292186a5a864ddc1
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

Conflicts:
	arch/arm64/Kconfig

PM / devfreq: governor_bw_hwmon: Find the correct hwmon for a devfreq dev

The governor pointer match is only present to support legacy BW HW monitors
(Krait) to maintain the legacy governor name. The newer BW HW monitor
devices do the matching using device pointer or of_node. So, the governor
pointer should not be checked if the BW HW monitor devices provide device
or of_node pointers.

Change-Id: Ic48f53e39a9b176b7dec415489bd1657d0df02e2
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Revert addition of devfreq_governor_data and related changes

This reverts commit 0e5b3352b5f04919a250bba695444c3a29672525 and fixes
other code that resulted from it. The devfreq_governor_data field is not
needed since the governors can use other means (like container_of) to find
out device specific private data

Change-Id: Ib7e5db29b40d9676bbb5cf6057e0072665ace821
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

PM / devfreq: Get priv data for msm-adreno-tz governor from dev_profile

The devfreq infrastructure has several bugs and general design flaws
with private governor data support. As a workaround the msm-adreno-tz
governor gets it's private data from the device driver by doing a
container_of on the dev_profile registered by the device driver.

Change-Id: Ib9dffd063fe3aa5cfd55727d41f7633e88b3f998
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>

cpufreq: Notify all policy->cpus in cpufreq_notify_transition()

policy->cpus contains all online cpus that have single shared clock line. And
their frequencies are always updated together.

Many SMP system's cpufreq drivers take care of this in individual drivers but
the best place for this code is in cpufreq core.

This patch modifies cpufreq_notify_transition() to notify frequency change for
all cpus in policy->cpus and hence updates all users of this API.

Change-Id: Ib8699ed8fd47e0e90fadf00a4d966d8df9b62747
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

cpufreq: Don't check cpu_online(policy->cpu)

policy->cpu or cpus in policy->cpus can't be offline anymore. And so we don't
need to check if they are online or not.

Change-Id: Iade8855dfef39ce734bf0f3b7e241275d0f7107f
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>