tick: Dynamically set broadcast irq affinity

When a cpu goes to a deep idle state where its local timer is
shutdown, it notifies the time frame work to use the broadcast timer
instead.  Unfortunately, the broadcast device could wake up any CPU,
including an idle one which is not concerned by the wake up at all. So
in the worst case an idle CPU will wake up to send an IPI to the CPU
whose timer expired.

Provide an opt-in feature CLOCK_EVT_FEAT_DYNIRQ which tells the core
that is should set the interrupt affinity of the broadcast interrupt
to the cpu which has the earliest expiry time. This avoids unnecessary
spurious wakeups and IPIs.

[ tglx: Adopted to cpumask rework, silenced an uninitialized warning,
  massaged changelog ]

Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: viresh.kumar@linaro.org
Cc: jacob.jun.pan@linux.intel.com
Cc: linux-arm-kernel@lists.infradead.org
Cc: santosh.shilimkar@ti.com
Cc: linaro-kernel@lists.linaro.org
Cc: patches@linaro.org
Cc: rickard.andersson@stericsson.com
Cc: vincent.guittot@linaro.org
Cc: linus.walleij@stericsson.com
Cc: john.stultz@linaro.org
Link: http://lkml.kernel.org/r/1362219013-18173-3-git-send-email-daniel.lezcano@linaro.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

Change-Id: I6a880a39dd595526b80a6d72b88be74163513da9
Signed-off-by: Karthik Parsha <kparsha@codeaurora.org>
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
cpuidle: make a single register function for all

The usual scheme to initialize a cpuidle driver on a SMP is:

	cpuidle_register_driver(drv);
	for_each_possible_cpu(cpu) {
		device = &per_cpu(cpuidle_dev, cpu);
		cpuidle_register_device(device);
	}

This code is duplicated in each cpuidle driver.

On UP systems, it is done this way:

	cpuidle_register_driver(drv);
	device = &per_cpu(cpuidle_dev, cpu);
	cpuidle_register_device(device);

On UP, the macro 'for_each_cpu' does one iteration:

        for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask)

Hence, the initialization loop is the same for UP than SMP.

Beside, we saw different bugs / mis-initialization / return code unchecked in
the different drivers, the code is duplicated including bugs. After fixing all
these ones, it appears the initialization pattern is the same for everyone.

Please note, some drivers are doing dev->state_count = drv->state_count. This is
not necessary because it is done by the cpuidle_enable_device function in the
cpuidle framework. This is true, until you have the same states for all your
devices. Otherwise, the 'low level' API should be used instead with the specific
initialization for the driver.

Let's add a wrapper function doing this initialization with a cpumask parameter
for the coupled idle states and use it for all the drivers.

That will save a lot of LOC, consolidate the code, and the modifications in the
future could be done in a single place. Another benefit is the consolidation of
the cpuidle_device variable which is now in the cpuidle framework and no longer
spread accross the different arch specific drivers.

Change-Id: I8f75188d5770871620f758c2973b74384ac570d5
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Karthik Parsha <kparsha@codeaurora.org>
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
ARM: mm: implement LoUIS API for cache maintenance ops

ARM v7 architecture introduced the concept of cache levels and related
control registers. New processors like A7 and A15 embed an L2 unified cache
controller that becomes part of the cache level hierarchy. Some operations in
the kernel like cpu_suspend and __cpu_disable do not require a flush of the
entire cache hierarchy to DRAM but just the cache levels belonging to the
Level of Unification Inner Shareable (LoUIS), which in most of ARM v7 systems
correspond to L1.

The current cache flushing API used in cpu_suspend and __cpu_disable,
flush_cache_all(), ends up flushing the whole cache hierarchy since for
v7 it cleans and invalidates all cache levels up to Level of Coherency
(LoC) which cripples system performance when used in hot paths like hotplug
and cpuidle.

Therefore a new kernel cache maintenance API must be added to cope with
latest ARM system requirements.

This patch adds flush_cache_louis() to the ARM kernel cache maintenance API.

This function cleans and invalidates all data cache levels up to the
Level of Unification Inner Shareable (LoUIS) and invalidates the instruction
cache for processors that support it (> v7).

This patch also creates an alias of the cache LoUIS function to flush_kern_all
for all processor versions prior to v7, so that the current cache flushing
behaviour is unchanged for those processors.

v7 cache maintenance code implements a cache LoUIS function that cleans and
invalidates the D-cache up to LoUIS and invalidates the I-cache, according
to the new API.

Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>

Change-Id: I561c3b187b5b87b25de07ea82a4ef73a42761e27
Signed-off-by: Karthik Parsha <kparsha@codeaurora.org>
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
timekeeping: Add missing update call in timekeeping_resume()

The leap second rework unearthed another issue of inconsistent data.

On timekeeping_resume() the timekeeper data is updated, but nothing
calls timekeeping_update(), so now the update code in the timer
interrupt sees stale values.

This has been the case before those changes, but then the timer
interrupt was using stale data as well so this went unnoticed for quite
some time.

Add the missing update call, so all the data is consistent everywhere.

CRs-fixed: 536881
Change-Id: I5f7d5219749a3f6330d074ef8866533ebda3bfca
Reported-by: Andreas Schwab <schwab@linux-m68k.org>
Reported-and-tested-by: "Rafael J. Wysocki" <rjw@sisk.pl>
Reported-and-tested-by: Martin Steigerwald <Martin@lichtvoll.de>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: Linux PM list <linux-pm@vger.kernel.org>
Cc: John Stultz <johnstul@us.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>,
Cc: Prarit Bhargava <prarit@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: John Stultz <johnstul@us.ibm.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Git-commit: 3e997130bd2e8c6f5aaa49d6e3161d4d29b43ab0
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
timekeeping: utilize the suspend-nonstop clocksource to count suspended time

There are some new processors whose TSC clocksource won't stop during
suspend. Currently, after system resumes, kernel will use persistent
clock or RTC to compensate the sleep time, but with these nonstop
clocksources, we could skip the special compensation from external
sources, and just use current clocksource for time recounting.

This can solve some time drift bugs caused by some not-so-accurate or
error-prone RTC devices.

The current way to count suspended time is first try to use the persistent
clock, and then try the RTC if persistent clock can't be used. This
patch will change the trying order to:
	suspend-nonstop clocksource -> persistent clock -> RTC

When counting the sleep time with nonstop clocksource, use an accurate way
suggested by Jason Gunthorpe to cover very large delta cycles.

CRs-fixed: 536881
Change-Id: I59c21b1ce5ae81cebc8cf3e817d9b3f951302a3d
Signed-off-by: Feng Tang <feng.tang@intel.com>
[jstultz: Small optimization, avoiding re-reading the clocksource]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Git-commit: e445cf1c4257cc0238d72e4129eb4739f46fd3de)
[sboyd: Took tk and clock local variables]
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
clocksource: Add new feature flag CLOCK_SOURCE_SUSPEND_NONSTOP

Some x86 processors have a TSC clocksource, which continues to run
even when system is suspended. Also most OMAP platforms have a
32 KHz timer which has similar capability. Add a feature flag so that
it could be utilized.

CRs-fixed: 536881
Change-Id: I863678fec5a3021158240f2c5b144c552ba0b16a
Signed-off-by: Feng Tang <feng.tang@intel.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Git-commit: 5caf4636259ae3af0efbb9bfc4cd97874b547c7d
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
clocksource: arm_arch_timer: Use clocksource for suspend timekeeping

The ARM architected timers keep counting during suspend so we can
mark this clocksource with the CLOCK_SOURCE_SUSPEND_NONSTOP. This
flag will indicate that this clocksource can be used for
calculating suspend time and injecting sleep time into the
timekeeping core. This will usually be more accurate than using
an external RTC or architecture specific persistent clock.

CRs-fixed: 536881
Change-Id: I882bf19a87ceb642aa069bbf26802c66c96b4426
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
msm: pm-8x60: Fix suspend time tracking

Calls to sched_clock() after syscore devices have been suspend
will return the same value until the syscored devices are
resumed. Furthermore, the value won't jump forward to account for
the time left in suspend so we can't use it to calculate how much
time we spent in suspend.

We could expose the hardware directly to the pm code but it's
better to use generic interfaces. Move to using the timekeeping
APIs to determine how long we were in suspend. Unfortunately, we
can't use the timekeeping APIs this early in the resume path, so
we must read the current time of day before we suspend the
timekeeping core and after we resume the timekeeping core. Move
the calls to the suspend_late() and wake() callbacks so we can
properly read the amount of suspend time.

CRs-fixed: 536881
Change-Id: I3b47511afb75926e51e82fc83929f405404fbf07
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
tracing/sched: add load balancer tracepoint

When doing performance analysis it can be useful to see exactly
what is going on with the load balancer - when it runs and why
exactly it may not be redistributing load.

This additional tracepoint will show the idle context of the
load balance operation (idle, not idle, newly idle), various
values from the load balancing operation, the final result,
and the new balance interval.

Change-Id: I9e5c97ae3878bea44e60d189ff3cec2275f2c75e
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
sched: Fix clear NOHZ_BALANCE_KICK

I have faced a sequence where the Idle Load Balance was sometime not
triggered for a while on my platform, in the following scenario:

 CPU 0 and CPU 1 are running tasks and CPU 2 is idle

 CPU 1 kicks the Idle Load Balance
 CPU 1 selects CPU 2 as the new Idle Load Balancer
 CPU 2 sets NOHZ_BALANCE_KICK for CPU 2
 CPU 2 sends a reschedule IPI to CPU 2

 While CPU 3 wakes up, CPU 0 or CPU 1 migrates a waking up task A on CPU 2

 CPU 2 finally wakes up, runs task A and discards the Idle Load Balance
       task A quickly goes back to sleep (before a tick occurs on CPU 2)
 CPU 2 goes back to idle with NOHZ_BALANCE_KICK set

Whenever CPU 2 will be selected as the ILB, no reschedule IPI will be sent
because NOHZ_BALANCE_KICK is already set and no Idle Load Balance will be
performed.

We must wait for the sched softirq to be raised on CPU 2 thanks to another
part the kernel to come back to clear NOHZ_BALANCE_KICK.

The proposed solution clears NOHZ_BALANCE_KICK in schedule_ipi if
we can't raise the sched_softirq for the Idle Load Balance.

Change since V1:

- move the clear of NOHZ_BALANCE_KICK in got_nohz_idle_kick if the ILB
  can't run on this CPU (as suggested by Peter)

Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1370419991-13870-1-git-send-email-vincent.guittot@linaro.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Git-commit: 873b4c65b519fd769940eb281f77848227d4e5c1
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[smuckle@codeaurora.org: minor merge resolution for 3.4 in scheduler_ipi()]
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
Change-Id: I3548612057cccc2ecc29429c129c44183083831f
msm: kgsl: Print a message when tz_init fails

When the tz_init fails to initialize the GPU busy percentage
based algorithm, print an error message and revert to the old
algorithm.

Change-Id: I55a09db7d6f33f6496584fd24c9cfc8a70d13e3d
Signed-off-by: Suman Tatiraju <sumant@codeaurora.org>
msm: kgsl: convert pwrscale framework to use devfreq

Convert the clock frequency scaling infrastructure to
be based on devfreq.

Change-Id: I1a60ba339db5715a8836b835bd1b29b46e151af6
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
msm: kgsl: make msm-adreno-tz the default governor for adreno

On targets where trustzone is available, use the trustzone
based governor instead of simple_ondemand.

Change-Id: Ie11c57684fc63a26e82a5c4816a35a327a1bf945
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
PM / devfreq: improve devfreq->data support

This field is treated as governer specific data for
a devfreq instance. But there's currently no way to
set the correct data when switching governors through
sysfs. Add support for optionally passing a set of
name / data pairs in struct devfreq_dev_profile,
representing the data for each governor.

Change-Id: I5523ce94f8b0045974f0635fb734cb1282512f91
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
PM / devfreq: allow governors to use devfreq_get_freq_level

Level based governors may need to perform this lookup to
interpret the current frequency of the device.

Change-Id: Idf7246b05775a52f088c52b898d98fbab4fd942c
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
PM / devfreq: fix find_governor_data() function

find_governor_data() was added to the devfreq infrastructure earlier.
It uses an incorrect condition in an if-statement when it searches
a governor by its name. Fixing the condition.

Change-Id: Ib297caad3e23984651217a43d2fdde1f6fcbdf7a
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
PM / devfreq: add msm-adreno-tz governor

This governor can be used to control adreno GPUs.
It is unlikely to be useful for other devices.

Change-Id: Icf481322454b814d2f41019f2f01286062409952
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
PM / devfreq: Allow the governors to set the target flag

The devfreq framework calls a frequency targeting function with
a flag parameter.  Allow the governors to influence that parameter.

Change-Id: I4058bd9dcd027dd246ccdb90d25c68f1dc055901
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>
msm: kgsl: convert pwrscale to use bus_mod

Use a bus specific speed flag rather than overloading
the least upper bound flag.

Change-Id: I343726e6e0d9885f39c343ed56c1667ab2008aee
Signed-off-by: Lucille Sylvester <lsylvest@codeaurora.org>
msm: kgsl: Don't spam the kernel log if qcom,step-pwrlevel isn't found

qcom,step-pwrlevel is a specific parameter that only applies to
certain targets.  Don't spam the world if it isn't specified.

CRs-fixed: 556418
Change-Id: Ic0dedbad65dbdcf9bc227fef4a796e22131ba68f
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>