cpufreq: interactive: Fix null pointer dereference in interactive governor

For the sync_freq feature currently we check pcpu->policy->cur frequency
for each online cpu.  But for a CPU that isn't using interactive governor
or for an offline CPU, pcpu->policy can be null or an invalid value.
This patch tries to avoid that scenario by using pcpu->target_freq
instead of policy->cur to get the frequency of an online CPU.

Kernel crash without this patch:
[   20.132373] Unable to handle kernel NULL pointer dereference at virtual address 00000028
[   20.132375] pgd = c34f34c0
[   20.132377] pgd = ef6f2440
[   20.132383] [00000028] *pgd=00000000
[   20.132385]
[   20.132388] [00000028] *pgd=2e98f003, *pmd=00000000
[   20.132390] Internal error: Oops: 205 [#1] PREEMPT SMP ARM
[   20.132394] Modules linked in:
[   20.132398] CPU: 0 PID: 1560 Comm: chown Tainted: G        W    3.10.0-perf-gb12057b-00001-ga2c6c16-dirty #7
[   20.132401] task: ef9af300 ti: ee49c000 task.ti: ee49c000
[   20.132411] PC is at cpufreq_interactive_timer+0x10c/0x650
[   20.132415] LR is at cpufreq_interactive_timer+0x128/0x650
<snip>
[   20.133002] [<c07eb204>] (cpufreq_interactive_timer+0x10c/0x650) from [<c02804d8>] (call_timer_fn+0x80/0x198)
[   20.133012] [<c02804d8>] (call_timer_fn+0x80/0x198) from [<c0280acc>] (run_timer_softirq+0x1f8/0x270)
[   20.133019] [<c0280acc>] (run_timer_softirq+0x1f8/0x270) from [<c0279e20>] (__do_softirq+0x12c/0x2d4)
[   20.133025] [<c0279e20>] (__do_softirq+0x12c/0x2d4) from [<c027a2d4>] (irq_exit+0x74/0xc8)
[   20.133034] [<c027a2d4>] (irq_exit+0x74/0xc8) from [<c0206a00>] (handle_IRQ+0x68/0x8c)
[   20.133041] [<c0206a00>] (handle_IRQ+0x68/0x8c) from [<c02004b8>] (gic_handle_irq+0x3c/0x60)
[   20.133051] [<c02004b8>] (gic_handle_irq+0x3c/0x60) from [<c0ac6900>] (__irq_svc+0x40/0x70)
<snip>

Change-Id: Ie834f5d383de4d41e0fe6fbd40c8b0a0c05d82f5
Signed-off-by: Vijay Ganti <viganti@codeaurora.org>
softirq: Use hotplug thread infrastructure

[ paulmck: Call rcu_note_context_switch() with interrupts enabled. ]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/20120716103948.456416747@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
softirq: reduce latencies

In various network workloads, __do_softirq() latencies can be up
to 20 ms if HZ=1000, and 200 ms if HZ=100.

This is because we iterate 10 times in the softirq dispatcher,
and some actions can consume a lot of cycles.

This patch changes the fallback to ksoftirqd condition to :

- A time limit of 2 ms.
- need_resched() being set on current task

When one of this condition is met, we wakeup ksoftirqd for further
softirq processing if we still have pending softirqs.

Using need_resched() as the only condition can trigger RCU stalls,
as we can keep BH disabled for too long.

I ran several benchmarks and got no significant difference in
throughput, but a very significant reduction of latencies (one order
of magnitude) :

In following bench, 200 antagonist "netperf -t TCP_RR" are started in
background, using all available cpus.

Then we start one "netperf -t TCP_RR", bound to the cpu handling the NIC
IRQ (hard+soft)

Before patch :

# netperf -H 7.7.7.84 -t TCP_RR -T2,2 -- -k
RT_LATENCY,MIN_LATENCY,MAX_LATENCY,P50_LATENCY,P90_LATENCY,P99_LATENCY,MEAN_LATENCY,STDDEV_LATENCY
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
to 7.7.7.84 () port 0 AF_INET : first burst 0 : cpu bind
RT_LATENCY=550110.424
MIN_LATENCY=146858
MAX_LATENCY=997109
P50_LATENCY=305000
P90_LATENCY=550000
P99_LATENCY=710000
MEAN_LATENCY=376989.12
STDDEV_LATENCY=184046.92

After patch :

# netperf -H 7.7.7.84 -t TCP_RR -T2,2 -- -k
RT_LATENCY,MIN_LATENCY,MAX_LATENCY,P50_LATENCY,P90_LATENCY,P99_LATENCY,MEAN_LATENCY,STDDEV_LATENCY
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
to 7.7.7.84 () port 0 AF_INET : first burst 0 : cpu bind
RT_LATENCY=40545.492
MIN_LATENCY=9834
MAX_LATENCY=78366
P50_LATENCY=33583
P90_LATENCY=59000
P99_LATENCY=69000
MEAN_LATENCY=38364.67
STDDEV_LATENCY=12865.26

Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: David Miller <davem@davemloft.net>
Cc: Tom Herbert <therbert@google.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
msm: dma: Moving queue_work() function within spinlock

Currently we are adding a dma command to the staged list with in a
spinlock and then adding to workqueue using queue_work after unlocking
the spinlock. With this there is chance of executing DMA commands in out
of order in below concurrency case.

Tread1                                Thread2
__msm_dmov_enqueue_cmd_ext
   spin_lock_irqsave(..)
   list_add_tail(..)
   spin_unlock_irqrestore(..)
       --PREEMPT--
                                      __msm_dmov_enqueue_cmd_ext
                                         spin_lock_irqsave(..)
                                         list_add_tail(..)
                                         spin_unlock_irqrestore(..)
                                         queue_work()
                                         ..
   queue_work()
So adding queue_work with in spin_lock will make sure that the
work added in the work_queue is processed in the same order as they
are added in staged_commands.

CRs-Fixed: 423190
Change-Id: I2ffd1327fb5f0cd1f06db7de9c026d1c4997fe4d
Acked-by: Gopi Krishna Nedanuri <gnedanur@qti.qualcomm.com>
Signed-off-by: Utsab Bose <ubose@codeaurora.org>
msm: dma: Refactored driver to avoid queuing of cmds in workq.

Highlights of the changes are:
1. A workqueue is used to configure ADM only when
   the clocks need to be prepared & enabled.
   A function call is used to configure ADM in
   cases where the ADM driver is invoked for data
   transfer when the clock is already prepared &
   enabled.
2. Replaced threaded irqs with hard irqs.

Change-Id: Ifaa5efdde8150f932a12d4f9eeccfb36ecb8f88f
Acked-by: John Nicholas <jnichola@qti.qualcomm.com>
Signed-off-by: Venkatesh Yadav Abbarapu <quicvenkat@codeaurora.org>
rcu: Yield simpler

The rcu_yield() code is amazing. It's there to avoid starvation of the
system when lots of (boosting) work is to be done.

Now looking at the code it's functionality is:

 Make the thread SCHED_OTHER and very nice, i.e. get it out of the way
 Arm a timer with 2 ticks
 schedule()

Now if the system goes idle the rcu task returns, regains SCHED_FIFO
and plugs on. If the systems stays busy the timer fires and wakes a
per node kthread which in turn makes the per cpu thread SCHED_FIFO and
brings it back on the cpu. For the boosting thread the "make it FIFO"
bit is missing and it just runs some magic boost checks. Now this is a
lot of code with extra threads and complexity.

It's way simpler to let the tasks when they detect overload schedule
away for 2 ticks and defer the normal wakeup as long as they are in
yielded state and the cpu is not idle.

That solves the same problem and the only difference is that when the
cpu goes idle it's not guaranteed that the thread returns right away,
but it won't be longer out than two ticks, so no harm is done. If
that's an issue than it is way simpler just to wake the task from
idle as RCU has callbacks there anyway.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Namhyung Kim <namhyung@kernel.org>
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20120716103948.131256723@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
rcu: Use smp_hotplug_thread facility for RCUs per-CPU kthread

Bring RCU into the new-age CPU-hotplug fold by modifying RCU's per-CPU
kthread code to use the new smp_hotplug_thread facility.

[ tglx: Adapted it to use callbacks and to the simplified rcu yield ]

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/20120716103948.673354828@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
drivers: cpufreq: Send a uevent when governor changes

 * Useful so userspace tools can reconfigure.

Change-Id: Ib423910b8b9ac791ebe81a75bf399f58272f64f2
cpufreq: Fix broken uevents for cpufreq governor and cpu devices

cyanogens uevent commit when the governor changes was rendered
non-working since kitkat (or so) as the uevent filter function
caused events to be dropped. Now hook into cpufreq_core_init()
and cpufreq_add_dev_interface() to create our basic ksets for
cpufreq and cpu devices. Also, we don't need to set environmental
data, so clean it up a bit.

This commit requires a change in ueventd.rc that add rules for
several files of interest.

Change-Id: I3aafa0d4e18363e1d68535f513099ecd27024007
msm: clock-rpm: Make rpm clocks sleeping clocks

Now that we have clk_prepare/unprepare we can make the RPM clocks
sleepable. This allows us to move the sometimes costly busy wait
that RPM clocks incur when enabling and disabling or changing
rates.

CRs-Fixed: 552223

Change-Id: I8ac53c0b7fc79e56051b19fedb6910ac3f1cda42
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Git-commit: b500badb5dc821dd92f93833003170cb9ae106b0
Git-repo: https://android.googlesource.com/kernel/msm/
Signed-off-by: Srinivasarao P <spathi@codeaurora.org>
sched,cgroup: Fix up task_groups list

With multiple instances of task_groups, for_each_rt_rq() is a noop,
no task groups having been added to the rt.c list instance.  This
renders __enable/disable_runtime() and print_rt_stats() noop, the
user (non) visible effect being that rt task groups are missing in
/proc/sched_debug.

Change-Id: Ib337ebda0e752f838d787c03fe1b0a8f03ea0551
Signed-off-by: Mike Galbraith <efault@gmx.de>
Cc: stable@kernel.org # v3.3+
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1344308413.6846.7.camel@marge.simpson.net
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Git-commit: 35cf4e50b16331def6cfcbee11e49270b6db07f5
Git-repo: git://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git
Signed-off-by: Syed Rameez Mustafa <rameezmustafa@codeaurora.org>
MDP: correct clock calculation logic for up-scaling scenario

Part of yscale calculation logic in mdp4_calc_req_mdp_clk
function is only applicable for downscale scenarios. The
incorrect clock calculation was unnecessarily creating
blt mode requirement even for upscale scenarios.

Change-Id: I036be2ceb2f59fb159885b2d15b208706808198b
Signed-off-by: Nirmal Abraham <nabrah@codeaurora.org>
Merge branch 'cm-11.0' of https://github.com/CyanogenMod/android_kernel_google_msm into kk4.4
cpufreq: ondemand: Remove unnecessary synchronization attempt

Commit 1ce28d6b (ondemand: updated tune for hardware coordination)
brings in an attempt to synchronize expiry of per-cpu ondemand timer
at same time across all online cpus. This is meant to help systems
where "multiple cores share P-state via Hardware Coordination".

Synchronization is achieved by having timers expire when jiffy value
would be a perfect multiple of sampling window size. While such
synchronozation attemps are unnecessary on systems where cores don't
share P-states, it can hurt on power as cpus can now work off shorter
sampling windows (when percentage utilization over short window can be
observed to be high, causing frequency to be boosted).

Remove this unnecessary attempt to synchronize expiry of ondemand
timer across online cpus. A more acceptable solution could be to have
low-level architecture specific cpufreq driver indicate the need for
synchronization.

Change-Id: I6cb1cf3472ca106f029dc01248e0df7cfbb495b8
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
cpufreq: interactive: delete timers for GOV_START

Make sure that timers cpu_timer and cpu_slack_timer
deactivated before addition of new.

Change-Id: If31c4049606871df6f00efdc24b1d713c86a6f69
Signed-off-by: Shridhar Rasal <srasal@nvidia.com>
Signed-off-by: Bharat Nihalani <bnihalani@nvidia.com>
Git-commit: b430123367e78a6557bac3cf1558bcb85193fb12
Git-repo: https://android.googlesource.com/kernel/common/
[mattw@codeaurora.org: resolved trivial context conflict]
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
cpufreq: Sync on thread migration optimizations

When threads migrate between CPUs the destination CPU will temporarily
have a boosted frequency and the source CPU will have a residual higher
frequency even though the load could have just been reduced. To avoid
leaving CPUs at a high frequencies notify the source CPU after a thread
migration and make the interactive governor's timer reschedule
conditionally.

Change-Id: Id55055f2a116ca7ffd2a34dc3d0ec519c888a111
Signed-off-by: Patrick Cain <pcain@codeaurora.org>
cpufreq: interactive: Don't reset sampling down factor at max freq

A previous patch sets the min_sample_time to zero if the cpufreq
min/max limits have changed which would bring down the frequency
in the next sample even if min_sample_time doesnt allow it to do so.
This could severly affect benchmarks if this is applied to max
frequencies as min_sample_time for max frequencies is set to
sampling_down_factor.
This patch resets the limits_changed flag for cases where timer fires
while running at max frequencies so that sampling_down_factor is
still applicable as before.

Change-Id: I397133cb15eea4ab999d0f69617b4922438c6e11
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
cpufreq: interactive: Remove unnecessary cpu_online() check

Cpufreq no longer calls governor callback for offlined cpus. i.e. All
policy->cpus are guaranteed to be online. Hence we don't need explicit check to
see if cpu is online or not.

Change-Id: I9ad85ea4addd5b4a40952e59ed730dd15e328690
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
cpufreq: cpu-boost: Re-issue boosts above minimum frequency

Frequency boosts where the source CPU frequency is greater than CPU's
minimum frequency should always go through regardless of the destination
CPU's current frequency. This fixes a performance issue where the governor
lowers the CPU frequency shortly after a thread is migrated to it because
the boost wasn't re-issued.

Change-Id: I449545a688d84b0a6e834f5a51dcb499caa84d29
Signed-off-by: Patrick Cain <pcain@codeaurora.org>
msm: cpufreq: Fix msm_cpufreq_get_freq() for sync CPUs

When CPUs are in sync, only CPU0 has a non-NULL cpu_clk. clk_get_rate()
on other cpu_clk will return 0. msm_cpufreq_get_freq() is used in multiple
places in cpufreq framework to determine current hardware frequency.

Fix msm_cpufreq_get_freq() to return the right hardware frequency for
systems with synchronous CPUs.

Change-Id: I6a880764ad51fa9a257a328798b38d574750686a
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>