softirq: Use hotplug thread infrastructure

[ paulmck: Call rcu_note_context_switch() with interrupts enabled. ]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/20120716103948.456416747@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
rcu: Yield simpler

The rcu_yield() code is amazing. It's there to avoid starvation of the
system when lots of (boosting) work is to be done.

Now looking at the code it's functionality is:

 Make the thread SCHED_OTHER and very nice, i.e. get it out of the way
 Arm a timer with 2 ticks
 schedule()

Now if the system goes idle the rcu task returns, regains SCHED_FIFO
and plugs on. If the systems stays busy the timer fires and wakes a
per node kthread which in turn makes the per cpu thread SCHED_FIFO and
brings it back on the cpu. For the boosting thread the "make it FIFO"
bit is missing and it just runs some magic boost checks. Now this is a
lot of code with extra threads and complexity.

It's way simpler to let the tasks when they detect overload schedule
away for 2 ticks and defer the normal wakeup as long as they are in
yielded state and the cpu is not idle.

That solves the same problem and the only difference is that when the
cpu goes idle it's not guaranteed that the thread returns right away,
but it won't be longer out than two ticks, so no harm is done. If
that's an issue than it is way simpler just to wake the task from
idle as RCU has callbacks there anyway.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Namhyung Kim <namhyung@kernel.org>
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20120716103948.131256723@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
rcu: Use smp_hotplug_thread facility for RCUs per-CPU kthread

Bring RCU into the new-age CPU-hotplug fold by modifying RCU's per-CPU
kthread code to use the new smp_hotplug_thread facility.

[ tglx: Adapted it to use callbacks and to the simplified rcu yield ]

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/20120716103948.673354828@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
cpufreq: Avoid calling cpufreq driver's target() routine if target_freq == policy->cur

Avoid calling cpufreq driver's target() routine if new frequency is same as
policies current frequency.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
cpufreq: interactive: Reset to kk_2.7 branch

* Going to upstream.

Signed-off-by: Zhao Wei Liew <zhaoweiliew@gmail.com>
Revert "cpufreq: fix interactive bug during hotplug"

This reverts commit 940d4b98eab2577d6c1e710f2939a2b0b6b3f028.

Change-Id: I16ec442f510c45951907417a5c87cd2e31c6c645
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: take idle notifications only when active

Register an idle notifier only when the governor is active.  Also
short-circuit work of idle end if the governor is not enabled.

Signed-off-by: Sam Leffler <sleffler@chromium.org>
Change-Id: I4cae36dd2e7389540d337d74745ffbaa0131870f
Git-Commit: 22a51ab6e21de740c1b555f9a03626d28aaebdf3
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: keep freezer happy when not current governor

Fix a problem where the hung task mechanism was deeming the interactive
clock boost thread as hung.  This was because the thread is created at
module init but never run/woken up until needed.  If the governor is not
being used this can be forever.  To workaround this explicitly wake up
the thread once all the necessary data structures are initialized.  The
latter required some minor code shuffle.

Signed-off-by: Sam Leffler <sleffler@chromium.org>
Change-Id: Ie2c058dd75dcb6460ea10e7ac997e46baf66b1fe
Git-Commit: 5ff8407798f1d8c44f3165bc535676a2a897666b
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: handle speed up and down in the realtime task

Not useful to have a separate, non-realtime workqueue for speed down
events, avoid priority inversion for speed up events.

Change-Id: Iddcd05545245c847aa1bbe0b8790092914c813d2
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: f8aa05d96a0d18416d2627c493f7dc9bc5324901
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: remove input_boost handling

Now handled in userspace Power HAL instead.

Change-Id: I78a4a2fd471308bfcd785bbefcc65fede27314cf
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: 4e27eb56e819745a1b184f278d7f3dd3887a456c
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: always limit initial speed bump to hispeed

First bump speed up to hispeed_freq whenever the current speed is below
hispeed_freq, instead of only when the current speed is the minimum speed.
The previous code made it too difficult to use hispeed_freq as a common
intermediate speed on systems that frequently run at speeds between
minimum and hispeed_freq.

Change-Id: I04ec30bafabf5741e267ff289209b8c2d846824b
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: b47148d9c2cc2f63eca7c7211b825501b2980bc0
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: run at fraction of hispeed_freq when load is low

When load is below go_hispeed_load, apply the percentage of CPU load to
a max frequency of hispeed_freq instead of the max speed.  This avoids
jumping too quickly to hispeed_freq when it is a relatively low
percentage of max speed.  This also allows go_hispeed_load to be set to
a high percentage relative to hispeed_freq (as a percentage of max speed,
again useful when hispeed_freq is a low fraction of max speed), to cap
larger loads at hispeed_freq.  For example, a load of 60% will typically
move to 60% of hispeed_freq, not 60% of max speed.  This causes the
governor to apply two different speed caps, depending on whether load is
below or above go_hispeed_load.

Also fix the type of hispeed_freq, which was u64, to match other
speed data types (and avoid overhead and allow division).

Change-Id: Ie2d0668be161c074aaad77db2037505431457b3a
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: d0ee6d303202e020bb7798c5aadb9ae3ed9b8a7a
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: pin timers to associated CPU

Helps avoid waking up other CPUs to react to activity on the local CPU.

Change-Id: Ife272aaa7916894a437705d44521b1a1693fbe8e
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: 430d3d4539a0d441679ca32cc7c3876f97a07da6
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: use deferrable timer by default

Avoid wakeups only to handle the governor timer when the system is otherwise
idle.

For platforms where the power cost of remaining in idle at higher CPU
speed may outweigh the cost of a governor wakeup from idle to lower the speed,
set parameter cpufreq_interactive.governidle=1.

Change-Id: Id6c43eb35caecf9b0574fcdd5b769711bc7e6de6
Signed-off-by: LianWei WANG <a22439@motorola.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: 0697ab54cab34e099711ceb4b07b2dfc6b507f0d
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: kick timer on idle exit past expiry

The deferrable timer list isn't checked on all idle exits, such as when
hi-res timers expire or ISRs schedule workers.  If the idle loop is
exited and it's past time to run the governor load polling timer,
run it immediately.  This ensures we handle load spikes caused by actvity
that does not run the normal timer list.

Rename the field that timestamps the "time_in_idle" value to be more
accurate.

Change-Id: Ied590ecbefc83c9a9ec5eb9e31903557f6fa1614
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: 9b29a6d49becf8caf0550373d84a1068c716e891
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: trace actual speed in target speed decisions

Tracing adds actual speed since this is expected to be key to the
choice of target speed.

Change-Id: Iec936102d0010c4e9dfa143c38a9fd0d551189c3
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: 36d7db744e7cfad5358bec1769bc21b317d97d94
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: change speed according to current speed and target load

Add a target_load attribute that specifies how aggressively the governor is
to adjust speed to meet the observed load.  New target speed is calculated
as the current actual speed (may be higher than target speed on SMP) times
the CPU load (as a fraction) divided by target load (fraction).

cpufreq_frequency_table_target() call use CPUFREQ_RELATION_L to set
the next higher speed rather than next lower speed.

Change-Id: If432451da82f5fed12e15c9421d7d27792376150
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: 11ecccb7afea50ff6aa69665f3c73efa1deee8a7
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: apply above_hispeed_delay to each step above hispeed

Apply above_hispeed_delay whenever increasing speed to a new speed above
hispeed (not just the first step above hispeed).

Change-Id: Ibb7add7db47f2a4306a9458c4e1ebabb60698636
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: 35a84ded7c31c35becc178f85d066e5fd2de6fb5
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: allow arbitrary speed / target load mappings

Accept a string of target loads and speeds at which to apply the
target loads, per the documentation update in this patch.  For example,
"85 1000000:90 1700000:99" targets CPU load 85% below speed 1GHz,  90%
at or above 1GHz, until 1.7GHz and above, at which load 99% is targeted.

Attempt to avoid oscillations by evaluating the current speed
weighted by current load against each new choice of speed, choosing a
higher speed if the current load requires a higher speed.

Change-Id: Ie3300206047c84eca5a26b0b63ea512e5207550e
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: dac27a6c1805ba1aa176d6e948c00e41489afbc6
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>
cpufreq: interactive: remove load since last speed change

The longer-term load since last speed change isn't terribly useful,
may delay recognition of dropping load, and would need forthcoming
changes to adjust load for changing CPU speeds.  Drop it.

Change-Id: Ic3cbb0542cc3484617031787e03ed9bdd632dec1
Signed-off-by: Todd Poynor <toddpoynor@google.com>
Git-Commit: 29f10ea7fd02a721935c701e96a0e0fb024d75e7
Git-Repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Dilip Gudlur <dgudlur@codeaurora.org>