sched: Make scheduler aware of cpu frequency state

Capacity of a cpu (how much performance it can deliver) is partly
determined by its frequency (P) state, both current frequency as well
as max frequency it can reach.  Knowing frequency state of cpus will
help scheduler optimize various functions such as tracking every
task's cpu demand and placing tasks on various cpus.

This patch has scheduler registering for cpufreq notifications to
become aware of cpu's frequency state. Subsequent patches will make
use of derived information for various purposes, such as task's scaled
load (cpu demand) accounting and task placement.

Change-Id: I376dffa1e7f3f47d0496cd7e6ef8b5642ab79016
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
sched: window-based load stats for tasks

Provide a metric per task that specifies how cpu bound a task is. Task
execution is monitored over several time windows and the fraction of
the window for which task was found to be executing or wanting to run
is recorded as task's demand. Windows over which task was sleeping are
ignored. We track last 5 recent windows for every task and the maximum
demand seen in any of the previous 5 windows (where task had some
activity) drives freq demand for every task.

A per-cpu metric (rq->cumulative_runnable_avg) is also provided which
is an aggregation of cpu demand of all tasks currently enqueued on it.
rq->cumulative_runnable_avg will be useful to know if cpu frequency
will need to be changed to match task demand.

Change-Id: Ib83207b9ba8683cd3304ee8a2290695c34f08fe2
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
sched: Call the notify_on_migrate notifier chain for wakeups as well

Add a change to send notify_on_migrate hints on wakeups of
foreground tasks from scheduler if their load is above
wakeup_load_thresholds (default value is 60).
These hints can be used to choose an appropriate CPU frequency
corresponding to the load of the task being woken up.

Change-Id: Ieca413c1a8bd2b14a15a7591e8e15d22925c42ca
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
cpufreq: cpu-boost: Make it less noisy
cpufreq: cpu-boost: Increase sync thread policy and priority

More or less all cpufreq related threads run in a high priority.

Also, CMs patch series was somehow out of sync and incomplete,
readd the unchanged policy update to source cpu bit. And readd
the check for a possible corner case in try_to_wake_up() that
was already missing before.
Revert dubious scheduler changes.

 * Rolled this series back on other devices as well. It's causing bad
   jank all over the system under a lot of use cases.

Revert "sched: reinitialize rq->next_balance when a CPU is hot-added"

This reverts commit 7c588520b259bf0a342058aa4a9e5ea677522a0d.

Revert "nohz: stat: Fix CPU idle time accounting"

This reverts commit 9446ae3986469a19a603344862e856d54f6df15b.

Revert "sched: remove redundant update_runtime notifier"

This reverts commit 1eb858976a30964ee1812f34f8a05056814ea8e9.

Revert "sched,rt: disable rt_runtime borrowing by default"

This reverts commit a731e214fe6606896de2b50a629bedbdbd95ced7.

Revert "sched/rt: Remove redundant nr_cpus_allowed test"

This reverts commit e169d922cc1fa6026225f9abc71ea28facafd43a.

Revert "sched: Micro-optimize the smart wake-affine logic"

This reverts commit 3bd0f8f8c045c77ca1d46a95ae09ab08c5d52c5d.

Revert "sched: Implement smarter wake-affine logic"

This reverts commit 8ea57b5895029c0694e8cb53295ce2cf735e9505.

Revert "sched: Reduce overestimating rq->avg_idle"

This reverts commit b49d8687515095bdf5089014c2c023d0e61de0cc.

Revert "sched: scale the busy and this queue's per-task load before compare"

This reverts commit a4d496cb8d418891f12cca0e5a3bd64dc2b83ea9.

Revert "sched: reduce calculation effort in fix_small_imbalance"

This reverts commit 65d048b5ef51f2104d3a26fb3e1fddfa8d401090.

Revert "mutex: back out architecture specific check for negative mutex count"

This reverts commit 2f983cc28c06648cc6a50b28a85b3008b51ec850.

Revert "mutex: Queue mutex spinners with MCS lock to reduce cacheline contention"

This reverts commit f6ab92b597a5d1cc6e4ffe82a787cd868140bc09.

Revert "mutex: Make more scalable by doing less atomic operations"

This reverts commit f425908d01a43e0e63e42d51b50c7b9b028e9702.

Change-Id: I7fcb7b2a7bb5657bf70a2189b2d04a5289987106

Conflicts:
	fs/proc/stat.c
sched: Remove get_online_cpus() usage

Remove get_online_cpus() usage from the scheduler; there's 4 sites that
use it:

 - sched_init_smp(); where its completely superfluous since we're in
   'early' boot and there simply cannot be any hotplugging.

 - sched_getaffinity(); we already take a raw spinlock to protect the
   task cpus_allowed mask, this disables preemption and therefore
   also stabilizes cpu_online_mask as that's modified using
   stop_machine. However switch to active mask for symmetry with
   sched_setaffinity()/set_cpus_allowed_ptr(). We guarantee active
   mask stability by inserting sync_rcu/sched() into _cpu_down.

 - sched_setaffinity(); we don't appear to need get_online_cpus()
   either, there's two sites where hotplug appears relevant:
    * cpuset_cpus_allowed(); for the !cpuset case we use possible_mask,
      for the cpuset case we hold task_lock, which is a spinlock and
      thus for mainline disables preemption (might cause pain on RT).
    * set_cpus_allowed_ptr(); Holds all scheduler locks and thus has
      preemption properly disabled; also it already deals with hotplug
      races explicitly where it releases them.

 - migrate_swap(); we can make stop_two_cpus() do the heavy lifting for
   us with a little trickery. By adding a sync_sched/rcu() after the
   CPU_DOWN_PREPARE notifier we can provide preempt/rcu guarantees for
   cpu_active_mask. Use these to validate that both our cpus are active
   when queueing the stop work before we queue the stop_machine works
   for take_cpu_down().

CRs-fixed: 647141
Change-Id: Id41e66659574f716de0e7c29f477e56a86db9404
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Cc: "Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Link: http://lkml.kernel.org/r/20131011123820.GV3081@twins.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Git-commit: 6acce3ef84520537f8a09a12c9ddbe814a584dd2
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[kaushalk@codeaurora.org: get_online_cpus has only 3 sites of usage in
 kernel/sched/core.c of msm-3.10 so migrate_swap changes are not
 applicable here. stop_two_cpus related change is not applicable to
 msm-3.10, so skip it.]
Signed-off-by: Kaushal Kumar <kaushalk@codeaurora.org>
sched: Remove extra put_online_cpus() inside sched_setaffinity()

Commit 6acce3ef8:

	sched: Remove get_online_cpus() usage

has left one extra put_online_cpus() inside sched_setaffinity(),
remove it to fix the WARN:

   ------------[ cut here ]------------
   WARNING: CPU: 0 PID: 3166 at kernel/cpu.c:84 put_online_cpus+0x43/0x70()
   ...
   [<ffffffff810c3fef>] put_online_cpus+0x43/0x70 [
   [<ffffffff810efd59>] sched_setaffinity+0x7d/0x1f9 [
   ...

CRs-fixed: 647141
Change-Id: I33f799f30a963db3e9459832832e9c786931c8c2
Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Tested-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Michael Wang <wangyun@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/526DD0EE.1090309@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Git-commit: ac9ff7997b6f2b31949dcd2495ac671fd9ddc990
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Kaushal Kumar <kaushalk@codeaurora.org>
sched: Remove get_online_cpus() usage
sched: Fix endless sync_sched/rcu() loop inside _cpu_down()

Commit 6acce3ef8:

	sched: Remove get_online_cpus() usage

tries to do sync_sched/rcu() inside _cpu_down() but triggers:

	INFO: task swapper/0:1 blocked for more than 120 seconds.
	...
	[<ffffffff811263dc>] synchronize_rcu+0x2c/0x30
	[<ffffffff81d1bd82>] _cpu_down+0x2b2/0x340
	...

It was caused by that in the rcu boost case we rely on smpboot thread to
finish the rcu callback, which has already been parked before sync in here
and leads to the endless sync_sched/rcu().

This patch exchanges the sequence of smpboot_park_threads() and
sync_sched/rcu() to fix the bug.

CRs-fixed: 647141
Change-Id: Ia8b325d7c3c778f428d9fb51271977c849df32ad
Reported-by: Fengguang Wu <fengguang.wu@intel.com>
Tested-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Michael Wang <wangyun@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/5282EDC0.6060003@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Git-commit: 106dd5afde3cd10db7e1370b6ddc77f0b2496a75
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Kaushal Kumar <kaushalk@codeaurora.org>
Revert "sched: Remove get_online_cpus() usage"

This reverts commit 7755290e4ee70b2ec1094452bb19fbd6053d5b68.

Revert "sched: Remove extra put_online_cpus() inside sched_setaffinity()"

This reverts commit 0372a8eb16e69345a12c1e1ce3c5e1aa4a758dca.

Revert "sched: Fix endless sync_sched/rcu() loop inside _cpu_down()"

This reverts commit 2baf8e44494bc6bd5e002d51740191f2c0da2470.

Revert "sched: Remove get_online_cpus() usage"

This reverts commit f0fbb1d0126e81adfb565474b5a3fd73425a98d0.