cpufreq: Avoid using smp_processor_id() in preemptible context

Even though this work item runs on only one cpu at a time (due to
queue_work_on()) it is possible for the work item to be preempted
and so use of smp_processor_id() is illegal.

BUG: using smp_processor_id() in preemptible [00000000]
code: kworker/3:1/4162 caller is dbs_refresh_callback+0xc/0x188
[<c00151b0>] (unwind_backtrace+0x0/0x120) from [<c0279058>]
(debug_smp_processor_id+0xbc/0xf0)
[<c0279058>] (debug_smp_processor_id+0xbc/0xf0) from [<c0454b54>]
(dbs_refresh_callback+0xc/0x188)
[<c0454b54>] (dbs_refresh_callback+0xc/0x188) from [<c0087290>]
(process_one_work+0x354/0x648)
[<c0087290>] (process_one_work+0x354/0x648) from [<c0089754>]
(worker_thread+0x1a8/0x2a8)
[<c0089754>] (worker_thread+0x1a8/0x2a8) from [<c008e480>]
(kthread+0x90/0xa0)
[<c008e480>] (kthread+0x90/0xa0) from [<c000f438>]
(kernel_thread_exit+0x0/0x8)

The intent of the code is to determine which CPU this work item
is running on, which we can easily do by passing that information
in a wrapper struct around the work struct. Do this so we avoid
this problem.

Change-Id: I05ca0ff2b3cbaa239930463ea0760e3e9d75145f
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Revert "cpufreq: Avoid using smp_processor_id() in preemptible context"

This reverts commit 7123e3046b6596c8190af14bd2d62e72a10ceb69.
msm: cpufreq: make the gov stop part fully atomic

In the function cpufreq_governor_dbs, the case CPUFREQ_GOV_STOP
is not completely atomic. The removal of sysfs is outside the
critical path. If a START/STOP done frequently from user space
then there is chance of adding the sysfs interface before removal.
This creates a warning from sysfs saying creating an already
existing entry.

Also once it fails to create, cpufreq_governor_dbs returns with
error code. The dbs_enable value is incremented which is not
rolled back and cpufreq core tries to set the old
governor. This might create further problems.

Hence the removal of sysfs entry should happen in the critical
path.

CRs-fixed: 440969
Change-Id: I98891756caf2ce0b85e39e9b4f58034af1ed97ce
Signed-off-by: Venkat Devarasetty <vdevaras@codeaurora.org>
cpufreq: Fix optimal_freq logic on ramp down

When ondemand governor decides to ramp down the frequency, the missing
check for frequency being lower than optimal_freq was causing frequency
to be ramped down to optimal_freq even if the frequency decided by ondemand
governor was higher.

Change-Id: I00f9ae4c61ccda4039f3338dee2c29b8cb790be4
Signed-off-by: Veena Sambasivan <veenas@codeaurora.org>
sched: provide per cpu-cgroup option to notify on migrations

On systems where CPUs may run asynchronously, task migrations
between CPUs running at grossly different speeds can cause
problems.

This change provides a mechanism to notify a subsystem
in the kernel if a task in a particular cgroup migrates to a
different CPU. Other subsystems (such as cpufreq) may then
register for this notifier to take appropriate action when
such a task is migrated.

The cgroup attribute to set for this behavior is
"notify_on_migrate" .

Change-Id: Ie1868249e53ef901b89c837fdc33b0ad0c0a4590
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>

Conflicts:
	kernel/sched/core.c
Revert "sched: provide per cpu-cgroup option to notify on migrations"

This reverts commit cc6281a07ef9284072df52d65b6210984fca562c.
sched: Reset rq->next_interval before going idle

next_balance, the point in jiffy time scale when a cpu will next load
balance, could have been calculated when the cpu was busy. A busy cpu
will apply its sched domain's busy_factor (usually > 1) in computing
next_balance for that sched domain, which causes the (busy) cpu to load
balance less frequently in its sched domains. However when the same cpu
is going idle, its next_balance needs to be reset without consideration
of busy_factor. Failure to do so would not trigger nohz idle balancer on
that cpu for unnecessarily long time (introducing additional scheduling
latencies for tasks). Fix bug in scheduler which aims to reset
next_balance before a cpu goes idle (as per existing comment) but is
clearly not doing so.

Change-Id: I7e027a51686528c4092d770c7d33c874d38f5df4
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
sched: Fix SCHED_HRTICK bug leading to late preemption of tasks

SCHED_HRTICK feature is useful to preempt SCHED_FAIR tasks on-the-dot
(just when they would have exceeded their ideal_runtime). It makes use
of a a per-cpu hrtimer resource and hence alarming that hrtimer should
be based on total SCHED_FAIR tasks a cpu has across its various cfs_rqs,
rather than being based on number of tasks in a particular cfs_rq (as
implemented currently). As a result, with current code, its possible for
a running task (which is the sole task in its cfs_rq) to be preempted
much after its ideal_runtime has elapsed, resulting in increased latency
for tasks in other cfs_rq on same cpu.

Fix this by alarming sched hrtimer based on total number of SCHED_FAIR
tasks a CPU has across its various cfs_rqs.

Change-Id: I1f23680a64872f8ce0f451ac4bcae28e8967918f
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
sched: Mark schedule_io_timeout() with EXPORT_SYMBOL

Make schedule_io_timeout() visible to modules.

Change-Id: Ic0dedbad8a591a9a721f0d2e8f6c372ec75bc4b2
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
sched: provide per cpu-cgroup option to notify on migrations

On systems where CPUs may run asynchronously, task migrations
between CPUs running at grossly different speeds can cause
problems.

This change provides a mechanism to notify a subsystem
in the kernel if a task in a particular cgroup migrates to a
different CPU. Other subsystems (such as cpufreq) may then
register for this notifier to take appropriate action when
such a task is migrated.

The cgroup attribute to set for this behavior is
"notify_on_migrate" .

Change-Id: Ie1868249e53ef901b89c837fdc33b0ad0c0a4590
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>

Conflicts:
	kernel/sched/core.c
sched: fix reference to wrong cfs_rq

Commit 7db16c8c (sched: Fix SCHED_HRTICK bug leading to late preemption
of tasks) introduced a bug in sched_slice() calculation by using wrong
cfs_rq for tasks. rq->cfs was incorrectly used as task's cfs_rq, rather
than the correct one to which they belonged.

Fix the bug by using correct cfs_rq for tasks.

Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
sched: remove migration notification from RT class

Commit 88a7e37d265 (sched: provide per cpu-cgroup option to
notify on migrations) added a notifier call when a task is moved
to a different CPU. Unfortunately the two call sites in the RT
sched class where this occurs happens with a runqueue lock held.
This can result in a deadlock if the notifier call attempts to do
something like wake up a task.

Fortunately the benefit of 88a7e37d265 comes mainly from notifying
on migration of non-RT tasks, so we can simply ignore the movements
of RT tasks.

CRs-Fixed: 491370
Change-Id: I8849d826bf1eeaf85a6f6ad872acb475247c5926
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
sched: re-calculate a cpu's next_balance point upon sched domain changes

Commit 55ddeb0f (sched: Reset rq->next_interval before going idle) reset
a cpu's rq->next_balance when pulled_task = 0, which will be true when
the cpu failed to pull any task, causing it go idle. However that patch
relied on next_balance being calculated as a result of traversing cpu's
sched domain hierarchy.

A cpu that is the only online cpu will however not be attached to any
sched domain hierarchy. When such a cpu calls into idle_balance(), we
will end up initializing next_balance to be 1sec away! Such a CPU will
defer load balance check for another 1sec, even though we may bring up
more cpus in the meantime requiring it to check for load imbalance more
frequently. This could then lead to increased scheduling latency for
some tasks.

This patch results in a cpu's next_balance being re-calculated when its
attaching to a new sched domain hierarchy.  This should let cpus call
load balance checks at the right time we expect them to!

Change-Id: I855cff8da5ca28d278596c3bb0163b839d4704bc
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
sched: Make sure to not re-read variables after validation

We could re-read rq->rt_avg after we validated it was smaller than
total, invalidating the check and resulting in an unintended negative.

Change-Id: I8543974aad539107768e9e513ca3a8c4cb79b2ff
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: David Rientjes <rientjes@google.com>
Link: http://lkml.kernel.org/r/1337688268.9698.29.camel@twins
Signed-off-by: Ingo Molnar <mingo@kernel.org>
CRs-Fixed: 497236
Git-commit: b654f7de41b0e3903ee2b51d3b8db77fe52ce728
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
tracing/sched: Track per-cpu rt and non-rt cpu_load.

Add a new tracepoint trace_sched_enq_deq_task to track
per-cpu rt and non-rt cpu_load during task enqueue
and dequeue.

This is useful to visualize and compare the load on
different cpus and also to understand how balanced
the load is at any point of time.

Note: We only print cpu_load[0] because we only care about
the most recent load history for tracking load balancer
effectiveness.

Change-Id: I46f0bb84e81652099ed5edf8c2686c70c8b8330c
Signed-off-by: Arun Bharadwaj <abharadw@codeaurora.org>
msm: Synchronize CPU frequency on thread migration

Sometimes when foreground threads, crucial for UI, migrate
from a core at higher frequency to the one running at a lower
frequency, it could badly affect performance for cases like
scrolling.
This change tries to raise the frequency of the destination core
to the same value as the source core, if it is not already running
at the same or higher frequency.  In order for the ondemand
governor to have a fair idea of load in the next sample, it also
reschedules the next sample one sampling interval from the
frequency transition after which the ondemand governor again
takes over.

Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Change-Id: I9d2fbb166b0cc042d3fd57cea75a5066df4c794b

Conflicts:
	drivers/cpufreq/cpufreq_ondemand.c
Revert "msm: Synchronize CPU frequency on thread migration"

This reverts commit 61552f47556dc8a33eed9ab28055210026a12022.
cpufreq: Avoid using smp_processor_id() in preemptible context

Even though this work item runs on only one cpu at a time (due to
queue_work_on()) it is possible for the work item to be preempted
and so use of smp_processor_id() is illegal.

BUG: using smp_processor_id() in preemptible [00000000]
code: kworker/3:1/4162 caller is dbs_refresh_callback+0xc/0x188
[<c00151b0>] (unwind_backtrace+0x0/0x120) from [<c0279058>]
(debug_smp_processor_id+0xbc/0xf0)
[<c0279058>] (debug_smp_processor_id+0xbc/0xf0) from [<c0454b54>]
(dbs_refresh_callback+0xc/0x188)
[<c0454b54>] (dbs_refresh_callback+0xc/0x188) from [<c0087290>]
(process_one_work+0x354/0x648)
[<c0087290>] (process_one_work+0x354/0x648) from [<c0089754>]
(worker_thread+0x1a8/0x2a8)
[<c0089754>] (worker_thread+0x1a8/0x2a8) from [<c008e480>]
(kthread+0x90/0xa0)
[<c008e480>] (kthread+0x90/0xa0) from [<c000f438>]
(kernel_thread_exit+0x0/0x8)

The intent of the code is to determine which CPU this work item
is running on, which we can easily do by passing that information
in a wrapper struct around the work struct. Do this so we avoid
this problem.

Change-Id: I05ca0ff2b3cbaa239930463ea0760e3e9d75145f
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
cpufreq: ondemand: Fix kernel warning in cpufreq_governor_dbs

GOV_STOP event destroys the timer_mutex and is not initialized back
upon GOV_START event.
The Kernel complains at mutex_lock(during GOV_LIMITS event) after
mutex_destroy.

[<c0014918>] (unwind_backtrace+0x0/0x11c)
[<c007c004>] (warn_slowpath_common+0x4c/0x64)
[<c007c034>] (warn_slowpath_null+0x18/0x1c)
[<c079eef8>] (__mutex_lock_slowpath+0x130/0x3c4)
[<c079f1ac>] (mutex_lock+0x20/0x3c) from [<c0540900>]
[<c0540900>] (cpufreq_governor_dbs+0x3c8/0x468)
[<c053d500>] (__cpufreq_governor+0x90/0xe0)
[<c053d774>] (__cpufreq_set_policy+0x224/0x258)
[<c053da58>] (cpufreq_add_dev_interface+0x2b0/0x308)
[<c053e784>] (cpufreq_add_dev+0x500/0x56c)
[<c079247c>] (cpufreq_cpu_callback+0x88/0x9c)
[<c07a2ca8>] (notifier_call_chain+0x38/0x68)
[<c007ded0>] (__cpu_notify+0x28/0x40)
[<c0790024>] (_cpu_up+0xe4/0x118)
[<c0790024>] (cpu_up+0x64/0x80)
[<c0765840>] (store_online+0x48/0x78)
[<c038d0c8>] (dev_attr_store+0x18/0x24)
[<c017be40>] (sysfs_write_file+0x108/0x13c)
[<c012cdfc>] (vfs_write+0xb0/0x128)
[<c012d034>](sys_write+0x38/0x64)

Change-Id: Ie2a796b17209243de731ccf2ab04b7f9e7048df8
Signed-off-by: Anji Jonnala <anjir@codeaurora.org>
msm: Synchronize CPU frequency on thread migration

Sometimes when foreground threads, crucial for UI, migrate
from a core at higher frequency to the one running at a lower
frequency, it could badly affect performance for cases like
scrolling.
This change tries to raise the frequency of the destination core
to the same value as the source core, if it is not already running
at the same or higher frequency.  In order for the ondemand
governor to have a fair idea of load in the next sample, it also
reschedules the next sample one sampling interval from the
frequency transition after which the ondemand governor again
takes over.

Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Change-Id: I9d2fbb166b0cc042d3fd57cea75a5066df4c794b