cpufreq: Retain only online cpus in managed_policy->cpus

cpufreq's handling of cpu online event can incorrectly initialize
managed_policy->cpus to include offline cpus. This causes cpufreq
governors to include offline cpus in their idle time computation, which
causes frequency to shoot up beyond normal levels (on systems where 2 or
more cpus need to be synchronized wrt frequency change).

This frequency shootup was not seen prior to commit 21111be8 which
changed idle time accounting for offline cpus. Prior to commit 21111be8,
idle time statistics for offline cpus would include their offline time
as well. Post commit 21111be8, offline time is not included in a cpu's
idle time statistics.

As a result, ondemand (or other cpufreq) governors would see unchanged
idle time for offline cpus, causing them to interpret those (offline)
cpus as 100% busy, which is wrong.

This patch fixes the issue by modifying cpufreq's hotplug handling to
retain only online cpus in managed_policy->cpus field. cpufreq governors
would then skip such offline cpus from idle time calculations.

Change-Id: Id2eba8c0ec3367609369b2e42a4e1386ff2e7ab5
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
cpufreq: Fix optimal_freq logic on ramp down

When ondemand governor decides to ramp down the frequency, the missing
check for frequency being lower than optimal_freq was causing frequency
to be ramped down to optimal_freq even if the frequency decided by ondemand
governor was higher.

Change-Id: I00f9ae4c61ccda4039f3338dee2c29b8cb790be4
Signed-off-by: Veena Sambasivan <veenas@codeaurora.org>
cpufreq: Resolve CPUFREQ_NOTIFY issue

When the cpufreq_stat receives the frequency policy notification,
it checks if the frequency table and stats table are present for the cpu.
If both are present, it returns with -EBUSY error to the notification.
Due to this, none of the other registered modules receive this
notification.

If the stat table and sysfs nodes are already created, return 0 instead
of returning EBUSY error.

Change-Id: I1fd4a46e6a65edfc6253c4271288f10912ae2ad7
Signed-off-by: Archana Sathyakumar <asathyak@codeaurora.org>
msm: audio: qdsp6v2: Add size safety check to ACDB driver

Check that the size sent by userspace is not larger
then the internal amount allowed. This protects
against overflowing the stack due to an invalid size.

Change-Id: I8230fdb00a7b57d398929e8ab0eb6587476f3db1
CRs-fixed: 470222
Signed-off-by: Ben Romberger <bromberg@codeaurora.org>
msm:rq_stats: Add hotplug disable attribute

Register for suspend and resume notifications from kernel. This should
be done as part of core_initcall so as to receive notification as close
as possible to the actual event. Add hotplug_disable attribute to
rq-stats node. Set/clear this attribute based on the suspend/resume events.

CRS-fixed: 452456
Change-Id: I76fee349b57f47df40785e9c2503e2e1cc6e064e
Signed-off-by: Archana Sathyakumar <asathyak@codeaurora.org>
msm:rq_stats: Fix hotplug attribute during the suspend failure

If the system suspend fails, the hotplug_disable attribute remains
set. This prevents any change to core status from mpdecision. Reset
the value of this attribute if the module receives PM_POST_RESTORE
notification.

Change-Id: I22fd639134adbeb8f8f96603e285151480f537ec
Signed-off-by: Archana Sathyakumar <asathyak@codeaurora.org>
msm:rq_stats: Register for Cpufreq policy notification

Currently the load of the system is calculated using the max
frequency that is read during startup(It does not get updated
when the policy changes). As a result, the average load of the
system is incorrectly calculated and the system might not bring
the cpus online to manage the additional load. This
reduces the performance of the system.

To fix this issue, register rq_stats to receive cpufreq policy
notifications. Update the policy's max frequency when a
notification is received and use that value to calculate the load.

Change-Id: Icc78e28c736c170e198f723fd96c13dfb2dafe8a
Signed-off-by: Archana Sathyakumar <asathyak@codeaurora.org>
Signed-off-by: Anji Jonnala <anjir@codeaurora.org>
workqueue: don't use WQ_HIGHPRI for unbound workqueues

Unbound wqs aren't concurrency-managed and try to execute work items
as soon as possible.  This is currently achieved by implicitly setting
%WQ_HIGHPRI on all unbound workqueues; however, WQ_HIGHPRI
implementation is about to be restructured and this usage won't be
valid anymore.

Add an explicit chain-wakeup path for unbound workqueues in
process_one_work() instead of piggy backing on %WQ_HIGHPRI.

Change-Id: Iecd17a9935ee28f856d8b726bb4c296762922bed
Signed-off-by: Tejun Heo <tj@kernel.org>
Git-commit: 974271c485a4d8bb801decc616748f90aafb07ec
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
(cherry picked from commit 5b92cea36d8328c17380d7997744284707134397)

Conflicts:
	kernel/workqueue.c
workqueue: factor out worker_pool from global_cwq

Move worklist and all worker management fields from global_cwq into
the new struct worker_pool.  worker_pool points back to the containing
gcwq.  worker and cpu_workqueue_struct are updated to point to
worker_pool instead of gcwq too.

This change is mechanical and doesn't introduce any functional
difference other than rearranging of fields and an added level of
indirection in some places.  This is to prepare for multiple pools per
gcwq.

v2: Comment typo fixes as suggested by Namhyung.

Change-Id: Iefae84798c2af580f425b92ed79117935d99f21f
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Git-commit: bd7bdd43dcb81bb08240b9401b36a104f77dc135
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
(cherry picked from commit e3676a67902c18021bd571aa3fcefc74b377b65f)

Conflicts:
	kernel/workqueue.c
workqueue: use @pool instead of @gcwq or @cpu where applicable

Modify all functions which deal with per-pool properties to pass
around @pool instead of @gcwq or @cpu.

The changes in this patch are mechanical and don't caues any
functional difference.  This is to prepare for multiple pools per
gcwq.

Change-Id: I4be6727e1cce6f9aa2a0057b96bdc725c84f1ea8
Signed-off-by: Tejun Heo <tj@kernel.org>
Git-commit: 63d95a9150ee3bbd4117fcd609dee40313b454d9
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
(cherry picked from commit ae73f6e6eae63e64d552227a109b6ccb802cdd04)

Conflicts:
	kernel/workqueue.c
workqueue: separate out worker_pool flags

GCWQ_MANAGE_WORKERS, GCWQ_MANAGING_WORKERS and GCWQ_HIGHPRI_PENDING
are per-pool properties.  Add worker_pool->flags and make the above
three flags per-pool flags.

The changes in this patch are mechanical and don't caues any
functional difference.  This is to prepare for multiple pools per
gcwq.

Change-Id: I1824fd1c509d8ac6b0619536621a22b15b316256
Signed-off-by: Tejun Heo <tj@kernel.org>
Git-commit: 11ebea50dbc1ade5994b2c838a096078d4c02399
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
(cherry picked from commit 1ab4f069d6d1255625a42209a42cf9d6773480d9)

Conflicts:
	kernel/workqueue.c
workqueue: introduce NR_WORKER_POOLS and for_each_worker_pool()

Introduce NR_WORKER_POOLS and for_each_worker_pool() and convert code
paths which need to manipulate all pools in a gcwq to use them.
NR_WORKER_POOLS is currently one and for_each_worker_pool() iterates
over only @gcwq->pool.

Note that nr_running is per-pool property and converted to an array
with NR_WORKER_POOLS elements and renamed to pool_nr_running.  Note
that get_pool_nr_running() currently assumes 0 index.  The next patch
will make use of non-zero index.

The changes in this patch are mechanical and don't caues any
functional difference.  This is to prepare for multiple pools per
gcwq.

v2: nr_running indexing bug in get_pool_nr_running() fixed.

v3: Pointer to array is stupid.  Don't use it in get_pool_nr_running()
    as suggested by Linus.

Change-Id: I46e9488601d764d25e4a6c707de129ab68f7064c
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Git-commit: 4ce62e9e30cacc26885cab133ad1de358dd79f21
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
(cherry picked from commit ca6a47d36f578a3c9ae47037413abca8319fd99f)

Conflicts:
	kernel/workqueue.c
msm: rpm-smd: Configure WQ for higer priority

When the workqueue runs at a lower priority, it gets starved by higher
priority threads. Bump the WQ priority to high to ensure rpm smd work
queue gets a fair chance

Change-Id: I06b864611cf45afe6931d6030327806032894663
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
(cherry picked from commit e0eccf8f6dece61cdb639c4b43374c34c3acf3ea)
msm: cpufreq: Configure WQ for higer priority

When the workqueue runs at a lower priority, it gets starved by higher
priority threads. Bump the WQ priority to high to ensure cpufreq work
queue gets a fair scheduling chance

Change-Id: I9e994da94a347dceb884e72ec3dd3da468a4471d
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
(cherry picked from commit c0a082fec5b016f0e2ea8b454de901d184401d9a)
modified for Mako kernel

Signed-off-by: Paul Reioux <reioux@gmail.com>

Conflicts:
	arch/arm/mach-msm/cpufreq.c

Conflicts:
	arch/arm/mach-msm/cpufreq.c
sched: smart wake-affine

wake-affine stuff is always trying to pull wakee close to waker, by theory,
this will bring benefit if waker's cpu cached hot data for wakee, or the
extreme ping-pong case.

And testing show it could benefit hackbench 15% at most.

However, the whole stuff is somewhat blindly and time-consuming, some
workload therefore suffer.

And testing show it could damage pgbench 50% at most.

Thus, wake-affine stuff should be smarter, and realise when to stop
it's thankless effort.

This patch introduced per task 'nr_wakee_switch', which will be increased
each time the task switch it's wakee.

So a high 'nr_wakee_switch' means the task has more than one wakee, and
less the wakee number, higher the wakeup frequency.

Now when making the decision on whether to pull or not, pay attention on
the wakee with a high 'nr_wakee_switch', pull such task may benefit wakee,
but that imply waker will face cruel competition later, it could be very
crule or very fast depends on the story behind 'nr_wakee_switch', whatever,
waker therefore suffer.

Furthermore, if waker also has a high 'nr_wakee_switch', that imply multiple
tasks rely on it, waker's higher latency will damage all those tasks, pull
wakee in such cases seems to be a bad deal.

Thus, when 'waker->nr_wakee_switch / wakee->nr_wakee_switch' become higher
and higher, the deal seems to be worse and worse.

This patch therefore help wake-affine stuff to stop it's work when:

	wakee->nr_wakee_switch > factor &&
	waker->nr_wakee_switch > (factor * wakee->nr_wakee_switch)
The factor here is the online cpu number, so more cpu will lead to more pull
since the trial become more severe.

After applied the patch, pgbench show 42% improvement at most.

Test:
	Test with 12 cpu X86 server and tip 3.10.0-rc1.

				base	smart

	| db_size | clients |  tps  | |  tps  |
	+---------+---------+-------+ +-------+
	| 21 MB   |       1 | 10749 | | 10337 |
	| 21 MB   |       2 | 21382 | | 21391 |
	| 21 MB   |       4 | 41570 | | 41808 |
	| 21 MB   |       8 | 52828 | | 58792 |
	| 21 MB   |      12 | 48447 | | 54553 |
	| 21 MB   |      16 | 46246 | | 56726 |	+22.66%
	| 21 MB   |      24 | 43850 | | 56853 |	+29.65%
	| 21 MB   |      32 | 43455 | | 55846 |	+28.51%
	| 7483 MB |       1 |  9290 | |  8848 |
	| 7483 MB |       2 | 19347 | | 19351 |
	| 7483 MB |       4 | 37135 | | 37511 |
	| 7483 MB |       8 | 47310 | | 50210 |
	| 7483 MB |      12 | 42721 | | 49396 |
	| 7483 MB |      16 | 41016 | | 51826 |	+26.36%
	| 7483 MB |      24 | 37540 | | 52579 |	+40.06%
	| 7483 MB |      32 | 36756 | | 51332 |	+39.66%
	| 15 GB   |       1 |  8758 | |  8670 |
	| 15 GB   |       2 | 19204 | | 19249 |
	| 15 GB   |       4 | 36997 | | 37199 |
	| 15 GB   |       8 | 46578 | | 50681 |
	| 15 GB   |      12 | 42141 | | 48671 |
	| 15 GB   |      16 | 40518 | | 51280 |	+26.56%
	| 15 GB   |      24 | 36788 | | 52329 |	+42.24%
	| 15 GB   |      32 | 36056 | | 50350 | +39.64%

CC: Ingo Molnar <mingo@kernel.org>
CC: Peter Zijlstra <peterz@infradead.org>
CC: Mike Galbraith <efault@gmx.de>
Signed-off-by: Michael Wang <wangyun@linux.vnet.ibm.com>
Signed-off-by: Paul Reioux <reioux@gmail.com>
CPU hotplug, writeback: Don't call writeback_set_ratelimit() too often during hotplug

The CPU hotplug callback related to writeback calls writeback_set_ratelimit()
during every state change in the hotplug sequence. This is unnecessary
since num_online_cpus() changes only once during the entire hotplug operation.

So invoke the function only once per hotplug, thereby avoiding the
unnecessary repetition of those costly calculations.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
ARM: 7467/1: mutex: use generic xchg-based implementation for ARMv6+

The open-coded mutex implementation for ARMv6+ cores suffers from a
severe lack of barriers, so in the uncontended case we don't actually
protect any accesses performed during the critical section.

Furthermore, the code is largely a duplication of the ARMv6+ atomic_dec
code but optimised to remove a branch instruction, as the mutex fastpath
was previously inlined. Now that this is executed out-of-line, we can
reuse the atomic access code for the locking (in fact, we use the xchg
code as this produces shorter critical sections).

This patch uses the generic xchg based implementation for mutexes on
ARMv6+, which introduces barriers to the lock/unlock operations and also
has the benefit of removing a fair amount of inline assembly code.

Change-Id: I58e3ca4d2740a834d30b54fc35742fa2df7792ad
Cc: <stable@vger.kernel.org> : 0bce9c46: mutex: Place lock in contended...
Cc: <stable@vger.kernel.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Nicolas Pitre <nico@linaro.org>
Reported-by: Shan Kang <kangshan0910@gmail.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Reviewed-on: http://git-master/r/130940
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Tested-by: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: Paul Reioux <reioux@gmail.com>
ARM: mutex: use generic atomic_dec-based implementation for ARMv6+

Commit a76d7bd96d65 ("ARM: 7467/1: mutex: use generic xchg-based
implementation for ARMv6+") removed the barrier-less, ARM-specific
mutex implementation in favour of the generic xchg-based code.

Since then, a bug was uncovered in the xchg code when running on SMP
platforms, due to interactions between the locking paths and the
MUTEX_SPIN_ON_OWNER code. This was fixed in 0bce9c46bf3b ("mutex: place
lock in contended state after fastpath_lock failure"), however, the
atomic_dec-based mutex algorithm is now marginally more efficient for
ARM (~0.5% improvement in hackbench scores on dual A15).

This patch moves ARMv6+ platforms to the atomic_dec-based mutex code.

Change-Id: I8f64e98ccb61cc1cb9cb68ee15e55d8a792792f5
Cc: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Reviewed-on: http://git-master/r/130941
Reviewed-by: Automatic_Commit_Validation_User
Reviewed-by: Bharat Nihalani <bnihalani@nvidia.com>
Tested-by: Bharat Nihalani <bnihalani@nvidia.com>
sched: Optimize build_sched_domains() for saving first SD node for a cpu

Date	Tue, 4 Jun 2013 16:50:18 +0530

We are saving first scheduling domain for a cpu in build_sched_domains() by
iterating over the nested sd->child list. We don't actually need to do it this
way.

*per_cpu_ptr(d.sd, i) is guaranteed to be NULL in the beginning as we have
called __visit_domain_allocation_hell() which does a memset to zero for struct
s_data.

So, save pointer to first SD while running the iteration loop over tl's.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Paul Reioux <reioux@gmail.com>
sched: Optimize build_sched_domains() for saving first SD node for a cpu

We are saving first scheduling domain for a cpu in build_sched_domains() by
iterating over the nested sd->child list. We don't actually need to do it this
way.

tl will be equal to sched_domain_topology for the first iteration and so we can
set *per_cpu_ptr(d.sd, i) based on that.  So, save pointer to first SD while
running the iteration loop over tl's.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Paul Reioux <reioux@gmail.com>