msm: cpufreq: fix a race between hotplug and cpufreq

When msm_cpufreq_target() is called without holding the policy
semaphore, there is a rare chance that the cpu gets hotplugged in
the acpuclk_set_rate. The probability of this issue increases
when cross-calling is used or using interactive governor which
tries to set all cpu frequencies from a single thread.

Add a hotplug notifier and set the device_suspended flag holding
the suspend mutex, when cpu is going down. This mutex is always
held when frequency changes are done.

Change-Id: Ide3aa6927ad60d2512d63761c51bb15117461250
Signed-off-by: Narayanan Gopalakrishnan <nargop@codeaurora.org>
msm: cpufreq: Remove cross-calling limitation

Remove cross-calling limitations imposed by cpufreq. Clock drivers
and SPM driver now allow changing of frequencies from any core. Using
cpus_allowed mask to check for core affinity works well with workqueues
but do not port so well when the core requesting frequency change is a
kthread with the flag PF_TASK_BOUND not set.

Change-Id: Iab664ce8657780814440a11dff77cc925df597f8
Signed-off-by: Praveen Chidambaram <pchidamb@codeaurora.org>
msm: cpufreq: increase priority of thread that increases frequencies

When the cpufreq governors request for increase in frequency from kworker
threads, there is chance that the kworker thread is cpu starved due to
other high priority threads, leading to undesirable increase in frequency
ramp-up latency. Add the calling thread to SCHED_FIFO policy when
requesting for increase in frequency and restore back to original policy
after ramp-up is completed.

Change-Id: Ie4fa199b7f9087717cb94dfcc2eddea27cc0012b
Signed-off-by: Narayanan Gopalakrishnan <nargop@codeaurora.org>

Conflicts:

	arch/arm/mach-msm/cpufreq.c
msm: cpufreq: Implement suspend/resume for cpufreq driver

Cpufreq suspend variable get's updated in kernel pm notifier
callback which gets called in process context. This will create
window where cpufreq governor may try to change the frequency
before suspend variable is updated while resume.

Define suspend/resume for cpufreq driver to update suspend variable
in atomic context.

Don't update the policy->cur upon cpufreq driver failure.

CRs-fixed: 423791
Change-Id: Id78705f5c3b8bf801c6253e9362299280903769f
Signed-off-by: Anji Jonnala <anjir@codeaurora.org>
Revert "msm: cpufreq: Remove cross-calling limitation"

This reverts commit be01a17afb6b6e85456217caa1ac11eb0b6674e1.

Re-introduce cross calling in cpufreq to prevent race conditions in the
hardware when changing PLLs.

A core could be in retention, while another core, could try and set the
frequency of the core in retention, causing a change in PLL and when the
core exits retention, the PLL mismatch could hang the SPM state machine.

Change-Id: I99b454a0f146652f2d8a369f79f6d35748160dcb
Signed-off-by: Praveen Chidambaram <pchidamb@codeaurora.org>
msm: cpufreq: Initialize cpufreq driver early at boot

MSM cpufreq driver registering with the CPUFreq framework during
late_initcall() may cause the device to be available a few seconds after
the clock driver is ready for changing frequency. This could be bad for
boot and/or thermal performance, since the cpu will continue to boot at
whatever frequency the bootloaders set it to.

Initialize cpufreq driver as part of device_initcalL() but only after
all the clock drivers have had a chance to initialize their devices.

Change-Id: Iaa4867108f59a368ee5b511d35ea2e8e219d39d3
Signed-off-by: Praveen Chidambaram <pchidamb@codeaurora.org>

Conflicts:

	arch/arm/mach-msm/Makefile
msm: cpufreq: Always update frequency limits when core is online

Remove condition in msm_cpufreq_init() so that when cpu is set to
online, it will call set_cpu_freq() and update the frequency limits.

CRs-Fixed: 462816
Change-Id: I40ac33c9cb4b42590a83f9d61b74f23379f5642e
Signed-off-by: Jennifer Liu <chihning@codeaurora.org>

Conflicts:

	arch/arm/mach-msm/cpufreq.c
msm: cpufreq: Configure WQ for higer priority

When the workqueue runs at a lower priority, it gets starved by higher
priority threads. Bump the WQ priority to high to ensure cpufreq work
queue gets a fair scheduling chance

Change-Id: I9e994da94a347dceb884e72ec3dd3da468a4471d
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
workqueue: don't use WQ_HIGHPRI for unbound workqueues

Unbound wqs aren't concurrency-managed and try to execute work items
as soon as possible.  This is currently achieved by implicitly setting
%WQ_HIGHPRI on all unbound workqueues; however, WQ_HIGHPRI
implementation is about to be restructured and this usage won't be
valid anymore.

Add an explicit chain-wakeup path for unbound workqueues in
process_one_work() instead of piggy backing on %WQ_HIGHPRI.

Change-Id: Iecd17a9935ee28f856d8b726bb4c296762922bed
Signed-off-by: Tejun Heo <tj@kernel.org>
Git-commit: 974271c485a4d8bb801decc616748f90aafb07ec
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
workqueue: factor out worker_pool from global_cwq

Move worklist and all worker management fields from global_cwq into
the new struct worker_pool.  worker_pool points back to the containing
gcwq.  worker and cpu_workqueue_struct are updated to point to
worker_pool instead of gcwq too.

This change is mechanical and doesn't introduce any functional
difference other than rearranging of fields and an added level of
indirection in some places.  This is to prepare for multiple pools per
gcwq.

v2: Comment typo fixes as suggested by Namhyung.

Change-Id: Iefae84798c2af580f425b92ed79117935d99f21f
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Git-commit: bd7bdd43dcb81bb08240b9401b36a104f77dc135
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
workqueue: use @pool instead of @gcwq or @cpu where applicable

Modify all functions which deal with per-pool properties to pass
around @pool instead of @gcwq or @cpu.

The changes in this patch are mechanical and don't caues any
functional difference.  This is to prepare for multiple pools per
gcwq.

Change-Id: I4be6727e1cce6f9aa2a0057b96bdc725c84f1ea8
Signed-off-by: Tejun Heo <tj@kernel.org>
Git-commit: 63d95a9150ee3bbd4117fcd609dee40313b454d9
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
workqueue: separate out worker_pool flags

GCWQ_MANAGE_WORKERS, GCWQ_MANAGING_WORKERS and GCWQ_HIGHPRI_PENDING
are per-pool properties.  Add worker_pool->flags and make the above
three flags per-pool flags.

The changes in this patch are mechanical and don't caues any
functional difference.  This is to prepare for multiple pools per
gcwq.

Change-Id: I1824fd1c509d8ac6b0619536621a22b15b316256
Signed-off-by: Tejun Heo <tj@kernel.org>
Git-commit: 11ebea50dbc1ade5994b2c838a096078d4c02399
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
workqueue: introduce NR_WORKER_POOLS and for_each_worker_pool()

Introduce NR_WORKER_POOLS and for_each_worker_pool() and convert code
paths which need to manipulate all pools in a gcwq to use them.
NR_WORKER_POOLS is currently one and for_each_worker_pool() iterates
over only @gcwq->pool.

Note that nr_running is per-pool property and converted to an array
with NR_WORKER_POOLS elements and renamed to pool_nr_running.  Note
that get_pool_nr_running() currently assumes 0 index.  The next patch
will make use of non-zero index.

The changes in this patch are mechanical and don't caues any
functional difference.  This is to prepare for multiple pools per
gcwq.

v2: nr_running indexing bug in get_pool_nr_running() fixed.

v3: Pointer to array is stupid.  Don't use it in get_pool_nr_running()
    as suggested by Linus.

Change-Id: I46e9488601d764d25e4a6c707de129ab68f7064c
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Git-commit: 4ce62e9e30cacc26885cab133ad1de358dd79f21
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
workqueue: reimplement WQ_HIGHPRI using a separate worker_pool

WQ_HIGHPRI was implemented by queueing highpri work items at the head
of the global worklist.  Other than queueing at the head, they weren't
handled differently; unfortunately, this could lead to execution
latency of a few seconds on heavily loaded systems.

Now that workqueue code has been updated to deal with multiple
worker_pools per global_cwq, this patch reimplements WQ_HIGHPRI using
a separate worker_pool.  NR_WORKER_POOLS is bumped to two and
gcwq->pools[0] is used for normal pri work items and ->pools[1] for
highpri.  Highpri workers get -20 nice level and has 'H' suffix in
their names.  Note that this change increases the number of kworkers
per cpu.

POOL_HIGHPRI_PENDING, pool_determine_ins_pos() and highpri chain
wakeup code in process_one_work() are no longer used and removed.

This allows proper prioritization of highpri work items and removes
high execution latency of highpri work items.

v2: nr_running indexing bug in get_pool_nr_running() fixed.

v3: Refreshed for the get_pool_nr_running() update in the previous
    patch.

Change-Id: Id843c0a425f51f84083786fbf413d999d35771b7
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Josh Hunt <joshhunt00@gmail.com>
LKML-Reference: <CAKA=qzaHqwZ8eqpLNFjxnO2fX-tgAOjmpvxgBFjv6dJeQaOW1w@mail.gmail.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>
Git-commit: 3270476a6c0ce322354df8679652f060d66526dc
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
build new wip