msm: cpufreq: increase priority of thread that increases frequencies

When the cpufreq governors request for increase in frequency from kworker
threads, there is chance that the kworker thread is cpu starved due to
other high priority threads, leading to undesirable increase in frequency
ramp-up latency. Add the calling thread to SCHED_FIFO policy when
requesting for increase in frequency and restore back to original policy
after ramp-up is completed.

Change-Id: Ie4fa199b7f9087717cb94dfcc2eddea27cc0012b
Signed-off-by: Narayanan Gopalakrishnan <nargop@codeaurora.org>
rcu: Fix day-one dyntick-idle stall-warning bug

Each grace period is supposed to have at least one callback waiting
for that grace period to complete.  However, if CONFIG_NO_HZ=n, an
extra callback-free grace period is no big problem -- it will chew up
a tiny bit of CPU time, but it will complete normally.  In contrast,
CONFIG_NO_HZ=y kernels have the potential for all the CPUs to go to
sleep indefinitely, in turn indefinitely delaying completion of the
callback-free grace period.  Given that nothing is waiting on this grace
period, this is also not a problem.

That is, unless RCU CPU stall warnings are also enabled, as they are
in recent kernels.  In this case, if a CPU wakes up after at least one
minute of inactivity, an RCU CPU stall warning will result.  The reason
that no one noticed until quite recently is that most systems have enough
OS noise that they will never remain absolutely idle for a full minute.
But there are some embedded systems with cut-down userspace configurations
that consistently get into this situation.

All this begs the question of exactly how a callback-free grace period
gets started in the first place.  This can happen due to the fact that
CPUs do not necessarily agree on which grace period is in progress.
If a CPU still believes that the grace period that just completed is
still ongoing, it will believe that it has callbacks that need to wait for
another grace period, never mind the fact that the grace period that they
were waiting for just completed.  This CPU can therefore erroneously
decide to start a new grace period.  Note that this can happen in
TREE_RCU and TREE_PREEMPT_RCU even on a single-CPU system:  Deadlock
considerations mean that the CPU that detected the end of the grace
period is not necessarily officially informed of this fact for some time.

Once this CPU notices that the earlier grace period completed, it will
invoke its callbacks.  It then won't have any callbacks left.  If no
other CPU has any callbacks, we now have a callback-free grace period.

This commit therefore makes CPUs check more carefully before starting a
new grace period.  This new check relies on an array of tail pointers
into each CPU's list of callbacks.  If the CPU is up to date on which
grace periods have completed, it checks to see if any callbacks follow
the RCU_DONE_TAIL segment, otherwise it checks to see if any callbacks
follow the RCU_WAIT_TAIL segment.  The reason that this works is that
the RCU_WAIT_TAIL segment will be promoted to the RCU_DONE_TAIL segment
as soon as the CPU is officially notified that the old grace period
has ended.

This change is to cpu_needs_another_gp(), which is called in a number
of places.  The only one that really matters is in rcu_start_gp(), where
the root rcu_node structure's ->lock is held, which prevents any
other CPU from starting or completing a grace period, so that the
comparison that determines whether the CPU is missing the completion
of a grace period is stable.

Change-Id: Ic77e2bbe865059aa6f363db284473e9adcf8797e
Reported-by: Becky Bruce <bgillbruce@gmail.com>
Reported-by: Subodh Nijsure <snijsure@grid-net.com>
Reported-by: Paul Walmsley <paul@pwsan.com>
Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Paul Walmsley <paul@pwsan.com>  # OMAP3730, OMAP4430
Cc: stable@vger.kernel.org
Signed-off-by: Jin Hong <jinh@codeaurora.org>
sched: fix rq->lock recursion

Enabling SCHED_HRTICK currently results in rq->lock recursion and a hard
hang at bootup.  Essentially try_to_wakeup() grabs rq->lock and tries
arming a hrtimer via hrtimer_restart(), which deep down tries waking up
ksoftirqd, which leads to a recursive call to try_to_wakeup() and thus
attempt to take rq->lock recursively!!

This is fixed by having scheduler queue hrtimer via
__hrtimer_start_range_ns() which avoids waking up ksoftirqd.

Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
Change-Id: I11a13be1d9db3a749614ccf3d4f5fb7bf6f18fa1
ARM: Fix negative idle stats for offline cpu

We see negative idle stats because of cpu dying without
cleaning up idle entry statistics. When a cpu is offline,
the most immediate thing you'd want to do is just call
into cpu_die() and not waste time calling notifier(IDLE_START).

CRs-Fixed: 414554
Change-Id: Iadc6a3ca39997e0ccf65d2a29b004e24b1b211a1
Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
Signed-off-by: Taniya Das <tdas@codeaurora.org>
arm: Remove no-longer-required RCU_NONIDLE wrapper

Commit 21111be8 "ARM: Fix negative idle stats for offline cpu" moved call
to cpu_die() to occur outside of rcu_idle_enter()/rcu_idle_exit() section.
As a result, the RCU_NONIDLE() wrapper to complete() call in
arch/arm/kernel/process.c:cpu_die() is no longer required (and is
technically incorrect to have). Removing RCU_NONIDLE() wrapper also removes
this warning seen during CPU offline:

[Note: Below message has been edited to fit 75-char per line limit.
Insignificant portions of warning message has been removed in each line.]

------------[ cut here ]------------
WARNING: at kernel/rcutree.c:456 rcu_idle_exit_common+0x4c/0xe0()
Modules linked in:
(unwind_backtrace+0x0/0x120) from (warn_slowpath_common+0x4c/0x64)
(warn_slowpath_common+0x4c/0x64) from (warn_slowpath_null+0x18/0x1c)
(warn_slowpath_null+0x18/0x1c) from (rcu_idle_exit_common+0x4c/0xe0)
(rcu_idle_exit_common+0x4c/0xe0) from (rcu_idle_exit+0xa8/0xc0)
(rcu_idle_exit+0xa8/0xc0) from (cpu_die+0x24/0x5c)
(cpu_die+0x24/0x5c) from (cpu_idle+0xdc/0xf0)
(cpu_idle+0xdc/0xf0) from (0x8160)
---[ end trace 61bf21937a496a37 ]---

Signed-off-by: Srivatsa Vaddagiri <vatsa@codeaurora.org>
Change-Id: I721e6b20651674e6f6f584cf8d814af00b688c91
msm: SSR: Remove useless warning

This is a warning so that developers know to add more restart
order lists to SSR when a new chip is added. This is mostly
irrelevant now because we assume either entire SoC restart on SSR
or independent restart on SSR, not group restarts. Remove this
warning as it is mostly a reminder that nobody is listening for.

Change-Id: Icbf955cb18395d8d5d086b2167c5c329588b9256
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
arm: Move to upstream udelay via timer implementation

This is a squash of a handful of changes and reverts of the
Qualcomm specific implementation:

  Revert "arm: Implement a timer based __delay() loop"

  This reverts commit 976eafa8b18252876e15f861944acf693b07ce7e.

  Revert "arm: Allow machines to override __delay()"

  This reverts commit bc0ef8ab167272890f1aab62928b04a9aeb87ce9.

  Revert "arm: Translate delay.S into (mostly) C"

  This reverts commit 8d5868d8205d10a0a8e423f53e9cc9bb3e9d1a34.

  ARM: 7451/1: arch timer: implement read_current_timer and get_cycles

  This patch implements read_current_timer using the architected timers
  when they are selected via CONFIG_ARM_ARCH_TIMER. If they are detected
  not to be usable at runtime, we return -ENXIO to the caller.

  Furthermore, if read_current_timer is exported then we can implement
  get_cycles in terms of it for use as both an entropy source and for
  implementing __udelay and friends.

  Tested-by: Shinya Kuribayashi <shinya.kuribayashi.px@renesas.com>
  Reviewed-by: Stephen Boyd <sboyd@codeaurora.org>
  Signed-off-by: Will Deacon <will.deacon@arm.com>
  Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

  ARM: 7452/1: delay: allow timer-based delay implementation to be
  selected

  This patch allows a timer-based delay implementation to be selected by
  switching the delay routines over to use get_cycles, which is
  implemented in terms of read_current_timer. This further allows us to
  skip the loop calibration and have a consistent delay function in the
  face of core frequency scaling.

  To avoid the pain of dealing with memory-mapped counters, this
  implementation uses the co-processor interface to the architected timers
  when they are available. The previous loop-based implementation is
  kept around for CPUs without the architected timers and we retain both
  the maximum delay (2ms) and the corresponding conversion factors for
  determining the number of loops required for a given interval. Since the
  indirection of the timer routines will only work when called from C,
  the sa1100 sleep routines are modified to branch to the loop-based delay
  functions directly.

  Tested-by: Shinya Kuribayashi <shinya.kuribayashi.px@renesas.com>
  Reviewed-by: Stephen Boyd <sboyd@codeaurora.org>
  Signed-off-by: Will Deacon <will.deacon@arm.com>
  Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

  ARM: delay: set loops_per_jiffy when moving to timer-based loop

  The delay functions may be called by some platforms between switching to
  the timer-based delay loop but before calibration. In this case, the
  initial loops_per_jiffy may not be suitable for the timer (although a
  compromise may be achievable) and delay times may be considered too
  inaccurate.

  This patch updates loops_per_jiffy when switching to the timer-based
  delay loop so that delays are consistent prior to calibration.

  Signed-off-by: Will Deacon <will.deacon@arm.com>

  ARM: delay: add registration mechanism for delay timer sources

  The current timer-based delay loop relies on the architected timer to
  initiate the switch away from the polling-based implementation. This is
  unfortunate for platforms without the architected timers but with a
  suitable delay source (that is, constant frequency, always powered-up
  and ticking as long as the CPUs are online).

  This patch introduces a registration mechanism for the delay timer
  (which provides an unconditional read_current_timer implementation) and
  updates the architected timer code to use the new interface.

  Signed-off-by: Jonathan Austin <jonathan.austin@arm.com>
  Signed-off-by: Will Deacon <will.deacon@arm.com>

  ARM: export default read_current_timer

  read_current_timer is used by get_cycles since "ARM: 7538/1: delay:
  add registration mechanism for delay timer sources", and get_cycles
  can be used by device drivers in loadable modules, so it has to
  be exported.

  Without this patch, building imote2_defconfig fails with

  ERROR: "read_current_timer" [crypto/tcrypt.ko] undefined!

  Signed-off-by: Arnd Bergmann <arnd@arndb.de>
  Cc: Stephen Boyd <sboyd@codeaurora.org>
  Cc: Jonathan Austin <jonathan.austin@arm.com>
  Cc: Will Deacon <will.deacon@arm.com>
  Cc: Russell King <rmk+kernel@arm.linux.org.uk>

Change-Id: If1ad095d6852f5966ea995856103e06de6ab2f59
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>

Conflicts:
	arch/arm/lib/delay.c
block: row: Re-design urgent request notification mechanism

When ROW scheduler reports to the block layer that there is an urgent
request pending, the device driver may decide to stop the transmission
of the current request in order to handle the urgent one. This is done
in order to reduce the latency of an urgent request. For example:
long WRITE may be stopped to handle an urgent READ.

This patch updates the ROW URGENT notification policy to apply with the
below:

- Don't notify URGENT if there is an un-completed URGENT request in driver
- After notifying that URGENT request is present, the next request
  dispatched is the URGENT one.
- At every given moment only 1 request can be marked as URGENT.
  Independent of it's location (driver or scheduler)

Other changes to URGENT policy:
- Only READ queues are allowed to notify of an URGENT request pending.

CR fix:
If a pending urgent request (A) gets merged with another request (B)
A is removed from scheduler queue but is not removed from
rd->pending_urgent_rq.

CRs-Fixed: 453712
Change-Id: I321e8cf58e12a05b82edd2a03f52fcce7bc9a900
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
block: urgent request: remove unnecessary urgent marking

An urgent request is marked by the scheduler in rq->cmd_flags with the
REQ_URGENT flag. There is no need to add an additional marking by
the block layer.

Change-Id: I05d5e9539d2f6c1bfa80240b0671db197a5d3b3f
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
mmc: fix read latency of urgent request

In order to improve read request latency, urgent notification allowed
anytime, when current or previous requests are existing in the mmc layer.
It is not allowed to stop following running requests:
- urgent request: block layer should serialize urgent request notification.
- read request: it can't be packed, therefore it is better
to wait for its completion.
- REQ_FUA: interruption and delay in such request will impact sync process
of upper layers.

Change-Id: Id7d1480cce2059c1f23a5f29ad8f74e858be1ee6
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
Signed-off-by: Maya Erez <merez@codeaurora.org>
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
mmc: msm_sdcc: enable support to handle Urgent data transfer request

In order to decrease a latency of a prioritized requests (aka Urgent
request), we might want to stop the transmission of a running "low
priority" request in order to handle the Urgent request.

URGENT_REQUEST handling has following dependencies:
1. Host controller driver should support mmc_host op named "stop_request"
2. Block I/O scheduler should support re-insert API
3. eMMC card should support HPI (High Priority Interrupt) command
If any of the above dependencies are not met then urgent request mechanism
will not become operational.

Change-Id: I976af4fa794fb0fac701fe12f249c36d78080c6e
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
d2: Update defconfigs

 * Useful debugging options
 * QCOM's recommended power-saving options (RCU_FAST_NO_HZ, SCHED_MC)
 * Enable recommended networking options for JB-MR1

Change-Id: Ie204b95a92f1cac2a8516a6c7c9a173e29465dda