writeback: Do not sort b_io list only because of block device inode

It is very likely that block device inode will be part of BDI dirty list
as well. However it doesn't make sence to sort inodes on the b_io list
just because of this inode (as it contains buffers all over the device
anyway). So save some CPU cycles which is valuable since we hold relatively
contented wb->list_lock.

Signed-off-by: Jan Kara <jack@suse.cz>
rtc: Keep system awake until all expired RTC timers are handled

Current implementation of RTC interface allows for system suspend to
occur in the following cases:
(a) if a timer is set in the past and rtc_timer_do_work() is scheduled
to handle it, and
(b) if rtc_timer_do_work() is called to handle expired timers whose
handlers implement a preemption point.

A pending suspend request may be honoured in the above cases causing
timer handling to be delayed until after the next resume. This is
undesirable since timer handlers may have time-critical code to execute.

This patch makes sure that the system stays awake until all expired
timers are handled.

Note that all calls to pm_stay_awake() are eventually paired with
the single pm_relax() call in rtc_timer_do_work(), which is launched
using schedule_work().

Cc: Alessandro Zummo <a.zummo@towertech.it>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Arve Hjonnevag <arve@android.com>
Cc: Todd Poynor <toddpoynor@google.com>
Signed-off-by: Zoran Markovic <zoran.markovic@linaro.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
clocksource: Fix !CONFIG_CLOCKSOURCE_WATCHDOG compile

If I explicitly disable the clocksource watchdog in the x86 Kconfig,
the x86 kernel will not compile unless this is properly defined.

Cc: John Stultz <john.stultz@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: x86@kernel.org
Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
ARM: suspend: use hash of cpu_logical_map value to index into save array

Currently we hash the MPIDR of the CPU being suspended to determine which
entry in the sleep_save_sp array to use. In some situations, such as when
we want to resume on another physical CPU, the MPIDR of another CPU should
be used instead.

So let's use the value of cpu_logical_map(smp_processor_id()) in place
of the MPIDR in the suspend path.  This will result in the same index
being used as with the previous code unless the caller has modified
cpu_logical_map() beforehand with the MPIDR of the physical CPU the
suspending logical CPU will resume on.

Consequently, if doing a physical CPU migration, cpu_logical_map() must
be updated appropriately somewhere between cpu_pm_enter() and
cpu_suspend().

The register allocation in __cpu_suspend is reworked in order to better
accommodate the additional argument.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
ARM: gic: add CPU migration support

This is required by the big.LITTLE switcher code.

The gic_migrate_target() changes the CPU interface mapping for the
current CPU to redirect SGIs to the specified interface, and it also
updates the target CPU for each interrupts to that CPU interface
if they were targeting the current interface.  Finally, pending
SGIs for the current CPU are forwarded to the new interface.

Because Linux does not use it, the SGI source information for the
forwarded SGIs is not preserved.  Neither is the source information
for the SGIs sent by the current CPU to other CPUs adjusted to match
the new CPU interface mapping.  The required registers are banked so
only the target CPU could do it.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: b.L: core switcher code

This is the core code implementing big.LITTLE switcher functionality.
Rationale for this code is available here:

http://lwn.net/Articles/481055/

The main entry point for a switch request is:

void bL_switch_request(unsigned int cpu, unsigned int new_cluster_id)

If the calling CPU is not the wanted one, this wrapper takes care of
sending the request to the appropriate CPU with schedule_work_on().

At the moment the core switch operation is handled by bL_switch_to()
which must be called on the CPU for which a switch is requested.

What this code does:

  * Return early if the current cluster is the wanted one.

  * Close the gate in the kernel entry vector for both the inbound
    and outbound CPUs.

  * Wake up the inbound CPU so it can perform its reset sequence in
    parallel up to the kernel entry vector gate.

  * Migrate all interrupts in the GIC targeting the outbound CPU
    interface to the inbound CPU interface, including SGIs. This is
    performed by gic_migrate_target() in drivers/irqchip/irq-gic.c.

  * Call cpu_pm_enter() which takes care of flushing the VFP state to
    RAM and save the CPU interface config from the GIC to RAM.

  * Modify the cpu_logical_map to refer to the inbound physical CPU.

  * Call cpu_suspend() which saves the CPU state (general purpose
    registers, page table address) onto the stack and store the
    resulting stack pointer in an array indexed by the updated
    cpu_logical_map, then call the provided shutdown function.
    This happens in arch/arm/kernel/sleep.S.

At this point, the provided shutdown function executed by the outbound
CPU ungates the inbound CPU. Therefore the inbound CPU:

  * Picks up the saved stack pointer in the array indexed by its MPIDR
    in arch/arm/kernel/sleep.S.

  * The MMU and caches are re-enabled using the saved state on the
    provided stack, just like if this was a resume operation from a
    suspended state.

  * Then cpu_suspend() returns, although this is on the inbound CPU
    rather than the outbound CPU which called it initially.

  * The function cpu_pm_exit() is called which effect is to restore the
    CPU interface state in the GIC using the state previously saved by
    the outbound CPU.

  * Exit of bL_switch_to() to resume normal kernel execution on the
    new CPU.

However, the outbound CPU is potentially still running in parallel while
the inbound CPU is resuming normal kernel execution, hence we need
per CPU stack isolation to execute bL_do_switch().  After the outbound
CPU has ungated the inbound CPU, it calls mcpm_cpu_power_down() to:

  * Clean its L1 cache.

  * If it is the last CPU still alive in its cluster (last man standing),
    it also cleans its L2 cache and disables cache snooping from the other
    cluster.

  * Power down the CPU (or whole cluster).

Code called from bL_do_switch() might end up referencing 'current' for
some reasons.  However, 'current' is derived from the stack pointer.
With any arbitrary stack, the returned value for 'current' and any
dereferenced values through it are just random garbage which may lead to
segmentation faults.

The active page table during the execution of bL_do_switch() is also a
problem.  There is no guarantee that the inbound CPU won't destroy the
corresponding task which would free the attached page table while the
outbound CPU is still running and relying on it.

To solve both issues, we borrow some of the task space belonging to
the init/idle task which, by its nature, is lightly used and therefore
is unlikely to clash with our usage.  The init task is also never going
away.

Right now the logical CPU number is assumed to be equivalent to the
physical CPU number within each cluster. The kernel should also be
booted with only one cluster active.  These limitations will be lifted
eventually.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: bL_switcher: add clockevent save/restore support

Per-CPU timers that are shutdown when a CPU is switched over must be disabled
upon switching and reprogrammed on the inbound CPU by relying on the
clock events management API. save/restore sequence is executed with irqs
disabled as mandated by the clock events API.

The next_event is an absolute time, hence, when the inbound CPU resumes,
if the timer has expired the min delta is forced into the tick device to
fire after few cycles.

This patch adds switching support for clock events that are per-CPU and
have to be migrated when a switch takes place; the cpumask of the clock
event device is checked against the cpumask of the current cpu, and if
they match, the clockevent device mode is saved and it is put in
shutdown mode. Resume code reprogrammes the tick device accordingly.

Tested on A15/A7 fast models and architected timers.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: bL_switcher: move to dedicated threads rather than workqueues

The workqueues are problematic as they may be contended.
They can't be scheduled with top priority either.  Also the optimization
in bL_switch_request() to skip the workqueue entirely when the target CPU
and the calling CPU were the same didn't allow for bL_switch_request() to
be called from atomic context, as might be the case for some cpufreq
drivers.

Let's move to dedicated kthreads instead.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: bL_switcher: simplify stack isolation

We now have a dedicated thread for each logical CPU.  That's plenty
of stack space for our needs.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: bL_switcher: hot-unplug half of the available CPUs

In a regular kernel configuration, all the CPUs are initially available.
But the switcher execution model uses half of them at any time.  Instead
of hacking the DTB to remove half of the CPUs, let's remove them at
run time and make sure we still have a working switcher configuration.
This way, the same DTB can be used whether or not the switcher is used.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: bL_switcher: do not hardcode GIC IDs in the code

Currently, GIC IDs are hardcoded making the code dependent on the 4+4 b.L
configuration.  Let's allow for GIC IDs to be discovered upon switcher
initialization to support other b.L configurations such as the 1+1 one,
or 2+3 as on the VExpress TC2.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: bL_switcher: ability to enable and disable the switcher via sysfs

The /sys/kernel/bL_switcher/enable file allows to enable or disable
the switcher by writing 1 or 0 to it respectively.  It is still enabled
by default on boot.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: bL_switcher: add kernel cmdline param to disable the switcher on boot

By adding no_bL_switcher to the kernel cmdline string, the switcher
won't be activated automatically at boot time.  It is still possible
to activate it later with:

	echo 1 > /sys/kernel/bL_switcher/active

Signed-off-by: Nicolas Pitre <nico@linaro.org>
clocksource: Extract max nsec calculation into separate function

We need to calculate the same number in the clocksource code and
the sched_clock code, so extract this code into its own function.
We also drop the min_t and just use min() because the two types
are the same.

Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
sched_clock: Use seqcount instead of rolling our own

We're going to increase the cyc value to 64 bits in the near
future. Doing that is going to break the custom seqcount
implementation in the sched_clock code because 64 bit numbers
aren't guaranteed to be atomic. Replace the cyc_copy with a
seqcount to avoid this problem.

Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
sched_clock: Use an hrtimer instead of timer

In the next patch we're going to increase the number of bits that
the generic sched_clock can handle to be greater than 32. With
more than 32 bits the wraparound time can be larger than what can
fit into the units that msecs_to_jiffies takes (unsigned int).
Luckily, the wraparound is initially calculated in nanoseconds
which we can easily use with hrtimers, so switch to using an
hrtimer.

Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
[jstultz: Fixup hrtimer intitialization order issue]
Signed-off-by: John Stultz <john.stultz@linaro.org>
sched_clock: Add support for >32 bit sched_clock

The ARM architected system counter has at least 56 usable bits.
Add support for counters with more than 32 bits to the generic
sched_clock implementation so we can increase the time between
wakeups due to dealing with wrap-around on these devices while
benefiting from the irqtime accounting and suspend/resume
handling that the generic sched_clock code already has. On my
system using 56 bits over 32 bits changes the wraparound time
from a few minutes to an hour. For faster running counters (GHz
range) this is even more important because we may not be able to
execute the timer in time to deal with the wraparound if only 32
bits are used.

We choose a maxsec value of 3600 seconds because we assume no
system will go idle for more than an hour. In the future we may
need to increase this value.

Note: All users should switch over to the 64-bit read function so
we can remove setup_sched_clock() in favor of sched_clock_register().

Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
ARM: bL_switcher: remove assumptions between logical and physical CPUs

Up to now, the logical CPU was somehow tied to the physical CPU number
within a cluster.  This causes problems when forcing the boot CPU to be
different from the first enumerated CPU in the device tree creating a
discrepancy between logical and physical CPU numbers.

Let's make the pairing completely independent from physical CPU numbers.

Let's keep only those logical CPUs with same initial CPU cluster to create
a uniform scheduler profile without having to modify any of the probed
topology and compute capacity data.  This has the potential to create
a non contiguous CPU numbering space when the switcher is active with
potential impact on buggy user space tools.  It is however better to fix
those tools rather than making the switcher code more intrusive.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
Reviewed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
ARM: bL_switcher: filter CPU hotplug requests when the switcher is active

Trying to support both the switcher and CPU hotplug at the same time
is tricky due to ambiguous semantics.  So let's at least prevent users
from messing around with those logical CPUs the switcher has removed
and those which were not active when the switcher was activated.

Signed-off-by: Nicolas Pitre <nico@linaro.org>
ARM: bL_switcher: add a simple /dev user interface for debugging purposes

Only the basic call to aid debugging.

*** NOT FOR PRODUCTION ***

Usage:

	echo <cpuid>,<clusterid> > /dev/b.L_switcher

where <cpuid> is the logical CPU number, and <clusterid> is 0 for the
first cluster and 1 for the second cluster.

Signed-off-by: nicolas Pitre <nico@linaro.org>