ARM: SMP: use a timing out completion for cpu hotplug

    ARM: SMP: use a timing out completion for cpu hotplug

    Rather than open-coding the jiffy-based wait, and polling for the
    secondary CPU to come online, use a completion instead.  This
    removes the need to poll, instead we will be notified when the
    secondary CPU has initialized.

    Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7060/1: smp: populate logical CPU mapping during boot

To allow booting Linux on a CPU with physical ID != 0, we need to
provide a mapping from the logical CPU number to the physical CPU
number.

This patch adds such a mapping and populates it during boot.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7011/1: Add ARM cpu topology definition

The affinity between ARM processors is defined in the MPIDR register.
We can identify which processors are in the same cluster,
and which ones have performance interdependency. We can define the
cpu topology of ARM platform, that is then used by sched_mc and sched_smt.

The default state of sched_mc and sched_smt config is disable.
When enabled, the behavior of the scheduler can be modified with
sched_mc_power_savings and sched_smt_power_savings sysfs interfaces.

Changes since v4 :
*  Remove unnecessary parentheses and blank lines

Changes since v3 :
* Update the format of printk message
* Remove blank line

Changes since v2 :
* Update the commit message and some comments

Changes since v1 :
* Update the commit message
* Add read_cpuid_mpidr in arch/arm/include/asm/cputype.h
* Modify header of arch/arm/kernel/topology.c
* Modify tests and manipulation of MPIDR's bitfields
* Modify the place and dependancy of the config
* Modify Noop functions

Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Amit Kucheria <amit.kucheria@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7123/1: smp: Add an IPI handler callable from C code

In order to be able to handle IPI directly from C code instead of
assembly code, introduce handle_IPI(), which is modeled after handle_IRQ().

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7124/1: smp: Add a localtimer handler callable from C code

In order to be able to handle localtimer directly from C code instead of
assembly code, introduce handle_local_timer(), which is modeled after
handle_IRQ().

Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: gic: consolidate PPI handling

PPI handling is a bit of an odd beast. It uses its own low level
handling code and is hardwired to the local timers (hence lacking
a registration interface).

Instead, switch the low handling to the normal SPI handling code.
PPIs are handled by the handle_percpu_devid_irq flow.

This also allows the removal of some duplicated code.

Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: David Brown <davidb@codeaurora.org>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: David Brown <davidb@codeaurora.org>
Tested-by: David Brown <davidb@codeaurora.org>
Tested-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>

Conflicts:
	arch/arm/kernel/smp.c
	arch/arm/mach-exynos/include/mach/entry-macro.S
fiq_debugger.c: Fix local_timer_irqs errors
ARM: restart: remove argument to setup_mm_for_reboot()

setup_mm_for_reboot() doesn't make use of its argument, so remove it.

Acked-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: H Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: Tony Lindgren <tony@atomide.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: idmap: populate identity map pgd at init time using .init.text

When disabling and re-enabling the MMU, it is necessary to take out an
identity mapping for the code that manipulates the SCTLR in order to
avoid it disappearing from under our feet. This is useful when soft
rebooting and returning from CPU suspend.

This patch allocates a set of page tables during boot and populates them
with an identity mapping for the .idmap.text section. This means that
users of the identity map do not need to manage their own pgd and can
instead annotate their functions with __idmap or, in the case of assembly
code, place them in the correct section.

Acked-by: Dave Martin <dave.martin@linaro.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
ARM: idmap: use idmap_pgd when setting up mm for reboot

For soft-rebooting a system, it is necessary to map the MMU-off code
with an identity mapping so that execution can continue safely once the
MMU has been switched off.

Currently, switch_mm_for_reboot takes out a 1:1 mapping from 0x0 to
TASK_SIZE during reboot in the hope that the reset code lives at a
physical address corresponding to a userspace virtual address.

This patch modifies the code so that we switch to the idmap_pgd tables,
which contain a 1:1 mapping of the cpu_reset code. This has the
advantage of only remapping the code that we need and also means we
don't need to worry about allocating a pgd from an atomic context in the
case that the physical address of the cpu_reset code aliases with the
virtual space used by the kernel.

Acked-by: Dave Martin <dave.martin@linaro.org>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
ARM: SMP: use idmap_pgd for mapping MMU enable during secondary booting

The ARM SMP booting code allocates a temporary set of page tables
containing an identity mapping of the kernel image and provides this
to secondary CPUs for initial booting.

In reality, we only need to include the __turn_mmu_on function in the
identity mapping since the rest of the kernel is executing from virtual
addresses after this point.

This patch adds __turn_mmu_on to the .idmap.text section, allowing the
SMP booting code to use the idmap_pgd directly and not have to populate
its own set of page table.

As a result of this patch, we can make the identity_mapping_add function
static (since it is only used within mm/idmap.c) and also remove the
identity_mapping_del function. The identity map population is moved to
an early initcall so that it is setup in time for secondary CPU bringup.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
ARM: stop: execute platform callback from cpu_stop code

Sending IPI_CPU_STOP to a CPU causes it to execute a busy cpu_relax
loop forever. This makes it impossible to kexec successfully on an SMP
system since the secondary CPUs do not reset.

This patch adds a callback to platform_cpu_kill, defined when
CONFIG_HOTPLUG_CPU=y, from the ipi_cpu_stop handling code. This function
currently just returns 1 on all platforms that define it but allows them
to do something more sophisticated in the future.

Signed-off-by: Will Deacon <will.deacon@arm.com>

Conflicts:
	arch/arm/kernel/smp.c
ARM: 7293/1: logical_cpu_map: decouple CPU mapping from SMP

It turns out that the logical CPU mapping is useful even when !CONFIG_SMP
for manipulation of devices like interrupt and power controllers when
running a UP kernel on a CPU other than 0. This can happen when kexecing
a UP image from an SMP kernel.

In the future, multi-cluster systems running AMP configurations will
require something similar for mapping cluster IDs, so it makes sense to
decouple this logic in preparation for this support.

Acked-by: Yang Bai <hamo.by@gmail.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reported-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

Conflicts:
	arch/arm/mach-exynos/hotplug.c
	arch/arm/mach-exynos/platsmp.c
remove references to cpu_*_map in arch/

This has been obsolescent for a while; time for the final push.

In adjacent context, replaced old cpus_* with cpumask_*.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Acked-by: David S. Miller <davem@davemloft.net> (arch/sparc)
Acked-by: Chris Metcalf <cmetcalf@tilera.com> (arch/tile)
Cc: user-mode-linux-devel@lists.sourceforge.net
Cc: Russell King <linux@arm.linux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Richard Kuo <rkuo@codeaurora.org>
Cc: linux-hexagon@vger.kernel.org
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: Kyle McMartin <kyle@mcmartin.ca>
Cc: Helge Deller <deller@gmx.de>
Cc: sparclinux@vger.kernel.org
ARM: 7405/1: kexec: call platform_cpu_kill on the killer rather than the victim

When performing a kexec on an SMP system, the secondary cores are stopped
by calling machine_shutdown(), which in turn issues IPIs to offline the
other CPUs. Unfortunately, this isn't enough to reboot the cores into
a new kernel (since they are just executing a cpu_relax loop somewhere
in memory) so we make use of platform_cpu_kill, part of the CPU hotplug
implementation, to place the cores somewhere safe. This function expects
to be called on the killing CPU for each core that it takes out.

This patch moves the platform_cpu_kill callback out of the IPI handler
and into smp_send_stop, therefore ensuring that it executes on the
killing CPU rather than on the victim, matching what the hotplug code
requires.

Cc: stable@vger.kernel.org
Reported-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
arm: use clear_tasks_mm_cpumask()

Checking for process->mm is not enough because process' main thread may
exit or detach its mm via use_mm(), but other threads may still have a
valid mm.

To fix this we would need to use find_lock_task_mm(), which would walk up
all threads and returns an appropriate task (with task lock held).

clear_tasks_mm_cpumask() has this issue fixed, so let's use it.

Suggested-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Anton Vorontsov <anton.vorontsov@linaro.org>
Cc: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
ARM: 7480/1: only call smp_send_stop() on SMP

On reboot or poweroff (machine_shutdown()) a call to smp_send_stop() is
made (to stop the others CPU's) when CONFIG_SMP=y.

arch/arm/kernel/process.c:

void machine_shutdown(void)
{
 #ifdef CONFIG_SMP
       smp_send_stop();
 #endif
}

smp_send_stop() calls the function pointer smp_cross_call(), which is set
on the smp_init_cpus() function for OMAP processors.

arch/arm/mach-omap2/omap-smp.c:

void __init smp_init_cpus(void)
{
...
	set_smp_cross_call(gic_raise_softirq);
...
}

But the ARM setup_arch() function only calls smp_init_cpus()
if CONFIG_SMP=y && is_smp().

arm/kernel/setup.c:

void __init setup_arch(char **cmdline_p)
{
...
 #ifdef CONFIG_SMP
	if (is_smp())
		smp_init_cpus();
 #endif
...
}

Newer OMAP CPU's are SMP machines so omap2plus_defconfig sets
CONFIG_SMP=y. Unfortunately on an OMAP UP machine is_smp()
returns false and smp_init_cpus() is never called and the
smp_cross_call() function remains NULL.

If the machine is rebooted or powered off, smp_send_stop() will
be called (since CONFIG_SMP=y) leading to the following error:

[   42.815551] Restarting system.
[   42.819030] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[   42.827667] pgd = d7a74000
[   42.830566] [00000000] *pgd=96ce7831, *pte=00000000, *ppte=00000000
[   42.837249] Internal error: Oops: 80000007 [#1] SMP ARM
[   42.842773] Modules linked in:
[   42.846008] CPU: 0    Not tainted  (3.5.0-rc3-next-20120622-00002-g62e87ba-dirty #44)
[   42.854278] PC is at 0x0
[   42.856994] LR is at smp_send_stop+0x4c/0xe4
[   42.861511] pc : [<00000000>]    lr : [<c00183a4>]    psr: 60000013
[   42.861511] sp : d6c85e70  ip : 00000000  fp : 00000000
[   42.873626] r10: 00000000  r9 : d6c84000  r8 : 00000002
[   42.879150] r7 : c07235a0  r6 : c06dd2d0  r5 : 000f4241  r4 : d6c85e74
[   42.886047] r3 : 00000000  r2 : 00000000  r1 : 00000006  r0 : d6c85e74
[   42.892944] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   42.900482] Control: 10c5387d  Table: 97a74019  DAC: 00000015
[   42.906555] Process reboot (pid: 1166, stack limit = 0xd6c842f8)
[   42.912902] Stack: (0xd6c85e70 to 0xd6c86000)
[   42.917510] 5e60:                                     c07235a0 00000000 00000000 d6c84000
[   42.926177] 5e80: 01234567 c00143d0 4321fedc c00511bc d6c85ebc 00000168 00000460 00000000
[   42.934814] 5ea0: c1017950 a0000013 c1017900 d8014390 d7ec3858 c0498e48 c1017950 00000000
[   42.943481] 5ec0: d6ddde10 d6c85f78 00000003 00000000 d6ddde10 d6c84000 00000000 00000000
[   42.952117] 5ee0: 00000002 00000000 00000000 c0088c88 00000002 00000000 00000000 c00f4b90
[   42.960784] 5f00: 00000000 d6c85ebc d8014390 d7e311c8 60000013 00000103 00000002 d6c84000
[   42.969421] 5f20: c00f3274 d6e00a00 00000001 60000013 d6c84000 00000000 00000000 c00895d4
[   42.978057] 5f40: 00000002 d8007c80 d781f000 c00f6150 d8010cc0 c00f3274 d781f000 d6c84000
[   42.986694] 5f60: c0013020 d6e00a00 00000001 20000010 0001257c ef000000 00000000 c00895d4
[   42.995361] 5f80: 00000002 00000001 00000003 00000000 00000001 00000003 00000000 00000058
[   43.003997] 5fa0: c00130c8 c0012f00 00000001 00000003 fee1dead 28121969 01234567 00000002
[   43.012634] 5fc0: 00000001 00000003 00000000 00000058 00012584 0001257c 00000001 00000000
[   43.021270] 5fe0: 000124bc bec5cc6c 00008f9c 4a2f7c40 20000010 fee1dead 00000000 00000000
[   43.029968] [<c00183a4>] (smp_send_stop+0x4c/0xe4) from [<c00143d0>] (machine_restart+0xc/0x4c)
[   43.039154] [<c00143d0>] (machine_restart+0xc/0x4c) from [<c00511bc>] (sys_reboot+0x144/0x1f0)
[   43.048278] [<c00511bc>] (sys_reboot+0x144/0x1f0) from [<c0012f00>] (ret_fast_syscall+0x0/0x3c)
[   43.057464] Code: bad PC value
[   43.060760] ---[ end trace c3988d1dd0b8f0fb ]---

Add a check so smp_cross_call() is only called when there is more than one CPU on-line.

Cc: <stable@vger.kernel.org>
Signed-off-by: Javier Martinez Canillas <javier at dowhile0.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Added missing function for last commit.
ondemand: cpuidle detection

I found a problem with the ondemand governor while in earlysuspend
suspend mode (ondemand keeping the freq close to the max at all times
despite near 0% load). The problem is that the cpu starts to go in
long cpuidle cycles, a sampling_rate of 10000 (usec) is applied only
once in a while and the real sampling_rate becomes 1 second (or more).

So when a wakeup happens we return to the 10000usec sampling rate. The
ondemand sees lots of activity after the wakeup but those "loads" must
be adjusted down if the previous wall_time delta was huge and the
current one is tiny. We're too close to long cpuidle to worry about
the cpu freq anyway and it may be just a jitter load that if we take
into account without adjusting it down, will lead to the next long
cpuidle to be entered at the max freq again, and this repeats forever.

So my solution is to tweak the ondemand to scale down the "load"
according to the decrease in the wall_time delta ratio
(deep_sleep_ratio variable). This allows the CPU to stay at the lowest
freq during the deep sleeps even when the ondemand governor is enabled
and the sampling_rate is set to 10000. (setting the sampling_rate to
values >100000 would also tend to hide the problem but it'd screw the
interactive behavior by running at the lowest frequency for too long
during interactive usage)

During interactive usage with the screen on (not in earlysuspend
mode), or during playback or voip with screen off the deep_sleep_rate
is set to 1 practically all the time so it won't alter the behavior of
the ondemand governor unless the system is very idle (and in turn when
we want it to stay at low freq).

Change-Id: If0391e6d6c41c1c8c9fa590502e88e681a800b04

Signed-off-by: Andrea Arcangeli <andrea@cpushare.com>
mutex: Place lock in contended state after fastpath_lock failure

ARM recently moved to asm-generic/mutex-xchg.h for its mutex
implementation after the previous implementation was found to be missing
some crucial memory barriers. However, this has revealed some problems
running hackbench on SMP platforms due to the way in which the
MUTEX_SPIN_ON_OWNER code operates.

The symptoms are that a bunch of hackbench tasks are left waiting on an
unlocked mutex and therefore never get woken up to claim it. This boils
down to the following sequence of events:

        Task A        Task B        Task C        Lock value
0                                                     1
1       lock()                                        0
2                     lock()                          0
3                     spin(A)                         0
4       unlock()                                      1
5                                   lock()            0
6                     cmpxchg(1,0)                    0
7                     contended()                    -1
8       lock()                                        0
9       spin(C)                                       0
10                                  unlock()          1
11      cmpxchg(1,0)                                  0
12      unlock()                                      1

At this point, the lock is unlocked, but Task B is in an uninterruptible
sleep with nobody to wake it up.

This patch fixes the problem by ensuring we put the lock into the
contended state if we fail to acquire it on the fastpath, ensuring that
any blocked waiters are woken up when the mutex is released.

Signed-off-by: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: <stable@vger.kernel.org>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-6e9lrw2avczr0617fzl5vqb8@git.kernel.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
(cherry picked from commit 8c0fa5328bc6a981025020be011296c05d6fac8b)