smp: Fix SMP function call empty cpu mask race

I get the following warning every day with v3.7, once or
twice a day:

  [ 2235.186027] WARNING: at /mnt/sda7/kernel/linux/arch/x86/kernel/apic/ipi.c:109 default_send_IPI_mask_logical+0x2f/0xb8()

As explained by Linus as well:

 |
 | Once we've done the "list_add_rcu()" to add it to the
 | queue, we can have (another) IPI to the target CPU that can
 | now see it and clear the mask.
 |
 | So by the time we get to actually send the IPI, the mask might
 | have been cleared by another IPI.
 |

This patch also fixes a system hang problem, if the data->cpumask
gets cleared after passing this point:

        if (WARN_ONCE(!mask, "empty IPI mask"))
                return;

then the problem in commit 83d349f35e1a ("x86: don't send an IPI to
the empty set of CPU's") will happen again.

Signed-off-by: Wang YanQing <udknight@gmail.com>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: peterz@infradead.org
Cc: mina86@mina86.org
Cc: srivatsa.bhat@linux.vnet.ibm.com
Cc: <stable@kernel.org>
Link: http://lkml.kernel.org/r/20130126075357.GA3205@udknight
[ Tidied up the changelog and the comment in the code. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>

smp: make smp_call_function_many() use logic similar to smp_call_function_single()

I'm testing swapout workload in a two-socket Xeon machine.  The workload
has 10 threads, each thread sequentially accesses separate memory
region.  TLB flush overhead is very big in the workload.  For each page,
page reclaim need move it from active lru list and then unmap it.  Both
need a TLB flush.  And this is a multthread workload, TLB flush happens
in 10 CPUs.  In X86, TLB flush uses generic smp_call)function.  So this
workload stress smp_call_function_many heavily.

Without patch, perf shows:
+  24.49%  [k] generic_smp_call_function_interrupt
-  21.72%  [k] _raw_spin_lock
   - _raw_spin_lock
      + 79.80% __page_check_address
      + 6.42% generic_smp_call_function_interrupt
      + 3.31% get_swap_page
      + 2.37% free_pcppages_bulk
      + 1.75% handle_pte_fault
      + 1.54% put_super
      + 1.41% grab_super_passive
      + 1.36% __swap_duplicate
      + 0.68% blk_flush_plug_list
      + 0.62% swap_info_get
+   6.55%  [k] flush_tlb_func
+   6.46%  [k] smp_call_function_many
+   5.09%  [k] call_function_interrupt
+   4.75%  [k] default_send_IPI_mask_sequence_phys
+   2.18%  [k] find_next_bit

swapout throughput is around 1300M/s.

With the patch, perf shows:
-  27.23%  [k] _raw_spin_lock
   - _raw_spin_lock
      + 80.53% __page_check_address
      + 8.39% generic_smp_call_function_single_interrupt
      + 2.44% get_swap_page
      + 1.76% free_pcppages_bulk
      + 1.40% handle_pte_fault
      + 1.15% __swap_duplicate
      + 1.05% put_super
      + 0.98% grab_super_passive
      + 0.86% blk_flush_plug_list
      + 0.57% swap_info_get
+   8.25%  [k] default_send_IPI_mask_sequence_phys
+   7.55%  [k] call_function_interrupt
+   7.47%  [k] smp_call_function_many
+   7.25%  [k] flush_tlb_func
+   3.81%  [k] _raw_spin_lock_irqsave
+   3.78%  [k] generic_smp_call_function_single_interrupt

swapout throughput is around 1400M/s.  So there is around a 7%
improvement, and total cpu utilization doesn't change.

Without the patch, cfd_data is shared by all CPUs.
generic_smp_call_function_interrupt does read/write cfd_data several times
which will create a lot of cache ping-pong.  With the patch, the data
becomes per-cpu.  The ping-pong is avoided.  And from the perf data, this
doesn't make call_single_queue lock contend.

Next step is to remove generic_smp_call_function_interrupt() from arch
code.

Signed-off-by: Shaohua Li <shli@fusionio.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Conflicts:
	include/linux/smp.h

Change-Id: I71e497695fa343b3edc6ec77a6dc1698f0e07a86

[franciscofranco: slightly altered the original patch to fix merge
conflict]
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
cpuidle: check for need_resched() after the idle entry trace event

trace_cpu_idle_rcuidle() can end up waking ksoftirqd under certain
conditions. Since there is no check for need_resched() following
this wake up, the cpu ends up going into an idle state despite there
being a runnable task on that runqueue. Furthermore, since the
resched flag is set, any sucessive waking up tasks on the idle cpu
will not result in an IPI being sent to that cpu. Consequently tasks
on that runqueue will starve until the cpu exits idle by itself. Fix
this problem by adding a need_resched() check after the idle entry
trace event.

A side effect of this change is that the idle trace entry and exit
event may be recorded for an extremely short duration even though
the cpu does not actually enter idle.

Change-Id: I370c39a1e39ecd829e44ac783b4888a00ab05eae
Signed-off-by: Syed Rameez Mustafa <rameezmustafa@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>
ARM: gic: Bring inline with upstream

Change-Id: Ic2cce8da16c81b9448e1f2ea8813ae05b48f1e31

ARM: gic: fix restore order of distributor registers in gic_cpu_restore()

Similar to gic_dist_restore(), the GIC config, priority and enable
registers should be restored in that order. Fix this in gic_cpu_restore()
which currently restores the enable registers before the configuration
registers.

Change-Id: Ib75640849bef81aa87a3b110fe1bd7ea4371d659
Signed-off-by: Osvaldo Banuelos <osvaldob@codeaurora.org>

ARM: GIC: fix GIC cpumask initialization

Punit Agrawal reports:
> I was trying to boot 3.8-rc5 on Realview EB 11MPCore using
> realview-smp_defconfig as a starting point but the kernel failed to
> progress past the log below (config attached).
>
> Pawel suggested I try reverting 384a290283f - "ARM: gic: use a private
> mapping for CPU target interfaces" that you've authored. With this
> commit reverted the kernel boots.
>
> I am not quite sure why the commit breaks 11MPCore but Pawel (cc'd)
> might be able to shed light on that.

Some early GIC implementations return zero for the first distributor
CPU routing register.  This means we can't rely on that telling us
which CPU interface we're connected to.  We know that these platforms
implement PPIs for IRQs 29-31 - but we shouldn't assume that these
will always be populated.

So, instead, scan for a non-zero CPU routing register in the first
32 IRQs and use that as our CPU mask.

Change-Id: I264edd7273353e300d7d1ef096709f9d87b9ec78
Reported-by: Punit Agrawal <punit.agrawal@arm.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

irqchip: gic: Perform the gic_secondary_init() call via CPU notifier

All the calls to gic_secondary_init() pass 0 as the first argument.
Since this function is called on each CPU when starting, it can be done
in a platform-independent way via a CPU notifier registered by the GIC
code.

Change-Id: I729e09097230eb2f93fb730f1d58a279c5ef51f8
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Rob Herring <rob.herring@calxeda.com>
Acked-by: Simon Horman <horms+renesas@verge.net.au>
Tested-by: Simon Horman <horms+renesas@verge.net.au>
Acked-by: Srinidhi Kasagar <srinidhi.kasagar@stericsson.com>
Tested-by: Dinh Nguyen <dinguyen@altera.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Tested-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Sascha Hauer <kernel@pengutronix.de>
Cc: David Brown <davidb@codeaurora.org>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Shiraz Hashim <shiraz.hashim@st.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Barry Song <baohua.song@csr.com>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>

Conflicts:
	arch/arm/common/gic.c
qseecom: change the number of bytes to compare app names

keymaster app has two names with different length: "keymaster" and
"keymaste". To compare the app names correctly, the shorter name
length is choosen for memcmp in __qseecom_send_cmd.

Change-Id: I5f0dff7f9756362be40cc64985b65bc2268d734f
Signed-off-by: Zhen Kong <zkong@codeaurora.org>
Signed-off-by: William Clark <wclark@codeaurora.org>
Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>