mm: prevent mmap_cache race in find_vma()

find_vma() can be called by multiple threads with read lock
held on mm->mmap_sem and any of them can update mm->mmap_cache.
Prevent compiler from re-fetching mm->mmap_cache, because other
readers could update it in the meantime:

               thread 1                             thread 2
                                        |
  find_vma()                            |  find_vma()
    struct vm_area_struct *vma = NULL;  |
    vma = mm->mmap_cache;               |
    if (!(vma && vma->vm_end > addr     |
        && vma->vm_start <= addr)) {    |
                                        |    mm->mmap_cache = vma;
    return vma;                         |
     ^^ compiler may optimize this      |
        local variable out and re-read  |
        mm->mmap_cache                  |

This issue can be reproduced with gcc-4.8.0-1 on s390x by running
mallocstress testcase from LTP, which triggers:

  kernel BUG at mm/rmap.c:1088!
    Call Trace:
     ([<000003d100c57000>] 0x3d100c57000)
      [<000000000023a1c0>] do_wp_page+0x2fc/0xa88
      [<000000000023baae>] handle_pte_fault+0x41a/0xac8
      [<000000000023d832>] handle_mm_fault+0x17a/0x268
      [<000000000060507a>] do_protection_exception+0x1e2/0x394
      [<0000000000603a04>] pgm_check_handler+0x138/0x13c
      [<000003fffcf1f07a>] 0x3fffcf1f07a
    Last Breaking-Event-Address:
      [<000000000024755e>] page_add_new_anon_rmap+0xc2/0x168

Thanks to Jakub Jelinek for his insight on gcc and helping to
track this down.

Signed-off-by: Jan Stancek <jstancek@redhat.com>
Acked-by: David Rientjes <rientjes@google.com>
Signed-off-by: Hugh Dickins <hughd@google.com>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
We have 16GB internal storage. best to set 1024 and tune the EXTERNAL!
 kernel:sched: LOAD_FREQ (4*HZ+61) avoids loadavg Moire

LOAD_FREQ is (5*HZ+1) to avoid high load average when idle:
http://kerneltrap.org/mailarchive/linux-kernel/2007/10/3/328568

I suggest (4*HZ+61) for a better distribution.

With some seconds based load (like SSL heartbeats)
and LOAD_FREQ at (5*HZ+1) I see Moire patterns like inverse sawtooth,
since 2 or 3 probes hit the jobs (load increases quickly),
followed by several probes missing it.

A 4.61 sec interval gives optimal distribution over when within a
second a probe is taken, as .61 is close to golden ratio phi 1.618...
(test in http://ripke.com/goldenratio.c).

12*4.61 = 55.32 secs is still close to a minute,
and 13*4.61=59.93 is even closer than the current 12*5.01=60.12
(with exponents EXP_x adjusted to a ratio of 13 instead of 12).

DM: corrected to default all the #define EXP_*
 arm: Add auto hotplug driver This driver is simple and cheap and aims…

… to avoid unnecessary hotplug events and unnecessary task migrations. The design is simple. The value of nr_running() (which is the current number of runnable threads) is averaged over X samples by a ringbuffer. If the value is over a dynamically calculated value, all CPUs are onlined. When the value falls under another dynamically calculated value, the CPUs are offlined with a 2 second delay.

If there is persistent load on the system, the CPUs will not continually
thrash between onlining and offlining and continually migrate tasks.
The number of online CPUs will remain constant until the load has dropped
below the dynamically calculated value.

The online and offline values are dynamically scaled based on the number of
CPUs online, as is the sampling rate of the driver. This aims to reduce the
expense of the driver and also reduce the number of unless hotplug transitions
which could have been avoided had cpufreq been given the chance to ramp the
online CPUs up and deal with the load.

This driver will also offline secondary cores when the screen is turned off,
providing earlysuspend is available.
It also has a module param to set the minimum number of online CPUs:
/sys/module/auto_hotplug/parameters/min_online_cpus
This value is ignored when the device enters earlysuspend, as the secondary
CPUs are all offlined, but is set again when the screen is turned back on.

Signed-off-by: Seongmin Park <r_data@naver.com>
arm: auto_hotplug:
UPDATES:
Step1:
 v0.200 Add hotplug_boostpulse() function which is called from cpufreq and onlines an additional CPU instantly providing <50% of CPUs are online. This improves interactivity substantially.

Change schedule_work() functions to schedule_work_on() to ensure they are
always queued on the 1st CPU.

Fix the utter ugliness inside hotplug_offline_work_fn(), we now use
for_each_online_cpu() and break out after the first cpu_down() is called.

Fix a few variable names.

STEP2:
v0.201 Increase sampling periods to 25 with a minimum sampling rate of 20ms. This will give a best case average for any 500ms period, which works well with boostpulse, and it makes the overhead of the driver cheaper.

Don't attempt to online CPUs with boostpulse when early_suspend() has
been called.

Add some TODOs and comments to the code.

STEP3:
v0.210

*Use dedicated workqueue for better control and ability to easily flush.
*Implement HOTPLUG_PAUSED flag which allows sampling to continue but stops
any hotplug events from occurring unless the load spikes drastically high.
*Implement HOTPLUG_DISABLED flag which can be called externally as nvidia
likes to do at shutdown for T3.
*Add boostpulse sysfs interface (kanged from cpufreq.c) at:
/sys/devices/system/cpu/auto_hotplug/boostpulse
*Rewrite boostpulse to work in oneshot mode always. Fixed mode isn't
required any longer.
*Use 10 sampling periods of min 20ms each. Too many sampling periods was
causing spurious onlines because too much history was being averaged.
*Add ability to online only 1 CPU instead of all CPUs if load dictates.
*Change some of the delayed work items into normal work.
*Fix averaging error caused by avg_running not being reinitialised at each
invocation of hotplug_decision_work_fn()
*Rewrite the decision code to implement paused and disabled flags and use
else if instead of if if if.
*Tidy up the onlining and offlining work functions.
*Fix the bug causing all secondary cores to be offlined when hotplugging
was started. We now start sampling with the paused flag set which allows
the array to fill so the average will be representative of the current
system load. We clear the pause flag after 10 seconds to allow hotplugging.

STEP4:
v0.211 tentatively working

STEP5:
v0.220 * Now that it actually works properly, the locking is not required * Fixup decision logic and use consistent variables throughout * Use cancel_delayed_work() when necessary instead of blindly using cancel_delayed_work_sync() * Increase ENABLE_LOAD_THRESHOLD slightly to 225 to avoid most spurious cpu_up() calls

STEP6:
wrap all pr_info into the existing debug mode so we don't spam the kernel log
Enabled ARM_AUTO_HOTPLUG for testing + adapt to our structure.
auto_hotplug driver updates:
UPDATE1:
change the thresholds a little bit. Also changed how it behaves when it has 2 cores online. When the screen is touched the avg_running goes up and it onlines cpu1, but we don't want it to go up and down continuously while we are interacting with the device so now while the avg_running is >= 125 (disable_load/2) cpu1 stays online. Only when the avg_running goes below 125 which is mostly possible when we're not interacting with the device (e.g. reading a book or a website) cpu1 goes down, but also takes 3 seconds to go down just in case theres a new interaction in between. This should help a lot on battery life because hotplug is very costy and now it will be called much less than before.

Signed-off-by: franciscofranco <franciscofranco.1990@gmail.com>

UPDATE2:
remove the extra second for the scheduled delayed work or we could get into trouble.

UPDATE3:
Tune for battery life
Moved the auto_hotplug code to arch/arm/mach-exynos
+ patch-3.0.71-72 (& ext4-part from patch-3.4.38-39)
mm: prevent mmap_cache race in find_vma()

find_vma() can be called by multiple threads with read lock
held on mm->mmap_sem and any of them can update mm->mmap_cache.
Prevent compiler from re-fetching mm->mmap_cache, because other
readers could update it in the meantime:

               thread 1                             thread 2
                                        |
  find_vma()                            |  find_vma()
    struct vm_area_struct *vma = NULL;  |
    vma = mm->mmap_cache;               |
    if (!(vma && vma->vm_end > addr     |
        && vma->vm_start <= addr)) {    |
                                        |    mm->mmap_cache = vma;
    return vma;                         |
     ^^ compiler may optimize this      |
        local variable out and re-read  |
        mm->mmap_cache                  |

This issue can be reproduced with gcc-4.8.0-1 on s390x by running
mallocstress testcase from LTP, which triggers:

  kernel BUG at mm/rmap.c:1088!
    Call Trace:
     ([<000003d100c57000>] 0x3d100c57000)
      [<000000000023a1c0>] do_wp_page+0x2fc/0xa88
      [<000000000023baae>] handle_pte_fault+0x41a/0xac8
      [<000000000023d832>] handle_mm_fault+0x17a/0x268
      [<000000000060507a>] do_protection_exception+0x1e2/0x394
      [<0000000000603a04>] pgm_check_handler+0x138/0x13c
      [<000003fffcf1f07a>] 0x3fffcf1f07a
    Last Breaking-Event-Address:
      [<000000000024755e>] page_add_new_anon_rmap+0xc2/0x168

Thanks to Jakub Jelinek for his insight on gcc and helping to
track this down.

Signed-off-by: Jan Stancek <jstancek@redhat.com>
Acked-by: David Rientjes <rientjes@google.com>
Signed-off-by: Hugh Dickins <hughd@google.com>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Kernel IRQ: updates from 3.9.Y kernel.
ARM: smp: Make SMP operations mandatory

Now that all SMP platforms have been converted to use struct
smp_operations, remove the "weak" attribute from the hooks
in smp.c, and make the functions static wherever possible.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
ARM: exynos: remove incorrect BSYM usage

BSYM macro is only needed for assembly files and its usage in c files is
wrong, so remove it. The linker will correctly set bit 0 for Thumb2
kernels.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Cc: Dave Martin <dave.martin@linaro.org>
Cc: Kukjin Kim <kgene.kim@samsung.com>
ARM: SoC: convert Exynos4 to SMP operations

Convert Exynos4 to use struct smp_operations to provide its SMP
and CPU hotplug operations.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Tested-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
ARM: consolidate pen_release instead of having per platform definitions

Almost each SMP platform defines pen_release to manage booting secondary
CPUs. This of course clashes with the single zImage effort.

Add the pen_release definition to the ARM SMP code, and remove all others.
This should only be used by platforms which lack any kind of CPU power
management...

Reported-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
 ARM: GIC: remove direct use of gic_raise_softirq

In preparation of moving gic code to drivers/irqchip, remove the direct
platform dependencies on gic_raise_softirq. Move the setup of
smp_cross_call into the gic code and use arch_send_wakeup_ipi_mask
function to trigger wake-up IPIs.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Sascha Hauer <kernel@pengutronix.de>
Cc: David Brown <davidb@codeaurora.org>
Cc: Daniel Walker <dwalker@fifo99.com>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Acked-by: Tony Lindgren <tony@atomide.com>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Magnus Damm <magnus.damm@gmail.com>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Shiraz Hashim <shiraz.hashim@st.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
Cc: Srinidhi Kasagar <srinidhi.kasagar@stericsson.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Acked-by: Olof Johansson <olof@lixom.net>
ARM: virt: Add boot-time diagnostics

In order to easily detect pathological cases, print some diagnostics
when the kernel boots.

This also provides helpers to detect that HYP mode is actually available,
which can be used by other subsystems to enable HYP specific features.

Signed-off-by: Dave Martin <dave.martin@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
ARM: 7536/1: smp: Formalize an IPI for wakeup

Remove the offset from ipi_msg_type and assume that SGI0 is the
wakeup interrupt now that all WFI hotplug users call
gic_raise_softirq() with 0 instead of 1. This allows us to
track how many wakeup interrupts are sent and also removes the
unknown IPI printk message for WFI hotplug based systems.

Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7408/1: cacheflush: return error to userspace when flushing syscall fails

The cacheflush syscall can fail for two reasons:

(1) The arguments are invalid (nonsensical address range or no VMA)

(2) The region generates a translation fault on a VIPT or PIPT cache

This patch allows do_cache_op to return an error code to userspace in
the case of the above. The various coherent_user_range implementations
are modified to return 0 in the case of VIVT caches or -EFAULT in the
case of an abort on v6/v7 cores.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: mm: implement LoUIS API for cache maintenance ops

ARM v7 architecture introduced the concept of cache levels and related
control registers. New processors like A7 and A15 embed an L2 unified cache
controller that becomes part of the cache level hierarchy. Some operations in
the kernel like cpu_suspend and __cpu_disable do not require a flush of the
entire cache hierarchy to DRAM but just the cache levels belonging to the
Level of Unification Inner Shareable (LoUIS), which in most of ARM v7 systems
correspond to L1.

The current cache flushing API used in cpu_suspend and __cpu_disable,
flush_cache_all(), ends up flushing the whole cache hierarchy since for
v7 it cleans and invalidates all cache levels up to Level of Coherency
(LoC) which cripples system performance when used in hot paths like hotplug
and cpuidle.

Therefore a new kernel cache maintenance API must be added to cope with
latest ARM system requirements.

This patch adds flush_cache_louis() to the ARM kernel cache maintenance API.

This function cleans and invalidates all data cache levels up to the
Level of Unification Inner Shareable (LoUIS) and invalidates the instruction
cache for processors that support it (> v7).

This patch also creates an alias of the cache LoUIS function to flush_kern_all
for all processor versions prior to v7, so that the current cache flushing
behaviour is unchanged for those processors.

v7 cache maintenance code implements a cache LoUIS function that cleans and
invalidates the D-cache up to LoUIS and invalidates the I-cache, according
to the new API.

Reviewed-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: Shawn Guo <shawn.guo@linaro.org>