mm: retry page fault when blocking on disk transfer

This change reduces mmap_sem hold times that are caused by waiting for
disk transfers when accessing file mapped VMAs.

It introduces the VM_FAULT_ALLOW_RETRY flag, which indicates that the call
site wants mmap_sem to be released if blocking on a pending disk transfer.
In that case, filemap_fault() returns the VM_FAULT_RETRY status bit and
do_page_fault() will then re-acquire mmap_sem and retry the page fault.

It is expected that the retry will hit the same page which will now be
cached, and thus it will complete with a low mmap_sem hold time.

Tests:

- microbenchmark: thread A mmaps a large file and does random read accesses
  to the mmaped area - achieves about 55 iterations/s. Thread B does
  mmap/munmap in a loop at a separate location - achieves 55 iterations/s
  before, 15000 iterations/s after.

- We are seeing related effects in some applications in house, which show
  significant performance regressions when running without this change.

[akpm@linux-foundation.org: fix warning & crash]
Signed-off-by: Michel Lespinasse <walken@google.com>
Acked-by: Rik van Riel <riel@redhat.com>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Reviewed-by: Wu Fengguang <fengguang.wu@intel.com>
Cc: Ying Han <yinghan@google.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: "H. Peter Anvin" <hpa@zytor.com>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
sched_fair: Remove cpu from nohz idle set if not active

Currently, when determining which cpu should do load balancing in nohz
mode, we still consider cpus that have been hotremoved. This is a
problem because, if we decide to try and move the load balancing to a
hotremoved cpu, we will spin and stop doing load balancing at all. This
will also make us cease doing any idle power collapse until the cpu has
been brought back online.

Fix this by removing a cpu from the nohz idle mask if we find that it
isn't active when we consider it for load balancing.

Change-Id: I93a86fdcab88964a452cb3f38e4815db13d0ee6c
CRs-Fixed: 290042
Signed-off-by: Jeff Ohlstein <johlstei@codeaurora.org>
Signed-off-by: Raj Kushwaha <rajk@codeaurora.org>
ARM: 6528/1: Use CTR for the I-cache line size on ARMv7

The current implementation of the v7_coherent_*_range function assumes
that the D and I cache lines have the same size, which is incorrect
architecturally. This patch adds the icache_line_size macro which reads
the CTR register. The main loop in v7_coherent_*_range is split in two
independent loops or the D and I caches. This also has the performance
advantage that the DSB is moved outside the main loop.

Change-Id: Ibba459ab2b9f0310cdbe02ce9adaedd396bb2175
CRs-fixed: 294396
Reported-by: Kevin Sapp <ksapp@quicinc.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
[smuckle@codeaurora.org: added change to Sc45nm workaround]
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
Signed-off-by: Jin Hong <jinh@codeaurora.org>

Conflicts:

	arch/arm/mm/cache-v7.S