nfs: writeback pages wait queue

The generic writeback routines are departing from congestion_wait()
in preference of get_request_wait(), aka. waiting on the block queues.

Introduce the missing writeback wait queue for NFS, otherwise its
writeback pages will grow greedily, exhausting all PG_dirty pages.

Tests show that it can effectively reduce stalls in the disk-network
pipeline, improve performance and reduce delays.

The test cases are basically

	for run in 1 2 3
	for nr_dd in 1 10 100
	for dirty_thresh in 10M 100M 1000M 2G
		start $nr_dd dd's writing to a 1-disk mem=12G NFS server

During all tests, nfs_congestion_kb is set to 1/8 dirty_thresh.

    3.2.0-rc1    3.2.0-rc1-ioless-full+
  (w/o patch)                (w/ patch)
  -----------  ------------------------
        20.66      +136.7%        48.90  thresh=1000M/nfs-100dd-1
        20.82      +147.5%        51.52  thresh=1000M/nfs-100dd-2
        20.57      +129.8%        47.26  thresh=1000M/nfs-100dd-3
        35.96       +96.5%        70.67  thresh=1000M/nfs-10dd-1
        37.47       +89.1%        70.85  thresh=1000M/nfs-10dd-2
        34.55      +106.1%        71.21  thresh=1000M/nfs-10dd-3
        58.24       +28.2%        74.63  thresh=1000M/nfs-1dd-1
        59.83       +18.6%        70.93  thresh=1000M/nfs-1dd-2
        58.30       +31.4%        76.61  thresh=1000M/nfs-1dd-3
        23.69       -10.0%        21.33  thresh=100M/nfs-100dd-1
        23.59        -1.7%        23.19  thresh=100M/nfs-100dd-2
        23.94        -1.0%        23.70  thresh=100M/nfs-100dd-3
        27.06        -0.0%        27.06  thresh=100M/nfs-10dd-1
        25.43        +4.8%        26.66  thresh=100M/nfs-10dd-2
        27.21        -0.8%        26.99  thresh=100M/nfs-10dd-3
        53.82        +4.4%        56.17  thresh=100M/nfs-1dd-1
        55.80        +4.2%        58.12  thresh=100M/nfs-1dd-2
        55.75        +2.9%        57.37  thresh=100M/nfs-1dd-3
        15.47        +1.3%        15.68  thresh=10M/nfs-10dd-1
        16.09        -3.5%        15.53  thresh=10M/nfs-10dd-2
        15.09        -0.9%        14.96  thresh=10M/nfs-10dd-3
        26.65       +13.0%        30.10  thresh=10M/nfs-1dd-1
        25.09        +7.7%        27.02  thresh=10M/nfs-1dd-2
        27.16        +3.3%        28.06  thresh=10M/nfs-1dd-3
        27.51       +78.6%        49.11  thresh=2G/nfs-100dd-1
        22.46      +131.6%        52.01  thresh=2G/nfs-100dd-2
        12.95      +289.8%        50.50  thresh=2G/nfs-100dd-3
        42.28       +81.0%        76.52  thresh=2G/nfs-10dd-1
        40.33       +78.8%        72.10  thresh=2G/nfs-10dd-2
        42.52       +67.6%        71.27  thresh=2G/nfs-10dd-3
        62.27       +34.6%        83.84  thresh=2G/nfs-1dd-1
        60.10       +35.6%        81.48  thresh=2G/nfs-1dd-2
        66.29       +17.5%        77.88  thresh=2G/nfs-1dd-3
      1164.97       +41.6%      1649.19  TOTAL write_bw
The local queue time for WRITE RPCs could be reduced by several orders!

    3.2.0-rc1    3.2.0-rc1-ioless-full+
  -----------  ------------------------
     90226.82       -99.9%        92.07  thresh=1000M/nfs-100dd-1
     88904.27       -99.9%        80.21  thresh=1000M/nfs-100dd-2
     97436.73       -99.9%        87.32  thresh=1000M/nfs-100dd-3
     62167.19       -99.3%       444.25  thresh=1000M/nfs-10dd-1
     64150.34       -99.2%       539.38  thresh=1000M/nfs-10dd-2
     78675.54       -99.3%       540.27  thresh=1000M/nfs-10dd-3
      5372.84       +57.8%      8477.45  thresh=1000M/nfs-1dd-1
     10245.66       -51.2%      4995.71  thresh=1000M/nfs-1dd-2
      4744.06      +109.1%      9919.55  thresh=1000M/nfs-1dd-3
      1727.29        -9.6%      1562.16  thresh=100M/nfs-100dd-1
      2183.49        +4.4%      2280.21  thresh=100M/nfs-100dd-2
      2201.49        +3.7%      2281.92  thresh=100M/nfs-100dd-3
      6213.73       +19.9%      7448.13  thresh=100M/nfs-10dd-1
      8127.01        +3.2%      8387.06  thresh=100M/nfs-10dd-2
      7255.35        +4.4%      7571.11  thresh=100M/nfs-10dd-3
      1144.67       +20.4%      1378.01  thresh=100M/nfs-1dd-1
      1010.02       +19.0%      1202.22  thresh=100M/nfs-1dd-2
       906.33       +15.8%      1049.76  thresh=100M/nfs-1dd-3
       642.82       +17.3%       753.80  thresh=10M/nfs-10dd-1
       766.82       -21.7%       600.18  thresh=10M/nfs-10dd-2
       575.95       +16.5%       670.85  thresh=10M/nfs-10dd-3
        21.91       +71.0%        37.47  thresh=10M/nfs-1dd-1
        16.70      +105.3%        34.29  thresh=10M/nfs-1dd-2
        19.05       -71.3%         5.47  thresh=10M/nfs-1dd-3
    123877.11       -99.0%      1187.27  thresh=2G/nfs-100dd-1
    122353.65       -98.8%      1505.84  thresh=2G/nfs-100dd-2
    101140.82       -98.4%      1641.03  thresh=2G/nfs-100dd-3
     78248.51       -98.9%       892.00  thresh=2G/nfs-10dd-1
     84589.42       -98.6%      1212.17  thresh=2G/nfs-10dd-2
     89684.95       -99.4%       495.28  thresh=2G/nfs-10dd-3
     10405.39        -6.9%      9684.57  thresh=2G/nfs-1dd-1
     16151.86       -48.5%      8316.69  thresh=2G/nfs-1dd-2
     16119.17       -49.0%      8214.84  thresh=2G/nfs-1dd-3
   1177306.98       -92.1%     93588.50  TOTAL nfs_write_queue_time
The average COMMIT size is not impacted too much.

    3.2.0-rc1    3.2.0-rc1-ioless-full+
  -----------  ------------------------
         5.56       +44.9%         8.06  thresh=1000M/nfs-100dd-1
         4.14      +109.1%         8.67  thresh=1000M/nfs-100dd-2
         5.46       +16.3%         6.35  thresh=1000M/nfs-100dd-3
        52.04        -8.4%        47.70  thresh=1000M/nfs-10dd-1
        52.33       -13.8%        45.09  thresh=1000M/nfs-10dd-2
        51.72        -9.2%        46.98  thresh=1000M/nfs-10dd-3
       484.63        -8.6%       443.16  thresh=1000M/nfs-1dd-1
       492.42        -8.2%       452.26  thresh=1000M/nfs-1dd-2
       493.13       -11.4%       437.15  thresh=1000M/nfs-1dd-3
        32.52       -72.9%         8.80  thresh=100M/nfs-100dd-1
        36.15       +26.1%        45.58  thresh=100M/nfs-100dd-2
        38.33        +0.4%        38.49  thresh=100M/nfs-100dd-3
         5.67        +0.5%         5.69  thresh=100M/nfs-10dd-1
         5.74        -1.1%         5.68  thresh=100M/nfs-10dd-2
         5.69        +0.9%         5.74  thresh=100M/nfs-10dd-3
        44.91        -1.0%        44.45  thresh=100M/nfs-1dd-1
        44.22        -0.6%        43.96  thresh=100M/nfs-1dd-2
        44.18        +0.2%        44.28  thresh=100M/nfs-1dd-3
         1.42        +1.1%         1.43  thresh=10M/nfs-10dd-1
         1.48        +0.3%         1.48  thresh=10M/nfs-10dd-2
         1.43        -1.0%         1.42  thresh=10M/nfs-10dd-3
         5.51        -6.8%         5.14  thresh=10M/nfs-1dd-1
         5.91        -8.1%         5.43  thresh=10M/nfs-1dd-2
         5.44        +3.0%         5.61  thresh=10M/nfs-1dd-3
         8.80        +6.6%         9.38  thresh=2G/nfs-100dd-1
         8.51       +65.2%        14.06  thresh=2G/nfs-100dd-2
        15.28       -13.2%        13.27  thresh=2G/nfs-100dd-3
       105.12       -24.9%        78.99  thresh=2G/nfs-10dd-1
       101.90        -9.1%        92.60  thresh=2G/nfs-10dd-2
       106.24       -29.7%        74.65  thresh=2G/nfs-10dd-3
       909.85        +0.4%       913.68  thresh=2G/nfs-1dd-1
      1030.45       -18.3%       841.68  thresh=2G/nfs-1dd-2
      1016.56       -11.6%       898.36  thresh=2G/nfs-1dd-3
      5222.74       -10.1%      4695.25  TOTAL nfs_commit_size
And here is the list of overall numbers.

    3.2.0-rc1    3.2.0-rc1-ioless-full+
  -----------  ------------------------
      1164.97       +41.6%      1649.19  TOTAL write_bw
     54799.00       +25.0%     68500.00  TOTAL nfs_nr_commits
   3543263.00        -3.3%   3425418.00  TOTAL nfs_nr_writes
      5222.74       -10.1%      4695.25  TOTAL nfs_commit_size
         7.62       +89.2%        14.42  TOTAL nfs_write_size
   1177306.98       -92.1%     93588.50  TOTAL nfs_write_queue_time
      5977.02       -16.0%      5019.34  TOTAL nfs_write_rtt_time
   1183360.15       -91.7%     98645.74  TOTAL nfs_write_execute_time
     51186.59       -62.5%     19170.98  TOTAL nfs_commit_queue_time
     81801.14        +3.6%     84735.19  TOTAL nfs_commit_rtt_time
    133015.32       -21.9%    103926.05  TOTAL nfs_commit_execute_time
Feng: do more coarse grained throttle on each ->writepages rather than
on each page, for better performance and avoid throttled-before-send-rpc
deadlock

Signed-off-by: Feng Tang <feng.tang@intel.com>
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
CFS Auto Group V2 - Changes since last:
- switch to per session vs tty
- make autogroups visible in /proc/sched_debug
- make autogroups visible in /proc/<pid>/autogroup
- add nice level bandwidth tweakability to /proc/<pid>/autogroup

Modulo "kill it" debate outcome...

A recurring complaint from CFS users is that parallel kbuild has a negative
impact on desktop interactivity.  This patch implements an idea from Linus,
to automatically create task groups.  This patch only per session autogroups,
but leaves the way open for enhancement.

Implementation: each task's signal struct contains an inherited pointer to a
refcounted autogroup struct containing a task group pointer, the default for
all tasks pointing to the init_task_group.  When a task calls setsid(), the
process wide reference to the default group is dropped, a new task group is
created, and the process is moved into the new task group.  Children thereafter
inherit this task group, and increase it's refcount.  On exit, a reference to the
current task group is dropped when the last reference to each signal struct is
dropped.  The task group is destroyed when the last signal struct referencing
it is freed.   At runqueue selection time, IFF a task has no cgroup assignment,
it's current autogroup is used.

Autogroup bandwidth is controllable via setting it's nice level through the
proc filesystem.  cat /proc/<pid>/autogroup displays the task's group and
the group's nice level.  echo <nice level> > /proc/<pid>/autogroup sets the
task group's shares to the weight of nice <level> task.  Setting nice level
is rate limited for !admin users due to the abuse risk of task group locking.

The feature is enabled from boot by default if CONFIG_SCHED_AUTOGROUP is
selected, but can be disabled via the boot option noautogroup, and can be
also be turned on/off on the fly via..
   echo [01] > /proc/sys/kernel/sched_autogroup_enabled.
..which will automatically move tasks to/from the root task group.

Signed-off-by: Mike Galbraith <efault@gmx.de>

Conflicts:
	kernel/sched.c
msm: Add sysfs interface for mfreq

Add mfreq interface in sysfs.  This forces the system to use the
maximum CPU clock on all CPU cores.
(mfreq=1 enables, mfreq=0 disables)

Signed-off-by: David Ng <dave@codeaurora.org>
(cherry picked from commit 14f38b6c25a3a63dc25fd4448c0b14171eabd59a)

Change-Id: I543098931798204cb3c431d245e279cc244b803b
Signed-off-by: Shruthi Krishna <skrish@codeaurora.org>
mmc: core: put eMMC in sleep (cmd5) mode before suspend

Put MMC to sleep if it supports SLEEP/AWAKE (CMD5) in the mmc suspend
to minimize power consumption.

Signed-off-by: Balaji T K <balajitk@ti.com>
Acked-by: Venkatraman S <svenkatr@ti.com>
Signed-off-by: Chris Ball <cjb@laptop.org>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
(cherry picked from commit 3765839119eb540b87bdbf8d6869c60658cb99e7)

Change-Id: I0c0d5d06c4204b15471d1ca95aec0c7fcaaae816
Signed-off-by: Shruthi Krishna <skrish@codeaurora.org>
Patched and Backported Joe's RCU to 2.6.35 kernel

Conflicts:
	include/linux/hardirq.h
	init/Kconfig
	kernel/rcupdate.c
ARM: 6527/1: Use CTR instead of CCSIDR for the D-cache line size on ARMv7

The current implementation of the dcache_line_size macro reads the L1
cache size from the CCSIDR register. This, however, is not guaranteed to
be the smallest cache line in the cache hierarchy. The patch changes to
the macro to use the more architecturally correct CTR register.

Change-Id: I9904d7830cb53fb493c52d1381fc5b78dd1a44ef
CRs-fixed: 294396
Reported-by: Kevin Sapp <ksapp@quicinc.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
staging:lowmemkiller add Fugeswap

fudgeswap acts as follows:

If set to non zero (defualt is 512k):
	Check for the amount of SWAP_FREE space avalible
	If > 0KB is avalible:
		if fudgeswap > swapfree:
			other_file += swapfree
		else:
			other_file += fugeswap

In short: we will add in fugeswap as long as its less then the free swap

Setting this to a very large positive number will indicate swap ought
to be fully used as free (and will slow the system down)

smaller numbers will allow you to put some pressure on SWAP without
slowing the system down as much.

small negitive numbers will allow the system to be faster at the same
minfree level.

default is 512 to give a very little bit of pressure to use some swap
but this can be modified at runtime via:
/sys/module/lowmemorykiller/parameters/fugeswap
originally by ezterry, modifiedy by faux123
sched_fair: Remove cpu from nohz idle set if not active

Currently, when determining which cpu should do load balancing in nohz
mode, we still consider cpus that have been hotremoved. This is a
problem because, if we decide to try and move the load balancing to a
hotremoved cpu, we will spin and stop doing load balancing at all. This
will also make us cease doing any idle power collapse until the cpu has
been brought back online.

Fix this by removing a cpu from the nohz idle mask if we find that it
isn't active when we consider it for load balancing.

Change-Id: I93a86fdcab88964a452cb3f38e4815db13d0ee6c
CRs-Fixed: 290042
Signed-off-by: Jeff Ohlstein <johlstei@codeaurora.org>
Signed-off-by: Raj Kushwaha <rajk@codeaurora.org>
CPU hotplug, PM: Remove unused symbol 'suspend_cpu_hotplug'

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
sched: Do not block when waiting to free old root domain

Instead of calling synchronize_sched(), directly invoke
call_rcu_sched() which will not block the cpu hotplug
operation but will call free_rootdomain when the rcu
process is complete. This is safe because the root
domains are allocated dynamically so any new readers
will have pointers to the new data and the new data
does not overwrite the old data.

This is a partial backport from 3.0.0 kernel.
I did not backport the dynamic allocation of the sched
domain and sched group data structures.  This would
require inclusion of 10 other patches and would not be
stable.

dce840a08702bd13a9a186e07e63d1ef82256b5e
Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
Date:   Thu Apr 7 14:09:50 2011 +0200

    sched: Dynamically allocate sched_domain/sched_group data-structures

    Instead of relying on static allocations for the sched_domain and
    sched_group trees, dynamically allocate and RCU free them.

    Allocating this dynamically also allows for some build_sched_groups()
    simplification since we can now (like with other simplifications) rely
    on the sched_domain tree instead of hard-coded knowledge.

    One tricky to note is that detach_destroy_domains() needs to hold
    rcu_read_lock() over the entire tear-down, per-cpu is not sufficient
    since that can lead to partial sched_group existance (could possibly
    be solved by doing the tear-down backwards but this is much more
    robust).

    A concequence of the above is that we can no longer print the
    sched_domain debug stuff from cpu_attach_domain() since that might now
    run with preemption disabled (due to classic RCU etc.) and
    sched_domain_debug() does some GFP_KERNEL allocations.

    Another thing to note is that we now fully rely on normal RCU and not
    RCU-sched, this is because with the new and exiting RCU flavours we
    grew over the years BH doesn't necessarily hold off RCU-sched grace
    periods (-rt is known to break this). This would in fact already cause
    us grief since we do sched_domain/sched_group iterations from softirq
    context.

    This patch is somewhat larger than I would like it to be, but I didn't
    find any means of shrinking/splitting this.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Nick Piggin <npiggin@kernel.dk>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
    Link: http://lkml.kernel.org/r/20110407122942.245307941@chello.nl
Signed-off-by: Ingo Molnar <mingo@elte.hu>

Signed-off-by: Maya Spivak <mspivak@codeaurora.org>
(cherry picked from commit 09d39b8ade298ffed33597926a1b30f4609400a6)

Change-Id: I7305af949900919f06cf3444daf1183370aa07b7
Signed-off-by: Swetha Basineni <sbasin@codeaurora.org>
partition_sched_domains: Do not destroy old sched domain on cpu_up

This is safe on a cpu_up only. Although a reader may still have
access to the old scheduling domain data, the data indicates that
the new CPU is not up, and therefore the only limitation is the
new CPU will not be schedulable by that reader until that reader
receives the new data.

Signed-off-by: Maya Spivak <mspivak@codeaurora.org>
(cherry picked from commit f963077fba323bd06814639a691bd905006bb541)

Change-Id: I15b3e0fd8c182e14db1c14f2a45b467f24d41cc5
Signed-off-by: Swetha Basineni <sbasin@codeaurora.org>
PM / Sleep: Remove pm_runtime_suspended() from __pm_generic_call()

From: Rafael J. Wysocki <rjw@sisk.pl>

The pm_runtime_suspended() check in __pm_generic_call() doesn't
really help and may cause problems to happen, because in some cases
the system suspend callbacks need to be called even if the given
device has been suspended by runtime PM.  For example, if the device
generally supports remote wakeup and is not enabled to wake up
the system from sleep, it should be prevented from generating wakeup
signals during system suspend and that has to be done by the
suspend callbacks that the pm_runtime_suspended() check prevents from
being executed.  For this reason, remove the pm_runtime_suspended()
check from __pm_generic_call().

Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
PM / Suspend: Fix bug in suspend statistics update

A pair of missing braces inside the state_store() function causes even
invalid arguments to suspend to be wrongly treated as failed suspend
attempts. Fix this.

Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
ARM: Add optimised swahb32() byteswap helper for v6 and above

ARMv6 and later processors have the REV16 instruction, which swaps
the bytes within each halfword of a register value.

This is already used to implement swab16(), but since the native
operation performaed by REV16 is actually swahb32(), this patch
renames the existing swab16() helper accordingly and defines
__arch_swab16() in terms of it.  This allows calls to both swab16()
and swahb32() to be optimised.

The compiler's generated code might improve someday, but as of
4.5.2 the code generated for pure C implementing these 16-bit
bytesswaps remains pessimal.

swahb32() is useful for converting 32-bit Thumb instructions
between integer and memory representation on BE8 platforms (among
other uses).

Signed-off-by: Dave Martin <dave.martin@xxxxxxxxxx>
Revert "msm: cpufreq: Create cpufreq workqueue for freq changes"

This reverts commit 1164a5c7c1088b9f22ddc73a2e67d866b9ab19d6.

Conflicts:

	arch/arm/mach-msm/cpufreq.c
Revert "partition_sched_domains: Do not destroy old sched domain on cpu_up"

This reverts commit ec8e19b8ec34a4d2b173ffcda3e1f5110d5647f8.
idr: make idr_get_next() good for rcu_read_lock()

Make one small adjustment to idr_get_next(): take the height from the
top layer (stable under RCU) instead of from the root (unprotected by
RCU), as idr_find() does: so that it can be used with RCU locking.
Copied comment on RCU locking from idr_find().

Signed-off-by: Hugh Dickins <hughd@google.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Acked-by: Li Zefan <lizf@cn.fujitsu.com>
memcg: let css_get_next() rely upon rcu_read_lock()

Remove lock and unlock around css_get_next()'s call to idr_get_next().
memcg iterators (only users of css_get_next) already did rcu_read_lock(),
and its comment demands that; but add a WARN_ON_ONCE to make sure of it.

Signed-off-by: Hugh Dickins <hughd@google.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Acked-by: Li Zefan <lizf@cn.fujitsu.com>
update defconfig