Revert "mm: micro-optimise slab to avoid a function call"

This reverts commit 381760eadc393bcb1bb328510ad75cf13431806d.
iov_iter: move into its own file

This moves the iov_iter functions in to their own file.  We're going to
be working on them in upcoming patches.  They become sufficiently large,
and remain self-contained, to justify seperating them from the rest of
the huge mm/filemap.c.

Signed-off-by: Dave Kleikamp <dave.kleikamp@oracle.com>
Acked-by: Jeff Moyer <jmoyer@redhat.com>
Cc: Zach Brown <zab@zabbo.net>
dio: Convert direct_IO to use iov_iter

Change the direct_IO aop to take an iov_iter argument rather than an iovec.
This will get passed down through most filesystems so that only the
__blockdev_direct_IO helper need be aware of whether user or kernel memory
is being passed to the function.

Signed-off-by: Dave Kleikamp <dave.kleikamp@oracle.com>
heavily backported for Mako from LKML reference

Signed-off-by: faux123 <reioux@gmail.com>
fs: pull iov_iter use higher up the stack

Right now only callers of generic_perform_write() pack their iovec
arguments into an iov_iter structure.  All the callers higher up in the
stack work on raw iovec arguments.

This patch introduces the use of the iov_iter abstraction higher up the
stack.  Private generic path functions are changed to operation on
iov_iter instead of on raw iovecs.  Exported interfaces that take iovecs
immediately pack their arguments into an iov_iter and call into the
shared functions.

File operation struct functions are added with iov_iter as an argument
so that callers to the generic file system functions can specify
abstract memory rather than iovec arrays only.

Almost all of this patch only transforms arguments and shouldn't change
functionality.  The buffered read path is the exception.  We add a
read_actor function which uses the iov_iter helper functions instead of
operating on each individual iovec element.  This may improve
performance as the iov_iter helper can copy multiple iovec elements from
one mapped page cache page.

As always, the direct IO path is special.  Sadly, it may still be
cleanest to have it work on the underlying memory structures directly
instead of working through the iov_iter abstraction.

Signed-off-by: Dave Kleikamp <dave.kleikamp@oracle.com>
Cc: Zach Brown <zab@zabbo.net>
modified for Mako from LKML reference

Signed-off-by: faux123 <reioux@gmail.com>
tmpfs: add support for read_iter and write_iter

Convert tmpfs do_shmem_file_read() to shmem_file_read_iter().
Make file_read_iter_actor() global so tmpfs can use it too: delete
file_read_actor(), which was made global in 2.4.4 for use by tmpfs.
Replace tmpfs generic_file_aio_write() by generic_file_write_iter().

Signed-off-by: Hugh Dickins <hughd@google.com>
modified for Mako from LKML reference

Signed-off-by: faux123 <reioux@gmail.com>
mm: Clear page active before releasing pages

Active pages should not be freed to the page allocator as it triggers
a bad page state warning. Fengguang Wu reported the following
bug and bisected it to the patch "mm: remove lru parameter from
__lru_cache_add and lru_cache_add_lru" which is currently in mmotm as
mm-remove-lru-parameter-from-__lru_cache_add-and-lru_cache_add_lru.patch

[   84.212960] BUG: Bad page state in process rm  pfn:0b0c9
[   84.214682] page:ffff88000d646240 count:0 mapcount:0 mapping:          (null) index:0x0
[   84.216883] page flags: 0x20000000004c(referenced|uptodate|active)
[   84.218697] CPU: 1 PID: 283 Comm: rm Not tainted 3.10.0-rc4-04361-geeb9bfc #49
[   84.220729]  ffff88000d646240 ffff88000d179bb8 ffffffff82562956 ffff88000d179bd8
[   84.223242]  ffffffff811333f1 000020000000004c ffff88000d646240 ffff88000d179c28
[   84.225387]  ffffffff811346a4 ffff880000270000 0000000000000000 0000000000000006
[   84.227294] Call Trace:
[   84.227867]  [<ffffffff82562956>] dump_stack+0x27/0x30
[   84.229045]  [<ffffffff811333f1>] bad_page+0x130/0x158
[   84.230261]  [<ffffffff811346a4>] free_pages_prepare+0x8b/0x1e3
[   84.231765]  [<ffffffff8113542a>] free_hot_cold_page+0x28/0x1cf
[   84.233171]  [<ffffffff82585830>] ? _raw_spin_unlock_irqrestore+0x6b/0xc6
[   84.234822]  [<ffffffff81135b59>] free_hot_cold_page_list+0x30/0x5a
[   84.236311]  [<ffffffff8113a4ed>] release_pages+0x251/0x267
[   84.237653]  [<ffffffff8112a88d>] ? delete_from_page_cache+0x48/0x9e
[   84.239142]  [<ffffffff8113ad93>] __pagevec_release+0x2b/0x3d
[   84.240473]  [<ffffffff8113b45a>] truncate_inode_pages_range+0x1b0/0x7ce
[   84.242032]  [<ffffffff810e76ab>] ? put_lock_stats.isra.20+0x1c/0x53
[   84.243480]  [<ffffffff810e77f5>] ? lock_release_holdtime+0x113/0x11f
[   84.244935]  [<ffffffff8113ba8c>] truncate_inode_pages+0x14/0x1d
[   84.246337]  [<ffffffff8119b3ef>] evict+0x11f/0x232
[   84.247501]  [<ffffffff8119c527>] iput+0x1a5/0x218
[   84.248607]  [<ffffffff8118f015>] do_unlinkat+0x19b/0x25a
[   84.249828]  [<ffffffff810ea993>] ? trace_hardirqs_on_caller+0x210/0x2ce
[   84.251382]  [<ffffffff8144372e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[   84.252879]  [<ffffffff8118f10d>] SyS_unlinkat+0x39/0x4c
[   84.254174]  [<ffffffff825874d6>] system_call_fastpath+0x1a/0x1f
[   84.255596] Disabling lock debugging due to kernel taint
The problem was that a page marked for activation was released via
pagevec. This patch clears the active bit before freeing in this case.

Signed-off-by: Mel Gorman <mgorman@suse.de>
Reported-and-Tested-by: Fengguang Wu <fengguang.wu@intel.com>
Signed-off-by: Paul Reioux <reioux@gmail.com>
bdi: use deferable timer for sync_supers task

sync_supers task currently wakes up periodically for superblock
writeback. This hurts power on battery driven devices. This patch
turns this housekeeping timer into a deferable timer so that it
does not fire when system is really idle.

CRs-Fixed: 353700
Change-Id: Idc7953b5d0580546808bc5832291ca570837ee7f
Signed-off-by: Yong Wang <yong.y.wang@intel.com>
Signed-off-by: Xia Wu <xia.wu@intel.com>
Signed-off-by: Krishna Vanka <kvanka@codeaurora.org>
Patch-mainline: v3.5
Git-commit: 19b52abe3c5d759661500a1dc810924369b2ad46
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
mm: cma: don't replace lowmem pages with highmem

The filesystem layer expects pages in the block device's mapping to not
be in highmem (the mapping's gfp mask is set in bdget()), but CMA can
currently replace lowmem pages with highmem pages, leading to crashes in
filesystem code such as the one below:

  Unable to handle kernel NULL pointer dereference at virtual address 00000400
  pgd = c0c98000
  [00000400] *pgd=00c91831, *pte=00000000, *ppte=00000000
  Internal error: Oops: 817 [#1] PREEMPT SMP ARM
  CPU: 0    Not tainted  (3.5.0-rc5+ #80)
  PC is at __memzero+0x24/0x80
  ...
  Process fsstress (pid: 323, stack limit = 0xc0cbc2f0)
  Backtrace:
  [<c010e3f0>] (ext4_getblk+0x0/0x180) from [<c010e58c>] (ext4_bread+0x1c/0x98)
  [<c010e570>] (ext4_bread+0x0/0x98) from [<c0117944>] (ext4_mkdir+0x160/0x3bc)
   r4:c15337f0
  [<c01177e4>] (ext4_mkdir+0x0/0x3bc) from [<c00c29e0>] (vfs_mkdir+0x8c/0x98)
  [<c00c2954>] (vfs_mkdir+0x0/0x98) from [<c00c2a60>] (sys_mkdirat+0x74/0xac)
   r6:00000000 r5:c152eb40 r4:000001ff r3:c14b43f0
  [<c00c29ec>] (sys_mkdirat+0x0/0xac) from [<c00c2ab8>] (sys_mkdir+0x20/0x24)
   r6:beccdcf0 r5:00074000 r4:beccdbbc
  [<c00c2a98>] (sys_mkdir+0x0/0x24) from [<c000e3c0>] (ret_fast_syscall+0x0/0x30)

Fix this by replacing only highmem pages with highmem.

Change-Id: I6af2d509af48b5a586037be14bd3593b3f269d95
Reported-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Rabin Vincent <rabin@rab.in>
Acked-by: Michal Nazarewicz <mina86@mina86.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Patch-mainline: v3.5
Git-commit: 6a6dccba2fdc2a69f1f36b8f1c0acc8598e7221b
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
mm: split_free_page ignore memory watermarks for CMA

Memory watermarks were sometimes preventing CMA allocations
in low memory.

Change-Id: I550ec987cbd6bc6dadd72b4a764df20cd0758479
Signed-off-by: Liam Mark <lmark@codeaurora.org>
mm: Use aligned zone start for pfn_to_bitidx calculation

The current calculation in pfn_to_bitidx assumes that
(pfn - zone->zone_start_pfn) >> pageblock_order will return the
same bit for all pfn in a pageblock. If zone_start_pfn is not
aligned to pageblock_nr_pages, this may not always be correct.

Consider the following with pageblock order = 10, zone start 2MB:

pfn     | pfn - zone start | (pfn - zone start) >> page block order
----------------------------------------------------------------
0x26000 | 0x25e00	   |  0x97
0x26100 | 0x25f00	   |  0x97
0x26200 | 0x26000	   |  0x98
0x26300 | 0x26100	   |  0x98

This means that calling {get,set}_pageblock_migratetype on a single
page will not set the migratetype for the full block. Fix this by
rounding down zone_start_pfn when doing the bitidx calculation.

Change-Id: I13e2f53f50db294f38ec86138c17c6fe29f0ee82
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: fix migration mode

__alloc_contig_migrate_range calls migrate_pages with wrong argument
for migrate_mode. Fix it.

Change-Id: I84697cf7c6aef6253e9ee7e5b3028c946b95e253
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>
Acked-by: Michal Nazarewicz <mina86@mina86.com>
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Patch-mainline: v3.5
Git-commit: 58f42fd54144346898e6dc6d6ae3acd4c591b42f
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
mm: Add is_cma_pageblock definition

Bring back the is_cma_pageblock definition for determining if a
page is CMA or not.

Change-Id: I39fd546e22e240b752244832c79514f109c8e84b
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
mm: Don't use CMA pages for writes

If CMA pages are used for writes, the writes may not complete
fast enough for CMA to be allocated within a reasonable amount
of time. If we get a CMA page, get another one to use instead.

Change-Id: I19d8ba655da7525d68d5947337d500566998971c
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma: fix race condition on a page

cruel, brute-force method for letting cma/migration to
finish its job without stealing the lock
migration_entry_wait() and creating a live-lock on the
faulted page. This patch solves the case of
page->_count == 2 migration failure.

Change-Id: Ia94542a80e44a213831291af289bbf5ee6880bfd
Signed-off-by: Heesub Shin <heesub.shin@samsung.com>
Reviewed-on: http://165.213.202.130:8080/39341
Tested-by: System S/W SCM <scm.systemsw@samsung.com>
Tested-by: Dongjun Shin <d.j.shin@samsung.com>
Reviewed-by: Hyunju Ahn <hyunju.ahn@samsung.com>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
cma : use migrate_prep() instead of migrate_prep_local()

__alloc_contig_migrate_range() should use all possible ways to get all the
pages migrated from the given memory range, so pruning per-cpu lru lists
for all CPUs is required, regadless the cost of such operation. Otherwise
some pages which got stuck at per-cpu lru list might get missed by
migration procedure causing the contiguous allocation to fail.

Change-Id: I70cc0864c57dd49e89f57797122a3fd0f300647a
Signed-off-by: woojoong.lee <woojoong.lee@samsung.com>
Reviewed-on: http://165.213.202.130:8080/43063
Tested-by: System S/W SCM <scm.systemsw@samsung.com>
Reviewed-by: daeho jeong <daeho.jeong@samsung.com>
Reviewed-by: Jeong-Ho Kim <jammer@samsung.com>
Tested-by: Jeong-Ho Kim <jammer@samsung.com>
[lauraa@codeaurora.org: Applied to correct file]
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
mm: make counts of CMA free pages correct

Both patches needed, second patch (among other things) fixes
a bug in the first.

commit 2139cbe627b8910ded55148f87ee10f7485408ed
Author: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Date:   Mon Oct 8 16:32:00 2012 -0700

    cma: fix counting of isolated pages

    Isolated free pages shouldn't be accounted to NR_FREE_PAGES counter.  Fix
    it by properly decreasing/increasing NR_FREE_PAGES counter in
    set_migratetype_isolate()/unset_migratetype_isolate() and removing counter
    adjustment for isolated pages from free_one_page() and split_free_page().

    Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
    Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
    Cc: Marek Szyprowski <m.szyprowski@samsung.com>
    Cc: Michal Nazarewicz <mina86@mina86.com>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Hugh Dickins <hughd@google.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    [lbassel@codeaurora.org: backport from 3.7, small changes needed]
    Signed-off-by: Larry Bassel <lbassel@codeaurora.org>

commit d1ce749a0db12202b711d1aba1d29e823034648d
Author: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Date:   Mon Oct 8 16:32:02 2012 -0700

    cma: count free CMA pages

    Add NR_FREE_CMA_PAGES counter to be later used for checking watermark in
    __zone_watermark_ok().  For simplicity and to avoid #ifdef hell make this
    counter always available (not only when CONFIG_CMA=y).

    [akpm@linux-foundation.org: use conventional migratetype naming]
    Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
    Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
    Cc: Marek Szyprowski <m.szyprowski@samsung.com>
    Cc: Michal Nazarewicz <mina86@mina86.com>
    Cc: Minchan Kim <minchan@kernel.org>
    Cc: Mel Gorman <mgorman@suse.de>
    Cc: Hugh Dickins <hughd@google.com>
    Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
    [lbassel@codeaurora.org: backport from 3.7, small changes needed]
    Signed-off-by: Larry Bassel <lbassel@codeaurora.org>

Change-Id: I3c694620021efa5ddee1b9dcd87f95b63ad8557c
Signed-off-by: Larry Bassel <lbassel@codeaurora.org>
fs: fuse: Workaround for CMA migration

The FUSE file system may hold references to pages for long
periods of time, preventing migration from occuring. If a CMA
page is used here, CMA allocations may fail. Work around this
by swapping out a CMA page for a non-CMA page when working with
the FUSE file system.

Change-Id: Id763ea833ee125c8732ae3759ec9e20d94aa8424
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
mm: Use correct define for CMA features

CMA features may ifdef out parts of the code with
CONFIG_CMA. Older code uses CONFIG_DMA_CMA. Switch
to using the newer CONFIG_CMA to ensure the code gets
compiled when needed.

Change-Id: I2ed8020e7b2f8319382b10c7c7c6818eb52995b6
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
mm: show migration types in show_mem

This is useful to diagnose the reason for page allocation failure for
cases where there appear to be several free pages.

Example, with this alloc_pages(GFP_ATOMIC) failure:

 swapper/0: page allocation failure: order:0, mode:0x0
 ...
 Mem-info:
 Normal per-cpu:
 CPU    0: hi:   90, btch:  15 usd:  48
 CPU    1: hi:   90, btch:  15 usd:  21
 active_anon:0 inactive_anon:0 isolated_anon:0
  active_file:0 inactive_file:84 isolated_file:0
  unevictable:0 dirty:0 writeback:0 unstable:0
  free:4026 slab_reclaimable:75 slab_unreclaimable:484
  mapped:0 shmem:0 pagetables:0 bounce:0
 Normal free:16104kB min:2296kB low:2868kB high:3444kB active_anon:0kB
 inactive_anon:0kB active_file:0kB inactive_file:336kB unevictable:0kB
 isolated(anon):0kB isolated(file):0kB present:331776kB mlocked:0kB
 dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:300kB
 slab_unreclaimable:1936kB kernel_stack:328kB pagetables:0kB unstable:0kB
 bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
 lowmem_reserve[]: 0 0

Before the patch, it's hard (for me, at least) to say why all these free
chunks weren't considered for allocation:

 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 1*256kB 1*512kB
 1*1024kB 1*2048kB 3*4096kB = 16128kB

After the patch, it's obvious that the reason is that all of these are
in the MIGRATE_CMA (C) freelist:

 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 1*256kB (C) 1*512kB
 (C) 1*1024kB (C) 1*2048kB (C) 3*4096kB (C) = 16128kB

Change-Id: Ic5fe77d762e0c03715bfb917774e7c4f03ac43f5
Signed-off-by: Rabin Vincent <rabin.vincent@stericsson.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Patch-mainline: v3.8
Git-commit: 377e4f167664d8bc390c04c911d846366000159c
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
mm: cma: on movable allocations try MIGRATE_CMA first

It has been observed that system tends to keep a lot of CMA free pages
even in very high memory pressure use cases.  The CMA fallback for
movable pages is used very rarely, only when system is completely
pruned from MOVABLE pages.  This means that the out-of-memory is
triggered for unmovable allocations even when there are many CMA pages
available.  This problem was not observed previously since movable
pages were used as a fallback for unmovable allocations.

To avoid such situation this commit changes the allocation order so
that on movable allocations the MIGRATE_CMA pageblocks are used first.

This change means that the MIGRATE_CMA can be removed from fallback
path of the MIGRATE_MOVABLE type.  This means that the
__rmqueue_fallback() function will never deal with CMA pages and thus
all the checks around MIGRATE_CMA can be removed from that function.

Change-Id: Ie13312d62a6af12d7aa78b4283ed25535a6d49fd
CRs-Fixed: 435287
Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
Reported-by: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Kyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>