mm: Mark free pages as read only

Drivers have a tendency to scribble on everything, including free
pages. Make life easier by marking free pages as read only when
on the buddy list and re-marking as read/write when allocating.

Change-Id: I978ed2921394919917307b9c99217fdc22f82c59
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Blechd0se <alex.christ@hotmail.de>
mm/compaction: Break out of loop on !PageBuddy in isolate_freepages_block

We received several reports of bad page state when freeing CMA pages
previously allocated with alloc_contig_range:

 BUG: Bad page state in process Binder_A  pfn:63202
 page:d21130b0 count:0 mapcount:1 mapping:  (null) index:0x7dfbf
 page flags: 0x40080068(uptodate|lru|active|swapbacked)

Based on the page state, it looks like the page was still in use. The page
flags do not make sense for the use case though. Further debugging showed
that despite alloc_contig_range returning success, at least one page in the
range still remained in the buddy allocator.

There is an issue with isolate_freepages_block. In strict mode (which CMA
uses), if any pages in the range cannot be isolated,
isolate_freepages_block should return failure 0. The current check keeps
track of the total number of isolated pages and compares against the size
of the range:

        if (strict && nr_strict_required > total_isolated)
                total_isolated = 0;

After taking the zone lock, if one of the pages in the range is not
in the buddy allocator, we continue through the loop and do not
increment total_isolated. If in the last iteration of the loop we isolate
more than one page (e.g. last page needed is a higher order page), the
check for total_isolated may pass and we fail to detect that a page was
skipped. The fix is to bail out if the loop immediately if we are in
strict mode. There's no benfit to continuing anyway since we need all
pages to be isolated. Additionally, drop the error checking based on
nr_strict_required and just check the pfn ranges. This matches with
what isolate_freepages_range does.

Change-Id: I2a8bc4d381ae35aa15a4d2a2424f0c2de801d9ba
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Blechd0se <alex.christ@hotmail.de>
mm: compaction: do not accidentally skip pageblocks in the migrate scanner

Compaction uses the ALIGN macro incorrectly with the migrate scanner by
adding pageblock_nr_pages to a PFN.  It happened to work when initially
implemented as the starting PFN was also aligned but with caching
restarts and isolating in smaller chunks this is no longer always true.

The impact is that the migrate scanner scans outside its current
pageblock.  As pfn_valid() is still checked properly it does not cause
any failure and the impact of the bug is that in some cases it will scan
more than necessary when it crosses a page boundary but by no more than
COMPACT_CLUSTER_MAX.  It is highly unlikely this is even measurable but
it's still wrong so this patch addresses the problem.

Change-Id: Id3972a7ec0d49e3fe1d225d466bff0f3d55b90bb
Signed-off-by: Mel Gorman <mgorman@suse.de>
Reviewed-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Blechd0se <alex.christ@hotmail.de>
mm: compaction: reset cached scanner pfn's before reading them

commit d3132e4b83e6bd383c74d716f7281d7c3136089c upstream.

Compaction caches pfn's for its migrate and free scanners to avoid
scanning the whole zone each time.  In compact_zone(), the cached values
are read to set up initial values for the scanners.  There are several
situations when these cached pfn's are reset to the first and last pfn
of the zone, respectively.  One of these situations is when a compaction
has been deferred for a zone and is now being restarted during a direct
compaction, which is also done in compact_zone().

However, compact_zone() currently reads the cached pfn's *before*
resetting them.  This means the reset doesn't affect the compaction that
performs it, and with good chance also subsequent compactions, as
update_pageblock_skip() is likely to be called and update the cached
pfn's to those being processed.  Another chance for a successful reset
is when a direct compaction detects that migration and free scanners
meet (which has its own problems addressed by another patch) and sets
update_pageblock_skip flag which kswapd uses to do the reset because it
goes to sleep.

This is clearly a bug that results in non-deterministic behavior, so
this patch moves the cached pfn reset to be performed *before* the
values are read.

Change-Id: Ife5b6dfd7148246eb84f8442391f70467ae95478
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Mel Gorman <mgorman@suse.de>
Acked-by: Rik van Riel <riel@redhat.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Blechd0se <alex.christ@hotmail.de>
mm: compaction: detect when scanners meet in isolate_freepages

commit 7ed695e069c3cbea5e1fd08f84a04536da91f584 upstream.

Compaction of a zone is finished when the migrate scanner (which begins
at the zone's lowest pfn) meets the free page scanner (which begins at
the zone's highest pfn).  This is detected in compact_zone() and in the
case of direct compaction, the compact_blockskip_flush flag is set so
that kswapd later resets the cached scanner pfn's, and a new compaction
may again start at the zone's borders.

The meeting of the scanners can happen during either scanner's activity.
However, it may currently fail to be detected when it occurs in the free
page scanner, due to two problems.  First, isolate_freepages() keeps
free_pfn at the highest block where it isolated pages from, for the
purposes of not missing the pages that are returned back to allocator
when migration fails.  Second, failing to isolate enough free pages due
to scanners meeting results in -ENOMEM being returned by
migrate_pages(), which makes compact_zone() bail out immediately without
calling compact_finished() that would detect scanners meeting.

This failure to detect scanners meeting might result in repeated
attempts at compaction of a zone that keep starting from the cached
pfn's close to the meeting point, and quickly failing through the
-ENOMEM path, without the cached pfns being reset, over and over.  This
has been observed (through additional tracepoints) in the third phase of
the mmtests stress-highalloc benchmark, where the allocator runs on an
otherwise idle system.  The problem was observed in the DMA32 zone,
which was used as a fallback to the preferred Normal zone, but on the
4GB system it was actually the largest zone.  The problem is even
amplified for such fallback zone - the deferred compaction logic, which
could (after being fixed by a previous patch) reset the cached scanner
pfn's, is only applied to the preferred zone and not for the fallbacks.

The problem in the third phase of the benchmark was further amplified by
commit 81c0a2bb515f ("mm: page_alloc: fair zone allocator policy") which
resulted in a non-deterministic regression of the allocation success
rate from ~85% to ~65%.  This occurs in about half of benchmark runs,
making bisection problematic.  It is unlikely that the commit itself is
buggy, but it should put more pressure on the DMA32 zone during phases 1
and 2, which may leave it more fragmented in phase 3 and expose the bugs
that this patch fixes.

The fix is to make scanners meeting in isolate_freepage() stay that way,
and to check in compact_zone() for scanners meeting when migrate_pages()
returns -ENOMEM.  The result is that compact_finished() also detects
scanners meeting and sets the compact_blockskip_flush flag to make
kswapd reset the scanner pfn's.

The results in stress-highalloc benchmark show that the "regression" by
commit 81c0a2bb515f in phase 3 no longer occurs, and phase 1 and 2
allocation success rates are also significantly improved.

Change-Id: Ifaaff38f0016a586ee52101aa95ca6788fbcbb2e
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Blechd0se <alex.christ@hotmail.de>
ion: Add missing unlock on error path

A mutex_unlock was missing for the alloc_lock on an error
path. Add it.

Change-Id: If88b72fbc2fd18504347b784d5daa2bc9170c55f
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
(cherry picked from commit 76d17e36bbb9f1a3acde26e02cfa4ee20903f4f5)
Signed-off-by: Blechd0se <alex.christ@hotmail.de>
ion: retry during race with shrinking/allocating

There exists a race between shrinking/allocating

Thread 1:                           Thread:2
mutex_lock(&sheap->alloc_lock)
ion_secure_cma_add_to_pool()
mutex_lock(&sheap->chunk_lock)
chunk->cnt = 0
mutex_unlock(&sheap->chunk_lock)
                                    ion_secure_cma_shrinker
                                    mutex_lock(&sheap->chunk_lock)
                                    __ion_secure_cma_shrink_pool
                                    chunk->cnt == 0 for the
                                       just allocated chunk
                                    ion_secure_cma-free_chunk
ion_secure_cma_alloc_from_pool
no memory to allocate!

If this race happens, just retry adding memory to the pool instead
of BUGing out the system.

Change-Id: Ic95782db6d88a33f01dcc5248fc957ca0f378078
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
(cherry picked from commit 17dcb6e11705de9e23c47538924b54ef7a131c7a)
Signed-off-by: Blechd0se <alex.christ@hotmail.de>