decompressor: Add LZ4 decompressor module

Date	Tue, 26 Feb 2013 15:24:27 +0900

This patch adds support for LZ4 decompression in the Linux Kernel.
LZ4 Decompression APIs for kernel are based on LZ4 implementation
by Yann Collet.

LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
LZ4 source repository : http://code.google.com/p/lz4/

Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>

v2:
- Clean up code
- Enable unaligned access for ARM v6 and above with
  CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
- Add lz4_decompress() for faster decompression with
  uncompressed output size

Signed-off-by: Paul Reioux <reioux@gmail.com>
lib: Add support for LZ4-compressed kernel

Date	Tue, 26 Feb 2013 15:24:28 +0900

This patch adds support for extracting LZ4-compressed kernel images,
as well as LZ4-compressed ramdisk images in the kernel boot process.

This depends on the patch below
decompressor: Add LZ4 decompressor module

Signed-off-by: Kyungsik Lee <kyungsik.lee@lge.com>

v2:
- Clean up code
- Use lz4_decompress() for LZ4-compressed kernel during boot-process

Signed-off-by: Paul Reioux <reioux@gmail.com>
lib: Add lz4 compressor module

From	Chanho Min <>

This patch adds support for LZ4 compression in the Linux Kernel.
LZ4 Compression APIs for kernel are based on LZ4 implementation
by Yann Collet and changed with kernel coding style.

LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
LZ4 source repository : http://code.google.com/p/lz4/
svn revision : r90

Two APIs are added:
lz4_compress() support basic lz4 compression whereas lz4hc_compress() support
high compression or CPU performance get lower but compression ratio get higher.
Also, we require the pre-allocated working memory with the defined size and
destination buffer must be allocated with the size of lz4_compressbound.

Signed-off-by: Chanho Min <chanho.min@lge.com>

Signed-off-by: Paul Reioux <reioux@gmail.com>
lib/lz4/lz4_compress: fix macro usage error

Signed-off-by: Paul Reioux <reioux@gmail.com>
LZ4: compression/decompression signedness mismatch (v2)

LZ4 compression and decompression functions require different
in signedness input/output parameters: unsigned char for
compression and signed char for decompression.

Change decompression API to require "(const) unsigned char *".

v2: minor coding style fix.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>

Signed-off-by: Paul Reioux <reioux@gmail.com>
crypto: Add lz4 Cryptographic API

From	Chanho Min <>

This patch adds support for lz4 and lz4hc compression algorithm
using the lib/lz4/* codebase.
Signed-off-by: Chanho Min <chanho.min@lge.com>

Signed-off-by: Paul Reioux <reioux@gmail.com>
 staging: Add Snappy compression library (v3)

Google's Snappy data compression library is a faster alternative to LZO,
optimized for x86-64. On compressible input it compresses ~2.5x faster than LZO
and decompresses ~1.5-2x faster than LZO. On incompressible input, it skips the
input at 100x faster than LZO and decompresses ~4x faster than LZO.
It is released under BSD license.
This is a kernel port from user space C++ code.
The current intended use is with zram (see next patch in series).

Signed-off-by: Zeev Tarantov <zeev.tarantov@gmail.com>
Signed-off-by: Willi Ye <williye97@gmail.com>

Conflicts:
	drivers/staging/Makefile
csnappy: Update to newest version

Pulled on 18/04/2013
fs: Add exFAT file system

Change-Id: I7b611a7290e13b382597b34e68f05c78e19c4895
CHROMIUM: zsmalloc: add MODULE_LICENSE

It won't load as a module without MODULE_LICENSE.

BUG=chromium-os:32328
TEST=modprobe zsmalloc works with this change

Change-Id: Ic62fe20816e5079d92ce1da6a9d8a9cdbc7a5074
Signed-off-by: Luigi Semenzato <semenzato@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/26627
Reviewed-by: Olof Johansson <olofj@chromium.org>
CHROMIUM: BACKPORT: staging: zsmalloc: Finish conversion to a separate module

Need BACKPORT, rather than UPSTREAM, since it won't apply cleanly because
we had already fixed the MODULE_LICENSE problem.

---original comment:

ZSMALLOC is tristate, but the code has no MODULE_LICENSE and since it
depends on GPL-only symbols it cannot be loaded as a module.  This in
turn breaks zram which now depends on it.  I assume it's meant to be
Dual BSD/GPL like the other z-stuff.

There is also no module_exit, which will make it impossible to unload.
Add the appropriate module_init and module_exit declarations suggested
by comments.

Reported-by: Christian Ohm <chr.ohm@gmx.net>
References: http://bugs.debian.org/677273
Cc: stable@vger.kernel.org # v3.4
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Conflicts:

	drivers/staging/zsmalloc/zsmalloc-main.c

BUG=chromium-os:36829
TEST=extensively tested manually

Change-Id: I82bc7ffffe3007b82601ec71f43873be2059586c
Signed-off-by: Luigi Semenzato <semenzato@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/38998
Reviewed-by: Olof Johansson <olofj@chromium.org>
CHROMIUM: UPSTREAM: staging: zsmalloc: remove x86 dependency

This patch replaces the page table assisted object mapping
method, which has x86 dependencies, with a arch-independent
method that does a simple copy into a temporary per-cpu
buffer.

While a copy seems like it would be worse than mapping the pages,
tests demonstrate the copying is always faster and, in the case of
running inside a KVM guest, roughly 4x faster.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

BUG=chromium-os:36829
TEST=extensive manual testing

Change-Id: I8b41c92eb146c5cf0af40682802ba443f7810a0b
Signed-off-by: Luigi Semenzato <semenzato@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/38999
Reviewed-by: Olof Johansson <olofj@chromium.org>
CHROMIUM: UPSTREAM: staging: zram/zcache: swtich Kconfig dependency from X86 to ZSMALLOC

upstream: commit 6e2361720b9da9ec830d407da058ca1827e62b12

This patch switches zcache and zram dependency to ZSMALLOC
rather than X86.  There is no net change since ZSMALLOC
depends on X86, however, this prevent further changes to
these files as zsmalloc dependencies change.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

BUG=chromium-os:36829
TEST=compile, verify that zram module is updated and /boot/config is correct

Change-Id: I91f6ed46549fe62a9fe3035063d98beb2f5d966a
Signed-off-by: Luigi Semenzato <semenzato@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39289
Tested-by: Luigi Semenzato <semenzato@google.com>
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Ready: Luigi Semenzato <semenzato@google.com>
CHROMIUM: zram: fix 32-bit overflow

The 64-bit division operator is not available in 32-bit kernels,
so have to use do_div.

BUG=chromium:245703
TEST=compiled

Change-Id: I44ab763ea9e115cef6212f6524518cf7d9eac8c7
Signed-off-by: Luigi Semenzato <semenzato@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/64695
Reviewed-by: Doug Anderson <dianders@chromium.org>
CHROMIUM: zram: finish fixing 32-bit overflow

Nasty bug.  PAGE_MASK is unsigned, so it extends to 64 bit with zeros.
The first read in the zram block device is done by mkswap, which
tries to read at the end of device.  zram_set_disksize is a side
effect of the first read, so the disk size is now silently changed
from 6GB to 2GB.  The read fails, but mkswap does not complain either:
it finds the size of the device by trial and error and continues.
Swapon detects the mismatch and fails.

BUG=chromium:245703
TEST=ran before and after change with various printks
Signed-off-by: Luigi Semenzato <semenzato@chromium.org>

Change-Id: I01750554964756c2759375df1f14d7bb8b859ccf
Reviewed-on: https://gerrit.chromium.org/gerrit/65752
Tested-by: Luigi Semenzato <semenzato@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Luigi Semenzato <semenzato@chromium.org>
exFAT: Update to v1.2.5

Change-Id: I6a43a674e5edc599b00a902247539b11569f4e6f
fs/dyn_sync_cntrl: dynamic sync control

The dynamic sync control interface uses Android kernel's unique early
suspend / lat resume interface.

While screen is on, file sync is disabled
when screen is off, a file sync is called to flush all outstanding writes
and restore file sync operation as normal.

Signed-off-by: Paul Reioux <reioux@gmail.com>
fs/dyn_sync_cntrl: Linux 3.4 Kernel compatibility fixup

Signed-off-by: faux123 <reioux@gmail.com>
writeback: fix writeback cache thrashing

Consider Process A: huge I/O on sda
        doing heavy write operation - dirty memory becomes more
        than dirty_background_ratio
        on HDD - flusher thread flush-8:0

Consider Process B: small I/O on sdb
        doing while [1]; read 1024K + rewrite 1024K + sleep 2sec
        on Flash device - flusher thread flush-8:16

As Process A is a heavy dirtier, dirty memory becomes more
than dirty_background_thresh. Due to this, below check becomes
true(checking global_page_state in over_bground_thresh)
for all bdi devices(even for very small dirtied bdi - sdb):

In this case, even small cached data on 'sdb' is forced to flush
and writeback cache thrashing happens.

When we added debug prints inside above 'if' condition and ran
above Process A(heavy dirtier on bdi with flush-8:0) and
Process B(1024K frequent read/rewrite on bdi with flush-8:16)
we got below prints:

[Test setup: ARM dual core CPU, 512 MB RAM]

[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  56064 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  56704 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE = 84720 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE = 94720 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   384 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   960 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =    64 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE = 92160 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   256 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   768 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =    64 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   256 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   320 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =     0 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE = 92032 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE = 91968 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   192 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =  1024 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =    64 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   192 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   576 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =     0 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE = 84352 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   192 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =   512 KB
[over_bground_thresh]: wakeup flush-8:16 : BDI_RECLAIMABLE =     0 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE = 92608 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE = 92544 KB

As mentioned in above log, when global dirty memory > global background_thresh
small cached data is also forced to flush by flush-8:16.
If removing global background_thresh checking code, we can reduce cache
thrashing of frequently used small data.
And It will be great if we can reserve a portion of writeback cache using
min_ratio.

After applying patch:
$ echo 5 > /sys/block/sdb/bdi/min_ratio
$ cat /sys/block/sdb/bdi/min_ratio
5
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  56064 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  56704 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  84160 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  96960 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  94080 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  93120 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  93120 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  91520 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  89600 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  93696 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  93696 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  72960 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  90624 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  90624 KB
[over_bground_thresh]: wakeup flush-8:0 : BDI_RECLAIMABLE =  90688 KB

As mentioned in the above logs, once cache is reserved for Process B,
and patch is applied there is less writeback cache thrashing on sdb
by frequent forced writeback by flush-8:16 in over_bground_thresh.

After all, small cached data will be flushed by periodic writeback
once every dirty_writeback_interval.

Suggested-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Signed-off-by: Namjae Jeon <namjae.jeon@samsung.com>
Signed-off-by: Vivek Trivedi <t.vivek@samsung.com>
fs/dyn_fsync: check dyn fsync control's active prior to performing fsync ops

Signed-off-by: Andrew Bartholomew <andrewb03@gmail.com>