ARM: implement optimized percpu variable access

From: Rob Herring <rob.herring@calxeda.com>

Use the previously unused TPIDRPRW register to store percpu offsets.
TPIDRPRW is only accessible in PL1, so it can only be used in the kernel.

This replaces 2 loads with a mrc instruction for each percpu variable
access. With hackbench, the performance improvement is 1.4% on Cortex-A9
(highbank). Taking an average of 30 runs of "hackbench -l 1000" yields:

Before: 6.2191
After: 6.1348

Will Deacon reported similar delta on v6 with 11MPCore.

The asm "memory" constraints are needed here to ensure the percpu offset
gets reloaded. Testing by Will found that this would not happen in
__schedule() which is a bit of a special case as preemption is disabled
but the execution can move cores.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: keep __my_cpu_offset consistent with generic one

Commit 14318efb(ARM: 7587/1: implement optimized percpu variable access)
introduces arm's __my_cpu_offset to optimize percpu vaiable access,
which really works well on hackbench, but will cause __my_cpu_offset
to return garbage value before it is initialized in cpu_init() called
by setup_arch, so accessing percpu variable before setup_arch may cause
kernel hang. But generic __my_cpu_offset always returns zero before
percpu area is brought up, and won't hang kernel.

So the patch tries to clear __my_cpu_offset on boot CPU early
to avoid boot hang.

At least now percpu variable is accessed by lockdep before
setup_arch(), and enabling CONFIG_LOCK_STAT or CONFIG_DEBUG_LOCKDEP
can trigger kernel hang.

Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Rob Herring <rob.herring@calxeda.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Nicolas Pitre <nico@linaro.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ext4: Speed up FITRIM by recording flags in ext4_group_info

In ext4, when FITRIM is called every time, we iterate all the
groups and do trim one by one. It is a bit time wasting if the
group has been trimmed and there is no change since the last
trim.

So this patch adds a new flag in ext4_group_info->bb_state to
indicate that the group has been trimmed, and it will be cleared
if some blocks is freed(in release_blocks_on_commit). Another
trim_minlen is added in ext4_sb_info to record the last minlen
we use to trim the volume, so that if the caller provide a small
one, we will go on the trim regardless of the bb_state.

A simple test with my intel x25m ssd:
df -h shows:
/dev/sdb1              40G   21G   17G  56% /mnt/ext4
Block size:               4096

run the FITRIM with the following parameter:
range.start = 0;
range.len = UINT64_MAX;
range.minlen = 1048576;

without the patch:
[root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
real	0m5.505s
user	0m0.000s
sys	0m1.224s
[root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
real	0m5.359s
user	0m0.000s
sys	0m1.178s
[root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
real	0m5.228s
user	0m0.000s
sys	0m1.151s

with the patch:
[root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
real	0m5.625s
user	0m0.000s
sys	0m1.269s
[root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
real	0m0.002s
user	0m0.000s
sys	0m0.001s
[root@boyu-tm linux-2.6]# time ./ftrim /mnt/ext4/a
real	0m0.002s
user	0m0.000s
sys	0m0.001s

A big improvement for the 2nd and 3rd run.

Even after I delete some big image files, it is still much
faster than iterating the whole disk.

[root@boyu-tm test]# time ./ftrim /mnt/ext4/a
real	0m1.217s
user	0m0.000s
sys	0m0.196s

Cc: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Andreas Dilger <adilger.kernel@dilger.ca>
Signed-off-by: Tao Ma <boyu.mt@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Signed-off-by: Dan Trifan <jimsth@gmail.com>

Conflicts:
	fs/ext4/mballoc.c

Signed-off-by: Dan Trifan <jimsth@gmail.com>
mm: remove compressed copy from zram in-memory

Swap subsystem does lazy swap slot free with expecting the page
would be swapped out again so we can avoid unnecessary write.

But the problem in in-memory swap(ex, zram) is that it consumes
memory space until vm_swap_full(ie, used half of all of swap device)
condition meet. It could be bad if we use multiple swap device,
small in-memory swap and big storage swap or in-memory swap alone.

This patch makes swap subsystem free swap slot as soon as swap-read
is completed and make the swapcache page dirty so the page should
be written out the swap device to reclaim it.
It means we never lose it.

I tested this patch with kernel compile workload.

1. before

compile time : 9882.42
zram max wasted space by fragmentation: 13471881 byte
memory space consumed by zram: 174227456 byte
the number of slot free notify: 206684

2. after

compile time : 9653.90
zram max wasted space by fragmentation: 11805932 byte
memory space consumed by zram: 154001408 byte
the number of slot free notify: 426972

Cc: Hugh Dickins <hughd@google.com>
Cc: Seth Jennings <sjenning@linux.vnet.ibm.com>
Cc: Nitin Gupta <ngupta@vflare.org>
Cc: Konrad Rzeszutek Wilk <konrad@darnok.org>
Cc: Shaohua Li <shli@kernel.org>
Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>

boype: added necessary swap_info_struct to mm/swapfile.c
and include/linux/swap.h

Signed-off-by: Dan Trifan <jimsth@gmail.com>
Linux v3.0.96

This is the 3.0.96 stable release.

Signed-off-by: Dan Trifan <jimsth@gmail.com>
mm: Dynamic management of dirty page writebacks

This feature allows to have two different intervals for dirty page
writebacks and to dynamically manage them when the system has been
resumed (it's active) or when has been suspended.

Three new procfs parameters are exposed inside /proc/sys/vm path:
- "dynamic_dirty_writeback" is the activation status of this feature,
  set 1 to enable it, set 0 to disable it and use the standard behaviour
- "dirty_writeback_active_centisecs" is the interval for the dirty page
  writebacks when the system is active (screen on)
- "dirty_writeback_suspend_centisecs" is the interval for the dirty page
  writebacks when the system is suspended (screen off)

This feature is in part inspired on Francisco Franco's patch:
https://github.com/franciscofranco/mako/commit/34d79548cf16b69c7d0154836018e7f258a22947

Signed-off-by: Cristoforo Cataldo <cristoforo.cataldo@gmail.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
defconfig: enable DYNAMIC_PAGE_WRITEBACK

Signed-off-by: Dan Trifan <jimsth@gmail.com>
mm: finetune the filesystem dynamic read-ahead

Signed-off-by: Dan Trifan <jimsth@gmail.com>
kernel: cgroup_timer_slack: Dynamic management of timer slack

This feature allows to use different minimal timer slack values and to
dynamically manage them when the system has been resumed (it's active)
or when has been suspended.

A new file is provided in addition to the two files provided by the
original timer slack controller:
- "timer_slack.min_slack_suspend_ns" is the minimal timer slack value
  to use when the system is suspended (screen off)

For further info, please refers to the timer slack documentation:
Documentation/cgroupd/timer_slack.txt

Signed-off-by: Cristoforo Cataldo <cristoforo.cataldo@gmail.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
defconfig: enable dynamic CGROUP_TIMER_SLACK

Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: 7746/1: mm: lazy cache flushing on non-mapped pages

Currently flush_dcache_page() thinks pages as non-mapped if
mapping_mapped(mapping) return false. This approach is very
coase:
	- mmap on part of file may cause all pages backed on
	the file being thought as mmaped

	- file-backed pages aren't mapped into user space actually
	if the memory mmaped on the file isn't accessed

This patch uses page_mapped() to decide if the page has been
mapped.

From the attached test code, I find there is much performance
improvement(>25%) when accessing page caches via read under this
situations, so memcpy benefits a lot from not flushing cache
under this situation.

No.   read time without the patch	No. read time with the patch
================================================================
No. 0, time  22615636 us		No. 0, time  22014717 us
No. 1, time  4387851 us 		No. 1, time  3113184 us
No. 2, time  4276535 us 		No. 2, time  3005244 us
No. 3, time  4259821 us 		No. 3, time  3001565 us
No. 4, time  4263811 us 		No. 4, time  3002748 us
No. 5, time  4258486 us 		No. 5, time  3004104 us
No. 6, time  4253009 us 		No. 6, time  3002188 us
No. 7, time  4262809 us 		No. 7, time  2998196 us
No. 8, time  4264525 us 		No. 8, time  3007255 us
No. 9, time  4267795 us 		No. 9, time  3005094 us

1), No.0. is to read the file from storage device, and others are
to read the file from page caches basically.
2), file size is 512M, and is on ext4 over usb mass storage.
3), the test is done on Pandaboard.

unsigned int  sum = 0;
unsigned long sum_val = 0;

static unsigned long tv_diff(struct timeval *tv1, struct timeval *tv2)
{
	return (tv2->tv_sec - tv1->tv_sec) * 1000000 +
		(tv2->tv_usec - tv1->tv_usec);
}

int main(int argc, char *argv[])
{
	char *mbuf, fbuf;
	int fd;
	int i;
	unsigned long page_size, size;
	struct stat stat;
	struct timeval t1, t2;
	unsigned char *rbuf = malloc(32 * page_size);

	if (!rbuf) {
		printf("	%sn", "malloc failed");
		exit(-1);
	}

	page_size = getpagesize();
	fd = open(argv[1], O_RDWR);
	assert(fd >= 0);

	fstat(fd, &stat);
	size = stat.st_size;
	printf("%s: file %s, size %lu, page size %lun",
		argv[0],
		argv[1], size, page_size);

	gettimeofday(&t1, NULL);
	mbuf = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
	if (!mbuf) {
		printf("	%sn", "mmap failed");
		exit(-1);
	}

	for (i = 0 ; i < size ; i += (page_size * 32)) {
		int rcnt;
		lseek(fd, i, SEEK_SET);
		rcnt = read(fd, rbuf, page_size * 32);
		if (rcnt != page_size * 32) {
			printf("%s: read faildn", __func__);
			exit(-1);
		}
	}
	free(rbuf);
	munmap(mbuf, size);
	gettimeofday(&t2, NULL);
	printf("tread mmaped time: %luusn", tv_diff(&t1, &t2));

	close(fd);
}

Cc: Michel Lespinasse <walken@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Nicolas Pitre <nicolas.pitre@linaro.org>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Ming Lei <ming.lei@canonical.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
ARM: 6998/2: kernel: use proper memory barriers for bitops

Improve scalability by avoiding costly and unnecessary L2 cache sync
in handling bitops.

Signed-off-by: Heechul Yun <hyun@nvidia.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sync: allow async waits to be canceled

In order to allow drivers to cleanly handled teardown we need to allow them
to cancel pending async waits.  To do this cleanly, we move allocation of
sync_fence_waiter to the driver calling sync_async_wait().

Change-Id: Ifcd95648be6ec07026d67f810070a4310f099989
Signed-off-by: Erik Gilling <konkers@android.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sync: export sync API symbols

This is needed to allow modules to link against the sync subsystem

Change-Id: I15c1818de329f24e4113ef1d0923413b22fd0eff
Signed-off-by: Erik Gilling <konkers@android.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sw_sync: export sw_sync API

Needed to let modules link against sw_sync.

Change-Id: I71d3e52677ef21b6e1ecdb84f831491be1f4fbe6
Signed-off-by: Erik Gilling <konkers@android.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sync: reorder sync_fence_release

Previously fence's pts were freed before the were the fence was removed from the
global fence list.  This led to a race with the debugfs support where it would
iterate over sync_pts that had been freed.

Change-Id: Ia3ddbf77de42ca593fc2dc353b5d04e42ddf3946
Signed-off-by: Erik Gilling <konkers@android.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sync: optimize fence merges

If the two fences being merged contain sync_pts from the same timeline,
those two pts will be collapsed into a single pt representing the latter
of the two.

Change-Id: Iced7ebb7e5a17a0c8b1a2969ba3388a4edb9ecaf
Signed-off-by: Erik Gilling <konkers@android.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sync: add internal refcounting to fences

If a fence is released while a timeline that one of it's pts is on is being
signaled, it is possible for that fence to be deleted before it is signaled.
This patch adds a refcount for internal references such as signaled pt
processing.

Change-Id: Ie8605e6fd2ac026c207220a03d84e1c1078ec719
Signed-off-by: Erik Gilling <konkers@android.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sync: add reference counting to timelines

If a timeline is destroyed while fences still hold pts on it, the reworked
fence release handler can cause the timeline to be freed before all it's points
are freed.

Change-Id: I1cd8ddb638eded7db9db446ff6b37f3dd165d6c4
Signed-off-by: Erik Gilling <konkers@android.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>
sync: Fix error paths

Check the return value of get_unused_fd to make sure a valid
file descriptor is returned.

Make sure to call put_unused_fd even if an error occurs before
the fd can be used.

Change-Id: I0fe8f78d9e30ecfc4d271c5d875424543dae2d0f
Signed-off-by: Rebecca Schultz Zavin <rebecca@android.com>
Signed-off-by: Dan Trifan <jimsth@gmail.com>