BUILD: v004

Signed-off-by: poondog <markj338@gmail.com>
add basic CFLAGS for Krait

Signed-off-by: poondog <markj338@gmail.com>
add vfpv4 optimizations for Krait

Signed-off-by: poondog <markj338@gmail.com>
remove unsafe-math-optimizations

Signed-off-by: poondog <markj338@gmail.com>
ARM: makefile: remove march=armv7-a for satisfy targeting MSM Krait CPUs

Signed-off-by: poondog <markj338@gmail.com>
slub.c: Remove PAGE_ALLOC_COSTLY_ORDER - +3% boost

Signed-off-by: poondog <markj338@gmail.com>
timer: optimize apply_slack()

__fls(mask) is equivalent to find_last_bit(&mask, BITS_PER_LONG), but cheaper.
find_last_bit was showing up high on the list when I was profiling for stalls
on icache misses on a system with very small cache size (MIPS).

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
Signed-off-by: poondog <markj338@gmail.com>
softirq: reduce latencies

In various network workloads, __do_softirq() latencies can be up
to 20 ms if HZ=1000, and 200 ms if HZ=100.

This is because we iterate 10 times in the softirq dispatcher,
and some actions can consume a lot of cycles.

This patch changes the fallback to ksoftirqd condition to :

- A time limit of 2 ms.
- need_resched() being set on current task

When one of this condition is met, we wakeup ksoftirqd for further
softirq processing if we still have pending softirqs.

Using need_resched() as the only condition can trigger RCU stalls,
as we can keep BH disabled for too long.

I ran several benchmarks and got no significant difference in
throughput, but a very significant reduction of latencies (one order
of magnitude) :

In following bench, 200 antagonist "netperf -t TCP_RR" are started in
background, using all available cpus.

Then we start one "netperf -t TCP_RR", bound to the cpu handling the NIC
IRQ (hard+soft)

Before patch :

# netperf -H 7.7.7.84 -t TCP_RR -T2,2 -- -k
RT_LATENCY,MIN_LATENCY,MAX_LATENCY,P50_LATENCY,P90_LATENCY,P99_LATENCY,MEAN_LATENCY,STDDEV_LATENCY
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
to 7.7.7.84 () port 0 AF_INET : first burst 0 : cpu bind
RT_LATENCY=550110.424
MIN_LATENCY=146858
MAX_LATENCY=997109
P50_LATENCY=305000
P90_LATENCY=550000
P99_LATENCY=710000
MEAN_LATENCY=376989.12
STDDEV_LATENCY=184046.92

After patch :

# netperf -H 7.7.7.84 -t TCP_RR -T2,2 -- -k
RT_LATENCY,MIN_LATENCY,MAX_LATENCY,P50_LATENCY,P90_LATENCY,P99_LATENCY,MEAN_LATENCY,STDDEV_LATENCY
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET
to 7.7.7.84 () port 0 AF_INET : first burst 0 : cpu bind
RT_LATENCY=40545.492
MIN_LATENCY=9834
MAX_LATENCY=78366
P50_LATENCY=33583
P90_LATENCY=59000
P99_LATENCY=69000
MEAN_LATENCY=38364.67
STDDEV_LATENCY=12865.26

Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: David Miller <davem@davemloft.net>
Cc: Tom Herbert <therbert@google.com>
Cc: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Signed-off-by: poondog <markj338@gmail.com>
msm: memutils: memcpy, memmove, copy_page optimization

Preload farther to take advantage of the memory bus, and assume
64-byte cache lines.  Unroll some pairs of ldm/stm as well, for
unexplainable reasons.

Future enhancements should include,

- #define for how far to preload, possibly defined separately for
  memcpy, copy_*_user
- Tuning for misaligned buffers
- Tuning for memmove
- Tuning for small buffers
- Understanding mechanism behind ldm/stm unroll causing some gains
  in copy_to_user

BASELINE (msm8960pro):
======================================================================
memcpy 1000MB at 5MB       : took 808850 usec, bandwidth 1236.236 MB/s
copy_to_user 1000MB at 5MB : took 810071 usec, bandwidth 1234.234 MB/s
copy_from_user 1000MB at 5M: took 942926 usec, bandwidth 1060.060 MB/s
memmove 1000GB at 5MB      : took 848588 usec, bandwidth 1178.178 MB/s
copy_to_user 1000GB at 4kB : took 847916 usec, bandwidth 1179.179 MB/s
copy_from_user 1000GB at 4k: took 935113 usec, bandwidth 1069.069 MB/s
copy_page 1000GB at 4kB    : took 779459 usec, bandwidth 1282.282 MB/s

THIS PATCH:
======================================================================
memcpy 1000MB at 5MB       : took 346223 usec, bandwidth 2888.888 MB/s
copy_to_user 1000MB at 5MB : took 348084 usec, bandwidth 2872.872 MB/s
copy_from_user 1000MB at 5M: took 348176 usec, bandwidth 2872.872 MB/s
memmove 1000GB at 5MB      : took 348267 usec, bandwidth 2871.871 MB/s
copy_to_user 1000GB at 4kB : took 377018 usec, bandwidth 2652.652 MB/s
copy_from_user 1000GB at 4k: took 371829 usec, bandwidth 2689.689 MB/s
copy_page 1000GB at 4kB    : took 383763 usec, bandwidth 2605.605 MB/s

Change-Id: I843fe77192d61ce35a81e75d76fc09f0a472e98a
Signed-off-by: Chris Fries <C.Fries@motorola.com>
Signed-off-by: poondog <markj338@gmail.com>
msm: memutils: memcpy, memmove optimization

1. To fit 8x26 64-bytes cache line size, we change preload steps as 64Bytes.
2. According to tune result, change preload distance as 5 cache lines.
3. According to tune result, re-arrange the ld/str order as back-to-back
   for copy_from_user.

The markable improvement:
	memcpy : 5%
	copy_to_user : 9%
	copy_from_user : 13%
	memmove : 37%

Raw data is as below:

BASELINE:
	memcpy 1000MB at 5MB       : took 1547098 usec, bandwidth 646.646 MB/s
	copy_to_user 1000MB at 5MB : took 1704308 usec, bandwidth 586.586 MB/s
	copy_from_user 1000MB at 5M: took 1777090 usec, bandwidth 562.562 MB/s
	memmove 1000GB at 5MB      : took 1066205 usec, bandwidth 937.937 MB/s
	copy_to_user 1000GB at 4kB : took 1774866 usec, bandwidth 563.563 MB/s
	copy_from_user 1000GB at 4k: took 1797654 usec, bandwidth 556.556 MB/s
	copy_page 1000GB at 4kB    : took 1644606 usec, bandwidth 608.608 MB/s
	memmove 1000GB at 4kB      : took 1236227 usec, bandwidth 808.808 MB/s

THIS PATCH:
	memcpy 1000MB at 5MB       : took 1475835 usec, bandwidth 677.677 MB/s
	copy_to_user 1000MB at 5MB : took 1559060 usec, bandwidth 641.641 MB/s
	copy_from_user 1000MB at 5M: took 1561603 usec, bandwidth 640.640 MB/s
	memmove 1000GB at 5MB      : took 861664 usec, bandwidth 1160.160 MB/s
	copy_to_user 1000GB at 4kB : took 1673501 usec, bandwidth 597.597 MB/s
	copy_from_user 1000GB at 4k: took 1674006 usec, bandwidth 597.597 MB/s
	copy_page 1000GB at 4kB    : took 1691358 usec, bandwidth 591.591 MB/s
	memmove 1000GB at 4kB      : took 882985 usec, bandwidth 1132.132 MB/s

Change-Id: I83bec3b7a9dd9cd88890eaa7ec423363e230a651
Signed-off-by: Hong-Mei Li <a21834@motorola.com>
Reviewed-on: http://gerrit.pcs.mot.com/550383
SLT-Approved: Slta Waiver <sltawvr@motorola.com>
Tested-by: Jira Key <jirakey@motorola.com>
Reviewed-by: Klocwork kwcheck <klocwork-kwcheck@sourceforge.mot.com>
Reviewed-by: Check Patch <CHEKPACH@motorola.com>
Reviewed-by: Yi-Wei Zhao <gbjc64@motorola.com>
Submit-Approved: Jira Key <jirakey@motorola.com>
Signed-off-by: poondog <markj338@gmail.com>
arm: lib: Fix makefile issue

Fix the typo introduced by commit fb7051a183cba3925f1f00c9bdadc2adcc682020.

Change-Id: Ia23c80f903ed431e60e4ba17c9c6d527e6febd87
Signed-off-by: Hong-Mei Li <a21834@motorola.com>
Reviewed-on: http://gerrit.pcs.mot.com/527915
SLT-Approved: Gerrit Code Review <gerrit-blurdev@motorola.com>
Tested-by: Jira Key <jirakey@motorola.com>
Reviewed-by: Christopher Fries <qcf001@motorola.com>
Submit-Approved: Jira Key <jirakey@motorola.com>
Signed-off-by: poondog <markj338@gmail.com>
BUILD: v005

Signed-off-by: poondog <markj338@gmail.com>
BUILD: make defconfig instead of me having to do it all the time :P

Signed-off-by: poondog <markj338@gmail.com>
binfmt_elf.c: use get_random_int() to fix entropy depleting

Changes:
--------
v4->v3:
- s/random_stack_user()/get_atrandom_bytes()/
- Move this function to ahead of its use to avoid the predeclaration.

v3->v2:
- Tweak code comments of random_stack_user().
- Remove redundant bits mask and shift upon the random variable.

v2->v1:
- Fix random copy to check up buffer length that are not 4-byte multiples.

v3 can be found at:
http://www.spinics.net/lists/linux-fsdevel/msg59597.html
v2 can be found at:
http://www.spinics.net/lists/linux-fsdevel/msg59418.html
v1 can be found at:
http://www.spinics.net/lists/linux-fsdevel/msg59128.html

Thanks,
-Jeff

Entropy is quickly depleted under normal operations like ls(1), cat(1),
etc...  between 2.6.30 to current mainline, for instance:

$ cat /proc/sys/kernel/random/entropy_avail
3428
$ cat /proc/sys/kernel/random/entropy_avail
2911
$cat /proc/sys/kernel/random/entropy_avail
2620

We observed this problem has been occurring since 2.6.30 with
fs/binfmt_elf.c: create_elf_tables()->get_random_bytes(), introduced by
f06295b44c296c8f ("ELF: implement AT_RANDOM for glibc PRNG seeding").

/*
 * Generate 16 random bytes for userspace PRNG seeding.
 */
get_random_bytes(k_rand_bytes, sizeof(k_rand_bytes));

The patch introduces a wrapper around get_random_int() which has lower
overhead than calling get_random_bytes() directly.

With this patch applied:
$ cat /proc/sys/kernel/random/entropy_avail
2731
$ cat /proc/sys/kernel/random/entropy_avail
2802
$ cat /proc/sys/kernel/random/entropy_avail
2878

Analyzed by John Sobecki.

Signed-off-by: Jie Liu <jeff.liu@oracle.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Andreas Dilger <aedilger@gmail.com>
Cc: Alan Cox <alan@linux.intel.com>
Cc: Arnd Bergmann <arnn@arndb.de>
Cc: John Sobecki <john.sobecki@oracle.com>
Cc: James Morris <james.l.morris@oracle.com>
Cc: Jakub Jelinek <jakub@redhat.com>
Cc: Ted Ts'o <tytso@mit.edu>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Kees Cook <keescook@chromium.org>
Cc: Ulrich Drepper <drepper@redhat.com>
Signed-off-by: poondog <markj338@gmail.com>
SCHED: LOAD_FREQ (4*HZ+61) avoids loadavg Moire
LOAD_FREQ is (5*HZ+1) to avoid high load average when idle:
http://kerneltrap.org/mailarchive/linux-kernel/2007/10/3/328568

I suggest (4*HZ+61) for a better distribution.

With some seconds based load (like SSL heartbeats)
and LOAD_FREQ at (5*HZ+1) I see Moire patterns like inverse sawtooth,
since 2 or 3 probes hit the jobs (load increases quickly),
followed by several probes missing it.

A 4.61 sec interval gives optimal distribution over when within a
second a probe is taken, as .61 is close to golden ratio phi 1.618...
(test in http://ripke.com/goldenratio.c).

12*4.61 = 55.32 secs is still close to a minute,
and 13*4.61=59.93 is even closer than the current 12*5.01=60.12
(with exponents EXP_x adjusted to a ratio of 13 instead of 12).

best
Klaus

More here:
http://ripke.com/loadavg/moire

Signed-off-by: poondog <markj338@gmail.com>
BUILD: v006

Signed-off-by: poondog <markj338@gmail.com>
FS: fix derp from prev commit

Signed-off-by: poondog <markj338@gmail.com>
debug: remove: remove some dmesg logspam from Linux mainline 3.4 fs

Signed-off-by: poondog <markj338@gmail.com>
switch do_fsync() to fget_light()

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: poondog <markj338@gmail.com>
fs/sync: Make sync() satisfy many requests with one invocation

Dave Jones reported RCU stalls, overly long hrtimer interrupts, and
amazingly long NMI handlers from a trinity-induced workload involving
lots of concurrent sync() calls (https://lkml.org/lkml/2013/7/23/369).
There are any number of things that one might do to make sync() behave
better under high levels of contention, but it is also the case that
multiple concurrent sync() system calls can be satisfied by a single
sys_sync() invocation.

Given that this situation is reminiscent of rcu_barrier(), this commit
applies the rcu_barrier() approach to sys_sync().  This approach uses
a global mutex and a sequence counter.  The mutex is held across the
sync() operation, which eliminates contention between concurrent sync()
operations.  The counter is incremented at the beginning and end of
each sync() operation, so that it is odd while a sync() operation is in
progress and even otherwise, just like sequence locks.

The code that used to be in sys_sync() is now in do_sync(), and sys_sync()
now handles the concurrency.  The sys_sync() function first takes a
snapshot of the counter, then acquires the mutex, and then takes another
snapshot of the counter.  If the values of the two snapshots indicate that
a full do_sync() executed during the mutex acquisition, the sys_sync()
function releases the mutex and returns ("Our work is done!").  Otherwise,
sys_sync() increments the counter, invokes do_sync(), and increments
the counter again.

This approach allows a single call to do_sync() to satisfy an arbitrarily
large number of sync() system calls, which should eliminate issues due
to large numbers of concurrent invocations of the sync() system call.

Changes since v1 (https://lkml.org/lkml/2013/7/24/683):

o	Add a pair of memory barriers to keep the increments from
	bleeding into the do_sync() code.  (The failure probability
	is insanely low, but when you have several hundred million
	devices running Linux, you can expect several hundred instances
	of one-in-a-million failures.)

o	Actually CC some people who have experience in this area.

Reported-by: Dave Jones <davej@redhat.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jan Kara <jack@suse.cz>
Cc: Curt Wohlgemuth <curtw@google.com>
Cc: Jens Axboe <jaxboe@fusionio.com>
Cc: linux-fsdevel@vger.kernel.org

Signed-off-by: Paul Reioux <reioux@gmail.com>
Signed-off-by: poondog <markj338@gmail.com>