ARM: net: JIT compiler for packet filters

based of Matt Evans's PPC64 implementation.

The compiler generates ARM instructions but interworking is
supported for Thumb2 kernels.

Supports both little and big endian. Unaligned loads are emitted
for ARMv6+. Not all the BPF opcodes that deal with ancillary data
are supported. The scratch memory of the filter lives on the stack.
Hardware integer division is used if it is available.

Enabled in the same way as for x86-64 and PPC64:

	echo 1 > /proc/sys/net/core/bpf_jit_enable

A value greater than 1 enables opcode output.

Signed-off-by: Mircea Gherzan <mgherzan@xxxxxxxxx>
---

Changes in v7:
 * fix the intruction generation for LDX_MSH, OR_X, LSH_K,
   RSH_K and JMP_JA
 * fix the condition for saving the A register
 * use fls() instead of the compiler builtin
 * punt to the interpreter on absolute loads with K < 0
 * check for invalid data references
 * support the NEG opcode
 * clear X in the prologue based on a context flag
 * simplify the conditional jumps

Changes in v6:
 * fix the code generation for the ANC_CPU opcode

Changes in v5:
 * replace SEEN_LEN with SEEN_SKB
 * set ctx->seen when handling some ancillary data opcodes

Changes in v4:
 * first check if the JIT compiler is enabled
 * fix the code generation for the LDX_MSH opcode

Changes in v3:
 * no longer depend on EABI and !Thumb2
 * add BLX "emulation" for ARMv4 without Thumb
 * use the integer divide instruction on Cortex-A15
 * fix the handling of the DIV_K opcode
 * use a C wrapper for __aeabi_uidiv
 * fix the generation of the epilogue (non-FP case)

Changes in v2:
 * enable the compiler only for ARMv5+ because of the BLX instruction
 * use the same comparison for the ARM version checks
 * use misaligned accesses on ARMv6
 * fix the SEEN_MEM
 * fix the mem_words_used()
ARM: net: JIT compiler for packet filters enabled by default
staging/zcache: always enable when selected
staging: zcache: reduce tmem bucket lock contention

tmem uses hash buckets each with their own rbtree and lock to
quickly lookup tmem objects.  tmem has TMEM_HASH_BUCKETS (256)
buckets per pool.  However, because of the way the tmem_oid is
generated for frontswap pages, only 16 unique tmem_oids are being
generated, resulting in only 16 of the 256 buckets being used.
This cause high lock contention for the per bucket locks.

This patch changes SWIZ_BITS to include more bits of the offset.
The result is that all 256 hash buckets are potentially used resulting in a
95% drop in hash bucket lock contention.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
staging: zcache: fix serialization bug in zv stats

In a multithreaded workload, the zv_curr_dist_counts
and zv_cumul_dist_counts statistics are being corrupted
because the increments and decrements in zv_create
and zv_free are not atomic.

This patch converts these statistics and their corresponding
increments/decrements/reads to atomic operations.

Based on v3.2-rc7

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
zcache: Set SWIZ_BITS to 8 to reduce tmem bucket lock contention.

SWIZ_BITS > 8 results in a much larger number of "tmem_obj"
allocations, likely one per page-placed-in-frontswap.  The
tmem_obj is not huge (roughly 100 bytes), but it is large
enough to add a not-insignificant memory overhead to zcache.
The SWIZ_BITS=8  will get roughly the same lock contention
without the space wastage.

The effect of SWIZ_BITS can be thought of as "2^SWIZ_BITS is
the number of unique oids that be generated" (This concept is
limited to frontswap's use of tmem).

CC: Seth Jennings <sjenning@linux.vnet.ibm.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
zcache: fix deadlock condition

I discovered this deadlock condition awhile ago working on RAMster
but it affects zcache as well.  The list spinlock must be
locked prior to the page spinlock and released after.  As
a result, the page copy must also be done while the locks are held.

Applies to 3.2.  Konrad, please push (via GregKH?)...
this is definitely a bug fix so need not be pushed during
a -rc0 window.

Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
staging/zram: fix compilation warning due to lack of pointer type cast
init: apply SCHED_FIFO to kthreadd

Hotplug (cpu_up) latency currently suffers because the system
must wait for kthreadd to spawn certain kthreads (such as the
migration and workqueue kthreads) and for them to run for
the first time. Setting SCHED_FIFO will cause kthreadd to run
immediately. The newly created kthreads will inherit the scheduler
policy and run immediately also.

The scheduling policy of newly created kthreads is already set
back to SCHED_NORMAL in kthread_create_on_node, so nothing needs
to be done to restore normal scheduling behavior for the kthreads
once spawned.

Change-Id: I236ceb29845cf58ea1ea886fc3210ccaab2dd792
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
kernel: reduce sleep duration in wait_task_inactive

Sleeping for an entire tick adds unnecessary latency to
hotplugging a cpu (cpu_up).

Change-Id: Iab323a79f4048bc9101ecfd368e0f275827ed4ab
Signed-off-by: Steve Muckle <smuckle@codeaurora.org>
lib/kstrtox: common code between kstrto*() and simple_strto*() functions

Currently termination logic (\0 or \n\0) is hardcoded in _kstrtoull(),
avoid that for code reuse between kstrto*() and simple_strtoull().
Essentially, make them different only in termination logic.

simple_strtoull() (and scanf(), BTW) ignores integer overflow, that's a
bug we currently don't have guts to fix, making KSTRTOX_OVERFLOW hack
necessary.

Almost forgot: patch shrinks code size by about ~80 bytes on x86_64.

Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
vsprintf: make %pV handling compatible with kasprintf()

kasprintf() (and potentially other functions that I didn't run across so
far) want to evaluate argument lists twice.  Caring to do so for the
primary list is obviously their job, but they can't reasonably be
expected to check the format string for instances of %pV, which however
need special handling too: On architectures like x86-64 (as opposed to
e.g.  ix86), using the same argument list twice doesn't produce the
expected results, as an internally managed cursor gets updated during
the first run.

Fix the problem by always acting on a copy of the original list when
handling %pV.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
procfs: add num_to_str() to speed up /proc/stat

== stat_check.py
num = 0
with open("/proc/stat") as f:
        while num < 1000 :
                data = f.read()
                f.seek(0, 0)
                num = num + 1
==

perf shows

    20.39%  stat_check.py  [kernel.kallsyms]    [k] format_decode
    13.41%  stat_check.py  [kernel.kallsyms]    [k] number
    12.61%  stat_check.py  [kernel.kallsyms]    [k] vsnprintf
    10.85%  stat_check.py  [kernel.kallsyms]    [k] memcpy
     4.85%  stat_check.py  [kernel.kallsyms]    [k] radix_tree_lookup
     4.43%  stat_check.py  [kernel.kallsyms]    [k] seq_printf

This patch removes most of calls to vsnprintf() by adding num_to_str()
and seq_print_decimal_ull(), which prints decimal numbers without rich
functions provided by printf().

On my 8cpu box.
== Before patch ==
[root@bluextal test]# time ./stat_check.py

real    0m0.150s
user    0m0.026s
sys     0m0.121s

== After patch ==
[root@bluextal test]# time ./stat_check.py

real    0m0.055s
user    0m0.022s
sys     0m0.030s

[akpm@linux-foundation.org: remove incorrect comment, use less statck in num_to_str(), move comment from .h to .c, simplify seq_put_decimal_ull()]
[andrea@betterlinux.com: avoid breaking the ABI in /proc/stat]
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrea Righi <andrea@betterlinux.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Glauber Costa <glommer@parallels.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Turner <pjt@google.com>
Cc: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
backported from Linux 3.4-rc4 to Linux 3.0.x by faux123
vsprintf: optimize decimal conversion (again)

    This patch is based upon an original from Michal Nazarewicz.

    Previous code was using optimizations which were developed
    to work well even on narrow-word CPUs (by today's standards).
    But Linux runs only on 32-bit and wider CPUs. We can use that.

    First: using 32x32->64 multiply and trivial 32-bit shift,
    we can correctly divide by 10 much larger numbers, and thus
    we can print groups of 9 digits instead of groups of 5 digits.

    Next: there are two algorithms to print larger numbers.
    One is generic: divide by 1000000000 and repeatedly print
    groups of (up to) 9 digits. It's conceptually simple,
    but requires an (unsigned long long) / 1000000000 division.
    Second algorithm splits 64-bit unsigned long long into 16-bit chunks,
    manipulates them cleverly and generates groups of 4 decimal digits.
    It so happens that it does NOT require long long division.

    If long is > 32 bits, division of 64-bit values is relatively easy,
    and we will use the first algorithm.
    If long long is > 64 bits (strange architecture with VERY large long long),
    second algorithm can't be used, and we again use the first one.
    Else (if long is 32 bits and long long is 64 bits) we use second one.

    And third: there is a simple optimization which takes fast path
    not only for zero as was done before, but for all one-digit numbers.

    In all tested cases new code is faster than old one, in many cases by 30%,
    in few cases by more than 50% (for example, on x86-32, conversion of 12345678).
    Code growth is ~0 in 32-bit case and ~130 bytes in 64-bit case.

    Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
    Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
    Cc: Douglas W Jones <jones@cs.uiowa.edu>a
    backported from Linux 3.4+ to Linux 3.0 by faux123
netfilter: xt_qtaguid: fix ipv6 protocol lookup

When updating the stats for a given uid it would incorrectly assume
IPV4 and pick up the wrong protocol when IPV6.

Change-Id: Iea4a635012b4123bf7aa93809011b7b2040bb3d5
Signed-off-by: JP Abgrall <jpa@google.com>
Staging: android: binder: Don't call dump_stack in binder_vma_open

If user-space partially unmaps the driver, binder_vma_open
would dump the kernel stack. This is not a kernel bug however
and will be treated as if the whole area was unmapped once
binder_vma_close gets called.

Signed-off-by: Arve HjÃ¸nnevÃ¥g <arve@android.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
timer: optimize apply_slack()

__fls(mask) is equivalent to find_last_bit(&mask, BITS_PER_LONG), but cheaper.
find_last_bit was showing up high on the list when I was profiling for stalls
on icache misses on a system with very small cache size (MIPS).

Signed-off-by: Felix Fietkau <nbd@openwrt.org>
word-at-a-time: make the interfaces truly generic

This changes the interfaces in <asm/word-at-a-time.h> to be a bit more
complicated, but a lot more generic.

In particular, it allows us to really do the operations efficiently on
both little-endian and big-endian machines, pretty much regardless of
machine details.  For example, if you can rely on a fast population
count instruction on your architecture, this will allow you to make your
optimized <asm/word-at-a-time.h> file with that.

NOTE! The "generic" version in include/asm-generic/word-at-a-time.h is
not truly generic, it actually only works on big-endian.  Why? Because
on little-endian the generic algorithms are wasteful, since you can
inevitably do better. The x86 implementation is an example of that.

(The only truly non-generic part of the asm-generic implementation is
the "find_zero()" function, and you could make a little-endian version
of it.  And if the Kbuild infrastructure allowed us to pick a particular
header file, that would be lovely)

The <asm/word-at-a-time.h> functions are as follows:

 - WORD_AT_A_TIME_CONSTANTS: specific constants that the algorithm
   uses.

 - has_zero(): take a word, and determine if it has a zero byte in it.
   It gets the word, the pointer to the constant pool, and a pointer to
   an intermediate "data" field it can set.

   This is the "quick-and-dirty" zero tester: it's what is run inside
   the hot loops.

 - "prep_zero_mask()": take the word, the data that has_zero() produced,
   and the constant pool, and generate an *exact* mask of which byte had
   the first zero.  This is run directly *outside* the loop, and allows
   the "has_zero()" function to answer the "is there a zero byte"
   question without necessarily getting exactly *which* byte is the
   first one to contain a zero.

   If you do multiple byte lookups concurrently (eg "hash_name()", which
   looks for both NUL and '/' bytes), after you've done the prep_zero_mask()
   phase, the result of those can be or'ed together to get the "either
   or" case.

 - The result from "prep_zero_mask()" can then be fed into "find_zero()"
   (to find the byte offset of the first byte that was zero) or into
   "zero_bytemask()" (to find the bytemask of the bytes preceding the
   zero byte).

   The existence of zero_bytemask() is optional, and is not necessary
   for the normal string routines.  But dentry name hashing needs it, so
   if you enable DENTRY_WORD_AT_A_TIME you need to expose it.

This changes the generic strncpy_from_user() function and the dentry
hashing functions to use these modified word-at-a-time interfaces.  This
gets us back to the optimized state of the x86 strncpy that we lost in
the previous commit when moving over to the generic version.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
modified for Linux 3.0 from Linux 3.4+ by faux123
Clean DUAL code. we got it at lines 1200
ARM: use generic strnlen_user and strncpy_from_user
 functions

This patch implements the word-at-a-time interface for ARM using the
same algorithm as x86. Although we have a clz instruction from ARMv5,
this only saves us one mov instruction when building with Thumb-2 and
makes no difference when targetting ARM, so we use the magic 0x0ff0001
constant for all CPUs. For big-endian configurations, we use the
implementation from asm-generic.

With this implemented, we can replace our byte-at-a-time strnlen_user
and strncpy_from_user functions with the optimised generic versions.

Signed-off-by: Will Deacon <will.deacon@arm.com>
modified for Linux 3.0 from Linux 3.4+ by faux123