ARM: add support for bit sliced AES using NEON instructions

Bit sliced AES gives around 45% speedup on Cortex-A15 for encryption
and around 25% for decryption. This implementation of the AES algorithm
does not rely on any lookup tables so it is believed to be invulnerable
to cache timing attacks.

This algorithm processes up to 8 blocks in parallel in constant time. This
means that it is not usable by chaining modes that are strictly sequential
in nature, such as CBC encryption. CBC decryption, however, can benefit from
this implementation and runs about 25% faster. The other chaining modes
implemented in this module, XTS and CTR, can execute fully in parallel in
both directions.

The core code has been adopted from the OpenSSL project (in collaboration
with the original author, on cc). For ease of maintenance, this version is
identical to the upstream OpenSSL code, i.e., all modifications that were
required to make it suitable for inclusion into the kernel have been made
upstream. The original can be found here:

    http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=6f6a6130

Note to integrators:
While this implementation is significantly faster than the existing table
based ones (generic or ARM asm), especially in CTR mode, the effects on
power efficiency are unclear as of yet. This code does fundamentally more
work, by calculating values that the table based code obtains by a simple
lookup; only by doing all of that work in a SIMD fashion, it manages to
perform better.

Cc: Andy Polyakov <appro@openssl.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: poondog <markj338@gmail.com>
crypto: create generic version of ablk_helper

Create a generic version of ablk_helper so it can be reused
by other architectures.

Acked-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Conflicts:
	crypto/Kconfig
	crypto/Makefile
Signed-off-by: poondog <markj338@gmail.com>
crypto: ablk_helper - Replace memcpy with struct assignment

tree:   git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git master
head:   48e6dc1b2a1ad8186d48968d5018912bdacac744
commit: a62b01cd6cc1feb5e80d64d6937c291473ed82cb [20/24] crypto: create generic version of ablk_helper

coccinelle warnings: (new ones prefixed by >>)

>> crypto/ablk_helper.c:97:2-8: Replace memcpy with struct assignment
>> crypto/ablk_helper.c:78:2-8: Replace memcpy with struct assignment

Please consider folding the attached diff :-)

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: poondog <markj338@gmail.com>
ARM: move AES typedefs and function prototypes to separate header

Put the struct definitions for AES keys and the asm function prototypes in a
separate header and export the asm functions from the module.
This allows other drivers to use them directly.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: poondog <markj338@gmail.com>
ARM: pull in <asm/simd.h> from asm-generic

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: poondog <markj338@gmail.com>
ARM: 7626/1: arm/crypto: Make asm SHA-1 and AES code Thumb-2 compatible

This patch fixes aes-armv4.S and sha1-armv4-large.S to work
natively in Thumb.  This allows ARM/Thumb interworking workarounds
to be removed.

I also take the opportunity to convert some explicit assembler
directives for exported functions to the standard
ENTRY()/ENDPROC().

For the code itself:

  * In sha1_block_data_order, use of TEQ with sp is deprecated in
    ARMv7 and not supported in Thumb.  For the branches back to
    .L_00_15 and .L_40_59, the TEQ is converted to a CMP, under the
    assumption that clobbering the C flag here will not cause
    incorrect behaviour.

    For the first branch back to .L_20_39_or_60_79 the C flag is
    important, so sp is moved temporarily into another register so
    that TEQ can be used for the comparison.

  * In the AES code, most forms of register-indexed addressing with
    shifts and rotates are not permitted for loads and stores in
    Thumb, so the address calculation is done using a separate
    instruction for the Thumb case.

The resulting code is unlikely to be optimally scheduled, but it
should not have a large impact given the overall size of the code.
I haven't run any benchmarks.

Signed-off-by: Dave Martin <dave.martin@linaro.org>
Tested-by: David McCullough <ucdevel@gmail.com> (ARM only)
Acked-by: David McCullough <ucdevel@gmail.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: poondog <markj338@gmail.com>
ARM: 7837/3: fix Thumb-2 bug in AES assembler code

Patch 638591c enabled building the AES assembler code in Thumb2 mode.
However, this code used arithmetic involving PC rather than adr{l}
instructions to generate PC-relative references to the lookup tables,
and this needs to take into account the different PC offset when
running in Thumb mode.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Cc: stable@vger.kernel.org
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: poondog <markj338@gmail.com>
ARM: 7723/1: crypto: sha1-armv4-large.S: fix SP handling

Make the SHA1 asm code ABI conformant by making sure all stack
accesses occur above the stack pointer.

Origin:
http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=1a9d60d2

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Cc: stable@vger.kernel.org
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: poondog <markj338@gmail.com>
ARM: add .gitignore entry for aesbs-core.S

This avoids this file being incorrectly added to git.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: poondog <markj338@gmail.com>
defconfig: Enable NEON optimized crypto routines

Signed-off-by: poondog <markj338@gmail.com>
arm: vfpmodule: Fix warning procfs vfp_bounce reporting failed

Creation of procfs cpu/vfp_bounce fails because we're initialized too early. Fix
this by creating it on rootfs_initcall as before the NEON patches.

<6>[    0.130770] VFP support v0.3: implementor 51 architecture 64 part 6f varia
nt 2 rev 0
<4>[    0.130795] ------------[ cut here ]------------
<4>[    0.130813] WARNING: at fs/proc/generic.c:323 __xlate_proc_name+0xac/0xcc(
)
<4>[    0.130822] name 'cpu/vfp_bounce'
<4>[    0.130855] [<c010e26c>] (unwind_backtrace+0x0/0x144) from [<c0a20f58>] (d
ump_stack+0x20/0x24)
<4>[    0.130879] [<c0a20f58>] (dump_stack+0x20/0x24) from [<c019b670>] (warn_sl
owpath_common+0x58/0x70)
<4>[    0.130899] [<c019b670>] (warn_slowpath_common+0x58/0x70) from [<c019b704>
] (warn_slowpath_fmt+0x40/0x48)
<4>[    0.130919] [<c019b704>] (warn_slowpath_fmt+0x40/0x48) from [<c02c2ad8>] (
__xlate_proc_name+0xac/0xcc)
<4>[    0.130938] [<c02c2ad8>] (__xlate_proc_name+0xac/0xcc) from [<c02c2b50>] (
__proc_create+0x58/0x100)
<4>[    0.130956] [<c02c2b50>] (__proc_create+0x58/0x100) from [<c02c2ed0>] (pro
c_create_data+0x5c/0xc0)
<4>[    0.130979] [<c02c2ed0>] (proc_create_data+0x5c/0xc0) from [<c0f03484>] (v
fp_init+0x19c/0x200)
<4>[    0.131000] [<c0f03484>] (vfp_init+0x19c/0x200) from [<c0f00c98>] (do_one_
initcall+0x98/0x168)
<4>[    0.131020] [<c0f00c98>] (do_one_initcall+0x98/0x168) from [<c0f00e60>] (k
ernel_init+0xf8/0x1b4)
<4>[    0.131043] [<c0f00e60>] (kernel_init+0xf8/0x1b4) from [<c01081a0>] (kerne
l_thread_exit+0x0/0x8)
<4>[    0.131076] ---[ end trace ea6d9a9b5e947151 ]---
<3>[    0.131086] Failed to create procfs node for VFP bounce reporting

Signed-off-by: poondog <markj338@gmail.com>
ARCH: ARM: VFP: fix build

Signed-off-by: poondog <markj338@gmail.com>
BUILD: v028

Signed-off-by: poondog <markj338@gmail.com>
DEFCONFIG: support for new crypto

Signed-off-by: poondog <markj338@gmail.com>
INCLUDE: add back some comments

Signed-off-by: poondog <markj338@gmail.com>
2w: don't vibrate in response to sweep when screen is on, there is enough vibration from button feedback

Signed-off-by: poondog <markj338@gmail.com>
MM: remove redundant statement

Signed-off-by: poondog <markj338@gmail.com>
workqueue: add missing smp_wmb() in process_one_work()

commit 959d1af8cffc8fd38ed53e8be1cf4ab8782f9c00 upstream.

WORK_STRUCT_PENDING is used to claim ownership of a work item and
process_one_work() releases it before starting execution.  When
someone else grabs PENDING, all pre-release updates to the work item
should be visible and all updates made by the new owner should happen
afterwards.

Grabbing PENDING uses test_and_set_bit() and thus has a full barrier;
however, clearing doesn't have a matching wmb.  Given the preceding
spin_unlock and use of clear_bit, I don't believe this can be a
problem on an actual machine and there hasn't been any related report
but it still is theretically possible for clear_pending to permeate
upwards and happen before work->entry update.

Add an explicit smp_wmb() before work_clear_pending().

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: poondog <markj338@gmail.com>
workqueue: exit rescuer_thread() as TASK_RUNNING

commit 412d32e6c98527078779e5b515823b2810e40324 upstream.

A rescue thread exiting TASK_INTERRUPTIBLE can lead to a task scheduling
off, never to be seen again.  In the case where this occurred, an exiting
thread hit reiserfs homebrew conditional resched while holding a mutex,
bringing the box to its knees.

PID: 18105  TASK: ffff8807fd412180  CPU: 5   COMMAND: "kdmflush"
 #0 [ffff8808157e7670] schedule at ffffffff8143f489
 #1 [ffff8808157e77b8] reiserfs_get_block at ffffffffa038ab2d [reiserfs]
 #2 [ffff8808157e79a8] __block_write_begin at ffffffff8117fb14
 #3 [ffff8808157e7a98] reiserfs_write_begin at ffffffffa0388695 [reiserfs]
 #4 [ffff8808157e7ad8] generic_perform_write at ffffffff810ee9e2
 #5 [ffff8808157e7b58] generic_file_buffered_write at ffffffff810eeb41
 #6 [ffff8808157e7ba8] __generic_file_aio_write at ffffffff810f1a3a
 #7 [ffff8808157e7c58] generic_file_aio_write at ffffffff810f1c88
 #8 [ffff8808157e7cc8] do_sync_write at ffffffff8114f850
 #9 [ffff8808157e7dd8] do_acct_process at ffffffff810a268f
    [exception RIP: kernel_thread_helper]
    RIP: ffffffff8144a5c0  RSP: ffff8808157e7f58  RFLAGS: 00000202
    RAX: 0000000000000000  RBX: 0000000000000000  RCX: 0000000000000000
    RDX: 0000000000000000  RSI: ffffffff8107af60  RDI: ffff8803ee491d18
    RBP: 0000000000000000   R8: 0000000000000000   R9: 0000000000000000
    R10: 0000000000000000  R11: 0000000000000000  R12: 0000000000000000
    R13: 0000000000000000  R14: 0000000000000000  R15: 0000000000000000
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018

Signed-off-by: Mike Galbraith <mgalbraith@suse.de>
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: poondog <markj338@gmail.com>
workqueue: convert BUG_ON()s in __queue_delayed_work() to WARN_ON_ONCE()s

commit fc4b514f2727f74a4587c31db87e0e93465518c3 upstream.

8852aac25e ("workqueue: mod_delayed_work_on() shouldn't queue timer on
0 delay") unexpectedly uncovered a very nasty abuse of delayed_work in
megaraid - it allocated work_struct, casted it to delayed_work and
then pass that into queue_delayed_work().

Previously, this was okay because 0 @delay short-circuited to
queue_work() before doing anything with delayed_work.  8852aac25e
moved 0 @delay test into __queue_delayed_work() after sanity check on
delayed_work making megaraid trigger BUG_ON().

Although megaraid is already fixed by c1d390d8e6 ("megaraid: fix
BUG_ON() from incorrect use of delayed work"), this patch converts
BUG_ON()s in __queue_delayed_work() to WARN_ON_ONCE()s so that such
abusers, if there are more, trigger warning but don't crash the
machine.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Xiaotian Feng <xtfeng@gmail.com>
Signed-off-by: Shuah Khan <shuah.khan@hp.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: poondog <markj338@gmail.com>