ARM: move VFP init to an earlier boot stage

In order to use the NEON unit in the kernel, we should
initialize it a bit earlier in the boot process so NEON users
that like to do a quick benchmark at load time (like the
xor_blocks or RAID-6 code) find the NEON/VFP unit already
enabled.

Replaced late_initcall() with core_initcall().

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>

Conflicts:
	arch/arm/vfp/vfpmodule.c
ARM: be strict about FP exceptions in kernel mode

The support code in vfp_support_entry does not care whether the
exception that caused it to be invoked occurred in kernel mode or
in user mode. However, neither condition that could trigger this
exception (lazy restore and VFP bounce to support code) is
currently allowable in kernel mode.

In either case, print a message describing the condition before
letting the undefined instruction handler run its course and trigger
an oops.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>

Conflicts:
	arch/arm/vfp/vfpmodule.c
Signed-off-by: poondog <markj338@gmail.com>

Conflicts:
	arch/arm/vfp/vfphw.S
ARM: add support for kernel mode NEON

In order to safely support the use of NEON instructions in
kernel mode, some precautions need to be taken:
- the userland context that may be present in the registers (even
  if the NEON/VFP is currently disabled) must be stored under the
  correct task (which may not be 'current' in the UP case),
- to avoid having to keep track of additional vfpstates for the
  kernel side, disallow the use of NEON in interrupt context
  and run with preemption disabled,
- after use, re-enable preemption and re-enable the lazy restore
  machinery by disabling the NEON/VFP unit.

This patch adds the functions kernel_neon_begin() and
kernel_neon_end() which take care of the above. It also adds
the Kconfig symbol KERNEL_MODE_NEON to enable it.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>

Conflicts:
	arch/arm/vfp/vfpmodule.c
Signed-off-by: poondog <markj338@gmail.com>
ARM: crypto: add NEON accelerated XOR implementation

Add a source file xor-neon.c (which is really just the reference
C implementation passed through the GCC vectorizer) and hook it
up to the XOR framework.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: poondog <markj338@gmail.com>
ARM: 7835/2: fix modular build of xor_blocks() with NEON enabled

Commit 0195659 introduced a NEON accelerated version of the xor_blocks()
function, but it needs the changes in this patch to allow it to be built
as a module rather than statically into the kernel.

This patch creates a separate module xor-neon.ko which exports the NEON
inner xor_blocks() functions depended upon by the regular xor.ko if it
is built with CONFIG_KERNEL_MODE_NEON=y

Reported-by: Josh Boyer <jwboyer@fedoraproject.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: poondog <markj338@gmail.com>
defconfig: Enable support for NEON in kernel mode

Signed-off-by: poondog <markj338@gmail.com>
ARM: add support for bit sliced AES using NEON instructions

Bit sliced AES gives around 45% speedup on Cortex-A15 for encryption
and around 25% for decryption. This implementation of the AES algorithm
does not rely on any lookup tables so it is believed to be invulnerable
to cache timing attacks.

This algorithm processes up to 8 blocks in parallel in constant time. This
means that it is not usable by chaining modes that are strictly sequential
in nature, such as CBC encryption. CBC decryption, however, can benefit from
this implementation and runs about 25% faster. The other chaining modes
implemented in this module, XTS and CTR, can execute fully in parallel in
both directions.

The core code has been adopted from the OpenSSL project (in collaboration
with the original author, on cc). For ease of maintenance, this version is
identical to the upstream OpenSSL code, i.e., all modifications that were
required to make it suitable for inclusion into the kernel have been made
upstream. The original can be found here:

    http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=6f6a6130

Note to integrators:
While this implementation is significantly faster than the existing table
based ones (generic or ARM asm), especially in CTR mode, the effects on
power efficiency are unclear as of yet. This code does fundamentally more
work, by calculating values that the table based code obtains by a simple
lookup; only by doing all of that work in a SIMD fashion, it manages to
perform better.

Cc: Andy Polyakov <appro@openssl.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: poondog <markj338@gmail.com>
defconfig: Enable NEON optimized crypto routines

Signed-off-by: poondog <markj338@gmail.com>
arm: vfpmodule: Fix warning procfs vfp_bounce reporting failed

Creation of procfs cpu/vfp_bounce fails because we're initialized too early. Fix
this by creating it on rootfs_initcall as before the NEON patches.

<6>[    0.130770] VFP support v0.3: implementor 51 architecture 64 part 6f varia
nt 2 rev 0
<4>[    0.130795] ------------[ cut here ]------------
<4>[    0.130813] WARNING: at fs/proc/generic.c:323 __xlate_proc_name+0xac/0xcc(
)
<4>[    0.130822] name 'cpu/vfp_bounce'
<4>[    0.130855] [<c010e26c>] (unwind_backtrace+0x0/0x144) from [<c0a20f58>] (d
ump_stack+0x20/0x24)
<4>[    0.130879] [<c0a20f58>] (dump_stack+0x20/0x24) from [<c019b670>] (warn_sl
owpath_common+0x58/0x70)
<4>[    0.130899] [<c019b670>] (warn_slowpath_common+0x58/0x70) from [<c019b704>
] (warn_slowpath_fmt+0x40/0x48)
<4>[    0.130919] [<c019b704>] (warn_slowpath_fmt+0x40/0x48) from [<c02c2ad8>] (
__xlate_proc_name+0xac/0xcc)
<4>[    0.130938] [<c02c2ad8>] (__xlate_proc_name+0xac/0xcc) from [<c02c2b50>] (
__proc_create+0x58/0x100)
<4>[    0.130956] [<c02c2b50>] (__proc_create+0x58/0x100) from [<c02c2ed0>] (pro
c_create_data+0x5c/0xc0)
<4>[    0.130979] [<c02c2ed0>] (proc_create_data+0x5c/0xc0) from [<c0f03484>] (v
fp_init+0x19c/0x200)
<4>[    0.131000] [<c0f03484>] (vfp_init+0x19c/0x200) from [<c0f00c98>] (do_one_
initcall+0x98/0x168)
<4>[    0.131020] [<c0f00c98>] (do_one_initcall+0x98/0x168) from [<c0f00e60>] (k
ernel_init+0xf8/0x1b4)
<4>[    0.131043] [<c0f00e60>] (kernel_init+0xf8/0x1b4) from [<c01081a0>] (kerne
l_thread_exit+0x0/0x8)
<4>[    0.131076] ---[ end trace ea6d9a9b5e947151 ]---
<3>[    0.131086] Failed to create procfs node for VFP bounce reporting

Signed-off-by: poondog <markj338@gmail.com>
ARCH: ARM: VFP: fix build

Signed-off-by: poondog <markj338@gmail.com>
LIB: use generic strnlen_user and strncpy_from_user functions (Chad Goodman)

Conflicts:
	arch/arm/Kconfig
ARM: dcache: select DCACHE_WORD_ACCESS for little-endian ARMv6+ CPUs (Chad Goodman)

Conflicts:
	arch/arm/Kconfig

Conflicts:
	arch/arm/Kconfig
ASM: word-at-a-time: make the interfaces truly generic (Chad Goodman)

Signed-off-by: poondog <markj338@gmail.com>