ARM: add support for bit sliced AES using NEON instructions

Bit sliced AES gives around 45% speedup on Cortex-A15 for encryption
and around 25% for decryption. This implementation of the AES algorithm
does not rely on any lookup tables so it is believed to be invulnerable
to cache timing attacks.

This algorithm processes up to 8 blocks in parallel in constant time. This
means that it is not usable by chaining modes that are strictly sequential
in nature, such as CBC encryption. CBC decryption, however, can benefit from
this implementation and runs about 25% faster. The other chaining modes
implemented in this module, XTS and CTR, can execute fully in parallel in
both directions.

The core code has been adopted from the OpenSSL project (in collaboration
with the original author, on cc). For ease of maintenance, this version is
identical to the upstream OpenSSL code, i.e., all modifications that were
required to make it suitable for inclusion into the kernel have been made
upstream. The original can be found here:

    http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=6f6a6130

Note to integrators:
While this implementation is significantly faster than the existing table
based ones (generic or ARM asm), especially in CTR mode, the effects on
power efficiency are unclear as of yet. This code does fundamentally more
work, by calculating values that the table based code obtains by a simple
lookup; only by doing all of that work in a SIMD fashion, it manages to
perform better.

Cc: Andy Polyakov <appro@openssl.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
crypto: create generic version of ablk_helper

Create a generic version of ablk_helper so it can be reused
by other architectures.

Acked-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Conflicts:
	crypto/Kconfig
	crypto/Makefile
crypto: ablk_helper - Replace memcpy with struct assignment

tree:   git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git master
head:   48e6dc1b2a1ad8186d48968d5018912bdacac744
commit: a62b01cd6cc1feb5e80d64d6937c291473ed82cb [20/24] crypto: create generic version of ablk_helper

coccinelle warnings: (new ones prefixed by >>)

>> crypto/ablk_helper.c:97:2-8: Replace memcpy with struct assignment
>> crypto/ablk_helper.c:78:2-8: Replace memcpy with struct assignment

Please consider folding the attached diff :-)

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
ARM: move AES typedefs and function prototypes to separate header

Put the struct definitions for AES keys and the asm function prototypes in a
separate header and export the asm functions from the module.
This allows other drivers to use them directly.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
ARM: 7626/1: arm/crypto: Make asm SHA-1 and AES code Thumb-2 compatible

This patch fixes aes-armv4.S and sha1-armv4-large.S to work
natively in Thumb.  This allows ARM/Thumb interworking workarounds
to be removed.

I also take the opportunity to convert some explicit assembler
directives for exported functions to the standard
ENTRY()/ENDPROC().

For the code itself:

  * In sha1_block_data_order, use of TEQ with sp is deprecated in
    ARMv7 and not supported in Thumb.  For the branches back to
    .L_00_15 and .L_40_59, the TEQ is converted to a CMP, under the
    assumption that clobbering the C flag here will not cause
    incorrect behaviour.

    For the first branch back to .L_20_39_or_60_79 the C flag is
    important, so sp is moved temporarily into another register so
    that TEQ can be used for the comparison.

  * In the AES code, most forms of register-indexed addressing with
    shifts and rotates are not permitted for loads and stores in
    Thumb, so the address calculation is done using a separate
    instruction for the Thumb case.

The resulting code is unlikely to be optimally scheduled, but it
should not have a large impact given the overall size of the code.
I haven't run any benchmarks.

Signed-off-by: Dave Martin <dave.martin@linaro.org>
Tested-by: David McCullough <ucdevel@gmail.com> (ARM only)
Acked-by: David McCullough <ucdevel@gmail.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7837/3: fix Thumb-2 bug in AES assembler code

Patch 638591c enabled building the AES assembler code in Thumb2 mode.
However, this code used arithmetic involving PC rather than adr{l}
instructions to generate PC-relative references to the lookup tables,
and this needs to take into account the different PC offset when
running in Thumb mode.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Cc: stable@vger.kernel.org
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: 7723/1: crypto: sha1-armv4-large.S: fix SP handling

Make the SHA1 asm code ABI conformant by making sure all stack
accesses occur above the stack pointer.

Origin:
http://git.openssl.org/gitweb/?p=openssl.git;a=commit;h=1a9d60d2

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Cc: stable@vger.kernel.org
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
ARM: add .gitignore entry for aesbs-core.S

This avoids this file being incorrectly added to git.

Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
CRYPTO: Fix more AES build errors
Building a multi-arch kernel results in:

arch/arm/crypto/built-in.o: In function `aesbs_xts_decrypt':
sha1_glue.c:(.text+0x15c8): undefined reference to `bsaes_xts_decrypt'
arch/arm/crypto/built-in.o: In function `aesbs_xts_encrypt':
sha1_glue.c:(.text+0x1664): undefined reference to `bsaes_xts_encrypt'
arch/arm/crypto/built-in.o: In function `aesbs_ctr_encrypt':
sha1_glue.c:(.text+0x184c): undefined reference to `bsaes_ctr32_encrypt_blocks'
arch/arm/crypto/built-in.o: In function `aesbs_cbc_decrypt':
sha1_glue.c:(.text+0x19b4): undefined reference to `bsaes_cbc_encrypt'

This code is already runtime-conditional on NEON being supported, so
there's no point compiling it out depending on the minimum build
architecture.

Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: LaboDJ <jacopolabardi@gmail.com>
include/asm: symbolic link to asm-generic to avoid errors
Linux 3.4.87

Signed-off-by: LaboDJ <jacopolabardi@gmail.com>
crypto: msm: check NULL pointer in _qcrypto_setkey_des()

add NULL pointer check before des_ekey()

Change-Id: I57c40b82fc5141c1666c3fd493b43e962f0d1ae9
Signed-off-by: Zhen Kong <zkong@codeaurora.org>
crypto: msm: Add support for aead

Add support for aead mode algorithms (simultaneous auth
and cipher operations)

Change-Id: I22cc530f8fe822504649cb3c60030a729e1305c2
Signed-off-by: Mona Hossain <mhossain@codeaurora.org>
Signed-off-by: Hariprasad Dhalinarasimha <hnamgund@codeaurora.org>
crypto: msm: Fix Uninitialized flags

Due to uninitialized "flags" field in the request structure,
incorrect choices are made in the qce driver layer with
regards to setting crypto hardware registers.

Initializing the flags fixes this incorrect setting of the
registers.

Change-Id: Ic40319ea5e15e66cbafd28f727a9bcc6ea41b1ca
Signed-off-by: Mona Hossain <mhossain@codeaurora.org>
crypto: msm: check potential integer overflow on AEAD req length

According to the specification of AEAD, AEAD request cryptlen is
not a Fixed maximum and assoclen is also same. This could lead to
potential integer overflow, thus allocating less memory. So we
need to check potential integer overflow on AEAD request length.

Change-Id: I58a0c5e1a6e890bad30f7865e96b7db46337158c
Signed-off-by: Zhen Kong <zkong@codeaurora.org>
crypto: msm: Use scnprintf() instead of snprintf()

snprintf can return more than the size of buffer, which is not
expected here, obviously. Use scnprintf instead.

Change-Id: I3b97a016df910728dfef5a0914ab8ad67c68e39c
Signed-off-by: AnilKumar Chimata <anilc@codeaurora.org>
crypto: msm: Fix null pointer exception in qcrypto_count_sg()

Fixes null pointer exception in count scatter gather function. Scatter
gather list pointer become NULL if the sg pointer reaches last but one
node, which results in NULL pointer exception.

Change-Id: I634baf30074e6e53ffa22635565da54e9ae43150
Signed-off-by: AnilKumar Chimata <anilc@codeaurora.org>
crypto: msm: multiple qce50 support

In some platforms such as fsm9900, multiple instances of qce50 hw
may be supported.  Each qce50 HW has an ndp-bam. Each qce50 HW
can support multiple instances of qce_dev platform device.
Each qce_dev has its own configured pipe set and associated qce50 HW,
defined by crypto-base and bam-pipe-pair fields in the device tree files.
This patch based on io address, it decides to create a new qce50 hw
instance or use the existing known instance at device probe time.

Furthermore, this patch does iomap for only once for each HW BAM instance
to avoid issue with bam driver. If modprobe, and rmmod of qcedev,
and qcrypto are not in first in last out order, kernel may crash in
bam driver in the last rmmod.
This saves virtual space for IO if multiple instances of qcedev
share the same instance of HW BAM.

Acked-by: Chemin Hsieh <cheminh@qti.qualcomm.com>
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Signed-off-by: Zhen Kong <zkong@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
crypto: msm: Fix issues related to non-zero byteoffset input

Fix buffer overflow for a non-zero byteoffset value.

Also fixes memory leak issue by adding data_len check in check_params.
One of the scenarios data_len can be less than byteoffset which results
in memory leak with huge data length, which might cause the kernel panic.

Signed-off-by: AnilKumar Chimata <anilc@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
crypto: msm: Remove invalid check in qcedev driver

Unsigned int variable is compared for < 0, which is an invalid if
condition. This patch removes the check.

Signed-off-by: AnilKumar Chimata <anilc@codeaurora.org>
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>