chacha20/SSSE3: clear XMM registers after use

* cipher/chacha20-ssse3-amd64.S (_gcry_chacha20_amd64_ssse3_blocks): On
return, clear XMM registers.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20/AVX2: check for ENABLE_AVX2_SUPPORT instead of HAVE_GCC_INLINE_ASM_AVX2

* cipher/chacha20.c (USE_AVX2): Enable depending on
ENABLE_AVX2_SUPPORT, not HAVE_GCC_INLINE_ASM_AVX2.
* cipher/chacha20-avx2-amd64.S: Ditto.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20/AVX2: clear upper-halfs of YMM registers on entry

* cipher/chacha20-avx2-amd64.S (_gcry_chacha20_amd64_avx2_blocks): Add
'vzeroupper' at beginning.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add Poly1305 MAC

* cipher/Makefile.am: Add 'mac-poly1305.c', 'poly1305.c' and
'poly1305-internal.h'.
* cipher/mac-internal.h (poly1305mac_context_s): New.
(gcry_mac_handle): Add 'u.poly1305mac'.
(_gcry_mac_type_spec_poly1305mac): New.
* cipher/mac-poly1305.c: New.
* cipher/mac.c (mac_list): Add Poly1305.
* cipher/poly1305-internal.h: New.
* cipher/poly1305.c: New.
* src/gcrypt.h.in: Add 'GCRY_MAC_POLY1305'.
* tests/basic.c (check_mac): Add Poly1035 test vectors; Allow
overriding lengths of data and key buffers.
* tests/bench-slope.c (mac_bench): Increase max algo number from 500 to
600.
* tests/benchmark.c (mac_bench): Ditto.
--

Patch adds Bernstein's Poly1305 message authentication code to libgcrypt.
Implementation is based on Andrew Moon's public domain implementation
from: https://github.com/floodyberry/poly1305-opt

The algorithm added by this patch is the plain Poly1305 without AES and
takes 32-bit key that must not be reused.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add Poly1305-AES (-Camellia, etc) MACs

* cipher/mac-internal.h (_gcry_mac_type_spec_poly1305_aes)
(_gcry_mac_type_spec_poly1305_camellia)
(_gcry_mac_type_spec_poly1305_twofish)
(_gcry_mac_type_spec_poly1305_serpent)
(_gcry_mac_type_spec_poly1305_seed): New.
* cipher/mac-poly1305.c (poly1305mac_context_s): Add 'hd' and
'nonce_set'.
(poly1305mac_open, poly1305mac_close, poly1305mac_setkey): Add handling
for Poly1305-*** MACs.
(poly1305mac_prepare_key, poly1305mac_setiv): New.
(poly1305mac_reset, poly1305mac_write, poly1305mac_read): Add handling
for 'nonce_set'.
(poly1305mac_ops): Add 'poly1305mac_setiv'.
(_gcry_mac_type_spec_poly1305_aes)
(_gcry_mac_type_spec_poly1305_camellia)
(_gcry_mac_type_spec_poly1305_twofish)
(_gcry_mac_type_spec_poly1305_serpent)
(_gcry_mac_type_spec_poly1305_seed): New.
* cipher/mac.c (mac_list): Add Poly1305-AES, Poly1305-Twofish,
Poly1305-Serpent, Poly1305-SEED and Poly1305-Camellia.
* src/gcrypt.h.in: Add 'GCRY_MAC_POLY1305_AES',
'GCRY_MAC_POLY1305_CAMELLIA', 'GCRY_MAC_POLY1305_TWOFISH',
'GCRY_MAC_POLY1305_SERPENT' and 'GCRY_MAC_POLY1305_SEED'.
* tests/basic.c (check_mac): Add Poly1305-AES test vectors.
* tests/bench-slope.c (bench_mac_init): Set IV for Poly1305-*** MACs.
* tests/bench-slope.c (mac_bench): Set IV for Poly1305-*** MACs.
--

Patch adds Bernstein's Poly1305-AES message authentication code to libgcrypt
and other variants of Poly1305-<128-bit block cipher>.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add Poly1305 based cipher AEAD mode

* cipher/Makefile.am: Add 'cipher-poly1305.c'.
* cipher/cipher-internal.h (gcry_cipher_handle): Add 'u_mode.poly1305'.
(_gcry_cipher_poly1305_encrypt, _gcry_cipher_poly1305_decrypt)
(_gcry_cipher_poly1305_setiv, _gcry_cipher_poly1305_authenticate)
(_gcry_cipher_poly1305_get_tag, _gcry_cipher_poly1305_check_tag): New.
* cipher/cipher-poly1305.c: New.
* cipher/cipher.c (_gcry_cipher_open_internal, cipher_setkey)
(cipher_reset, cipher_encrypt, cipher_decrypt, _gcry_cipher_setiv)
(_gcry_cipher_authenticate, _gcry_cipher_gettag)
(_gcry_cipher_checktag): Handle 'GCRY_CIPHER_MODE_POLY1305'.
(cipher_setiv): Move handling of 'GCRY_CIPHER_MODE_GCM' to ...
(_gcry_cipher_setiv): ... here, as with other modes.
* src/gcrypt.h.in: Add 'GCRY_CIPHER_MODE_POLY1305'.
* tests/basic.c (_check_poly1305_cipher, check_poly1305_cipher): New.
(check_ciphers): Add Poly1305 check.
(check_cipher_modes): Call 'check_poly1305_cipher'.
* tests/bench-slope.c (bench_gcm_encrypt_do_bench): Rename to
bench_aead_... and take nonce as argument.
(bench_gcm_decrypt_do_bench, bench_gcm_authenticate_do_bench): Ditto.
(bench_gcm_encrypt_do_bench, bench_gcm_decrypt_do_bench)
(bench_gcm_authenticate_do_bench, bench_poly1305_encrypt_do_bench)
(bench_poly1305_decrypt_do_bench)
(bench_poly1305_authenticate_do_bench, poly1305_encrypt_ops)
(poly1305_decrypt_ops, poly1305_authenticate_ops): New.
(cipher_modes): Add Poly1305.
(cipher_bench_one): Add special handling for Poly1305.
--

Patch adds Poly1305 based AEAD cipher mode to libgcrypt. ChaCha20 variant
of this mode is proposed for use in TLS and ipsec:
 https://tools.ietf.org/html/draft-agl-tls-chacha20poly1305-04
 http://tools.ietf.org/html/draft-nir-ipsecme-chacha20-poly1305-02

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
poly1305: add AMD64/SSE2 optimized implementation

* cipher/Makefile.am: Add 'poly1305-sse2-amd64.S'.
* cipher/poly1305-internal.h (POLY1305_USE_SSE2)
(POLY1305_SSE2_BLOCKSIZE, POLY1305_SSE2_STATESIZE)
(POLY1305_SSE2_ALIGNMENT): New.
(POLY1305_LARGEST_BLOCKSIZE, POLY1305_LARGEST_STATESIZE)
(POLY1305_STATE_ALIGNMENT): Use SSE2 versions when needed.
* cipher/poly1305-sse2-amd64.S: New.
* cipher/poly1305.c [POLY1305_USE_SSE2]
(_gcry_poly1305_amd64_sse2_init_ext)
(_gcry_poly1305_amd64_sse2_finish_ext)
(_gcry_poly1305_amd64_sse2_blocks, poly1305_amd64_sse2_ops): New.
(_gcry_polu1305_init) [POLY1305_USE_SSE2]: Use SSE2 version.
* configure.ac [host=x86_64]: Add 'poly1305-sse2-amd64.lo'.
--

Add Andrew Moon's public domain SSE2 implementation of Poly1305. Original
source is available at: https://github.com/floodyberry/poly1305-opt

Benchmarks on Intel i5-4570 (haswell):

Old:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.844 ns/B    1130.2 MiB/s      2.70 c/B

New:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.448 ns/B    2129.5 MiB/s      1.43 c/B

Benchmarks on Intel i5-2450M (sandy-bridge):

Old:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |      1.25 ns/B     763.0 MiB/s      3.12 c/B

New:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.605 ns/B    1575.9 MiB/s      1.51 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
poly1305: add AMD64/AVX2 optimized implementation

* cipher/Makefile.am: Add 'poly1305-avx2-amd64.S'.
* cipher/poly1305-avx2-amd64.S: New.
* cipher/poly1305-internal.h (POLY1305_USE_AVX2)
(POLY1305_AVX2_BLOCKSIZE, POLY1305_AVX2_STATESIZE)
(POLY1305_AVX2_ALIGNMENT): New.
(POLY1305_LARGEST_BLOCKSIZE, POLY1305_LARGEST_STATESIZE)
(POLY1305_STATE_ALIGNMENT): Use AVX2 versions when needed.
* cipher/poly1305.c [POLY1305_USE_AVX2]
(_gcry_poly1305_amd64_avx2_init_ext)
(_gcry_poly1305_amd64_avx2_finish_ext)
(_gcry_poly1305_amd64_avx2_blocks, poly1305_amd64_avx2_ops): New.
(_gcry_poly1305_init) [POLY1305_USE_AVX2]: Use AVX2 implementation if
AVX2 supported by CPU.
* configure.ac [host=x86_64]: Add 'poly1305-avx2-amd64.lo'.
--

Add Andrew Moon's public domain AVX2 implementation of Poly1305. Original
source is available at: https://github.com/floodyberry/poly1305-opt

Benchmarks on Intel i5-4570 (haswell):

Old:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.448 ns/B    2129.5 MiB/s      1.43 c/B

New:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.205 ns/B    4643.5 MiB/s     0.657 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20: add SSE2/AMD64 optimized implementation

* cipher/Makefile.am: Add 'chacha20-sse2-amd64.S'.
* cipher/chacha20-sse2-amd64.S: New.
* cipher/chacha20.c (USE_SSE2): New.
[USE_SSE2] (_gcry_chacha20_amd64_sse2_blocks): New.
(chacha20_do_setkey) [USE_SSE2]: Use SSE2 implementation for blocks
function.
* configure.ac [host=x86-64]: Add 'chacha20-sse2-amd64.lo'.
--

Add Andrew Moon's public domain SSE2 implementation of ChaCha20. Original
source is available at: https://github.com/floodyberry/chacha-opt

Benchmark on Intel i5-4570 (haswell),
with "--disable-hwf intel-avx2 --disable-hwf intel-ssse3":

Old:
 CHACHA20       |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |      1.97 ns/B     483.8 MiB/s      6.31 c/B
     STREAM dec |      1.97 ns/B     483.6 MiB/s      6.31 c/B

New:
 CHACHA20       |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     0.931 ns/B    1024.7 MiB/s      2.98 c/B
     STREAM dec |     0.930 ns/B    1025.0 MiB/s      2.98 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add Poly1305 to documentation

* doc/gcrypt.texi: Add documentation for Poly1305 MACs and AEAD mode.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>