Comment typo fix

--
mpi: Fix a subtle bug setting spurious bits with in mpi_set_bit.

* mpi/mpi-bit.c (_gcry_mpi_set_bit, _gcry_mpi_set_highbit): Clear
allocated but not used bits before resizing.
* tests/t-mpi-bits.c (set_bit_with_resize): New.
--

Reported-by: Martin Sewelies.

This bug is probably with us for many years.  Probably due to
different memory allocation patterns, it did first revealed itself
with 1.6.  It could be the reason for other heisenbugs.

Signed-off-by: Werner Koch <wk@gnupg.org>
Add ChaCha20 stream cipher

* cipher/Makefile.am: Add 'chacha20.c'.
* cipher/chacha20.c: New.
* cipher/cipher.c (cipher_list): Add ChaCha20.
* configure.ac: Add ChaCha20.
* doc/gcrypt.texi: Add ChaCha20.
* src/cipher.h (_gcry_cipher_spec_chacha20): New.
* src/gcrypt.h.in (GCRY_CIPHER_CHACHA20): Add new algo.
* tests/basic.c (MAX_DATA_LEN): Increase to 128 from 100.
(check_stream_cipher): Add ChaCha20 test-vectors.
(check_ciphers): Add ChaCha20.
--

Patch adds Bernstein's ChaCha20 cipher to libgcrypt. Implementation is based
on public domain implementations.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20: add SSSE3 assembly implementation

* cipher/Makefile.am: Add 'chacha20-ssse3-amd64.S'.
* cipher/chacha20-ssse3-amd64.S: New.
* cipher/chacha20.c (USE_SSSE3): New macro.
[USE_SSSE3] (_gcry_chacha20_amd64_ssse3_blocks): New.
(chacha20_do_setkey): Select SSSE3 implementation if there is HW
support.
* configure.ac [host=x86-64]: Add 'chacha20-ssse3-amd64.lo'.
--

Add SSSE3 optimized implementation for ChaCha20. Based on implementation
by Andrew Moon.

Before (Intel Haswell):

 CHACHA20       |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |      1.97 ns/B     483.6 MiB/s      6.31 c/B
     STREAM dec |      1.97 ns/B     484.0 MiB/s      6.31 c/B

After:

 CHACHA20       |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     0.742 ns/B    1284.8 MiB/s      2.38 c/B
     STREAM dec |     0.741 ns/B    1286.5 MiB/s      2.37 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20: add AVX2/AMD64 assembly implementation

* cipher/Makefile.am: Add 'chacha20-avx2-amd64.S'.
* cipher/chacha20-avx2-amd64.S: New.
* cipher/chacha20.c (USE_AVX2): New macro.
[USE_AVX2] (_gcry_chacha20_amd64_avx2_blocks): New.
(chacha20_do_setkey): Select AVX2 implementation if there is HW
support.
(selftest): Increase size of buf by 256.
* configure.ac [host=x86-64]: Add 'chacha20-avx2-amd64.lo'.
--

Add AVX2 optimized implementation for ChaCha20. Based on implementation by
Andrew Moon.

SSSE3 (Intel Haswell):

 CHACHA20       |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     0.742 ns/B    1284.8 MiB/s      2.38 c/B
     STREAM dec |     0.741 ns/B    1286.5 MiB/s      2.37 c/B

AVX2:

 CHACHA20       |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     0.393 ns/B    2428.0 MiB/s      1.26 c/B
     STREAM dec |     0.392 ns/B    2433.6 MiB/s      1.25 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20/SSSE3: clear XMM registers after use

* cipher/chacha20-ssse3-amd64.S (_gcry_chacha20_amd64_ssse3_blocks): On
return, clear XMM registers.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20/AVX2: check for ENABLE_AVX2_SUPPORT instead of HAVE_GCC_INLINE_ASM_AVX2

* cipher/chacha20.c (USE_AVX2): Enable depending on
ENABLE_AVX2_SUPPORT, not HAVE_GCC_INLINE_ASM_AVX2.
* cipher/chacha20-avx2-amd64.S: Ditto.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20/AVX2: clear upper-halfs of YMM registers on entry

* cipher/chacha20-avx2-amd64.S (_gcry_chacha20_amd64_avx2_blocks): Add
'vzeroupper' at beginning.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add Poly1305 MAC

* cipher/Makefile.am: Add 'mac-poly1305.c', 'poly1305.c' and
'poly1305-internal.h'.
* cipher/mac-internal.h (poly1305mac_context_s): New.
(gcry_mac_handle): Add 'u.poly1305mac'.
(_gcry_mac_type_spec_poly1305mac): New.
* cipher/mac-poly1305.c: New.
* cipher/mac.c (mac_list): Add Poly1305.
* cipher/poly1305-internal.h: New.
* cipher/poly1305.c: New.
* src/gcrypt.h.in: Add 'GCRY_MAC_POLY1305'.
* tests/basic.c (check_mac): Add Poly1035 test vectors; Allow
overriding lengths of data and key buffers.
* tests/bench-slope.c (mac_bench): Increase max algo number from 500 to
600.
* tests/benchmark.c (mac_bench): Ditto.
--

Patch adds Bernstein's Poly1305 message authentication code to libgcrypt.
Implementation is based on Andrew Moon's public domain implementation
from: https://github.com/floodyberry/poly1305-opt

The algorithm added by this patch is the plain Poly1305 without AES and
takes 32-bit key that must not be reused.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add Poly1305-AES (-Camellia, etc) MACs

* cipher/mac-internal.h (_gcry_mac_type_spec_poly1305_aes)
(_gcry_mac_type_spec_poly1305_camellia)
(_gcry_mac_type_spec_poly1305_twofish)
(_gcry_mac_type_spec_poly1305_serpent)
(_gcry_mac_type_spec_poly1305_seed): New.
* cipher/mac-poly1305.c (poly1305mac_context_s): Add 'hd' and
'nonce_set'.
(poly1305mac_open, poly1305mac_close, poly1305mac_setkey): Add handling
for Poly1305-*** MACs.
(poly1305mac_prepare_key, poly1305mac_setiv): New.
(poly1305mac_reset, poly1305mac_write, poly1305mac_read): Add handling
for 'nonce_set'.
(poly1305mac_ops): Add 'poly1305mac_setiv'.
(_gcry_mac_type_spec_poly1305_aes)
(_gcry_mac_type_spec_poly1305_camellia)
(_gcry_mac_type_spec_poly1305_twofish)
(_gcry_mac_type_spec_poly1305_serpent)
(_gcry_mac_type_spec_poly1305_seed): New.
* cipher/mac.c (mac_list): Add Poly1305-AES, Poly1305-Twofish,
Poly1305-Serpent, Poly1305-SEED and Poly1305-Camellia.
* src/gcrypt.h.in: Add 'GCRY_MAC_POLY1305_AES',
'GCRY_MAC_POLY1305_CAMELLIA', 'GCRY_MAC_POLY1305_TWOFISH',
'GCRY_MAC_POLY1305_SERPENT' and 'GCRY_MAC_POLY1305_SEED'.
* tests/basic.c (check_mac): Add Poly1305-AES test vectors.
* tests/bench-slope.c (bench_mac_init): Set IV for Poly1305-*** MACs.
* tests/bench-slope.c (mac_bench): Set IV for Poly1305-*** MACs.
--

Patch adds Bernstein's Poly1305-AES message authentication code to libgcrypt
and other variants of Poly1305-<128-bit block cipher>.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add Poly1305 based cipher AEAD mode

* cipher/Makefile.am: Add 'cipher-poly1305.c'.
* cipher/cipher-internal.h (gcry_cipher_handle): Add 'u_mode.poly1305'.
(_gcry_cipher_poly1305_encrypt, _gcry_cipher_poly1305_decrypt)
(_gcry_cipher_poly1305_setiv, _gcry_cipher_poly1305_authenticate)
(_gcry_cipher_poly1305_get_tag, _gcry_cipher_poly1305_check_tag): New.
* cipher/cipher-poly1305.c: New.
* cipher/cipher.c (_gcry_cipher_open_internal, cipher_setkey)
(cipher_reset, cipher_encrypt, cipher_decrypt, _gcry_cipher_setiv)
(_gcry_cipher_authenticate, _gcry_cipher_gettag)
(_gcry_cipher_checktag): Handle 'GCRY_CIPHER_MODE_POLY1305'.
(cipher_setiv): Move handling of 'GCRY_CIPHER_MODE_GCM' to ...
(_gcry_cipher_setiv): ... here, as with other modes.
* src/gcrypt.h.in: Add 'GCRY_CIPHER_MODE_POLY1305'.
* tests/basic.c (_check_poly1305_cipher, check_poly1305_cipher): New.
(check_ciphers): Add Poly1305 check.
(check_cipher_modes): Call 'check_poly1305_cipher'.
* tests/bench-slope.c (bench_gcm_encrypt_do_bench): Rename to
bench_aead_... and take nonce as argument.
(bench_gcm_decrypt_do_bench, bench_gcm_authenticate_do_bench): Ditto.
(bench_gcm_encrypt_do_bench, bench_gcm_decrypt_do_bench)
(bench_gcm_authenticate_do_bench, bench_poly1305_encrypt_do_bench)
(bench_poly1305_decrypt_do_bench)
(bench_poly1305_authenticate_do_bench, poly1305_encrypt_ops)
(poly1305_decrypt_ops, poly1305_authenticate_ops): New.
(cipher_modes): Add Poly1305.
(cipher_bench_one): Add special handling for Poly1305.
--

Patch adds Poly1305 based AEAD cipher mode to libgcrypt. ChaCha20 variant
of this mode is proposed for use in TLS and ipsec:
 https://tools.ietf.org/html/draft-agl-tls-chacha20poly1305-04
 http://tools.ietf.org/html/draft-nir-ipsecme-chacha20-poly1305-02

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
poly1305: add AMD64/SSE2 optimized implementation

* cipher/Makefile.am: Add 'poly1305-sse2-amd64.S'.
* cipher/poly1305-internal.h (POLY1305_USE_SSE2)
(POLY1305_SSE2_BLOCKSIZE, POLY1305_SSE2_STATESIZE)
(POLY1305_SSE2_ALIGNMENT): New.
(POLY1305_LARGEST_BLOCKSIZE, POLY1305_LARGEST_STATESIZE)
(POLY1305_STATE_ALIGNMENT): Use SSE2 versions when needed.
* cipher/poly1305-sse2-amd64.S: New.
* cipher/poly1305.c [POLY1305_USE_SSE2]
(_gcry_poly1305_amd64_sse2_init_ext)
(_gcry_poly1305_amd64_sse2_finish_ext)
(_gcry_poly1305_amd64_sse2_blocks, poly1305_amd64_sse2_ops): New.
(_gcry_polu1305_init) [POLY1305_USE_SSE2]: Use SSE2 version.
* configure.ac [host=x86_64]: Add 'poly1305-sse2-amd64.lo'.
--

Add Andrew Moon's public domain SSE2 implementation of Poly1305. Original
source is available at: https://github.com/floodyberry/poly1305-opt

Benchmarks on Intel i5-4570 (haswell):

Old:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.844 ns/B    1130.2 MiB/s      2.70 c/B

New:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.448 ns/B    2129.5 MiB/s      1.43 c/B

Benchmarks on Intel i5-2450M (sandy-bridge):

Old:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |      1.25 ns/B     763.0 MiB/s      3.12 c/B

New:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.605 ns/B    1575.9 MiB/s      1.51 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
poly1305: add AMD64/AVX2 optimized implementation

* cipher/Makefile.am: Add 'poly1305-avx2-amd64.S'.
* cipher/poly1305-avx2-amd64.S: New.
* cipher/poly1305-internal.h (POLY1305_USE_AVX2)
(POLY1305_AVX2_BLOCKSIZE, POLY1305_AVX2_STATESIZE)
(POLY1305_AVX2_ALIGNMENT): New.
(POLY1305_LARGEST_BLOCKSIZE, POLY1305_LARGEST_STATESIZE)
(POLY1305_STATE_ALIGNMENT): Use AVX2 versions when needed.
* cipher/poly1305.c [POLY1305_USE_AVX2]
(_gcry_poly1305_amd64_avx2_init_ext)
(_gcry_poly1305_amd64_avx2_finish_ext)
(_gcry_poly1305_amd64_avx2_blocks, poly1305_amd64_avx2_ops): New.
(_gcry_poly1305_init) [POLY1305_USE_AVX2]: Use AVX2 implementation if
AVX2 supported by CPU.
* configure.ac [host=x86_64]: Add 'poly1305-avx2-amd64.lo'.
--

Add Andrew Moon's public domain AVX2 implementation of Poly1305. Original
source is available at: https://github.com/floodyberry/poly1305-opt

Benchmarks on Intel i5-4570 (haswell):

Old:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.448 ns/B    2129.5 MiB/s      1.43 c/B

New:
                    |  nanosecs/byte   mebibytes/sec   cycles/byte
 POLY1305           |     0.205 ns/B    4643.5 MiB/s     0.657 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
chacha20: add SSE2/AMD64 optimized implementation

* cipher/Makefile.am: Add 'chacha20-sse2-amd64.S'.
* cipher/chacha20-sse2-amd64.S: New.
* cipher/chacha20.c (USE_SSE2): New.
[USE_SSE2] (_gcry_chacha20_amd64_sse2_blocks): New.
(chacha20_do_setkey) [USE_SSE2]: Use SSE2 implementation for blocks
function.
* configure.ac [host=x86-64]: Add 'chacha20-sse2-amd64.lo'.
--

Add Andrew Moon's public domain SSE2 implementation of ChaCha20. Original
source is available at: https://github.com/floodyberry/chacha-opt

Benchmark on Intel i5-4570 (haswell),
with "--disable-hwf intel-avx2 --disable-hwf intel-ssse3":

Old:
 CHACHA20       |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |      1.97 ns/B     483.8 MiB/s      6.31 c/B
     STREAM dec |      1.97 ns/B     483.6 MiB/s      6.31 c/B

New:
 CHACHA20       |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     0.931 ns/B    1024.7 MiB/s      2.98 c/B
     STREAM dec |     0.930 ns/B    1025.0 MiB/s      2.98 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add Poly1305 to documentation

* doc/gcrypt.texi: Add documentation for Poly1305 MACs and AEAD mode.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Fix ARM assembly when building __PIC__

* cipher/camellia-arm.S (GET_DATA_POINTER): New.
(_gcry_camellia_arm_encrypt_block): Use GET_DATA_POINTER.
(_gcry_camellia_arm_decrypt_block): Ditto.
* cipher/cast5-arm.S (GET_DATA_POINTER): New.
(_gcry_cast5_arm_encrypt_block, _gcry_cast5_arm_decrypt_block)
(_gcry_cast5_arm_enc_blk2, _gcry_cast5_arm_dec_blk2): Use
GET_DATA_POINTER.
* cipher/rijndael-arm.S (GET_DATA_POINTER): New.
(_gcry_aes_arm_encrypt_block, _gcry_aes_arm_decrypt_block): Use
GET_DATA_POINTER.
* cipher/sha1-armv7-neon.S (GET_DATA_POINTER): New.
(.LK_VEC): Move from .text to .data section.
(_gcry_sha1_transform_armv7_neon): Use GET_DATA_POINTER.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
sha512: fix ARM/NEON implementation

* cipher/sha512-armv7-neon.S
(_gcry_sha512_transform_armv7_neon): Byte-swap RW67q and RW1011q
correctly in multi-block loop.
* tests/basic.c (check_digests): Add large test vector for SHA512.
--

Patch fixes bug introduced to multi-block processing by commit df629ba53a6,
"Improve performance of SHA-512/ARM/NEON implementation". Patch also adds
multi-block test vector for SHA-512.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
tests: add larger test-vectors for hash algorithms

* tests/basic.c (check_digests): Add large test-vectors for MD5, SHA1,
SHA224, SHA256, SHA384, RMD160, CRC32, TIGER1, WHIRLPOOL and
GOSTR3411_94.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
GOST R 34.11-94 add OIDs

* cipher/gostr3411-94.c: Add OIDs for GOST R 34.11-94 from RFC 4357.

Signed-off-by: Dmitry Eremin-Solenikov <dbaryshkov@gmail.com>
gost28147: add OIDs used to define cipher mode

* cipher/gost28147 (oids_gost28147): Add OID from RFC4357.

Signed-off-by: Dmitry Eremin-Solenikov <dbaryshkov@gmail.com>