md: Fix hashing for data >= 256 GB

* cipher/hash-common.h (gcry_md_block_ctx): Add "nblocks_high".
* cipher/hash-common.c (_gcry_md_block_write): Bump NBLOCKS_HIGH.
* cipher/md4.c (md4_init, md4_final): Take care of NBLOCKS_HIGH.
* cipher/md5.c (md5_init, md5_final): Ditto.
* cipher/rmd160.c (_gcry_rmd160_init, rmd160_final): Ditto.
* cipher/sha1.c (sha1_init, sha1_final): Ditto.
* cipher/sha256.c (sha256_init, sha224_init, sha256_final): Ditto.
* cipher/sha512.c (sha512_init, sha384_init, sha512_final): Ditto.
* cipher/tiger.c (do_init, tiger_final): Ditto.
* cipher/whirlpool.c (whirlpool_final): Ditto.

* cipher/md.c (gcry_md_algo_info): Add GCRYCTL_SELFTEST.
(_gcry_md_selftest): Return "not implemented" as required.
* tests/hashtest.c: New.
* tests/genhashdata.c: New.
* tests/Makefile.am (TESTS): Add hashtest.
(noinst_PROGRAMS): Add genhashdata
--

Problem found by Denis Corbin and analyzed by Yuriy Kaminskiy.

sha512 and whirlpool should not have this problem because they use 64
bit types for counting the blocks. However, a similar fix has been
employed to allow for really huge sizes - despite that it will be very
hard to test them.

The test vectors have been produced by sha{1,224,256}sum and the
genhashdata tool.  A sequence of 'a' is used for them because a test
using one million 'a' is commonly used for test vectors.  More test
vectors are required.  Running the large tests needs to be done
manual for now:

  ./hashtest --gigs 256

tests all algorithms,

  ./hashtest --gigs 256 sha1 sha224 sha256

only the given ones.  A configure option to include these test in the
standard regression suite will be useful.  The tests will take looong.

Signed-off-by: Werner Koch <wk@gnupg.org>
Avoid unneeded stack burning with AES-NI and reduce number of 'decryption_prepared' checks

* cipher/rijndael.c (RIJNDAEL_context): Make 'decryption_prepared',
'use_padlock' and 'use_aesni' 1-bit members in bitfield.
(do_setkey): Move 'hwfeatures' inside [USE_AESNI || USE_PADLOCK].
(do_aesni_enc_aligned): Rename to...
(do_aesni_enc): ...this, as function does not require aligned input.
(do_aesni_dec_aligned): Rename to...
(do_aesni_dec): ...this, as function does not require aligned input.
(do_aesni): Remove.
(rijndael_encrypt): Call 'do_aesni_enc' instead of 'do_aesni'.
(rijndael_decrypt): Call 'do_aesni_dec' instead of 'do_aesni'.
(check_decryption_preparation): New.
(do_decrypt): Remove 'decryption_prepared' check.
(rijndael_decrypt): Ditto and call 'check_decryption_preparation'.
(_gcry_aes_cbc_dec): Ditto.
(_gcry_aes_cfb_enc): Add 'burn_depth' and burn stack only when needed.
(_gcry_aes_cbc_enc): Ditto.
(_gcry_aes_ctr_enc): Ditto.
(_gcry_aes_cfb_dec): Ditto.
(_gcry_aes_cbc_dec): Ditto and correct clearing of 'savebuf'.
--

Patch is mostly about reducing overhead for short buffers.

Results on Intel i5-4570:

After:
 $ tests/benchmark --cipher-repetitions 1000 --cipher-with-keysetup cipher aes
 Running each test 1000 times.
                 ECB/Stream         CBC             CFB             OFB             CTR             CCM
              --------------- --------------- --------------- --------------- --------------- ---------------
 AES            480ms   540ms  1750ms   300ms  1630ms   300ms  1640ms  1640ms   350ms   350ms  2130ms  2140ms

Before:
 $ tests/benchmark --cipher-repetitions 1000 --cipher-with-keysetup cipher aes
 Running each test 1000 times.
                 ECB/Stream         CBC             CFB             OFB             CTR             CCM
              --------------- --------------- --------------- --------------- --------------- ---------------
 AES            520ms   590ms  1760ms   310ms  1640ms   310ms  1610ms  1600ms   360ms   360ms  2150ms  2160ms

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Camellia: Add AVX/AES-NI key setup

* cipher/camellia-aesni-avx-amd64.S (key_bitlength, key_table): New
order of fields in ctx.
(camellia_f, vec_rol128, vec_ror128): New macros.
(__camellia_avx_setup128, __camellia_avx_setup256)
(_gcry_camellia_aesni_avx_keygen): New functions.
* cipher/camellia-aesni-avx2-amd64.S (key_bitlength, key_table): New
order of fields in ctx.
* cipher/camellia-arm.S (CAMELLIA_TABLE_BYTE_LEN, key_length): Remove
unused macros.
* cipher/camellia-glue.c (CAMELLIA_context): Move keytable to head for
better alignment; Make 'use_aesni_avx' and 'use_aesni_avx2' bitfield
members.
[USE_AESNI_AVX] (_gcry_camellia_aesni_avx_keygen): New prototype.
(camellia_setkey) [USE_AESNI_AVX || USE_AESNI_AVX2]: Read hw features
to variable 'hwf' and match features from it.
(camellia_setkey) [USE_AESNI_AVX]: Use AES-NI/AVX key setup if
available.
--

Use AVX/AES-NI for key-setup for small speed-up.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
cipher: use size_t for internal buffer lengths

* cipher/arcfour.c (do_encrypt_stream, encrypt_stream): Use 'size_t'
for buffer lengths.
* cipher/blowfish.c (_gcry_blowfish_ctr_enc, _gcry_blowfish_cbc_dec)
(_gcry_blowfish_cfb_dec): Ditto.
* cipher/camellia-glue.c (_gcry_camellia_ctr_enc)
(_gcry_camellia_cbc_dec, _gcry_blowfish_cfb_dec): Ditto.
* cipher/cast5.c (_gcry_cast5_ctr_enc, _gcry_cast5_cbc_dec)
(_gcry_cast5_cfb_dec): Ditto.
* cipher/cipher-aeswrap.c (_gcry_cipher_aeswrap_encrypt)
(_gcry_cipher_aeswrap_decrypt): Ditto.
* cipher/cipher-cbc.c (_gcry_cipher_cbc_encrypt)
(_gcry_cipher_cbc_decrypt): Ditto.
* cipher/cipher-ccm.c (_gcry_cipher_ccm_encrypt)
(_gcry_cipher_ccm_decrypt): Ditto.
* cipher/cipher-cfb.c (_gcry_cipher_cfb_encrypt)
(_gcry_cipher_cfb_decrypt): Ditto.
* cipher/cipher-ctr.c (_gcry_cipher_ctr_encrypt): Ditto.
* cipher/cipher-internal.h (gcry_cipher_handle->bulk)
(_gcry_cipher_cbc_encrypt, _gcry_cipher_cbc_decrypt)
(_gcry_cipher_cfb_encrypt, _gcry_cipher_cfb_decrypt)
(_gcry_cipher_ofb_encrypt, _gcry_cipher_ctr_encrypt)
(_gcry_cipher_aeswrap_encrypt, _gcry_cipher_aeswrap_decrypt)
(_gcry_cipher_ccm_encrypt, _gcry_cipher_ccm_decrypt): Ditto.
* cipher/cipher-ofb.c (_gcry_cipher_cbc_encrypt): Ditto.
* cipher/cipher-selftest.h (gcry_cipher_bulk_cbc_dec_t)
(gcry_cipher_bulk_cfb_dec_t, gcry_cipher_bulk_ctr_enc_t): Ditto.
* cipher/cipher.c (cipher_setkey, cipher_setiv, do_ecb_crypt)
(do_ecb_encrypt, do_ecb_decrypt, cipher_encrypt)
(cipher_decrypt): Ditto.
* cipher/rijndael.c (_gcry_aes_ctr_enc, _gcry_aes_cbc_dec)
(_gcry_aes_cfb_dec, _gcry_aes_cbc_enc, _gcry_aes_cfb_enc): Ditto.
* cipher/salsa20.c (salsa20_setiv, salsa20_do_encrypt_stream)
(salsa20_encrypt_stream, salsa20r12_encrypt_stream): Ditto.
* cipher/serpent.c (_gcry_serpent_ctr_enc, _gcry_serpent_cbc_dec)
(_gcry_serpent_cfb_dec): Ditto.
* cipher/twofish.c (_gcry_twofish_ctr_enc, _gcry_twofish_cbc_dec)
(_gcry_twofish_cfb_dec): Ditto.
* src/cipher-proto.h (gcry_cipher_stencrypt_t)
(gcry_cipher_stdecrypt_t, cipher_setiv_fuct_t): Ditto.
* src/cipher.h (_gcry_aes_cfb_enc, _gcry_aes_cfb_dec)
(_gcry_aes_cbc_enc, _gcry_aes_cbc_dec, _gcry_aes_ctr_enc)
(_gcry_blowfish_cfb_dec, _gcry_blowfish_cbc_dec)
(_gcry_blowfish_ctr_enc, _gcry_cast5_cfb_dec, _gcry_cast5_cbc_dec)
(_gcry_cast5_ctr_enc, _gcry_camellia_cfb_dec, _gcry_camellia_cbc_dec)
(_gcry_camellia_ctr_enc, _gcry_serpent_cfb_dec, _gcry_serpent_cbc_dec)
(_gcry_serpent_ctr_enc, _gcry_twofish_cfb_dec, _gcry_twofish_cbc_dec)
(_gcry_twofish_ctr_enc): Ditto.
--

On 64-bit platforms, cipher module internally converts 64-bit size_t values
to 32-bit unsigned integers.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Use correct blocksize of 32 bytes for GOSTR3411-94 HMAC

* cipher/md.c (md_open): Set macpads_Bsize to 32 for
GCRY_MD_GOST24311_94.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add new MAC API, initially with HMAC

* cipher/Makefile.am: Add 'mac.c', 'mac-internal.h' and 'mac-hmac.c'.
* cipher/bufhelp.h (buf_eq_const): New.
* cipher/cipher-ccm.c (_gcry_cipher_ccm_tag): Use 'buf_eq_const' for
constant-time compare.
* cipher/mac-hmac.c: New.
* cipher/mac-internal.h: New.
* cipher/mac.c: New.
* doc/gcrypt.texi: Add documentation for MAC API.
* src/gcrypt-int.h [GPG_ERROR_VERSION_NUMBER < 1.13]
(GPG_ERR_MAC_ALGO): New.
* src/gcrypt.h.in (gcry_mac_handle, gcry_mac_hd_t, gcry_mac_algos)
(gcry_mac_flags, gcry_mac_open, gcry_mac_close, gcry_mac_ctl)
(gcry_mac_algo_info, gcry_mac_setkey, gcry_mac_setiv, gcry_mac_write)
(gcry_mac_read, gcry_mac_verify, gcry_mac_get_algo_maclen)
(gcry_mac_get_algo_keylen, gcry_mac_algo_name, gcry_mac_map_name)
(gcry_mac_reset, gcry_mac_test_algo): New.
* src/libgcrypt.def (gcry_mac_open, gcry_mac_close, gcry_mac_ctl)
(gcry_mac_algo_info, gcry_mac_setkey, gcry_mac_setiv, gcry_mac_write)
(gcry_mac_read, gcry_mac_verify, gcry_mac_get_algo_maclen)
(gcry_mac_get_algo_keylen, gcry_mac_algo_name, gcry_mac_map_name): New.
* src/libgcrypt.vers (gcry_mac_open, gcry_mac_close, gcry_mac_ctl)
(gcry_mac_algo_info, gcry_mac_setkey, gcry_mac_setiv, gcry_mac_write)
(gcry_mac_read, gcry_mac_verify, gcry_mac_get_algo_maclen)
(gcry_mac_get_algo_keylen, gcry_mac_algo_name, gcry_mac_map_name): New.
* src/visibility.c (gcry_mac_open, gcry_mac_close, gcry_mac_ctl)
(gcry_mac_algo_info, gcry_mac_setkey, gcry_mac_setiv, gcry_mac_write)
(gcry_mac_read, gcry_mac_verify, gcry_mac_get_algo_maclen)
(gcry_mac_get_algo_keylen, gcry_mac_algo_name, gcry_mac_map_name): New.
* src/visibility.h (gcry_mac_open, gcry_mac_close, gcry_mac_ctl)
(gcry_mac_algo_info, gcry_mac_setkey, gcry_mac_setiv, gcry_mac_write)
(gcry_mac_read, gcry_mac_verify, gcry_mac_get_algo_maclen)
(gcry_mac_get_algo_keylen, gcry_mac_algo_name, gcry_mac_map_name): New.
* tests/basic.c (check_one_mac, check_mac): New.
(main): Call 'check_mac'.
* tests/bench-slope.c (bench_print_header, bench_print_footer): Allow
variable algorithm name width.
(_cipher_bench, hash_bench): Update to above change.
(bench_hash_do_bench): Add 'gcry_md_reset'.
(bench_mac_mode, bench_mac_init, bench_mac_free, bench_mac_do_bench)
(mac_ops, mac_modes, mac_bench_one, _mac_bench, mac_bench): New.
(main): Add 'mac' benchmark options.
* tests/benchmark.c (mac_repetitions, mac_bench): New.
(main): Add 'mac' benchmark options.
--

Add MAC API, with HMAC algorithms. Internally uses HMAC functionality of the
MD module.

[v2]:
 - Add documentation for MAC API.
 - Change length argument for gcry_mac_read from size_t to size_t* for
   returning number of written bytes.
[v3]:
 - HMAC algorithm ids start from 101.
 - Fix coding style for new files.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add CMAC (Cipher-based MAC) to MAC API

* cipher/Makefile.am: Add 'cipher-cmac.c' and 'mac-cmac.c'.
* cipher/cipher-cmac.c: New.
* cipher/cipher-internal.h (gcry_cipher_handle.u_mode): Add 'cmac'.
* cipher/cipher.c (gcry_cipher_open): Rename to...
(_gcry_cipher_open_internal): ...this and add CMAC.
(gcry_cipher_open): New wrapper that disallows use of internal
modes (CMAC) from outside.
(cipher_setkey, cipher_encrypt, cipher_decrypt)
(_gcry_cipher_authenticate, _gcry_cipher_gettag)
(_gcry_cipher_checktag): Add handling for CMAC mode.
(cipher_reset): Do not reset 'marks.key' and do not clear subkeys in
'u_mode' in CMAC mode.
* cipher/mac-cmac.c: New.
* cipher/mac-internal.h: Add CMAC support and algorithms.
* cipher/mac.c: Add CMAC algorithms.
* doc/gcrypt.texi: Add documentation for CMAC.
* src/cipher.h (gcry_cipher_internal_modes): New.
(_gcry_cipher_open_internal, _gcry_cipher_cmac_authenticate)
(_gcry_cipher_cmac_get_tag, _gcry_cipher_cmac_check_tag)
(_gcry_cipher_cmac_set_subkeys): New prototypes.
* src/gcrypt.h.in (gcry_mac_algos): Add CMAC algorithms.
* tests/basic.c (check_mac): Add CMAC test vectors.
--

Patch adds CMAC (Cipher-based MAC) as defined in RFC 4493 and NIST
Special Publication 800-38B.

Internally CMAC is added to cipher module, but is available to outside
only through MAC API.

[v2]:
 - Add documentation.
[v3]:
 - CMAC algorithm ids start from 201.
 - Coding style fixes.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Initial implementation of GCM

Currently it is still quite slow.

Still no support for generate_iv(). Is it really necessary?

TODO: Merge/reuse cipher-internal state used by CCM.

Changelog entry will be present in final patch submission.

Changes since v1:
- 6x-7x speedup.
- added bench-slope support

Signed-off-by: Dmitry Eremin-Solenikov <dbaryshkov@gmail.com>
[jk: mangle new file throught 'indent -nut']
[jk: few fixes]
[jk: changelog]
GCM optimizations
Add Intel PCLMUL acceleration for GCM
GCM: use counter mode code for speed-up
GCM: Tweak Intel PCLMUL ghash loop for small speed-up

#ifdef __x86_64__
      if (nblocks >= 4)
        {
          do
            {
              buf += 4 * blocksize;
              nblocks -= 4;
            }
          while (nblocks >= 4);
        }
#endif
Add aggregated bulk processing for GCM on x86-64

--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
GCM: Add stack burning
GCM: Add clearing and checking of marks.tag
GCM: add FIPS mode restrictions