Add new benchmarking utility, bench-slope

* tests/Makefile.am (TESTS): Add 'bench-slope'.
* tests/bench-slope.c: New.
--

Bench-slope is new benchmarking tool for libgcrypt for obtaining overheadless
cycles/byte speed of cipher and hash algorithms. Tool measures the time each
operation (hash/encrypt/decrypt/authentication) takes for different buffer
sizes of from ~0kB to ~4kB and calculates the slope for these data points.
The default output is then given as nanosecs/byte and mebibytes/sec. If user
provides the speed of used CPU, tool also outputs cycles/byte result (CPU-Ghz *
ns/B = c/B).

Output without CPU speed (with ARM Cortex-A8):

$ tests/bench-slope hash

Hash:
                |  nanosecs/byte   mebibytes/sec   cycles/byte
 MD5            |      7.35 ns/B     129.7 MiB/s         - c/B
 SHA1           |     12.30 ns/B     77.53 MiB/s         - c/B
 RIPEMD160      |     15.96 ns/B     59.77 MiB/s         - c/B
 TIGER192       |     55.55 ns/B     17.17 MiB/s         - c/B
 SHA256         |     24.38 ns/B     39.12 MiB/s         - c/B
 SHA384         |     34.24 ns/B     27.86 MiB/s         - c/B
 SHA512         |     34.19 ns/B     27.90 MiB/s         - c/B
 SHA224         |     24.38 ns/B     39.12 MiB/s         - c/B
 MD4            |      5.68 ns/B     168.0 MiB/s         - c/B
 CRC32          |      9.26 ns/B     103.0 MiB/s         - c/B
 CRC32RFC1510   |      9.20 ns/B     103.6 MiB/s         - c/B
 CRC24RFC2440   |     87.31 ns/B     10.92 MiB/s         - c/B
 WHIRLPOOL      |     253.3 ns/B      3.77 MiB/s         - c/B
 TIGER          |     55.55 ns/B     17.17 MiB/s         - c/B
 TIGER2         |     55.55 ns/B     17.17 MiB/s         - c/B
 GOSTR3411_94   |     212.0 ns/B      4.50 MiB/s         - c/B
 STRIBOG256     |     630.1 ns/B      1.51 MiB/s         - c/B
 STRIBOG512     |     630.1 ns/B      1.51 MiB/s         - c/B
                =

With CPU speed (with Intel i5-4570, 3.2Ghz when turbo-boost disabled):

$ tests/bench-slope --cpu-mhz 3201 cipher arcfour blowfish aes
Cipher:
 ARCFOUR        |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |      2.43 ns/B     392.1 MiB/s      7.79 c/B
     STREAM dec |      2.44 ns/B     390.2 MiB/s      7.82 c/B
                =
 BLOWFISH       |  nanosecs/byte   mebibytes/sec   cycles/byte
        ECB enc |      7.62 ns/B     125.2 MiB/s     24.38 c/B
        ECB dec |      7.63 ns/B     125.0 MiB/s     24.43 c/B
        CBC enc |      9.18 ns/B     103.9 MiB/s     29.38 c/B
        CBC dec |      2.60 ns/B     366.2 MiB/s      8.34 c/B
        CFB enc |      9.17 ns/B     104.0 MiB/s     29.35 c/B
        CFB dec |      2.66 ns/B     358.1 MiB/s      8.53 c/B
        OFB enc |      8.97 ns/B     106.3 MiB/s     28.72 c/B
        OFB dec |      8.97 ns/B     106.3 MiB/s     28.71 c/B
        CTR enc |      2.60 ns/B     366.5 MiB/s      8.33 c/B
        CTR dec |      2.60 ns/B     367.1 MiB/s      8.32 c/B
                =
 AES            |  nanosecs/byte   mebibytes/sec   cycles/byte
        ECB enc |     0.439 ns/B    2173.0 MiB/s      1.40 c/B
        ECB dec |     0.489 ns/B    1949.5 MiB/s      1.57 c/B
        CBC enc |      1.64 ns/B     580.8 MiB/s      5.26 c/B
        CBC dec |     0.219 ns/B    4357.6 MiB/s     0.701 c/B
        CFB enc |      1.53 ns/B     623.6 MiB/s      4.90 c/B
        CFB dec |     0.219 ns/B    4350.5 MiB/s     0.702 c/B
        OFB enc |      1.51 ns/B     629.9 MiB/s      4.85 c/B
        OFB dec |      1.51 ns/B     629.9 MiB/s      4.85 c/B
        CTR enc |     0.288 ns/B    3308.5 MiB/s     0.923 c/B
        CTR dec |     0.288 ns/B    3316.9 MiB/s     0.920 c/B
        CCM enc |      1.93 ns/B     493.8 MiB/s      6.18 c/B
        CCM dec |      1.93 ns/B     494.0 MiB/s      6.18 c/B
       CCM auth |      1.64 ns/B     580.1 MiB/s      5.26 c/B
                =

Note: It's highly recommented to disable turbo-boost and dynamic CPU frequency
features when making these kind of measurements to reduce variance.

Note: The results are maximum performance for each operation; the actual speed
in application depends on various matters, such as: used buffer sizes, cache
usage, etc.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add AMD64 assembly implementation of Salsa20

* cipher/Makefile.am: Add 'salsa20-amd64.S'.
* cipher/salsa20-amd64.S: New.
* cipher/salsa20.c (USE_AMD64): New macro.
[USE_AMD64] (_gcry_salsa20_amd64_keysetup, _gcry_salsa20_amd64_ivsetup)
(_gcry_salsa20_amd64_encrypt_blocks): New prototypes.
[USE_AMD64] (salsa20_keysetup, salsa20_ivsetup, salsa20_core): New.
[!USE_AMD64] (salsa20_core): Change 'src' to non-constant, update block
counter in 'salsa20_core' and return burn stack depth.
[!USE_AMD64] (salsa20_keysetup, salsa20_ivsetup): New.
(salsa20_do_setkey): Move generic key setup to 'salsa20_keysetup'.
(salsa20_setkey): Fix burn stack depth.
(salsa20_setiv): Move generic IV setup to 'salsa20_ivsetup'.
(salsa20_do_encrypt_stream) [USE_AMD64]: Process large buffers in AMD64
implementation.
(salsa20_do_encrypt_stream): Move stack burning to this function...
(salsa20_encrypt_stream, salsa20r12_encrypt_stream): ...from these
functions.
* configure.ac [x86-64]: Add 'salsa20-amd64.lo'.
--

Patch adds fast AMD64 assembly implementation for Salsa20. This implementation
is based on public domain code by D. J. Bernstein and it is available at
http://cr.yp.to/snuffle.html (amd64-xmm6). Implementation gains extra speed
by processing four blocks in parallel with help SSE2 instructions.

Benchmark results on Intel Core i5-4570 (3.2 Ghz):

Before:
SALSA20        |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |      3.88 ns/B     246.0 MiB/s     12.41 c/B
     STREAM dec |      3.88 ns/B     246.0 MiB/s     12.41 c/B
                =
 SALSA20R12     |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |      2.46 ns/B     387.9 MiB/s      7.87 c/B
     STREAM dec |      2.46 ns/B     387.7 MiB/s      7.87 c/B

After:
 SALSA20        |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     0.985 ns/B     967.8 MiB/s      3.15 c/B
     STREAM dec |     0.987 ns/B     966.5 MiB/s      3.16 c/B
                =
 SALSA20R12     |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     0.636 ns/B    1500.5 MiB/s      2.03 c/B
     STREAM dec |     0.636 ns/B    1499.2 MiB/s      2.04 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add ARM NEON assembly implementation of Salsa20

* cipher/Makefile.am: Add 'salsa20-armv7-neon.S'.
* cipher/salsa20-armv7-neon.S: New.
* cipher/salsa20.c [USE_ARM_NEON_ASM]: New macro.
(struct SALSA20_context_s, salsa20_core_t, salsa20_keysetup_t)
(salsa20_ivsetup_t): New.
(SALSA20_context_t) [USE_ARM_NEON_ASM]: Add 'use_neon'.
(SALSA20_context_t): Add 'keysetup', 'ivsetup' and 'core'.
(salsa20_core): Change 'src' argument to 'ctx'.
[USE_ARM_NEON_ASM] (_gcry_arm_neon_salsa20_encrypt): New prototype.
[USE_ARM_NEON_ASM] (salsa20_core_neon, salsa20_keysetup_neon)
(salsa20_ivsetup_neon): New.
(salsa20_do_setkey): Setup keysetup, ivsetup and core with default
functions.
(salsa20_do_setkey) [USE_ARM_NEON_ASM]: When NEON support detect,
set keysetup, ivsetup and core with ARM NEON functions.
(salsa20_do_setkey): Call 'ctx->keysetup'.
(salsa20_setiv): Call 'ctx->ivsetup'.
(salsa20_do_encrypt_stream) [USE_ARM_NEON_ASM]: Process large buffers
in ARM NEON implementation.
(salsa20_do_encrypt_stream): Call 'ctx->core' instead of directly
calling 'salsa20_core'.
(selftest): Add test to check large buffer processing and block counter
updating.
* configure.ac [neonsupport]: 'Add salsa20-armv7-neon.lo'.
--

Patch adds fast ARM NEON assembly implementation for Salsa20. Implementation
gains extra speed by processing three blocks in parallel with help of ARM
NEON vector processing unit.

This implementation is based on public domain code by Peter Schwabe and D. J.
Bernstein and it is available in SUPERCOP benchmarking framework. For more
details on this work, check paper "NEON crypto" by Daniel J. Bernstein and
Peter Schwabe:
    http://cryptojedi.org/papers/#neoncrypto

Benchmark results on Cortex-A8 (1008 Mhz):

Before:
 SALSA20        |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     18.88 ns/B     50.51 MiB/s     19.03 c/B
     STREAM dec |     18.89 ns/B     50.49 MiB/s     19.04 c/B
                =
 SALSA20R12     |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |     13.60 ns/B     70.14 MiB/s     13.71 c/B
     STREAM dec |     13.60 ns/B     70.13 MiB/s     13.71 c/B

After:
 SALSA20        |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |      5.48 ns/B     174.1 MiB/s      5.52 c/B
     STREAM dec |      5.47 ns/B     174.2 MiB/s      5.52 c/B
                =
 SALSA20R12     |  nanosecs/byte   mebibytes/sec   cycles/byte
     STREAM enc |      3.65 ns/B     260.9 MiB/s      3.68 c/B
     STREAM dec |      3.65 ns/B     261.6 MiB/s      3.67 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Add ARM NEON assembly implementation of Serpent

* cipher/Makefile.am: Add 'serpent-armv7-neon.S'.
* cipher/serpent-armv7-neon.S: New.
* cipher/serpent.c (USE_NEON): New macro.
(serpent_context_t) [USE_NEON]: Add 'use_neon'.
[USE_NEON] (_gcry_serpent_neon_ctr_enc, _gcry_serpent_neon_cfb_dec)
(_gcry_serpent_neon_cbc_dec): New prototypes.
(serpent_setkey_internal) [USE_NEON]: Detect NEON support.
(_gcry_serpent_neon_ctr_enc, _gcry_serpent_neon_cfb_dec)
(_gcry_serpent_neon_cbc_dec) [USE_NEON]: Use NEON implementations
to process eight blocks in parallel.
* configure.ac [neonsupport]: Add 'serpent-armv7-neon.lo'.
--

Patch adds ARM NEON optimized implementation of Serpent cipher
to speed up parallelizable bulk operations.

Benchmarks on ARM Cortex-A8 (armhf, 1008 Mhz):

Old:
 SERPENT128     |  nanosecs/byte   mebibytes/sec   cycles/byte
        CBC dec |     43.53 ns/B     21.91 MiB/s     43.88 c/B
        CFB dec |     44.77 ns/B     21.30 MiB/s     45.13 c/B
        CTR enc |     45.21 ns/B     21.10 MiB/s     45.57 c/B
        CTR dec |     45.21 ns/B     21.09 MiB/s     45.57 c/B
New:
 SERPENT128     |  nanosecs/byte   mebibytes/sec   cycles/byte
        CBC dec |     26.26 ns/B     36.32 MiB/s     26.47 c/B
        CFB dec |     26.21 ns/B     36.38 MiB/s     26.42 c/B
        CTR enc |     26.20 ns/B     36.40 MiB/s     26.41 c/B
        CTR dec |     26.20 ns/B     36.40 MiB/s     26.41 c/B

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Fix typos in documentation

* doc/gcrypt.texi: Fix some typos.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
ecc: Add flags "noparam" and "comp".

* src/cipher.h (PUBKEY_FLAG_NOPARAM, PUBKEY_FLAG_COMP): New.
* cipher/pubkey-util.c (_gcry_pk_util_parse_flaglist): Parse new flags
and change code for possible faster parsing.
* cipher/ecc.c (ecc_generate): Implement the "noparam" flag.
(ecc_sign): Ditto.
(ecc_verify): Ditto.
* tests/keygen.c (check_ecc_keys): Use the "noparam" flag.

* cipher/ecc.c (ecc_generate): Fix parsing of the deprecated
transient-flag parameter.
(ecc_verify): Do not make Q optional in the extract-param call.
--

Note that the "comp" flag has not yet any effect.

Signed-off-by: Werner Koch <wk@gnupg.org>
Typo fix.

--
ecc: Finish Ed25519/ECDSA hack.

* cipher/ecc.c (ecc_generate): Fix Ed25519/ECDSA case.
(ecc_verify): Implement ED25519/ECDSA uncompression.
--

With this change Ed25519 may be used with ECDSA while using the
Ed25519 standard compression technique.

Signed-off-by: Werner Koch <wk@gnupg.org>
tests: Add feature to skip benchmarks.

* tests/benchmark.c (main): Add feature to skip the test.
* tests/bench-slope.c (main): Ditto.
(get_slope): Repace C++ style comment.
(double_cmp, cipher_bench, _hash_bench): Repalce system reserved
symbols.
--

During development a quick run of the regression is often useful,
however the benchmarks take a lot of time and thus this feature
allows to skip theses tests.

Signed-off-by: Werner Koch <wk@gnupg.org>
bithelp: fix undefined behaviour with rol and ror

* cipher/bithelp.h (rol, ror): Mask shift with 31.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Make jump labels local in Salsa20 assembly

* cipher/salsa20-amd64.S: Rename '._labels' to '.L_labels'.
* cipher/salsa20-armv7-neon.S: Ditto.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Make test vectors 'static const'

* cipher/arcfour.c (selftest): Change test vectors to 'static const'.
* cipher/blowfish.c (selftest): Ditto.
* cipher/camellia-glue.c (selftest): Ditto.
* cipher/cast5.c (selftest): Ditto.
* cipher/des.c (selftest): Ditto.
* cipher/rijndael.c (selftest): Ditto.
* tests/basic.c (cipher_cbc_mac_cipher, check_aes128_cbc_cts_cipher)
(check_ctr_cipher, check_cfb_cipher, check_ofb_cipher)
(check_ccm_cipher, check_stream_cipher)
(check_stream_cipher_large_block, check_bulk_cipher_modes)
(check_ciphers, check_digests, check_hmac, check_pubkey_sign)
(check_pubkey_sign_ecdsa, check_pubkey_crypt, check_pubkey): Ditto.
--

Some test vectors have been defined without 'static' and thus end up being
initialized on runtime. Change these to 'static'. Also change test vectors
const where possible.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
mpi: Add function gcry_mpi_set_opaque_copy.

* src/gcrypt.h.in (gcry_mpi_set_opaque_copy): New.
* src/visibility.c (gcry_mpi_set_opaque_copy): New.
* src/visibility.h (gcry_mpi_set_opaque_copy): Mark visible.
* src/libgcrypt.def, src/libgcrypt.vers: Add new API.
* tests/mpitests.c (test_opaque): Add test.

Signed-off-by: Werner Koch <wk@gnupg.org>
ecc: Fully implement Ed25519 compression in ECDSA mode.

* src/ec-context.h (mpi_ec_ctx_s): Add field FLAGS.
* mpi/ec.c (ec_p_init): Add arg FLAGS.  Change all callers to pass it.
* cipher/ecc-curves.c (point_from_keyparam): Add arg EC, parse as
 opaque mpi and use eddsa decoding depending on the flag.
(_gcry_mpi_ec_new): Rearrange to parse Q and D after knowing the
curve.

Signed-off-by: Werner Koch <wk@gnupg.org>
ecc: Require "eddsa" flag for curve Ed25519.

* src/cipher.h (PUBKEY_FLAG_ECDSA): Remove.
* cipher/pubkey-util.c (_gcry_pk_util_parse_flaglist): Remove "ecdsa".
* cipher/ecc.c (ecc_generate, ecc_sign, ecc_verify): Require "eddsa" flag.
* cipher/ecc-misc.c (_gcry_ecc_compute_public): Depend "eddsa" flag.
* tests/benchmark.c, tests/keygen.c, tests/pubkey.c
* tests/t-ed25519.c, tests/t-mpi-point.c: Adjust for changed flags.
--

This changes make using ECDSA signatures the default for all curves.
If another signing algorithm is to be used, the corresponding flag
needs to be given.  In particular the flags "eddsa" is now always
required with curve Ed25519 to comply with the specs.  This change
makes the code better readable by not assuming a certain signature
algorithm depending on the curve.

Signed-off-by: Werner Koch <wk@gnupg.org>
pubkey: Add forward compatibility feature.

* cipher/pubkey-util.c (_gcry_pk_util_parse_flaglist): Add
"igninvflag".
--

If future versions of Libgcrypt want to add optional flags to a pubkey
s-expression, they may use the "igninvflag" flag to make the flag
parser ignore flags it does not know about.

Signed-off-by: Werner Koch <wk@gnupg.org>
Fix 'u32' build error with Camellia

* cipher/camellia.c: Add include for <config.h> and "types.h".
(u32): Remove.
(u8): Typedef as 'byte'.
--

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Fix __builtin_bswap32/64 checks

* configure.ac (gcry_cv_have_builtin_bswap32)
(gcry_cv_have_builtin_bswap64): Change compile checks to link checks.
--

Patch changes compile checks to link checks for __builtin_bswap(32|64).
Compiling obviously works with missing functions, linking not so much.

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Optimize Blowfish weak key check

* cipher/blowfish.c (hashset_elem, val_to_hidx, add_val): New.
(do_bf_setkey): Use faster algorithm for detecting weak keys.
(bf_setkey): Move stack burning to do_bf_setkey.
--

Patch optimizes the weak key check for Blowfish. Instead of iterating through
sbox-tables for duplicates, insert values to hash-set and detect collisions.

Old check code was taking slightly longer time than the actual key setup of
Blowfish, which by itself is already quite slow.


After:

 $ tests/benchmark --cipher-with-keysetup --cipher-repetitions 10 cipher blowfish
 Running each test 10 times.
                 ECB/Stream         CBC             CFB             OFB             CTR             CCM
              --------------- --------------- --------------- --------------- --------------- ---------------
 BLOWFISH       410ms   440ms   430ms   370ms   440ms   370ms   430ms   440ms   370ms   370ms       -       -


Before:

 $ tests/benchmark --cipher-with-keysetup --cipher-repetitions 10 cipher blowfish
 Running each test 10 times.
                 ECB/Stream         CBC             CFB             OFB             CTR             CCM
              --------------- --------------- --------------- --------------- --------------- ---------------
 BLOWFISH       780ms   770ms   780ms   730ms   780ms   730ms   780ms   790ms   720ms   730ms       -       -


Without key-setup:

 $ tests/benchmark --cipher-repetitions 10 cipher blowfish
 Running each test 10 times.
                 ECB/Stream         CBC             CFB             OFB             CTR             CCM
              --------------- --------------- --------------- --------------- --------------- ---------------
 BLOWFISH        70ms    70ms    80ms    30ms    80ms    30ms    80ms    90ms    20ms    30ms       -       -


Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>
Tweak bench-slope parameters

* tests/bench-slope.c (BUF_STEP_SIZE): Half step size to 64.
(NUM_MEASUREMENT_REPETITIONS): Double repetitions to 64.
--

Tweak parameters for better repeatability of results with fast ciphers
(AES-NI).

Signed-off-by: Jussi Kivilinna <jussi.kivilinna@iki.fi>