Repository: randombit/botan (cpp)

Issue:
ECDSA verification compatibility issue due to applying different hash truncation
Botan uses a slightly different ECDSA input truncation mechanism than OpenSSL/LibreSSL/BoringSSL/wolfCrypt.

https://nvlpubs.nist.gov/nistpubs/fips/nist.fips.186-4.pdf says:

> When the length of the output of the hash function is greater than the bit length of n, then the leftmost n bits of the hash function output block shall be used in any calculation using the hash function output during the generation or verification of a digital signature.



This is what Botan does.

OpenSSL and the others do this: https://github.com/openssl/openssl/blob/5357c10624bedaeed984ef4ff370096911ee2ddf/crypto/ec/ecdsa_ossl.c#L245-L258

So I think Botan is actually right and the others are doing it slightly wrong.

In my fuzzer, I apply the same truncation mechanism before calling Botan's `verify_message`: https://github.com/guidovranken/cryptofuzz/blob/7f9f35e20fcd8886cbcbf167474dd7f9954e5c02/modules/botan/module.cpp#L726-L739

So for me it's not a problem, but in practice this can lead to interoperability issues (Botan rejecting valid signatures).

Here's an example of a valid signature that Botan rejects:

```
operation name: ECDSA_Verify
ecc curve: x962_p239v1
public key X: 115014969141122710336858256331515905188079709631033705848897690363377891
public key Y: 81333916963110019576228330948951168219884247801435258672405011123948094
cleartext: {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} (30 bytes)
signature R: 336870737920796383793526100561731463195539556177859437212670821204992455
signature S: 328556355548848640685841521416759180891403359973782869535609514452377183

Module Botan result:

false

Module OpenSSL result:

true
```

(numbers are in decimal, cleartext must be passed through SHA256 first)

Pull request description:
Correct hash to integer conversions for ECDSA
When the hash and group sizes differ sometimes our conversion was different from standard. Closes #2415

Patch:
diff --git a/src/lib/math/bigint/bigint.cpp b/src/lib/math/bigint/bigint.cpp
index ad9e5a0da..8510da1d3 100644
--- a/src/lib/math/bigint/bigint.cpp
+++ b/src/lib/math/bigint/bigint.cpp
@@ -89,14 +89,13 @@ BigInt::BigInt(const uint8_t input[], size_t length, Base base)
 
 BigInt::BigInt(const uint8_t buf[], size_t length, size_t max_bits)
    {
-   const size_t max_bytes = std::min(length, (max_bits + 7) / 8);
-   binary_decode(buf, max_bytes);
+   if(8 * length > max_bits)
+      length = (max_bits + 7) / 8;
 
-   const size_t b = this->bits();
-   if(b > max_bits)
-      {
-      *this >>= (b - max_bits);
-      }
+   binary_decode(buf, length);
+
+   if(8 * length > max_bits)
+      *this >>= (8 - (max_bits % 8));
    }
 
 /*


Test patch:
diff --git a/src/tests/data/pubkey/ecdsa_rfc6979.vec b/src/tests/data/pubkey/ecdsa_rfc6979.vec
index b423624c2..f51069e3e 100644
--- a/src/tests/data/pubkey/ecdsa_rfc6979.vec
+++ b/src/tests/data/pubkey/ecdsa_rfc6979.vec
@@ -314,16 +314,16 @@ Hash = SHA-1
 Signature = 00CC9CD383D3EA80ADC9165C25F6F82A2ABB41943E00A74AEB2B96B8F7D6FC03F67ED226FCE0BAA6D7EB
 
 Hash = SHA-224
-Signature = 00AB01FAF31D1044023725665A95CE1AEF6680CE6E00ECEE1D6FED07ACF9ED7F197D2C29022D1592E5DF
+Signature = 00FAAD1A32DAC1B1743557B4C0C032A7D5CACAB66500A497A8057EE60D5F9EDD1DEA01843A9E6A8829C6
 
 Hash = SHA-256
-Signature = 00766B26E5C3201982F5B3AB1AC92C018F5A990CB6008C16BF4AB8CF6EE3ED4A821B3A890CD30E2F6925
+Signature = 00432E2CD75D7C8899591A433BC729476914ACCED5007F3373E5D488B639D36358FA0B2DE8FD66216A58
 
 Hash = SHA-384
-Signature = 009AD138B36636B550705112D14574EF6E7996263F00F360D79AD85661B305D56D64A20DA33182E1462B
+Signature = 005830E92CA044CDE5E975BA9E1ABE5CE7A89724920076F5EFC6846E3327268790EDDAD561C372038511
 
 Hash = SHA-512
-Signature = 0024A0B0069722A4A3DDAF40B0EE66BAD0B117D658003A6DB6DEEF2EB563115FA48D7E0FECB540802E13
+Signature = 00EBFD570A4A24093B6545486F0EEC08F29B591D7700DE6D60783A0658319E1999BE88EA367960FF7DA8
 
 Msg = 74657374
 
@@ -331,16 +331,16 @@ Hash = SHA-1
 Signature = 007FF876220B4EAF1E1F9C9694685CDA37B1DF45CA00D2A7917831648FB5A3351C22ED605BA8B8B6C16C
 
 Hash = SHA-224
-Signature = 00A616A5A4C482D60F44900916D9665BFD090854FC003BD6C529D45F5F70BD6F242981E23A119EEAE23A
+Signature = 002FA83E175EB1BC72EEAAF4F14C8F679C6B4C657D00BC92F83742A2361A60294D236BEB88C28306D058
 
 Hash = SHA-256
-Signature = 007F281E95A74F5EF24B40D489AD270284E8E52E5D00F4803ACFD86356DC95E9AD39930C09050C6CCA4B
+Signature = 00E761394E3DEF2EE5E5A4BEE3E8C3EAD8D1610012006918F13D4C16C251117FF2FD8725F2E8E986A841
 
 Hash = SHA-384
-Signature = 00C114E2B133AD8E725AF43AEFC381BE87063DB3FC00CAFAEB1949015CF4EC9EC2476F3970D3D97E08FF
+Signature = 0096EE7BC69C24BEEFA8057BFFBFAB848ECC3CF470009F3E1A1D176775F91AD8E6B8A4EB184D1D865044
 
 Hash = SHA-512
-Signature = 0080F3C4742E7341BE2E4D86E88757AB96F62427B500FD634D64F554BEC04211952798147420BBC98E15
+Signature = 0022DF62B101659EBC01176C57922A2AD2F2C12E3B0005CD8ADEB71B68A50A98D37B16D22FD60CE95263
 
 # frp256v1 (generated by botan)
 
diff --git a/src/tests/data/pubkey/ecdsa_verify.vec b/src/tests/data/pubkey/ecdsa_verify.vec
index df3b2a41c..213cc60d6 100644
--- a/src/tests/data/pubkey/ecdsa_verify.vec
+++ b/src/tests/data/pubkey/ecdsa_verify.vec
@@ -56,9 +56,15 @@ Msg = 04FEFDFCFBFAF9F8F7F6F5F4F3F2F1F0EFEEEDECEBEAE9E8E7E6E5E4E3E2E1E0DFDEDDDCDB
 Signature = 264A27F175848F0A110B2E7B03886B94777FB5ECBE2E8CD674AE196A6C80D8FFC8B5C6C00EACB1C76A3BAD03F8FFD4B58784BF35E104E721FF8F3F81AC9E6E91
 
 # https://github.com/randombit/botan/issues/2424
-
 Group = secp256k1
 Px = 55066263022277343669578718895168534326250603453777594175500187360389116729240
 Py = 83121579216557378445487899878180864668798711284981320763518679672151497189239
 Msg = 1111111111111111111111111111111111111111111111111111111111111111
 Signature = f490078fed0f0ca058fba78375f0e42678b25242238a664d4ed6784f7c284ec77cf9b997a778f8fe101f480d3c75cde95c5d712935fa5b9a4edf8e6654379b9c
+
+# https://github.com/randombit/botan/issues/2415
+Group = x962_p239v1
+Px = 115014969141122710336858256331515905188079709631033705848897690363377891
+Py = 81333916963110019576228330948951168219884247801435258672405011123948094
+Msg = 0679246D6C4216DE0DAA08E5523FB2674DB2B6599C3B72FF946B488A15290B62
+Signature = 30cf3ae9da8c18ef37664e358e43b07f93ded599653e64acd171e197a1c72f9ad521e5e2e091e9fe4c27f1110265ec5cbb701a6faf3569304774de5f