Repository: borgbackup/borg

Issue:
New encryption scheme may use HKDF in a sub-optimal (and cryptographically unproven) way.
<!--
Thank you for reporting an issue.

*IMPORTANT* -  *before* creating a new issue please look around:
 - Borgbackup documentation: http://borgbackup.readthedocs.io/en/stable/index.html
 - FAQ: https://borgbackup.readthedocs.io/en/stable/faq.html
 and
 - open issues in GitHub tracker: https://github.com/borgbackup/borg/issues
  
If you cannot find a similar problem, then create a new issue.

Please fill in as much of the template as possible.
-->

## Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

## Is this a BUG / ISSUE report or a QUESTION?

Issue

## Describe the problem you're observing.

Looking at Borg's encryption design document and the discussion in Issue #3814, it seems that Borg may be using HKDF in a way that is unproven by cryptanalysis and makes sub-optimal use of the randomness introduced by `SessionId` by using it as the `salt` parameter of HKDF instead of feeding it into the `info` parameter (called `context` in the diagram).

In the blog post by @soatok entitled [Understanding HKDF](https://soatok.blog/2021/11/17/understanding-hkdf/), he discusses the formal proofs provided for HKDF, and how many developers accidentally misuse the algorithm in ways that invalidate those proofs. It appears that the algorithm was designed so that the same "(salt, IKM)" would be used across all iterations of HKDF for a given key being used in a given context (in this case, the master keying material in the context of a Borg archive), and the cryptographic proofs do not extend to "same IKM, varying salt". If different keying material is desired for the same IKM within the same context, it should be produced by varying the `info` parameter, not the `salt`.

In fact, according to [RFC 5869 Section 3.3](https://datatracker.ietf.org/doc/html/rfc5869#section-3.3), if the IKM is already strong and uniformly-random, the `HKDF-Extract` stage can just be skipped altogether (which is the only step that uses `salt`). Borg's IKM already meets that requirement, so it would be permissible (and still within the cryptographic proof for HKDF) for Borg to do away with the first stage altogether (including eliminating the `salt` parameter) and just directly use the 512-bit `crypt_key` in the `HKDF-Expand` stage.

So, if we assume that it's okay to get rid of the `HKDF-Extract` stage (which RFC 5869 supports), and since we're only extracting a single 256-bit output block from `HKDF-Expand`, the session key derivation becomes a single iteration of `HKDF-Expand` with `crypt_key` as the PRK and `session_id | flavor_string` as the `info`:

`session_key = HMAC(crypt_key, session_id | flavor_string | 0x01)`

(Technically, you need to have the `0x01` concatenated in there to comply with the proofs from the paper, although I'd wager that it doesn't add much in terms of security, since it's already being concatenated with a random number).





<!--

If this complaint relates to borg performance, please include CRUD benchmark
results and any steps you took to troubleshoot.
How to run benchmark: http://borgbackup.readthedocs.io/en/stable/usage/benchmark.html

*IMPORTANT* - Please mark logs and text output from terminal commands 
or else GitHub will not display them correctly. 
An example is provided below.

Example:
```
this is an example how log text should be marked (wrap it with ```)
```
-->



Discussion:
Thanks for looking at borg2's crypto!

Just to be a bit more specific about what we are discussing, this is (besides our test suite) currently the only usage of hkdf in master branch in `borg.crypto.key` module:
```
    def _get_session_key(self, sessionid):
        assert len(sessionid) == 24  # 192bit
        key = hkdf_hmac_sha512(
            ikm=self.crypt_key,
            salt=sessionid,
            info=b"borg-session-key-" + self.CIPHERSUITE.__name__.encode(),
            output_length=32,
        )
        return key
```
What do we have there?
- self.crypt_key - this is the static key material from the (encrypted) on-disk borg key, at least 512b of random.
- sessionid - 192b of random, initialised freshly for each borg invocation (= for each backup)

Right, and using a static key (`self.crypt_key`) with a variable salt (`sessionid`) is the exact antipattern for HKDF that @marti4d reported above.

| Key | Salt | Info | Notes |
|-----|------|-----|--------|
| Static | Static | Static | Deterministic key derivation |
| Static | Variable | Static | **Anti-pattern**, only PRF security, not KDF security -- Borg2 is here |
| Variable | Variable | Static | OK |
| Variable | Static | Static | OK |
| Static | Static | Variable | Multiple keys derived from input key |
| Static | Variable | Variable | **Anti-pattern**, only PRF security, not KDF security |
| Variable | Variable | Variable | OK |
| Variable | Static | Variable | OK |

If I read the rfc correct, then the session id should go to info instead of salt 
The RFC in confusing in the sense that it recommended the salt to be random and the info to be a context identifier,the whole thing is pretty confusing 
The Blog post is most instructive

As far as I understand borg with actual keys doesn't need hkdf as presented the input key is already matching the requirements


> The Blog post is most instructive
> 
> As far as I understand borg with actual keys doesn't need hkdf as presented the input key is already matching the requirements

Exactly -- HKDF was really designed for deriving keys after Diffie–Hellman exchange, where the IKM is not uniformly random.

Since Borg's master key is already a PseudoRandom Key (PRK), there is no need for the "HKDF-Extract" stage. Only the "HKDF-Expand" stage is needed, with the random `sessionid` (and the crypto suite "flavor") as the `info` parameter, and only a single iteration.

Therefore, if you follow RFC 5869 definition for the `HKDF-Expand` stage, it simplifies to:

```
Input:
PRK = self.crypt_key
info = sessionid | flavor_string
L = 32 bytes (`session_key` length)

Output:
OKM = Will be the desired `session_key`

Other:
flavor_string = b"borg-session-key-" + self.CIPHERSUITE.__name__.encode()
HashLen = 64 bytes (length of a SHA-512 hash)

The output OKM is calculated as follows:

N = ceil(L/HashLen) = ceil(32/64)  = 1
T = T(1)
OKM = first 32 octets of T

where:
T(0) = empty string (zero length)
T(1) = HMAC-Hash(PRK, T(0) | info | 0x01)

Now, let's simplify:

T = T(1) = HMAC-Hash(PRK, (empty string) | info | 0x01)

OKM = first 32 octets of T

**Finally, substituting our values**

OKM = truncate32(HMAC-SHA-512(self.crypt_key, sessionid | flavor_string | 0x01))

```

One last note -- I'm pretty sure we can all agree (and the design paper for HKDF seems to second this) that there's really no reason to have the `0x01` byte in there -- They added it as an iterator in case you want to derive more than a single block of keying material.

Since that's not the case here, that simplifies the equation down to:

```
OKM = truncate32(HMAC-SHA-512(self.crypt_key, sessionid | flavor_string))
```

Essentially, when you are using HKDF with an already pseudorandom IKM, and you only want to derive a single hash block (or less) from it, it basically simplifies down to a simple `truncate(HMAC(key, info), output_size)` call.

I think it makes sense for Borg's session id to be derived as follows (in Python):

```python
def _get_session_key(self, sessionid):
    assert len(sessionid) == 24  # 192bit
    key_64 = hmac_sha512(
        key=self.crypt_key,
        message=sessionid + b"borg-session-key-" + self.CIPHERSUITE.__name__.encode(),
    )
    return key_64[:32]
```


Perhaps @soatok would know more than me, but since Borg is using HKDF with input key material that is already pseudorandom, I guess technically what it's doing now is sub-optimal, but likely not broken. IE there likely isn't an actual security problem here (although that fact is more difficult to prove than it would be if Borg were just to use the HMAC construction I suggested above). Does that sound right?
From the perspective of "I'm an academic and I want to write a formal model or security proof", only having PRF security when KDF security is desired is a minor nuisance.

From the real world perspective, PRFs are pretty damn good.

Resolving patch:
diff --git a/src/borg/crypto/key.py b/src/borg/crypto/key.py
--- a/src/borg/crypto/key.py
+++ b/src/borg/crypto/key.py
@@ -27,7 +27,7 @@
 from ..repoobj import RepoObj
 
 
-from .low_level import AES, bytes_to_int, num_cipher_blocks, hmac_sha256, blake2b_256, hkdf_hmac_sha512
+from .low_level import AES, bytes_to_int, num_cipher_blocks, hmac_sha256, blake2b_256
 from .low_level import AES256_CTR_HMAC_SHA256, AES256_CTR_BLAKE2b, AES256_OCB, CHACHA20_POLY1305
 from . import low_level
 
@@ -833,7 +833,7 @@ def decrypt(self, id, data):
         # to decrypt existing data, we need to get a cipher configured for the sessionid and iv from header
         self.assert_type(data[0], id)
         iv_48bit = data[2:8]
-        sessionid = data[8:32]
+        sessionid = bytes(data[8:32])
         iv = int.from_bytes(iv_48bit, "big")
         cipher = self._get_cipher(sessionid, iv)
         try:
@@ -857,15 +857,22 @@ def init_from_random_data(self):
             chunk_seed = chunk_seed - 0xFFFFFFFF - 1
         self.init_from_given_data(crypt_key=data[0:64], id_key=data[64:96], chunk_seed=chunk_seed)
 
-    def _get_session_key(self, sessionid):
+    def _get_session_key(self, sessionid, domain=None):
+        """
+        Derive a session key from the secret long-term static crypt_key (which is a fully random PRK)
+        and the session id (which is fully random also).
+        Optionally, a domain can be given for domain separation (defaults to a different binary string
+        per cipher suite).
+        """
+        # Performance note:
+        # While this is only invoked once per session to generate a new key for encrypting new data, it is invoked
+        # frequently (per encrypted repo object) to compute the corresponding key for decrypting existing data.
         assert len(sessionid) == 24  # 192bit
-        key = hkdf_hmac_sha512(
-            ikm=self.crypt_key,
-            salt=sessionid,
-            info=b"borg-session-key-" + self.CIPHERSUITE.__name__.encode(),
-            output_length=32,
-        )
-        return key
+        if domain is None:
+            domain = b"borg-session-key-" + self.CIPHERSUITE.__name__.encode()
+        # Because crypt_key is already a PRK, we do not need KDF security here, PRF security is good enough.
+        # See https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-56Cr2.pdf section 4 "one-step KDF".
+        return sha256(self.crypt_key + sessionid + domain).digest()
 
     def _get_cipher(self, sessionid, iv):
         assert isinstance(iv, int)
diff --git a/src/borg/selftest.py b/src/borg/selftest.py
--- a/src/borg/selftest.py
+++ b/src/borg/selftest.py
@@ -33,7 +33,7 @@
     ChunkerTestCase,
 ]
 
-SELFTEST_COUNT = 38
+SELFTEST_COUNT = 33
 
 
 class SelfTestResult(TestResult):
diff --git a/src/borg/testsuite/crypto.py b/src/borg/testsuite/crypto.py
--- a/src/borg/testsuite/crypto.py
+++ b/src/borg/testsuite/crypto.py
@@ -7,7 +7,6 @@
 
 from ..crypto.low_level import AES256_CTR_HMAC_SHA256, AES256_OCB, CHACHA20_POLY1305, UNENCRYPTED, IntegrityError
 from ..crypto.low_level import bytes_to_long, bytes_to_int, long_to_bytes
-from ..crypto.low_level import hkdf_hmac_sha512
 from ..crypto.low_level import AES, hmac_sha256
 from ..crypto.key import CHPOKeyfileKey, AESOCBRepoKey, FlexiKey
 from ..helpers import msgpack
@@ -195,74 +194,6 @@ def test_AEAD_with_more_AAD(self):
             cs = cs_cls(key, iv_int, header_len=len(header), aad_offset=0)
             self.assert_raises(IntegrityError, lambda: cs.decrypt(hdr_mac_iv_cdata, aad=b"incorrect_chunkid"))
 
-    # These test vectors come from https://www.kullo.net/blog/hkdf-sha-512-test-vectors/
-    # who claims to have verified these against independent Python and C++ implementations.
-
-    def test_hkdf_hmac_sha512(self):
-        ikm = b"\x0b" * 22
-        salt = bytes.fromhex("000102030405060708090a0b0c")
-        info = bytes.fromhex("f0f1f2f3f4f5f6f7f8f9")
-        length = 42
-
-        okm = hkdf_hmac_sha512(ikm, salt, info, length)
-        assert okm == bytes.fromhex(
-            "832390086cda71fb47625bb5ceb168e4c8e26a1a16ed34d9fc7fe92c1481579338da362cb8d9f925d7cb"
-        )
-
-    def test_hkdf_hmac_sha512_2(self):
-        ikm = bytes.fromhex(
-            "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f2021222324252627"
-            "28292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f"
-        )
-        salt = bytes.fromhex(
-            "606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868"
-            "788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeaf"
-        )
-        info = bytes.fromhex(
-            "b0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7"
-            "d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff"
-        )
-        length = 82
-
-        okm = hkdf_hmac_sha512(ikm, salt, info, length)
-        assert okm == bytes.fromhex(
-            "ce6c97192805b346e6161e821ed165673b84f400a2b514b2fe23d84cd189ddf1b695b48cbd1c838844"
-            "1137b3ce28f16aa64ba33ba466b24df6cfcb021ecff235f6a2056ce3af1de44d572097a8505d9e7a93"
-        )
-
-    def test_hkdf_hmac_sha512_3(self):
-        ikm = bytes.fromhex("0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b")
-        salt = None
-        info = b""
-        length = 42
-
-        okm = hkdf_hmac_sha512(ikm, salt, info, length)
-        assert okm == bytes.fromhex(
-            "f5fa02b18298a72a8c23898a8703472c6eb179dc204c03425c970e3b164bf90fff22d04836d0e2343bac"
-        )
-
-    def test_hkdf_hmac_sha512_4(self):
-        ikm = bytes.fromhex("0b0b0b0b0b0b0b0b0b0b0b")
-        salt = bytes.fromhex("000102030405060708090a0b0c")
-        info = bytes.fromhex("f0f1f2f3f4f5f6f7f8f9")
-        length = 42
-
-        okm = hkdf_hmac_sha512(ikm, salt, info, length)
-        assert okm == bytes.fromhex(
-            "7413e8997e020610fbf6823f2ce14bff01875db1ca55f68cfcf3954dc8aff53559bd5e3028b080f7c068"
-        )
-
-    def test_hkdf_hmac_sha512_5(self):
-        ikm = bytes.fromhex("0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c0c")
-        salt = None
-        info = b""
-        length = 42
-
-        okm = hkdf_hmac_sha512(ikm, salt, info, length)
-        assert okm == bytes.fromhex(
-            "1407d46013d98bc6decefcfee55f0f90b0c7f63d68eb1a80eaf07e953cfc0a3a5240a155d6e4daa965bb"
-        )
-
 
 def test_decrypt_key_file_argon2_chacha20_poly1305():
     plain = b"hello"


Test patch: