Repository: borgbackup/borg

Issue:
chunkid == id_hash(plaintext) - better always check?
Just a brain dump, so I don't forget, needs checking in the source.

## borg < 2.0

After reading a chunk from the repo, it calls `assert_id` to check whether `wanted_chunkid == H(plaintext_content_we_got_from_repo)`.

This is rather expensive on the CPU as all data is going through hmac-sha256 or blake2b, but back then this was the only way to make sure we really got the right data for the chunkid we wanted. Without doing that check, the repo could cheat and just give us data for another chunk (which would be also encrypted and authenticated, but the client would not know that it is the data for another chunkid).

## borg2 (master branch)

Currently omits the calls to `assert_id` because it does the crypto a bit differently (AEAD):
- when the client writes data to the repo, it puts the chunkid into the "AD" (additional data) part of AEAD. That part is not encrypted, but it is authenticated ("signed").
- When reading a chunk from the repo, the AEAD crypto will authenticate all data (including the chunkid) and if successful, will decrypt the encrypted part also. So, if we can successfully aead-auth-decrypt, we know that the chunkid is exactly as it was written by the client and that the data is also as written by the client. 
- if we assume the client was **well-behaved**, we do not need to call `assert_id`, because we can just check if the chunkid in the AD part is the same as the chunkid we wanted. This is good for performance.
- the repo can not cheat on the client by just returning a different chunk: it would be very obvious as it would have a different chunkid in the AD part.
- but, if we can not rely on that a well-behaved client has written the data correctly, I guess we still would need to call `assert_id`. E.g. there could be malware on the client at backup time that uses modified, evil borg code that writes a wrong chunkid into the AD part when writing data to the repo (to the same, wrong chunkid key). In that case, the AEAD would work as usual, but it would sign chunkid and data without `chunkid == H(plaintext)` being true.
- a well-behaved borg client could detect such a evil borg chunk when calling `assert_id`. but that requires that a well-behaved borg client is used (and not the same "evil borg" that made the bad chunks).

Discuss: how do we deal with that in borg2?

- always call `assert_id` and spend the additional cycles?
- only do it in some *paranoid mode* and otherwise omit it?




Discussion:
Sounds like something for borg check, not for regular operations.
> Sounds like something for borg check, not for regular operations.

This only applies when you have access to the plaintext - so decryption happens anyway - and then you can omit the additional hash by putting the chunk ID in the AAD. So, it is a performance win for borg check but also other operations which decrypt like restoring.
@horazont borg check repo part does not decrypt, so can't do it. borg check archives part goes over the chunks archive-by-archive and thus tends to be not very efficient for "all chunks once" operations.

@taqiohr as I said in top post, we have the chunkid in AAD now. But that does not defend against a evil borg client that puts a wrong chunkid into the AAD.
Ah I see, I wasn't aware that the archive check may read chunks multiple times.

Resolving patch:
diff --git a/src/borg/archive.py b/src/borg/archive.py
--- a/src/borg/archive.py
+++ b/src/borg/archive.py
@@ -1837,12 +1837,6 @@ def verify_data(self):
         chunks_count_index = len(self.chunks)
         chunks_count_segments = 0
         errors = 0
-        # for the new crypto, derived from AEADKeyBase, we know that it checks authenticity on
-        # the crypto.low_level level - invalid chunks will fail to AEAD authenticate.
-        # for these key types, we know that there is no need to decompress the data afterwards.
-        # for all other modes, we assume that we must decompress, so we can verify authenticity
-        # based on the plaintext MAC (via calling ._assert_id(id, plaintext)).
-        decompress = not isinstance(self.key, AEADKeyBase)
         defect_chunks = []
         pi = ProgressIndicatorPercent(
             total=chunks_count_index, msg="Verifying data %6.2f%%", step=0.01, msgid="check.verify_data"
@@ -1872,7 +1866,8 @@ def verify_data(self):
                         chunk_data_iter = self.repository.get_many(chunk_ids)
                 else:
                     try:
-                        self.repo_objs.parse(chunk_id, encrypted_data, decompress=decompress)
+                        # we must decompress, so it'll call assert_id() in there:
+                        self.repo_objs.parse(chunk_id, encrypted_data, decompress=True)
                     except IntegrityErrorBase as integrity_error:
                         self.error_found = True
                         errors += 1
@@ -1903,7 +1898,8 @@ def verify_data(self):
                     # from the underlying media.
                     try:
                         encrypted_data = self.repository.get(defect_chunk)
-                        self.repo_objs.parse(defect_chunk, encrypted_data, decompress=decompress)
+                        # we must decompress, so it'll call assert_id() in there:
+                        self.repo_objs.parse(defect_chunk, encrypted_data, decompress=True)
                     except IntegrityErrorBase:
                         # failed twice -> get rid of this chunk
                         del self.chunks[defect_chunk]
diff --git a/src/borg/crypto/key.py b/src/borg/crypto/key.py
--- a/src/borg/crypto/key.py
+++ b/src/borg/crypto/key.py
@@ -840,11 +840,23 @@ class AEADKeyBase(KeyBase):
     MAX_IV = 2**48 - 1
 
     def assert_id(self, id, data):
-        # note: assert_id(id, data) is not needed any more for the new AEAD crypto.
-        # we put the id into AAD when storing the chunk, so it gets into the authentication tag computation.
+        # Comparing the id hash here would not be needed any more for the new AEAD crypto **IF** we
+        # could be sure that chunks were created by normal (not tampered, not evil) borg code:
+        # We put the id into AAD when storing the chunk, so it gets into the authentication tag computation.
         # when decrypting, we provide the id we **want** as AAD for the auth tag verification, so
         # decrypting only succeeds if we got the ciphertext we wrote **for that chunk id**.
-        pass
+        # So, basically the **repository** can not cheat on us by giving us a different chunk.
+        #
+        # **BUT**, if chunks are created by tampered, evil borg code, the borg client code could put
+        # a wrong chunkid into AAD and then AEAD-encrypt-and-auth this and store it into the
+        # repository using this bad chunkid as key (violating the usual chunkid == id_hash(data)).
+        # Later, when reading such a bad chunk, AEAD-auth-and-decrypt would not notice any
+        # issue and decrypt successfully.
+        # Thus, to notice such evil borg activity, we must check for such violations here:
+        if id and id != Manifest.MANIFEST_ID:
+            id_computed = self.id_hash(data)
+            if not hmac.compare_digest(id_computed, id):
+                raise IntegrityError("Chunk %s: id verification failed" % bin_to_hex(id))
 
     def encrypt(self, id, data):
         # to encrypt new data in this session we use always self.cipher and self.sessionid


Test patch: