Repository: borgbackup/borg

Issue:
borg recreate --exclude triggers rechunking
I have a rather large, remote backup repository (4TB), and wanted to remove directories from old archives to free up some space.

I ran borg recreate -e "directory/to/remove/*" and a dry run seemed to do what I want after a few minutes. Now when I run the actual command, it didn't return for hours. nethogs shows that a lot of data is downloaded, so I assume that for some reason, the whole archive is downloaded. I assumed, since I only remove some files, the actual contents of the archive is not needed locally.

I didn't specify --recompress, and neither did I add --chunker-params. However, I do have different chunker params than the borg default when I create archives, but I assumed from the manpage that recreate is not re-chunking if no chunker params are specified.

I use borg 1.2.3.



Discussion:
Please give the complete borg command you used (you can edit paths if they are private, but keep semantics the same).

Indeed, I'ld also expect that not to download a lot of data. Can you reproduce this?

Oh, guess I need to check WHAT chunker params this will default to if one does not explicitly give them. Maybe it defaults to same value as `borg create` and thus would trigger rechunking in your case although you did not intend to do that.

You could try giving the same chunker params explicitly to `borg recreate` as you use with `borg create`.
This is the command

`borg recreate ssh://my.server/home/my.borg::myarchive -pse */var/lib/libvirt/* --comment novm --lock-wait 5
`
The manpage states that adding chunker params will cause re-chunking, that is why I didn't: --chunker-params will re-chunk all files in the archive
Maybe the help text / man page text is only true if one usually uses the default chunker params.

Resolving patch:
diff --git a/src/borg/archiver/recreate_cmd.py b/src/borg/archiver/recreate_cmd.py
--- a/src/borg/archiver/recreate_cmd.py
+++ b/src/borg/archiver/recreate_cmd.py
@@ -219,10 +219,10 @@ def build_parser_recreate(self, subparsers, common_parser, mid_common_parser):
             dest="chunker_params",
             action=Highlander,
             type=ChunkerParams,
-            default=CHUNKER_PARAMS,
-            help="specify the chunker parameters (ALGO, CHUNK_MIN_EXP, CHUNK_MAX_EXP, "
-            "HASH_MASK_BITS, HASH_WINDOW_SIZE) or `default` to use the current defaults. "
-            "default: %s,%d,%d,%d,%d" % CHUNKER_PARAMS,
+            default=None,
+            help="rechunk using given chunker parameters (ALGO, CHUNK_MIN_EXP, CHUNK_MAX_EXP, "
+            "HASH_MASK_BITS, HASH_WINDOW_SIZE) or `default` to use the chunker defaults. "
+            "default: do not rechunk",
         )
 
         subparser.add_argument(
diff --git a/src/borg/testsuite/archiver/recreate_cmd.py b/src/borg/testsuite/archiver/recreate_cmd.py
--- a/src/borg/testsuite/archiver/recreate_cmd.py
+++ b/src/borg/testsuite/archiver/recreate_cmd.py
@@ -171,6 +171,33 @@ def test_recreate_fixed_rechunkify(self):
         num_chunks = int(output)
         assert num_chunks == 2
 
+    def test_recreate_no_rechunkify(self):
+        with open(os.path.join(self.input_path, "file"), "wb") as fd:
+            fd.write(b"a" * 8192)
+        self.cmd(f"--repo={self.repository_location}", "rcreate", RK_ENCRYPTION)
+        # first create an archive with non-default chunker params:
+        self.cmd(f"--repo={self.repository_location}", "create", "test", "input", "--chunker-params", "7,9,8,128")
+        output = self.cmd(
+            f"--repo={self.repository_location}", "list", "test", "input/file", "--format", "{num_chunks}"
+        )
+        num_chunks = int(output)
+        # now recreate the archive and do NOT specify chunker params:
+        output = self.cmd(
+            f"--repo={self.repository_location}",
+            "recreate",
+            "--debug",
+            "--exclude",
+            "filename_never_matches",
+            "-a",
+            "test",
+        )
+        assert "Rechunking" not in output  # we did not give --chunker-params, so it must not rechunk!
+        output = self.cmd(
+            f"--repo={self.repository_location}", "list", "test", "input/file", "--format", "{num_chunks}"
+        )
+        num_chunks_after_recreate = int(output)
+        assert num_chunks == num_chunks_after_recreate
+
     def test_recreate_recompress(self):
         self.create_regular_file("compressible", size=10000)
         self.cmd(f"--repo={self.repository_location}", "rcreate", RK_ENCRYPTION)


Test patch: