internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: borgbackup/borg

Issue:
recreate without --chunker-params shall not rechunk (1.2)



Discussion:

Resolving patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -4652,10 +4652,10 @@ def define_borg_mount(parser):
                                         'If no MODE is given, `if-different` will be used. '
                                         'Not passing --recompress is equivalent to "--recompress never".')
         archive_group.add_argument('--chunker-params', metavar='PARAMS', dest='chunker_params', action=Highlander,
-                                   type=ChunkerParams, default=CHUNKER_PARAMS,
-                                   help='specify the chunker parameters (ALGO, CHUNK_MIN_EXP, CHUNK_MAX_EXP, '
-                                        'HASH_MASK_BITS, HASH_WINDOW_SIZE) or `default` to use the current defaults. '
-                                        'default: %s,%d,%d,%d,%d' % CHUNKER_PARAMS)
+                                   type=ChunkerParams, default=None,
+                                   help='rechunk using given chunker parameters (ALGO, CHUNK_MIN_EXP, CHUNK_MAX_EXP, '
+                                        'HASH_MASK_BITS, HASH_WINDOW_SIZE) or `default` to use the chunker defaults. '
+                                        'default: do not rechunk')
 
         subparser.add_argument('location', metavar='REPOSITORY_OR_ARCHIVE', nargs='?', default='',
                                type=location_validator(),
diff --git a/src/borg/testsuite/archiver.py b/src/borg/testsuite/archiver.py
--- a/src/borg/testsuite/archiver.py
+++ b/src/borg/testsuite/archiver.py
@@ -3080,6 +3080,23 @@ def test_recreate_fixed_rechunkify(self):
         num_chunks = int(output)
         assert num_chunks == 2
 
+    def test_recreate_no_rechunkify(self):
+        with open(os.path.join(self.input_path, 'file'), 'wb') as fd:
+            fd.write(b'a' * 8192)
+        self.cmd('init', '--encryption=repokey', self.repository_location)
+        # first create an archive with non-default chunker params:
+        self.cmd('create', '--chunker-params', '7,9,8,128', self.repository_location + '::test', 'input')
+        output = self.cmd('list', self.repository_location + '::test', 'input/file',
+                          '--format', '{num_chunks}')
+        num_chunks = int(output)
+        # now recreate the archive and do NOT specify chunker params:
+        output = self.cmd('recreate', '--debug', '--exclude', 'filename_never_matches', self.repository_location + '::test')
+        assert 'Rechunking' not in output  # we did not give --chunker-params, so it must not rechunk!
+        output = self.cmd('list', self.repository_location + '::test', 'input/file',
+                          '--format', '{num_chunks}')
+        num_chunks_after_recreate = int(output)
+        assert num_chunks == num_chunks_after_recreate
+
     def test_recreate_recompress(self):
         self.create_regular_file('compressible', size=10000)
         self.cmd('init', '--encryption=repokey', self.repository_location)


Test patch:
Repository: borgbackup/borg

Issue:
debug log all cache / index full paths
Considering issues like #5954 are bad to solve / debug without this information, borg should debug log full paths of all caches / indexes it uses.


Discussion:
Can I work on this issue if it's still open?
@Rsakhuja sure, just take it! :-)
@Rsakhuja are you still working upon this issue
@sumeetdeveloper if there is no response, i think we can just assume they are not working on this.
Shall I start working upon this issue then? if it is open?
Guess so, it's been a while since October. Thanks for helping!
@ThomasWaldmann can you help me to reproduce this issue or point any documentation or stuff
@sumeetdeveloper Sure.

To start, just do a global search for cache_dir in the source.
@ThomasWaldmann  How would you like this to be implemented? Should there be another command that logs the cache dir by using the `get_cache_dir()` function in `fs.py`? I need a little more context, thanks!
@diivi read how the ticket linked from top post ended.

quite some work there could have been avoided if it were clear that it used a different cache dir than the user thought it did. maybe the user would have found the issue themselves, if --debug had output the full cache file path.

Resolving patch:
diff --git a/src/borg/cache.py b/src/borg/cache.py
--- a/src/borg/cache.py
+++ b/src/borg/cache.py
@@ -262,6 +262,7 @@ class CacheConfig:
     def __init__(self, repository, path=None, lock_wait=None):
         self.repository = repository
         self.path = cache_dir(repository, path)
+        logger.debug("Using %s as cache", self.path)
         self.config_path = os.path.join(self.path, "config")
         self.lock = None
         self.lock_wait = lock_wait


Test patch:
Repository: borgbackup/borg

Issue:
test_create_read_special_symlink: fails with BlockingIOError: [Errno 35] Resource temporarily unavailable
borg 2.0.0 pre-b5: the `read` in the `finally` block fails on freebsd 13.

https://github.com/borgbackup/borg/blob/b04b4438bf2905cc95e4aec2caa8487414b47b0a/src/borg/testsuite/archiver/create_cmd.py#L800

Introduced by:

https://github.com/borgbackup/borg/commit/8ce3d223585ad68171e51839403c1fa3cb625d87

See also:

https://docs.python.org/3/library/exceptions.html#BlockingIOError


Discussion:
@pgerber I could add a `except BlockingIOError: pass` there.

Considering this is just a cleanup action, guess we can do it like that and just ignore that exception on freebsd?

Resolving patch:
diff --git a/src/borg/testsuite/archiver/create_cmd.py b/src/borg/testsuite/archiver/create_cmd.py
--- a/src/borg/testsuite/archiver/create_cmd.py
+++ b/src/borg/testsuite/archiver/create_cmd.py
@@ -13,7 +13,7 @@
 from ... import platform
 from ...constants import *  # NOQA
 from ...manifest import Manifest
-from ...platform import is_cygwin, is_win32
+from ...platform import is_cygwin, is_win32, is_darwin
 from ...repository import Repository
 from .. import has_lchflags
 from .. import changedir
@@ -713,7 +713,7 @@ def to_dict(borg_create_output):
         assert result["Modified files"] == 0
         # Archive a dir with two added files
         self.create_regular_file("testfile1", contents=b"test1")
-        time.sleep(0.01)  # testfile2 must have newer timestamps than testfile1
+        time.sleep(1.0 if is_darwin else 0.01)  # testfile2 must have newer timestamps than testfile1
         self.create_regular_file("testfile2", contents=b"test2")
         result = self.cmd(f"--repo={self.repository_location}", "create", "--stats", "test_archive2", self.input_path)
         result = to_dict(result)
@@ -798,6 +798,9 @@ def fifo_feeder(fifo_fn, data):
             fd = os.open(fifo_fn, os.O_RDONLY | os.O_NONBLOCK)
             try:
                 os.read(fd, len(data))
+            except OSError:
+                # fails on FreeBSD 13 with BlockingIOError
+                pass
             finally:
                 os.close(fd)
             t.join()


Test patch:
Repository: borgbackup/borg

Issue:
improve mount docs
>> borg mount -o versions
>>
>> Looks to be a (currently) undocumented function.
>
> https://borgbackup.readthedocs.io/en/stable/usage/mount.html
>
> See below "Additional mount options supported by borg".
>
These keywords should be printed with the other font, monospaced. 



Discussion:

Resolving patch:
diff --git a/src/borg/archiver/mount_cmds.py b/src/borg/archiver/mount_cmds.py
--- a/src/borg/archiver/mount_cmds.py
+++ b/src/borg/archiver/mount_cmds.py
@@ -89,14 +89,14 @@ def build_parser_mount_umount(self, subparsers, common_parser, mid_common_parser
 
         Additional mount options supported by borg:
 
-        - versions: when used with a repository mount, this gives a merged, versioned
+        - ``versions``: when used with a repository mount, this gives a merged, versioned
           view of the files in the archives. EXPERIMENTAL, layout may change in future.
-        - allow_damaged_files: by default damaged files (where missing chunks were
-          replaced with runs of zeros by borg check ``--repair``) are not readable and
+        - ``allow_damaged_files``: by default damaged files (where missing chunks were
+          replaced with runs of zeros by ``borg check --repair``) are not readable and
           return EIO (I/O error). Set this option to read such files.
-        - ignore_permissions: for security reasons the "default_permissions" mount
-          option is internally enforced by borg. "ignore_permissions" can be given to
-          not enforce "default_permissions".
+        - ``ignore_permissions``: for security reasons the ``default_permissions`` mount
+          option is internally enforced by borg. ``ignore_permissions`` can be given to
+          not enforce ``default_permissions``.
 
         The BORG_MOUNT_DATA_CACHE_ENTRIES environment variable is meant for advanced users
         to tweak the performance. It sets the number of cached data chunks; additional


Test patch:
Repository: borgbackup/borg

Issue:
improve mount docs
>> borg mount -o versions
>>
>> Looks to be a (currently) undocumented function.
>
> https://borgbackup.readthedocs.io/en/stable/usage/mount.html
>
> See below "Additional mount options supported by borg".
>
These keywords should be printed with the other font, monospaced. 



Discussion:

Resolving patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -3044,14 +3044,14 @@ def define_borg_mount(parser):
 
         Additional mount options supported by borg:
 
-        - versions: when used with a repository mount, this gives a merged, versioned
+        - ``versions``: when used with a repository mount, this gives a merged, versioned
           view of the files in the archives. EXPERIMENTAL, layout may change in future.
-        - allow_damaged_files: by default damaged files (where missing chunks were
-          replaced with runs of zeros by borg check ``--repair``) are not readable and
+        - ``allow_damaged_files``: by default damaged files (where missing chunks were
+          replaced with runs of zeros by ``borg check --repair``) are not readable and
           return EIO (I/O error). Set this option to read such files.
-        - ignore_permissions: for security reasons the "default_permissions" mount
-          option is internally enforced by borg. "ignore_permissions" can be given to
-          not enforce "default_permissions".
+        - ``ignore_permissions``: for security reasons the ``default_permissions`` mount
+          option is internally enforced by borg. ``ignore_permissions`` can be given to
+          not enforce ``default_permissions``.
 
         The BORG_MOUNT_DATA_CACHE_ENTRIES environment variable is meant for advanced users
         to tweak the performance. It sets the number of cached data chunks; additional


Test patch:
Repository: borgbackup/borg

Issue:
Add --format option to `borg diff`, resolve issue #4634
!!! warning
    this pull request may cause conflict with #7414 

- [x] add `--format` in `diff_cmd.py`
- [x] rewrite `ItemDIff` in `item.pyx`
- [x] keep `--content-only` and `--json-lines`
- [x] rewrite part of `BaseFormatter` and subclass
- [x] write `DiffFormatter`
- [x] use `DiffFormatter` in `diff_cmd`  
- [x] test



Discussion:

Resolving patch:
diff --git a/src/borg/archiver/delete_cmd.py b/src/borg/archiver/delete_cmd.py
--- a/src/borg/archiver/delete_cmd.py
+++ b/src/borg/archiver/delete_cmd.py
@@ -142,12 +142,6 @@ def build_parser_delete(self, subparsers, common_parser, mid_common_parser):
         subparser.add_argument(
             "-s", "--stats", dest="stats", action="store_true", help="print statistics for the deleted archive"
         )
-        subparser.add_argument(
-            "--cache-only",
-            dest="cache_only",
-            action="store_true",
-            help="delete only the local cache for the given repository",
-        )
         subparser.add_argument(
             "--force",
             dest="forced",


Test patch:
Repository: borgbackup/borg

Issue:
mode bug in tests (777 instead of 0o777)
testsuite.archiver:4531 (1.2-maint)


Discussion:
@Michael-Girma ^^ (please also check globally for same bug)
*bsd was so nice to tell that there is a problem: inappropriate file type or format (errno 79)
macOS:
- the ctime assertion in 4537 fails (even after fixing the mode)
- the mtime assertion in 4529 sometimes fails (guess hfs has 1s timestamp granularity)
macOS fixes needed:
```
$ git diff
diff --git a/src/borg/testsuite/archiver.py b/src/borg/testsuite/archiver.py
index 7eb5669b..6a05c1d2 100644
--- a/src/borg/testsuite/archiver.py
+++ b/src/borg/testsuite/archiver.py
@@ -4523,12 +4523,16 @@ def test_time_diffs(self):
         if is_win32:
             # Sleeping for 15s because Windows doesn't refresh ctime if file is deleted and recreated within 15 seconds.
             time.sleep(15)
+        elif is_darwin:
+            time.sleep(1)  # HFS has a 1s timestamp granularity
         self.create_regular_file("test_file", size=15)
         self.cmd('create', self.repository_location + '::archive2', 'input')
         output = self.cmd("diff", self.repository_location + "::archive1", "archive2")
         self.assert_in("mtime", output)
         self.assert_in("ctime", output)  # Should show up on windows as well since it is a new file.
-        os.chmod("input/test_file", 777)
+        if is_darwin:
+            time.sleep(1)  # HFS has a 1s timestamp granularity
+        os.chmod("input/test_file", 0o777)
         self.cmd('create', self.repository_location + '::archive3', 'input')
         output = self.cmd("diff", self.repository_location + "::archive2", "archive3")
         self.assert_not_in("mtime", output)
```
@Michael-Girma can you make a PR against 1.2-maint first?
@Michael-Girma can you work on that right now or shall I quickly fix it? Need it for 1.2.4...
Okay, I'll take a look right now.

Resolving patch:
diff --git a/src/borg/testsuite/archiver.py b/src/borg/testsuite/archiver.py
--- a/src/borg/testsuite/archiver.py
+++ b/src/borg/testsuite/archiver.py
@@ -4523,12 +4523,16 @@ def test_time_diffs(self):
         if is_win32:
             # Sleeping for 15s because Windows doesn't refresh ctime if file is deleted and recreated within 15 seconds.
             time.sleep(15)
+        elif is_darwin:
+            time.sleep(1)  # HFS has a 1s timestamp granularity
         self.create_regular_file("test_file", size=15)
         self.cmd('create', self.repository_location + '::archive2', 'input')
         output = self.cmd("diff", self.repository_location + "::archive1", "archive2")
         self.assert_in("mtime", output)
         self.assert_in("ctime", output)  # Should show up on windows as well since it is a new file.
-        os.chmod("input/test_file", 777)
+        if is_darwin:
+            time.sleep(1)  # HFS has a 1s timestamp granularity
+        os.chmod("input/test_file", 0o777)
         self.cmd('create', self.repository_location + '::archive3', 'input')
         output = self.cmd("diff", self.repository_location + "::archive2", "archive3")
         self.assert_not_in("mtime", output)


Test patch:
Repository: borgbackup/borg

Issue:
micro opt potential, also clarity
```
   def build_filter(matcher, peek_and_store_hardlink_masters, strip_components):
        if strip_components:
            def item_filter(item):
                matched = matcher.match(item.path) and os.sep.join(item.path.split(os.sep)[strip_components:])
                peek_and_store_hardlink_masters(item, matched)
                return matched
        else:
            def item_filter(item):
                matched = matcher.match(item.path)
                peek_and_store_hardlink_masters(item, matched)
                return matched
        return item_filter
```
the first matched expression looks like we want a path in matched, but when looking at the second and also at the name "matched", it seems we are rather just interested in a boolean.

then this could be replaced:
```
os.sep.join(item.path.split(os.sep)[strip_components:])
|
v
len(item.path.split(os.sep)) > strip_components
```


Discussion:
See #3406 (not merged, closed as there was no activity on it).
Here's the diff from the PR in case it goes away:
```
     def build_filter(matcher, peek_and_store_hardlink_masters, strip_components):
         if strip_components:
             def item_filter(item):
-                matched = matcher.match(item.path) and os.sep.join(item.path.split(os.sep)[strip_components:])
+                matched = matcher.match(item.path) and len(item.path.split(os.sep)) > strip_components
                 peek_and_store_hardlink_masters(item, matched)
                 return matched
         else:
```

Resolving patch:
diff --git a/src/borg/archiver/_common.py b/src/borg/archiver/_common.py
--- a/src/borg/archiver/_common.py
+++ b/src/borg/archiver/_common.py
@@ -591,7 +591,7 @@ def build_filter(matcher, strip_components):
     if strip_components:
 
         def item_filter(item):
-            matched = matcher.match(item.path) and os.sep.join(item.path.split(os.sep)[strip_components:])
+            matched = matcher.match(item.path) and len(item.path.split(os.sep)) > strip_components
             return matched
 
     else:
diff --git a/src/borg/testsuite/archiver/patterns.py b/src/borg/testsuite/archiver/patterns.py
--- a/src/borg/testsuite/archiver/patterns.py
+++ b/src/borg/testsuite/archiver/patterns.py
@@ -22,6 +22,5 @@ def test_strip_components(self):
         matcher = PatternMatcher(fallback=True)
         filter = build_filter(matcher, strip_components=1)
         assert not filter(Item(path="shallow"))
-        assert not filter(Item(path="shallow/"))  # can this even happen? paths are normalized...
         assert filter(Item(path="deep enough/file"))
         assert filter(Item(path="something/dir/file"))


Test patch:
Repository: borgbackup/borg

Issue:
highlander options "special effects"
borg has some cli options with `action=Highlander`, which raises an error if there is more than one (see also #6269).

E.g. it does not make sense to give `--umask` more than once (especially not with different values).

But, this error is not raised, if the first of these options happens to use the same value as the default value of the option.

E.g.:
```
borg --umask 077 --umask 007 rlist --repo ...  # no error (0o077 is the default)
borg --umask 007 --umask 077 rlist --repo ...  # error "there can be only one"
borg --umask 777 --umask 007 rlist --repo ...  # error "there can be only one"
borg --umask 007 --umask 777 rlist --repo ...  # error "there can be only one"
```


Discussion:
I'm working on this

Resolving patch:
diff --git a/src/borg/archiver/_common.py b/src/borg/archiver/_common.py
--- a/src/borg/archiver/_common.py
+++ b/src/borg/archiver/_common.py
@@ -249,9 +249,14 @@ def wrapper(self, args, repository, manifest, **kwargs):
 class Highlander(argparse.Action):
     """make sure some option is only given once"""
 
+    def __init__(self, *args, **kwargs):
+        self.__called = False
+        super().__init__(*args, **kwargs)
+
     def __call__(self, parser, namespace, values, option_string=None):
-        if getattr(namespace, self.dest, None) != self.default:
+        if self.__called:
             raise argparse.ArgumentError(self, "There can be only one.")
+        self.__called = True
         setattr(namespace, self.dest, values)
 
 
diff --git a/src/borg/testsuite/archiver/argparsing.py b/src/borg/testsuite/archiver/argparsing.py
--- a/src/borg/testsuite/archiver/argparsing.py
+++ b/src/borg/testsuite/archiver/argparsing.py
@@ -11,6 +11,23 @@ def test_bad_filters(self):
         self.cmd(f"--repo={self.repository_location}", "create", "test", "input")
         self.cmd(f"--repo={self.repository_location}", "delete", "--first", "1", "--last", "1", fork=True, exit_code=2)
 
+    def test_highlander(self):
+        self.cmd(f"--repo={self.repository_location}", "rcreate", RK_ENCRYPTION)
+        self.cmd(f"--repo={self.repository_location}", "create", "--comment", "comment 1", "test-1", __file__)
+        error_msg = "There can be only one"
+        # Default umask value is 0077
+        # Test that it works with a one time specified default or custom value
+        output_default = self.cmd(f"--repo={self.repository_location}", "--umask", "0077", "rlist")
+        assert error_msg not in output_default
+        output_custom = self.cmd(f"--repo={self.repository_location}", "--umask", "0007", "rlist")
+        assert error_msg not in output_custom
+        # Test that all combinations of custom and default values fail
+        for first, second in [("0007", "0007"), ("0007", "0077"), ("0077", "0007"), ("0077", "0077")]:
+            output_custom = self.cmd(
+                f"--repo={self.repository_location}", "--umask", first, "--umask", second, "rlist", exit_code=2
+            )
+            assert error_msg in output_custom
+
 
 def test_get_args():
     archiver = Archiver()


Test patch:
Repository: borgbackup/borg

Issue:
fix freebsd13 vagrant box, openssl issue
tox -e py39-none (py 3.9.10) and py310-none (3.10.2) fail to build borg because it does not find openssl lib/headers.

strange: py 3.11.2 works.


Discussion:

Resolving patch:
diff --git a/src/borg/testsuite/archiver/extract_cmd.py b/src/borg/testsuite/archiver/extract_cmd.py
--- a/src/borg/testsuite/archiver/extract_cmd.py
+++ b/src/borg/testsuite/archiver/extract_cmd.py
@@ -1,6 +1,7 @@
 import errno
 import os
 import shutil
+import time
 import unittest
 from unittest.mock import patch
 
@@ -643,6 +644,7 @@ def test_extract_continue(self):
             # make a hardlink, so it does not free the inode when unlinking input/file3
             os.link("input/file3", "hardlink-to-keep-inode-f3")
             os.remove("input/file3")
+        time.sleep(1)  # needed due to timestamp granularity of apple hfs+
         with changedir("output"):
             # now try to continue extracting, using the same archive, same output dir:
             self.cmd(f"--repo={self.repository_location}", "extract", "arch", "--continue")


Test patch:
Repository: borgbackup/borg

Issue:
reduce github actions usage
I just learned (by using a private repo) that the gha usage is limited to 3000 "minutes" there per month.

That sounds much, but it isn't, because these are not really wallclock minutes, but weighed depending on OS/hw they are used on:
- ubuntu: weight 1
- windows: weight 2
- macOS: weight 10

So, in this specific case, only the macOS runs ate more than half of the included "minutes" already.

So, guess we should try to:
- reduce the time on macOS
- reduce the time on windows
- reduce the time needed by tests in general



Discussion:

Resolving patch:
diff --git a/setup_docs.py b/setup_docs.py
--- a/setup_docs.py
+++ b/setup_docs.py
@@ -53,8 +53,7 @@ def run(self):
         import borg
 
         borg.doc_mode = "build_man"
-        if not os.path.exists("docs/usage"):
-            os.mkdir("docs/usage")
+        os.makedirs("docs/usage", exist_ok=True)
         # allows us to build docs without the C modules fully loaded during help generation
         from borg.archiver import Archiver
 


Test patch: