Repository: borgbackup/borg

Issue:
Add option to disable backup of atime and ctime
Hi, I just noticed that borg now backups atime and ctime since version 0.28.0 (issue #317), and this is a problem for me :
- I use rsync daily to update a mirror of my data, and less often create borg archives. Rsync sets atime, so it means that the metadata of my archives can never be deduplicated (and they are close to 400MB for each archive). I have found a patch for rsync to prevent it to set atime, but it has been around there for years and has never been integrated.
- It does not make a lot of sense to me to duplicate metadata because I've done a batch grep on a directory, or an index tool has visited them, or I even just read some files.
- Sometimes I create the borg archive from one of my mirrors, so all the files have different ctime, and again this prevents the metadata from being deduplicated with the archives of another disk, even if the file tree is identical.

Does it make sense to at least add an option to "borg create" to disable backup of atime and ctime ? (in this case they could be set to the same value as mtime).



Discussion:
Sounds like reasonable use cases and adding `--noatime` and `--noctime` to `borg create` might be easy.

As already noted above, atime and ctime are new since 0.28, so borg has compatibility code to live with archives that were created with < 0.28 and miss atime and/or ctime - it just falls back to using same value as mtime then when reading such archives.

Hi folks,
I'd like to have a shot at this issue. It would be my first contribution to open-source.

I've been trying out borgbackup for a week, it's a very worthwhile project. I spent a few hours today looking at the relevant code and tests, and as @ThomasWaldmann says, it *might* be easy to implement #1853. 

Please don't let the gray beard fool you; I'm fairly new at Python, with some C++ and C and UNIX/Linux in my background.
@jazcap53 cool, go for it! ;) If you have something and want first feedback, just open a pull request (against 1.0-maint, **not** master).
@ThomasWaldmann Will do!
Making progress? We plan rc1 for next saturday or so.
Have read a lot of code over the last 2 days and it's going well. Getting ready to try out some changes.
I'll have some specific questions within the next hour or so.
I have nothing but free time between now and saturday :)


If you like, join us on #borgbackup on freenode IRC.
OK

On Wed, Nov 23, 2016 at 8:08 PM, TW <notifications@github.com> wrote:

> If you like, join us on #borgbackup on freenode IRC.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/borgbackup/borg/issues/1853#issuecomment-262666838>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABYrl8SUJxjOxLLjRDnr8dAwjKzj0b5Hks5rBON1gaJpZM4KzJvK>
> .
>


Resolving patch:
diff --git a/borg/archive.py b/borg/archive.py
--- a/borg/archive.py
+++ b/borg/archive.py
@@ -185,7 +185,7 @@ class IncompatibleFilesystemEncodingError(Error):
         """Failed to encode filename "{}" into file system encoding "{}". Consider configuring the LANG environment variable."""
 
     def __init__(self, repository, key, manifest, name, cache=None, create=False,
-                 checkpoint_interval=300, numeric_owner=False, progress=False,
+                 checkpoint_interval=300, numeric_owner=False, noatime=False, noctime=False, progress=False,
                  chunker_params=CHUNKER_PARAMS, start=None, end=None):
         self.cwd = os.getcwd()
         self.key = key
@@ -198,6 +198,8 @@ def __init__(self, repository, key, manifest, name, cache=None, create=False,
         self.name = name
         self.checkpoint_interval = checkpoint_interval
         self.numeric_owner = numeric_owner
+        self.noatime = noatime
+        self.noctime = noctime
         if start is None:
             start = datetime.utcnow()
         self.start = start
@@ -571,10 +573,15 @@ def stat_attrs(self, st, path):
             b'mode': st.st_mode,
             b'uid': st.st_uid, b'user': uid2user(st.st_uid),
             b'gid': st.st_gid, b'group': gid2group(st.st_gid),
-            b'atime': int_to_bigint(st.st_atime_ns),
-            b'ctime': int_to_bigint(st.st_ctime_ns),
             b'mtime': int_to_bigint(st.st_mtime_ns),
         }
+        # borg can work with archives only having mtime (older attic archives do not have
+        # atime/ctime). it can be useful to omit atime/ctime, if they change without the
+        # file content changing - e.g. to get better metadata deduplication.
+        if not self.noatime:
+            item[b'atime'] = int_to_bigint(st.st_atime_ns)
+        if not self.noctime:
+            item[b'ctime'] = int_to_bigint(st.st_ctime_ns)
         if self.numeric_owner:
             item[b'user'] = item[b'group'] = None
         with backup_io():
diff --git a/borg/archiver.py b/borg/archiver.py
--- a/borg/archiver.py
+++ b/borg/archiver.py
@@ -281,7 +281,8 @@ def create_inner(archive, cache):
             with Cache(repository, key, manifest, do_files=args.cache_files, lock_wait=self.lock_wait) as cache:
                 archive = Archive(repository, key, manifest, args.location.archive, cache=cache,
                                   create=True, checkpoint_interval=args.checkpoint_interval,
-                                  numeric_owner=args.numeric_owner, progress=args.progress,
+                                  numeric_owner=args.numeric_owner, noatime=args.noatime, noctime=args.noctime,
+                                  progress=args.progress,
                                   chunker_params=args.chunker_params, start=t0)
                 create_inner(archive, cache)
         else:
@@ -1301,6 +1302,12 @@ def build_parser(self, args=None, prog=None):
         subparser.add_argument('--numeric-owner', dest='numeric_owner',
                                action='store_true', default=False,
                                help='only store numeric user and group identifiers')
+        subparser.add_argument('--noatime', dest='noatime',
+                               action='store_true', default=False,
+                               help='do not store atime into archive')
+        subparser.add_argument('--noctime', dest='noctime',
+                               action='store_true', default=False,
+                               help='do not store ctime into archive')
         subparser.add_argument('--timestamp', dest='timestamp',
                                type=timestamp, default=None,
                                metavar='yyyy-mm-ddThh:mm:ss',


Test patch: