Repository: borgbackup/borg

Issue:
Failing to open file with noatime
I have a mount point with `noatime` set. In this situation, the logic in `Archive._open_rb` is roughly:
1. Attempt to open the with noatime flags. 
2. If step 1 fails, try with normal flags and on success, assume step 1 failed because noatime was tried on a file we didn't have ownership on.  

The result is that files will be opened noatime only if we are the owner and are opened normal otherwise.

In my situation, the file fails to open with noatime even though I am the owner.  It seems like the code should check this scenario in the except clause. That is, rather than assume that opening noatime will work when I am the file owner, it should check that this is true. If not, then it should open normally.

I'm not sure why the file can't be opened noatime, even though I am the owner.  FWIW, the mount point is a Mac OS Extended filesystem that is made available to Linux using Parallels. 



Discussion:
The following got me going:

```
diff --git a/borg/archive.py b/borg/archive.py
index 99c9206..4b0cbd7 100644
--- a/borg/archive.py
+++ b/borg/archive.py
@@ -596,7 +596,12 @@ Number of files: {0.stats.nfiles}'''.format(
                 euid = os.geteuid()
                 # So in future, let's check whether the file is owned by us
                 # before attempting to use O_NOATIME.
-                Archive._open_rb = open_noatime_if_owner
+                if euid == 0 or s.st_uid == euid:
+                    # Then failure was not an ownership issue.
+                    Archive._open_rb = open_simple
+                else:
+                    # Assume failure was an ownership issue.
+                    Archive._open_rb = open_noatime_if_owner
             return fd
```

Interesting. One remaining problem might be that this seems to behave differently depending on the filesystem that is currently processed, so this one-time decision might not be correct for the rest of the run, when borg create recurses into different filesystems with different behaviour.

Right. Is the noatime check strictly for optimization? Perhaps it would be better not to guess and have the user explicitly request that files be opened with noatime. Otherwise, it seems like you might have to do this check on each file...which is probably slower than just opening in normal mode for every file?

It tries to use the not-atime-touching open() to not modify the file metadata for files it backs up in any way. In some cases (fs not mounted with noatime option), it is also better for performance.

Yes, i think for now doing the check for each file would be simplest. If it really slows down, it could be later optimized to only check when entering a new filesystem.

"In my situation, the file fails to open with noatime even though I am the owner."

Can you give the traceback for the original failure please?


Resolving patch:
diff --git a/borg/archive.py b/borg/archive.py
--- a/borg/archive.py
+++ b/borg/archive.py
@@ -41,6 +41,9 @@
 has_lchmod = hasattr(os, 'lchmod')
 has_lchflags = hasattr(os, 'lchflags')
 
+flags_normal = os.O_RDONLY | getattr(os, 'O_BINARY', 0)
+flags_noatime = flags_normal | getattr(os, 'O_NOATIME', 0)
+
 
 class DownloadPipeline:
 
@@ -544,7 +547,7 @@ def process_file(self, path, st, cache):
         item = {b'path': safe_path}
         # Only chunkify the file if needed
         if chunks is None:
-            fh = Archive._open_rb(path, st)
+            fh = Archive._open_rb(path)
             with os.fdopen(fh, 'rb') as fd:
                 chunks = []
                 for chunk in self.chunker.chunkify(fd, fh):
@@ -566,46 +569,16 @@ def list_archives(repository, key, manifest, cache=None):
             yield Archive(repository, key, manifest, name, cache=cache)
 
     @staticmethod
-    def _open_rb(path, st):
-        flags_normal = os.O_RDONLY | getattr(os, 'O_BINARY', 0)
-        flags_noatime = flags_normal | getattr(os, 'O_NOATIME', 0)
-        euid = None
-
-        def open_simple(p, s):
-            return os.open(p, flags_normal)
-
-        def open_noatime(p, s):
-            return os.open(p, flags_noatime)
-
-        def open_noatime_if_owner(p, s):
-            if euid == 0 or s.st_uid == euid:
-                # we are root or owner of file
-                return open_noatime(p, s)
-            else:
-                return open_simple(p, s)
-
-        def open_noatime_with_fallback(p, s):
-            try:
-                fd = os.open(p, flags_noatime)
-            except PermissionError:
-                # Was this EPERM due to the O_NOATIME flag?
-                fd = os.open(p, flags_normal)
-                # Yes, it was -- otherwise the above line would have thrown
-                # another exception.
-                nonlocal euid
-                euid = os.geteuid()
-                # So in future, let's check whether the file is owned by us
-                # before attempting to use O_NOATIME.
-                Archive._open_rb = open_noatime_if_owner
-            return fd
-
-        if flags_noatime != flags_normal:
-            # Always use O_NOATIME version.
-            Archive._open_rb = open_noatime_with_fallback
-        else:
-            # Always use non-O_NOATIME version.
-            Archive._open_rb = open_simple
-        return Archive._open_rb(path, st)
+    def _open_rb(path):
+        try:
+            # if we have O_NOATIME, this likely will succeed if we are root or owner of file:
+            return os.open(path, flags_noatime)
+        except PermissionError:
+            if flags_noatime == flags_normal:
+                # we do not have O_NOATIME, no need to try again:
+                raise
+            # Was this EPERM due to the O_NOATIME flag? Try again without it:
+            return os.open(path, flags_normal)
 
 
 # this set must be kept complete, otherwise the RobustUnpacker might malfunction:


Test patch: