Repository: borgbackup/borg

Issue:
UnboundLocalError: 'cmtime_ns' when using --files-cache=size
<!--
Thank you for reporting an issue.

*IMPORTANT* -  *before* creating a new issue please look around:
 - Borgbackup documentation: http://borgbackup.readthedocs.io/en/stable/index.html
 - FAQ: https://borgbackup.readthedocs.io/en/stable/faq.html
 and
 - open issues in GitHub tracker: https://github.com/borgbackup/borg/issues
  
If you cannot find a similar problem, then create a new issue.

Please fill in as much of the template as possible.
-->

## Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes

## Is this a BUG / ISSUE report or a QUESTION?

BUG

## System information. For client/server mode post info for both machines.

#### Your borg version (borg -V).

borg 1.2.4
(but also true for 1.2.2)

#### Operating system (distribution) and version.

```
Linux Mint 20.3 64Bit
> uname -a
Linux pc-name 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
```

#### Hardware / network configuration, and filesystems used.

#### How much data is handled by borg?

#### Full borg commandline that lead to the problem (leave away excludes and passwords)

BORG_RSH and BORG_REPO are set.

```
sys.argv: ['/usr/local/bin/borg', 'create', '--verbose', '--progress', '--stats', '--list', '--compression=none,', '--files-cache=size', '--filter=AMCE', '::SOME_ARCHIVE_NAME', 'DIR_TO_BACKUP']
```

**Attention: this is about the option `--files-cache=size`!**
**When using eg. `--files-cache=ctime,size`, the error does not appear.**

## Describe the problem you're observing.

**(I am rephrasing the error report, as the error seems to be completely unrelated to the data that is backed up.)**

I backup a directory to an archive. Then I backup the same directory, but for internal reasons I try to force borg to not read files from disk again by adding `--files-cache=size`. (I read the FAQ about the cache. Setting files-cache to `size` **was to investigate the reasons for borg to reread those files**)

Then borg crashes reproducible with this option every time it encounters a file with status flag `A`.

```
A DIR_TO_BACKUP/subdir3/someFilename                                                                               
Local Exception
Traceback (most recent call last):
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 5213, in main
    exit_code = archiver.run(args)
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 5144, in run
    return set_ec(func(args))
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 183, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 675, in do_create
    create_inner(archive, cache, fso)
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 608, in create_inner
    self._rec_walk(path=path, parent_fd=parent_fd, name=name,
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 823, in _rec_walk
    self._rec_walk(
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 823, in _rec_walk
    self._rec_walk(
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 823, in _rec_walk
    self._rec_walk(
  [Previous line repeated 5 more times]
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 786, in _rec_walk
    status = self._process_any(path=path, parent_fd=parent_fd, name=name, st=st, fso=fso, cache=cache,
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 688, in _process_any
    return fso.process_file(path=path, parent_fd=parent_fd, name=name, st=st, cache=cache)
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archive.py", line 1468, in process_file
    cache.memorize_file(hashed_path, path_hash, st, [c.id for c in item.chunks])
  File "/home/user1/.local/lib/python3.8/site-packages/borg/cache.py", line 1049, in memorize_file
    entry = FileCacheEntry(age=0, inode=st.st_ino, size=st.st_size, cmtime=int_to_bigint(cmtime_ns), chunk_ids=ids)
UnboundLocalError: local variable 'cmtime_ns' referenced before assignment

Platform: Linux pc-name 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64
Linux: Unknown Linux  
Borg: 1.2.4  Python: CPython 3.8.10 msgpack: 1.0.4 fuse: pyfuse3 3.2.2 [pyfuse3,llfuse]
PID: 3116549  CWD: /path/to/dir
sys.argv: ['/usr/local/bin/borg', 'create', '--verbose', '--progress', '--stats', '--list', '--compression=none,', '--files-cache=size', '--filter=AMCE', '::ARCHIVE_NAME', 'DIR_TO_BACKUP']
SSH_ORIGINAL_COMMAND: None
```
#### Do I need help?

I strippped private information and replaced files and directories names as careful as possible.

No, I guess I just let borg re-read all the files. I only want to point out an error that I suspect to be a bug worth fixing.


Discussion:
#### Reproducible with a fresh repo

```
borg init --encryption=none test2.borg
borg create --verbose --progress --stats --list --filter=AMCE --compression=none test2.borg::test1 ~/.config/fish/
# this works
borg create --verbose --progress --stats --list --filter=AMCE --compression=none --files-cache=size test2.borg::test2 ~/.config/fish/
# this crashes
Creating archive at "test2.borg::test2"
A /home/user1/.config/fish/fish_variables                                                                                                                                                                                                     
Local Exception
Traceback (most recent call last):
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 5213, in main
    exit_code = archiver.run(args)
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 5144, in run
    return set_ec(func(args))
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 183, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 675, in do_create
    create_inner(archive, cache, fso)
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 608, in create_inner
    self._rec_walk(path=path, parent_fd=parent_fd, name=name,
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 823, in _rec_walk
    self._rec_walk(
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 786, in _rec_walk
    status = self._process_any(path=path, parent_fd=parent_fd, name=name, st=st, fso=fso, cache=cache,
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archiver.py", line 688, in _process_any
    return fso.process_file(path=path, parent_fd=parent_fd, name=name, st=st, cache=cache)
  File "/home/user1/.local/lib/python3.8/site-packages/borg/archive.py", line 1468, in process_file
    cache.memorize_file(hashed_path, path_hash, st, [c.id for c in item.chunks])
  File "/home/user1/.local/lib/python3.8/site-packages/borg/cache.py", line 1049, in memorize_file
    entry = FileCacheEntry(age=0, inode=st.st_ino, size=st.st_size, cmtime=int_to_bigint(cmtime_ns), chunk_ids=ids)
UnboundLocalError: local variable 'cmtime_ns' referenced before assignment

Platform: Linux pc-name 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64
Linux: Unknown Linux  
Borg: 1.2.4  Python: CPython 3.8.10 msgpack: 1.0.4 fuse: pyfuse3 3.2.2 [pyfuse3,llfuse]
PID: 3119286  CWD: /home/user1
sys.argv: ['/home/user1/.local/bin/borg', 'create', '--verbose', '--progress', '--stats', '--list', '--filter=AMCE', '--compression=none', '--files-cache=size', 'test2.borg::test2', '/home/user1/.config/fish/']
SSH_ORIGINAL_COMMAND: None

```
I agree that it should not crash of course.

But: be aware that using `--files-cache=size` can be pretty risky. You are telling borg that if a file has same size as before, that its contents for sure did not change.

But, obviously one can change the content of a file without changing its size. In such a case, borg would not backup the current content of the file, but rely on it being unchanged.

> But: be aware that using `--files-cache=size` can be pretty risky. You are telling borg that if a file has same size as before, that its contents for sure did not change.

I absolutely understand.

In my special case that made me aware of the bug, the data itself would never have changed as the source is another fuse mounted borg repository. (with v1.x using `borg mount` is my meanwhile long-term approach for transfer archive content between repositories with "perfectly conserving" all metadata, and for backing up data to multiple repos.)

Therefor I was testing borg's behaviour, as I suspected that the re-reading from disk might be caused because I unmounted and mounted that borg archive since the first backup and inodes will have changed.

As `file` size is not actual metadata (like mtime and ctime), I considered it to be the easiset way to force borg to assume that files have not been changed since last run.
hmm, some feedback about the borg mount filesystem:
- that should work with ctime or mtime also, as these are the same as they come from same archived metadata
- not totally sure about inode numbers, but guess if you use same archive they will stay same, but if you switch to a different archive of same data, guess they might shift somehow
- the borg mount fs is **not** perfectly conserving all metadata, see our docs (e.g. ACL support is missing)


Resolving patch:
diff --git a/src/borg/cache.py b/src/borg/cache.py
--- a/src/borg/cache.py
+++ b/src/borg/cache.py
@@ -1046,6 +1046,9 @@ def memorize_file(self, hashed_path, path_hash, st, ids):
         elif 'm' in cache_mode:
             cmtime_type = 'mtime'
             cmtime_ns = safe_ns(st.st_mtime_ns)
+        else:  # neither 'c' nor 'm' in cache_mode, avoid UnboundLocalError
+            cmtime_type = 'ctime'
+            cmtime_ns = safe_ns(st.st_ctime_ns)
         entry = FileCacheEntry(age=0, inode=st.st_ino, size=st.st_size, cmtime=int_to_bigint(cmtime_ns), chunk_ids=ids)
         self.files[path_hash] = msgpack.packb(entry)
         self._newest_cmtime = max(self._newest_cmtime or 0, cmtime_ns)


Test patch: