Repository: borgbackup/borg

Issue:
TypeError: 'int' object is not iterable
Hello,

unfortunately, I had a "corrupted chunks index" due to filesystem corruption (that I myself caused - but that's another story). To fix it, I invoked `borg delete --cache-only` that removed the local cache. However, I now get the following error when trying to run the backup:

```
Aug 15 14:02:19 darkroom borgmatic[7640]: Synchronizing chunks cache...
Aug 15 14:02:19 darkroom borgmatic[7640]: Archives: 99, w/ cached Idx: 99, w/ outdated Idx: 1, w/o cached Idx: 1.
Aug 15 14:02:19 darkroom borgmatic[7640]: Reading cached archive chunk index for darkroom-2017-07-30T05:00:00.804255 ...
Aug 15 14:02:19 darkroom borgmatic[7640]: Merging into master chunks index ...
Aug 15 14:02:19 darkroom borgmatic[7640]: Reading cached archive chunk index for switchy-2017-08-09T21:32:04.884748 ...
Aug 15 14:02:19 darkroom borgmatic[7640]: Merging into master chunks index ...
Aug 15 14:02:19 darkroom borgmatic[7640]: Reading cached archive chunk index for darkroom-2017-07-31T01:00:00.800344 ...
... snip ...
Aug 15 14:02:42 darkroom borgmatic[7640]: Reading cached archive chunk index for darkroom-2017-07-31T05:00:00.814492 ...
Aug 15 14:02:42 darkroom borgmatic[7640]: Merging into master chunks index ...
Aug 15 14:02:42 darkroom borgmatic[7640]: Reading cached archive chunk index for switchy-2017-02-28T23:30:21.442136 ...
Aug 15 14:02:42 darkroom borgmatic[7640]: Merging into master chunks index ...
Aug 15 14:02:42 darkroom borgmatic[7640]: Done.
Aug 15 14:02:43 darkroom borgmatic[7640]: Local Exception.
Aug 15 14:02:43 darkroom borgmatic[7640]: Traceback (most recent call last):
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 2168, in main
Aug 15 14:02:43 darkroom borgmatic[7640]:     exit_code = archiver.run(args)
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 2104, in run
Aug 15 14:02:43 darkroom borgmatic[7640]:     return set_ec(func(args))
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 107, in wrapper
Aug 15 14:02:43 darkroom borgmatic[7640]:     return method(self, args, repository=repository, **kwargs)
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 331, in do_create
Aug 15 14:02:43 darkroom borgmatic[7640]:     create_inner(archive, cache)
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 304, in create_inner
Aug 15 14:02:43 darkroom borgmatic[7640]:     read_special=args.read_special, dry_run=dry_run)
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 380, in _process
Aug 15 14:02:43 darkroom borgmatic[7640]:     read_special=read_special, dry_run=dry_run)
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 380, in _process
Aug 15 14:02:43 darkroom borgmatic[7640]:     read_special=read_special, dry_run=dry_run)
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archiver.py", line 361, in _process
Aug 15 14:02:43 darkroom borgmatic[7640]:     status = archive.process_file(path, st, cache, self.ignore_inode)
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/archive.py", line 679, in process_file
Aug 15 14:02:43 darkroom borgmatic[7640]:     ids = cache.file_known_and_unchanged(path_hash, st, ignore_inode)
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/cache.py", line 489, in file_known_and_unchanged
Aug 15 14:02:43 darkroom borgmatic[7640]:     self._read_files()
Aug 15 14:02:43 darkroom borgmatic[7640]:   File "/usr/lib/python3.6/site-packages/borg/cache.py", line 211, in _read_files
Aug 15 14:02:43 darkroom borgmatic[7640]:     for path_hash, item in u:
Aug 15 14:02:43 darkroom borgmatic[7640]: TypeError: 'int' object is not iterable
Aug 15 14:02:43 darkroom borgmatic[7640]: Platform: Linux darkroom 4.12.4-1-ARCH #1 SMP PREEMPT Fri Jul 28 18:54:18 UTC 2017 x86_64
Aug 15 14:02:43 darkroom borgmatic[7640]: Linux: arch
Aug 15 14:02:43 darkroom borgmatic[7640]: Borg: 1.0.11  Python: CPython 3.6.2
Aug 15 14:02:43 darkroom borgmatic[7640]: PID: 7645  CWD: /
Aug 15 14:02:43 darkroom borgmatic[7640]: sys.argv: ['/usr/sbin/borg', 'create', '/volume1/Backup/borg-systems::darkroom-2017-08-15T14:02:18.282818', '/', '--exclude-from', '/etc/borgmatic/rootfs.excludes', '--compression', 'lz4', '--one-file-system', '--info', '--stats']
Aug 15 14:02:43 darkroom borgmatic[7640]: SSH_ORIGINAL_COMMAND: None
Aug 15 14:02:43 darkroom borgmatic[7640]: Command '('borg', 'create', '/volume1/Backup/borg-systems::darkroom-2017-08-15T14:02:18.282818', '/', '--exclude-from', '/etc/borgmatic/rootfs.excludes', '--compression', 'lz4', '--one-file-system', '--info', '--stats')' returned non-zero exit status 2.
Aug 15 14:02:43 darkroom systemd[1]: borgmatic.service: Main process exited, code=exited, status=1/FAILURE
Aug 15 14:02:43 darkroom systemd[1]: Failed to start borgmatic backup.
Aug 15 14:02:43 darkroom systemd[1]: borgmatic.service: Unit entered failed state.
Aug 15 14:02:43 darkroom systemd[1]: borgmatic.service: Triggering OnFailure= dependencies.
Aug 15 14:02:43 darkroom systemd[1]: borgmatic.service: Failed with result 'exit-code'.
```

I really don't know if it's a very stupid simple error that I'm causing or if there is something else going on here. As a background: I'm doing a backup of my root partition to a directory on a large RAID5 array which I later sync to another server. Until the corruption happened, the backup ran without a problem for at least half a year. You see two machines ("darkroom" and "switchy") as I backup also a portable device to the same repo to use deduplication as both use Archlinux. I run borg through borgmatic as a systemd timer job.

If there's anything else you need, please let me know. Thanks in advance for looking into this issue.

Best regards,
Mathias


Discussion:
That looks like a corrupted "files" cache. Maybe check if the cache was really removed (permissions?).

The `borg delete --cache-only repo` code should have killed the whole cache directory (in a `rm -rf` way), so it is strange that there seems to be an invalid `files` cache file left.
The cache is in `$HOME/.cache/borg/<repoid>/` assuming you did not point it somewhere else via env vars.


Resolving patch:
diff --git a/src/borg/cache.py b/src/borg/cache.py
--- a/src/borg/cache.py
+++ b/src/borg/cache.py
@@ -503,10 +503,16 @@ def _read_files(self):
                 if not data:
                     break
                 u.feed(data)
-                for path_hash, item in u:
-                    entry = FileCacheEntry(*item)
-                    # in the end, this takes about 240 Bytes per file
-                    self.files[path_hash] = msgpack.packb(entry._replace(age=entry.age + 1))
+                try:
+                    for path_hash, item in u:
+                        entry = FileCacheEntry(*item)
+                        # in the end, this takes about 240 Bytes per file
+                        self.files[path_hash] = msgpack.packb(entry._replace(age=entry.age + 1))
+                except (TypeError, ValueError) as exc:
+                    logger.warning('The files cache seems corrupt, ignoring it. '
+                                   'Expect lower performance. [%s]' % str(exc))
+                    self.files = {}
+                    return
 
     def begin_txn(self):
         # Initialize transaction snapshot


Test patch: