Repository: borgbackup/borg

Issue:
Corrupted repo causes borg check, delete fail
I have this corrupted repo created with broked Fedora23 package on ppc64, where it segfaulted shortly after start.

Now both borg delete and borg check are really unhappy about this repo even after I did borg break-lock on it and removed the now stale lock.

I am attaching a sample. which is a tar inside of zip since github apparently does not support tar.gz files.
[borg-repo.tar.zip](https://github.com/borgbackup/borg/files/265292/borg-repo.tar.zip)

The errors I get are:

```
[root@powermac ~]# borg delete localhost:/tmp/backup
root@localhost's password: 
Remote: Borg 1.0.2: exception in RPC call:
Remote: Traceback (most recent call last):
Remote:   File "/usr/lib64/python3.4/site-packages/borg/remote.py", line 96, in serve
Remote:     res = f(*args)
Remote:   File "/usr/lib64/python3.4/site-packages/borg/repository.py", line 447, in get
Remote:     raise self.ObjectNotFound(id_, self.path) from None
Remote: borg.repository.ObjectNotFound: (b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', '/tmp/backup')
Remote: Platform: Linux powermac.localnet 4.4.8-300.fc23.ppc64 #1 SMP Wed Apr 27 17:14:46 UTC 2016 ppc64 ppc64
Remote: Linux: Fedora 23 Twenty Three
Remote: Borg: 1.0.2  Python: CPython 3.4.3
Remote: PID: 22394  CWD: /root
Remote: sys.argv: ['/usr/bin/borg', 'serve', '--umask=077']
Remote: SSH_ORIGINAL_COMMAND: None
Remote: 
Object with key 82 not found in repository localhost:/tmp/backup.
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 1498, in main
    exit_code = archiver.run(args)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 1452, in run
    return args.func(args)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 74, in wrapper
    kwargs['manifest'], kwargs['key'] = Manifest.load(repository)
  File "/usr/lib64/python3.4/site-packages/borg/helpers.py", line 93, in load
    cdata = repository.get(cls.MANIFEST_ID)
  File "/usr/lib64/python3.4/site-packages/borg/remote.py", line 354, in get
    for resp in self.get_many([id_]):
  File "/usr/lib64/python3.4/site-packages/borg/remote.py", line 358, in get_many
    for resp in self.call_many('get', [(id_,) for id_ in ids], is_preloaded=is_preloaded):
  File "/usr/lib64/python3.4/site-packages/borg/remote.py", line 263, in call_many
    raise Repository.ObjectNotFound(res[0], self.location.orig)
borg.repository.ObjectNotFound: (82, 'localhost:/tmp/backup')

Platform: Linux powermac.localnet 4.4.8-300.fc23.ppc64 #1 SMP Wed Apr 27 17:14:46 UTC 2016 ppc64 ppc64
Linux: Fedora 23 Twenty Three
Borg: 1.0.2  Python: CPython 3.4.3
PID: 22374  CWD: /root
sys.argv: ['/usr/bin/borg', 'delete', 'localhost:/tmp/backup']
SSH_ORIGINAL_COMMAND: None

[root@powermac ~]# borg check localhost:/tmp/backup
root@localhost's password: 
Local Exception.                
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 1498, in main
    exit_code = archiver.run(args)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 1452, in run
    return args.func(args)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 80, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/usr/lib64/python3.4/site-packages/borg/archiver.py", line 147, in do_check
    last=args.last, prefix=args.prefix, save_space=args.save_space):
  File "/usr/lib64/python3.4/site-packages/borg/archive.py", line 663, in check
    self.key = self.identify_key(repository)
  File "/usr/lib64/python3.4/site-packages/borg/archive.py", line 696, in identify_key
    cdata = repository.get(next(self.chunks.iteritems())[0])
  File "borg/hashindex.pyx", line 341, in borg.hashindex.ChunkKeyIterator.__next__ (borg/hashindex.c:5407)
StopIteration

Platform: Linux powermac.localnet 4.4.8-300.fc23.ppc64 #1 SMP Wed Apr 27 17:14:46 UTC 2016 ppc64 ppc64
Linux: Fedora 23 Twenty Three
Borg: 1.0.2  Python: CPython 3.4.3
PID: 22419  CWD: /root
sys.argv: ['/usr/bin/borg', 'check', 'localhost:/tmp/backup']
SSH_ORIGINAL_COMMAND: None
```



Discussion:
The repository is completely empty (no segments) that's where the "borg check" crash comes from. Other commands give me this error:

```
borg.repository.Repository.ObjectNotFound: (
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
'/home/mabe/Projekte/_oss/borg/green1043')
```

This error you show is interesting: `
borg.repository.ObjectNotFound: (82, 'localhost:/tmp/backup')`

Right.
I imagine this is a pretty well known state, though, for a completely empty state and perhaps asserting like this is not the most ideal way of dealing with it?

Yes, a check should be added that aborts earlier and with a more clear error message when a repository is opened that has no commits.


Resolving patch:
diff --git a/borg/archive.py b/borg/archive.py
--- a/borg/archive.py
+++ b/borg/archive.py
@@ -695,7 +695,12 @@ def init_chunks(self):
                 self.chunks[id_] = (0, 0, 0)
 
     def identify_key(self, repository):
-        cdata = repository.get(next(self.chunks.iteritems())[0])
+        try:
+            some_chunkid, _ = next(self.chunks.iteritems())
+        except StopIteration:
+            # repo is completely empty, no chunks
+            return None
+        cdata = repository.get(some_chunkid)
         return key_factory(repository, cdata)
 
     def rebuild_manifest(self):
diff --git a/borg/archiver.py b/borg/archiver.py
--- a/borg/archiver.py
+++ b/borg/archiver.py
@@ -19,7 +19,7 @@
 from .helpers import Error, location_validator, archivename_validator, format_line, format_time, format_file_size, \
     parse_pattern, PathPrefixPattern, to_localtime, timestamp, safe_timestamp, \
     get_cache_dir, prune_within, prune_split, \
-    Manifest, remove_surrogates, update_excludes, format_archive, check_extension_modules, Statistics, \
+    Manifest, NoManifestError, remove_surrogates, update_excludes, format_archive, check_extension_modules, Statistics, \
     dir_is_tagged, bigint_to_int, ChunkerParams, CompressionSpec, is_slow_msgpack, yes, sysinfo, \
     EXIT_SUCCESS, EXIT_WARNING, EXIT_ERROR, log_multi, PatternMatcher
 from .logger import create_logger, setup_logging
@@ -397,10 +397,11 @@ def do_rename(self, args, repository, manifest, key, cache, archive):
         cache.commit()
         return self.exit_code
 
-    @with_repository(exclusive=True)
-    def do_delete(self, args, repository, manifest, key):
+    @with_repository(exclusive=True, manifest=False)
+    def do_delete(self, args, repository):
         """Delete an existing repository or archive"""
         if args.location.archive:
+            manifest, key = Manifest.load(repository)
             with Cache(repository, key, manifest, lock_wait=self.lock_wait) as cache:
                 archive = Archive(repository, key, manifest, args.location.archive, cache=cache)
                 stats = Statistics()
@@ -417,9 +418,15 @@ def do_delete(self, args, repository, manifest, key):
         else:
             if not args.cache_only:
                 msg = []
-                msg.append("You requested to completely DELETE the repository *including* all archives it contains:")
-                for archive_info in manifest.list_archive_infos(sort_by='ts'):
-                    msg.append(format_archive(archive_info))
+                try:
+                    manifest, key = Manifest.load(repository)
+                except NoManifestError:
+                    msg.append("You requested to completely DELETE the repository *including* all archives it may contain.")
+                    msg.append("This repository seems to have no manifest, so we can't tell anything about its contents.")
+                else:
+                    msg.append("You requested to completely DELETE the repository *including* all archives it contains:")
+                    for archive_info in manifest.list_archive_infos(sort_by='ts'):
+                        msg.append(format_archive(archive_info))
                 msg.append("Type 'YES' if you understand this and want to continue: ")
                 msg = '\n'.join(msg)
                 if not yes(msg, false_msg="Aborting.", truish=('YES', ),
diff --git a/borg/helpers.py b/borg/helpers.py
--- a/borg/helpers.py
+++ b/borg/helpers.py
@@ -65,6 +65,10 @@ class ExtensionModuleError(Error):
     """The Borg binary extension modules do not seem to be properly installed"""
 
 
+class NoManifestError(Error):
+    """Repository has no manifest."""
+
+
 def check_extension_modules():
     from . import platform
     if hashindex.API_VERSION != 2:
@@ -90,7 +94,11 @@ def __init__(self, key, repository):
     @classmethod
     def load(cls, repository, key=None):
         from .key import key_factory
-        cdata = repository.get(cls.MANIFEST_ID)
+        from .repository import Repository
+        try:
+            cdata = repository.get(cls.MANIFEST_ID)
+        except Repository.ObjectNotFound:
+            raise NoManifestError
         if not key:
             key = key_factory(repository, cdata)
         manifest = cls(key, repository)


Test patch: