Repository: borgbackup/borg

Issue:
memory leak in borg export-tar with remote (ssh:) repo
Archive size: 400-500G (20~30 archive dates, expected restore size of ~90% of the total archive size)

After restoring around 60-70G, RAM usage was going up exponentially (very fast: few minutes?), going from 100-200mb to a few gigs (10-16G) till killed by OOM while the system became unstable as also other things where killed by OOM (when no RAM or SWAP was free anymore).

When RAM went up, it seemed that nothing, or hardly anything was saved to the disk. I had the feeling that nothing was written to disk but the data was kept in memory. (but this last can be a wrong assumption, and might point to a memory leak or a recursion,..)

example vars:
```
backfrom=user@192.168.2.123
path=/mount/backups/borgarchivename
archive_backup_date=2020-05-01
restore_path=/mount/restored
```

command used with using the internal ssh of borg:
```
borg export-tar "${backfrom}:${path}::${archive_backup_date}" - | tar -C $restore_path -x
```

But using borg over ssh resolved that issue, and only ~200mb was used on the system that had the backup (where we connect to):
```
ssh ${backfrom} "borg export-tar --tar-filter='gzip -2' \"${path}::${archive_backup_date}\" -" | tar -C $restore_path -xz
```

The filesystem on source and target is ZFS.
Network transfer was at least 100Mbit, could have been 1Gbit. (transfer speed was for site ~10mb/s, maybe it was 2-3 times faster, but can't remember)

Checked and tested with 1.1.13.


Discussion:

Resolving patch:
diff --git a/src/borg/archive.py b/src/borg/archive.py
--- a/src/borg/archive.py
+++ b/src/borg/archive.py
@@ -463,6 +463,8 @@ def item_filter(self, item, filter=None):
         return filter(item) if filter else True
 
     def iter_items(self, filter=None, partial_extract=False, preload=False, hardlink_masters=None):
+        # note: when calling this with preload=True, later fetch_many() must be called with
+        # is_preloaded=True or the RemoteRepository code will leak memory!
         assert not (filter and partial_extract and preload) or hardlink_masters is not None
         for item in self.pipeline.unpack_many(self.metadata.items, partial_extract=partial_extract,
                                               preload=preload, hardlink_masters=hardlink_masters,
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -922,8 +922,8 @@ def _export_tar(self, args, archive, tar):
         hardlink_masters = {} if partial_extract else None
 
         def peek_and_store_hardlink_masters(item, matched):
-            if (partial_extract and not matched and hardlinkable(item.mode) and
-                    item.get('hardlink_master', True) and 'source' not in item):
+            if ((partial_extract and not matched and hardlinkable(item.mode)) and
+                    (item.get('hardlink_master', True) and 'source' not in item)):
                 hardlink_masters[item.get('path')] = (item.get('chunks'), None)
 
         filter = self.build_filter(matcher, peek_and_store_hardlink_masters, strip_components)
@@ -940,7 +940,8 @@ def item_content_stream(item):
             """
             Return a file-like object that reads from the chunks of *item*.
             """
-            chunk_iterator = archive.pipeline.fetch_many([chunk_id for chunk_id, _, _ in item.chunks])
+            chunk_iterator = archive.pipeline.fetch_many([chunk_id for chunk_id, _, _ in item.chunks],
+                                                         is_preloaded=True)
             if pi:
                 info = [remove_surrogates(item.path)]
                 return ChunkIteratorFileWrapper(chunk_iterator,
@@ -1024,7 +1025,8 @@ def item_to_tarinfo(item, original_path):
                 return None, stream
             return tarinfo, stream
 
-        for item in archive.iter_items(filter, preload=True, hardlink_masters=hardlink_masters):
+        for item in archive.iter_items(filter, partial_extract=partial_extract,
+                                       preload=True, hardlink_masters=hardlink_masters):
             orig_path = item.path
             if strip_components:
                 item.path = os.sep.join(orig_path.split(os.sep)[strip_components:])


Test patch: