Repository: borgbackup/borg

Issue:
Msg "creating archive at ..." does not expand placeholders in archive name
hi,
running bananian OS (debian) i downloaded https://borg.bauerj.eu/bin/borg-1.1.16-armv6

executing backup using #!/bin/bash script 
...

 $REPOSITORY::'{hostname}'-'{now:%Y-%m-%d_%T}'                  \

...

Creating archive at "heap@10.0.1.1:/x/banana-repo::{hostname}-{now:%Y-%m-%d_%T}"

The hostname and time is not "resolved" and replaced accordingly.

Any idea why?




Python 2.7.9 (default, Sep 16 2019, 12:56:14)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import socket
 >>> socket.gethostname()
'bananapi'


file /usr/bin/borg: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3,
              for GNU/Linux 3.2.0, BuildID[sha1]=584855b7096ef005ebbc274ffd8b61b126a2d202, stripped


Discussion:
codecov-commenter: # [Codecov](https://codecov.io/gh/borgbackup/borg/pull/6199?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup) Report
> Merging [#6199](https://codecov.io/gh/borgbackup/borg/pull/6199?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup) (5ccbd19) into [1.1-maint](https://codecov.io/gh/borgbackup/borg/commit/d67a1e77330f32e1be285cd30648b5e5c34754fd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup) (d67a1e7) will **decrease** coverage by `1.16%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/borgbackup/borg/pull/6199/graphs/tree.svg?width=650&height=150&src=pr&token=LssEpmW3o4&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup)](https://codecov.io/gh/borgbackup/borg/pull/6199?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup)

```diff
@@              Coverage Diff              @@
##           1.1-maint    #6199      +/-   ##
=============================================
- Coverage      80.01%   78.84%   -1.17%     
=============================================
  Files             27       27              
  Lines          10486    10391      -95     
  Branches        2147     2018     -129     
=============================================
- Hits            8390     8193     -197     
- Misses          1577     1667      +90     
- Partials         519      531      +12     
```


| [Impacted Files](https://codecov.io/gh/borgbackup/borg/pull/6199?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup) | Coverage Δ | |
|---|---|---|
| [src/borg/helpers.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcvaGVscGVycy5weQ==) | `86.99% <100.00%> (-0.60%)` | :arrow_down: |
| [src/borg/xattr.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcveGF0dHIucHk=) | `61.35% <0.00%> (-19.81%)` | :arrow_down: |
| [src/borg/platform/\_\_init\_\_.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcvcGxhdGZvcm0vX19pbml0X18ucHk=) | `73.68% <0.00%> (-16.32%)` | :arrow_down: |
| [src/borg/platform/base.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcvcGxhdGZvcm0vYmFzZS5weQ==) | `79.43% <0.00%> (-3.90%)` | :arrow_down: |
| [src/borg/archive.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcvYXJjaGl2ZS5weQ==) | `79.74% <0.00%> (-2.68%)` | :arrow_down: |
| [src/borg/nanorst.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcvbmFub3JzdC5weQ==) | `88.67% <0.00%> (-2.58%)` | :arrow_down: |
| [src/borg/upgrader.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcvdXBncmFkZXIucHk=) | `60.94% <0.00%> (-2.28%)` | :arrow_down: |
| [src/borg/archiver.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcvYXJjaGl2ZXIucHk=) | `80.79% <0.00%> (-0.75%)` | :arrow_down: |
| [src/borg/logger.py](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup#diff-c3JjL2JvcmcvbG9nZ2VyLnB5) | `72.03% <0.00%> (-0.24%)` | :arrow_down: |
| ... and [5 more](https://codecov.io/gh/borgbackup/borg/pull/6199/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/borgbackup/borg/pull/6199?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/borgbackup/borg/pull/6199?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup). Last update [d67a1e7...5ccbd19](https://codecov.io/gh/borgbackup/borg/pull/6199?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=borgbackup).


Patch:
diff --git a/src/borg/archiver.py b/src/borg/archiver.py
index 8676229d..54896892 100644
--- a/src/borg/archiver.py
+++ b/src/borg/archiver.py
@@ -581,7 +581,7 @@ def create_inner(archive, cache):
         dry_run = args.dry_run
         t0 = datetime.utcnow()
         t0_monotonic = time.monotonic()
-        logger.info('Creating archive at "%s"' % args.location.orig)
+        logger.info('Creating archive at "%s"' % args.location.processed)
         if not dry_run:
             with Cache(repository, key, manifest, do_files=args.cache_files, progress=args.progress,
                        lock_wait=self.lock_wait, permit_adhoc_cache=args.no_cache_sync,
diff --git a/src/borg/helpers.py b/src/borg/helpers.py
index 42c432bc..b33a69c3 100644
--- a/src/borg/helpers.py
+++ b/src/borg/helpers.py
@@ -1136,23 +1136,25 @@ class Location:
 
     def __init__(self, text='', overrides={}):
         if not self.parse(text, overrides):
-            raise ValueError('Invalid location format: "%s"' % self.orig)
+            raise ValueError('Invalid location format: "%s"' % self.processed)
 
     def parse(self, text, overrides={}):
-        self.orig = text
-        text = replace_placeholders(text, overrides)
+        self.raw = text  # as given by user, might contain placeholders
+        self.processed = text = replace_placeholders(text, overrides)  # after placeholder replacement
         valid = self._parse(text)
         if valid:
             return True
         m = self.env_re.match(text)
         if not m:
             return False
-        repo = os.environ.get('BORG_REPO')
-        if repo is None:
+        repo_raw = os.environ.get('BORG_REPO')
+        if repo_raw is None:
             return False
+        repo = replace_placeholders(repo_raw, overrides)
         valid = self._parse(repo)
         self.archive = m.group('archive')
-        self.orig = repo if not self.archive else '%s::%s' % (repo, self.archive)
+        self.raw = repo_raw if not self.archive else repo_raw + self.raw
+        self.processed = repo if not self.archive else '%s::%s' % (repo, self.archive)
         return valid
 
     def _parse(self, text):
@@ -1234,15 +1236,16 @@ def canonical_path(self):
                                            path)
 
     def with_timestamp(self, timestamp):
-        return Location(self.orig, overrides={
+        return Location(self.raw, overrides={
             'now': DatetimeWrapper(timestamp.astimezone(None)),
             'utcnow': DatetimeWrapper(timestamp),
         })
 
     def omit_archive(self):
-        loc = Location(self.orig)
+        loc = Location(self.raw)
         loc.archive = None
-        loc.orig = loc.orig.split("::")[0]
+        loc.raw = loc.raw.split("::")[0]
+        loc.processed = loc.processed.split("::")[0]
         return loc
 
 
diff --git a/src/borg/remote.py b/src/borg/remote.py
index 64d13331..56ef54c2 100644
--- a/src/borg/remote.py
+++ b/src/borg/remote.py
@@ -728,11 +728,11 @@ def handle_error(unpacked):
             args = unpacked.get(b'exception_args')
 
             if error == 'DoesNotExist':
-                raise Repository.DoesNotExist(self.location.orig)
+                raise Repository.DoesNotExist(self.location.processed)
             elif error == 'AlreadyExists':
-                raise Repository.AlreadyExists(self.location.orig)
+                raise Repository.AlreadyExists(self.location.processed)
             elif error == 'CheckNeeded':
-                raise Repository.CheckNeeded(self.location.orig)
+                raise Repository.CheckNeeded(self.location.processed)
             elif error == 'IntegrityError':
                 if old_server:
                     raise IntegrityError('(not available)')
@@ -752,9 +752,9 @@ def handle_error(unpacked):
                 raise Repository.ParentPathDoesNotExist(args[0].decode())
             elif error == 'ObjectNotFound':
                 if old_server:
-                    raise Repository.ObjectNotFound('(not available)', self.location.orig)
+                    raise Repository.ObjectNotFound('(not available)', self.location.processed)
                 else:
-                    raise Repository.ObjectNotFound(args[0].decode(), self.location.orig)
+                    raise Repository.ObjectNotFound(args[0].decode(), self.location.processed)
             elif error == 'InvalidRPCMethod':
                 if old_server:
                     raise InvalidRPCMethod('(not available)')


Test patch:
diff --git a/src/borg/testsuite/helpers.py b/src/borg/testsuite/helpers.py
index 04331c55..404f92bf 100644
--- a/src/borg/testsuite/helpers.py
+++ b/src/borg/testsuite/helpers.py
@@ -232,10 +232,12 @@ def test_bad_syntax(self):
             Location('ssh://user@host:/path')
 
     def test_omit_archive(self):
-        loc = Location('ssh://user@host:1234/some/path::archive')
+        from borg.platform import hostname
+        loc = Location('ssh://user@host:1234/repos/{hostname}::archive')
         loc_without_archive = loc.omit_archive()
         assert loc_without_archive.archive is None
-        assert loc_without_archive.orig == "ssh://user@host:1234/some/path"
+        assert loc_without_archive.raw == "ssh://user@host:1234/repos/{hostname}"
+        assert loc_without_archive.processed == "ssh://user@host:1234/repos/%s" % hostname
 
 
 class TestLocationWithEnv:
@@ -248,6 +250,16 @@ def test_ssh(self, monkeypatch):
         assert repr(Location()) == \
                "Location(proto='ssh', user='user', host='host', port=1234, path='/some/path', archive=None)"
 
+    def test_ssh_placeholder(self, monkeypatch):
+        from borg.platform import hostname
+        monkeypatch.setenv('BORG_REPO', 'ssh://user@host:1234/{hostname}')
+        assert repr(Location('::archive')) == \
+            "Location(proto='ssh', user='user', host='host', port=1234, path='/{}', archive='archive')".format(hostname)
+        assert repr(Location('::')) == \
+            "Location(proto='ssh', user='user', host='host', port=1234, path='/{}', archive=None)".format(hostname)
+        assert repr(Location()) == \
+            "Location(proto='ssh', user='user', host='host', port=1234, path='/{}', archive=None)".format(hostname)
+
     def test_file(self, monkeypatch):
         monkeypatch.setenv('BORG_REPO', 'file:///some/path')
         assert repr(Location('::archive')) == \
diff --git a/src/borg/testsuite/key.py b/src/borg/testsuite/key.py
index c37447f6..9a7e597f 100644
--- a/src/borg/testsuite/key.py
+++ b/src/borg/testsuite/key.py
@@ -85,10 +85,10 @@ def key(self, request, monkeypatch):
 
     class MockRepository:
         class _Location:
-            orig = '/some/place'
+            raw = processed = '/some/place'
 
             def canonical_path(self):
-                return self.orig
+                return self.processed
 
         _location = _Location()
         id = bytes(32)
diff --git a/src/borg/testsuite/repository.py b/src/borg/testsuite/repository.py
index 26a188d9..3018fd8f 100644
--- a/src/borg/testsuite/repository.py
+++ b/src/borg/testsuite/repository.py
@@ -864,19 +864,19 @@ def test_rpc_exception_transport(self):
             self.repository.call('inject_exception', {'kind': 'DoesNotExist'})
         except Repository.DoesNotExist as e:
             assert len(e.args) == 1
-            assert e.args[0] == self.repository.location.orig
+            assert e.args[0] == self.repository.location.processed
 
         try:
             self.repository.call('inject_exception', {'kind': 'AlreadyExists'})
         except Repository.AlreadyExists as e:
             assert len(e.args) == 1
-            assert e.args[0] == self.repository.location.orig
+            assert e.args[0] == self.repository.location.processed
 
         try:
             self.repository.call('inject_exception', {'kind': 'CheckNeeded'})
         except Repository.CheckNeeded as e:
             assert len(e.args) == 1
-            assert e.args[0] == self.repository.location.orig
+            assert e.args[0] == self.repository.location.processed
 
         try:
             self.repository.call('inject_exception', {'kind': 'IntegrityError'})
@@ -895,7 +895,7 @@ def test_rpc_exception_transport(self):
         except Repository.ObjectNotFound as e:
             assert len(e.args) == 2
             assert e.args[0] == s1
-            assert e.args[1] == self.repository.location.orig
+            assert e.args[1] == self.repository.location.processed
 
         try:
             self.repository.call('inject_exception', {'kind': 'InvalidRPCMethod'})