Repository: freedomofpress/securedrop

Issue:
Restoring from a v2 backup on an instance with v3 enabled breaks v3 onion services
## Description

When a v2 backup is used to restore an instance that has v3 services enabled (either v3 only or v2+v3), the v3 onion services are no longer enabled on the app server. 

## Steps to Reproduce

on prod vms or hardware:

- install a v2-only instance
- take a backup
- use `sdconfig` and `install` to enable v3
- restore from the v2backup 

## Expected Behavior
- data is restored, v3 services are still accessible

## Actual Behavior
- restore hangs when tor is restarted on the server, v3 services are no longer configured on the app server

## Comments
The backup tarball includes /etc/tor/torrc, which contains the onion service definitions. When it overwrites the v3 torrc, it breaks the services.

One approach to address this might be to break the onion service definitions out into files that are included in `torrc` via  %include. For example,a structure like:
```
/etc/tor/torrc.d/
                    10-v2-definitions.conf
                    20-v3-definitions.conf
```
and a line like:
```
%include /etc/tor/torrc.d
```
in the `torrc` proper.

Restoring from a v2 tarball would overwrite `/etc/tor/torrc.d/10-v2-definitions.conf` but shouldn't affect other files in the same directory.




Discussion:
Noting for the record: For 1.0.0, we've agreed that graceful and non-destructive failure is acceptable in this scenario. (Graceful: ideally include suggested remediation steps the admin can take, e.g., downgrade to v2 before restoring backup; non-destructive: fail early before any modifications to the system state.)

Resolving patch:
diff --git a/admin/securedrop_admin/__init__.py b/admin/securedrop_admin/__init__.py
--- a/admin/securedrop_admin/__init__.py
+++ b/admin/securedrop_admin/__init__.py
@@ -736,6 +736,10 @@ def restore_securedrop(args):
     # but the securedrop-admin
     # script will be invoked from the repo root, so preceding dirs are likely.
     restore_file_basename = os.path.basename(args.restore_file)
+
+    # Would like readable output if there's a problem
+    os.environ["ANSIBLE_STDOUT_CALLBACK"] = "debug"
+
     ansible_cmd = [
         'ansible-playbook',
         os.path.join(args.ansible_path, 'securedrop-restore.yml'),
diff --git a/install_files/ansible-base/roles/restore/files/compare_torrc.py b/install_files/ansible-base/roles/restore/files/compare_torrc.py
new file mode 100644
--- /dev/null
+++ b/install_files/ansible-base/roles/restore/files/compare_torrc.py
@@ -0,0 +1,69 @@
+#!/usr/bin/env python
+
+#
+# Compares Tor configurations on the app server and from a backup. If
+# restoring the backup would alter the server's Tor configuration,
+# print a warning and exit.
+#
+
+from __future__ import print_function
+
+import os
+import re
+import sys
+
+
+def get_tor_versions(path):
+    """
+    Determine which service versions are offered in the given torrc.
+    """
+    service_re = re.compile(r"HiddenServiceDir\s+(?:.*)/(.*)")
+    versions = set([])
+    with open(path) as f:
+        for line in f:
+            m = service_re.match(line)
+            if m:
+                service = m.group(1)
+                if "v3" in service:
+                    versions.add(3)
+                else:
+                    versions.add(2)
+
+    return versions
+
+
+def strset(s):
+    """
+    Sort the given set and join members with "and".
+    """
+    return " and ".join(str(v) for v in sorted(s))
+
+
+if __name__ == "__main__":
+    tempdir = sys.argv[1]
+
+    server_versions = get_tor_versions(os.path.join(tempdir, "app/etc/tor/torrc"))
+    backup_versions = get_tor_versions(os.path.join(tempdir, "backup/etc/tor/torrc"))
+
+    if server_versions == backup_versions:
+        print("The Tor configuration in the backup matches the server.")
+        sys.exit(0)
+
+    print(
+        "The Tor configuration on the app server offers version {} services.".format(
+            strset(server_versions)
+        )
+    )
+
+    print(
+        "The Tor configuration in this backup offers version {} services.".format(
+            strset(backup_versions)
+        )
+    )
+
+    print("\nRestoring a backup with a different Tor configuration than the server ")
+    print("is currently unsupported. If you require technical assistance, please ")
+    print("contact the SecureDrop team via the support portal or at ")
+    print("securedrop@freedom.press.")
+
+    sys.exit(1)
diff --git a/install_files/ansible-base/roles/restore/files/restore.py b/install_files/ansible-base/roles/restore/files/restore.py
deleted file mode 100755
--- a/install_files/ansible-base/roles/restore/files/restore.py
+++ /dev/null
@@ -1,57 +0,0 @@
-#!/opt/venvs/securedrop-app-code/bin/python
-"""
-This script and backup archive should be copied to the App server and run by
-the Ansible playbook. When run (as root), it restores the contents of the 0.3
-backup file to the machine it's run on.
-
-python restore.py sd-backup-TIMESTAMP.tar.gz
-"""
-
-import os
-import subprocess
-import sys
-import tarfile
-
-
-def verify_args():
-    usage = """
-Usage: restore.py <backup file>
-
-    <backup file>  Path to a SecureDrop 0.3 backup created by backup.py"
-    """
-    if len(sys.argv) != 2:
-        print(usage)
-        sys.exit(1)
-
-    if not os.path.exists(sys.argv[1]):
-        print("<backup file> '{}' not found".format(sys.argv[1]))
-        sys.exit(1)
-
-    if os.geteuid() != 0:
-        print("This program must be run as root!")
-        sys.exit(1)
-
-
-def main():
-    verify_args()
-
-    with tarfile.open(sys.argv[1], 'r:*') as backup:
-        # This assumes that both the old installation (source of the backup)
-        # and the new installation (destination of the restore) used the
-        # default paths for various locations.
-        backup.extractall(path='/')
-
-    # Apply database migrations (if backed-up version < version to restore)
-    subprocess.check_call(['dpkg-reconfigure', 'securedrop-app-code'])
-
-    # Update the configs
-    subprocess.check_call(['dpkg-reconfigure', 'securedrop-config'])
-
-    # Reload Tor and the web server so they pick up the new configuration
-    # If the process exits with a non-zero return code, raises an exception.
-    subprocess.check_call(['service', 'apache2', 'restart'])
-    subprocess.check_call(['service', 'tor', 'reload'])
-
-
-if __name__ == "__main__":
-    main()


Test patch: