Repository: Archmonger/django-dbbackup

Issue:
-O flag does not respect S3 URIs
### Current Situation

I have a Django project with dbbackup backed by S3. This is configured in my settings.py. When I am trying to do one-off backups, I want to run the command `python manage.py dbbackup -O s3://meshdb-backups/dev/`. However, I am getting an exception when I try to run that command:

```
INFO:dbbackup.command:Writing file to s3://meshdb-data-backups/meshdb-backups/dev/
ERROR:dbbackup:FileNotFoundError: [Errno 2] No such file or directory: 's3://meshdb-data-backups/meshdb-backups/dev/'
  File "/usr/local/lib/python3.11/site-packages/dbbackup/utils.py", line 116, in wrapper
    func(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/dbbackup/management/commands/dbbackup.py", line 100, in handle
    self._save_new_backup(database)
  File "/usr/local/lib/python3.11/site-packages/dbbackup/management/commands/dbbackup.py", line 142, in _save_new_backup
    self.write_local_file(outputfile, self.path)
  File "/usr/local/lib/python3.11/site-packages/dbbackup/management/commands/_base.py", line 108, in write_local_file
    with open(path, "wb") as fd:
         ^^^^^^^^^^^^^^^^
  File "/datadog-lib/ddtrace_pkgs/site-packages-ddtrace-py3.11-manylinux2014/ddtrace/appsec/_common_module_patches.py", line 85, in wrapped_open_CFDDB7ABBA9081B6
    raise e.with_traceback(
  File "/usr/local/lib/python3.11/site-packages/dbbackup/management/commands/_base.py", line 108, in write_local_file
    with open(path, "wb") as fd:
         ^^^^^^^^^^^^^^^^

Traceback (most recent call last):
  File "/opt/meshdb/manage.py", line 26, in <module>
    main()
  File "/opt/meshdb/manage.py", line 22, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dbbackup/utils.py", line 116, in wrapper
    func(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/dbbackup/management/commands/dbbackup.py", line 100, in handle
    self._save_new_backup(database)
  File "/usr/local/lib/python3.11/site-packages/dbbackup/management/commands/dbbackup.py", line 142, in _save_new_backup
    self.write_local_file(outputfile, self.path)
  File "/usr/local/lib/python3.11/site-packages/dbbackup/management/commands/_base.py", line 108, in write_local_file
    with open(path, "wb") as fd:
         ^^^^^^^^^^^^^^^^
  File "/datadog-lib/ddtrace_pkgs/site-packages-ddtrace-py3.11-manylinux2014/ddtrace/appsec/_common_module_patches.py", line 85, in wrapped_open_CFDDB7ABBA9081B6
    raise e.with_traceback(
  File "/usr/local/lib/python3.11/site-packages/dbbackup/management/commands/_base.py", line 108, in write_local_file
    with open(path, "wb") as fd:
         ^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 's3://meshdb-data-backups/meshdb-backups/dev/'
```

### Proposed Actions

Ensure that the `-O` flag understands S3.

### System Information

DBBackup version: https://github.com/jazzband/django-dbbackup/commit/3ccac962809214719cf1019a2189788f8e6d1a27

Django: "django==4.2.*",

Python: python:3.11-bookworm




Discussion:
@WillNilges did you manage to fix this? If so, how ?



Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6d64f7b2..ca112b8b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -32,6 +32,7 @@ Don't forget to remove deprecated code on each major release!
 
 ### Fixed
 
+- Fixed `-O` flag to properly handle S3 URIs. Now `python manage.py dbbackup -O s3://bucket/path/` and `python manage.py mediabackup -O s3://bucket/path/` correctly route S3 URIs to the storage backend instead of attempting to write to local filesystem.
 - Fix issues with parsing excess whitespace within `dbbackup -d "<COMMA_SEPARATED_ARGS>"`
 - Fix encryption support when using `gnupg==5.x`.
 - Resolve SQLite backup temporary file locking issues on Windows.
diff --git a/dbbackup/management/commands/dbbackup.py b/dbbackup/management/commands/dbbackup.py
index 34173b10..9a710088 100644
--- a/dbbackup/management/commands/dbbackup.py
+++ b/dbbackup/management/commands/dbbackup.py
@@ -52,7 +52,7 @@ class Command(BaseDbBackupCommand):
             "-O",
             "--output-path",
             default=None,
-            help="Specify where to store on local filesystem",
+            help="Specify where to store backup (local filesystem path or S3 URI like s3://bucket/path/)",
         ),
         make_option("-x", "--exclude-tables", default=None, help="Exclude tables from backup"),
         make_option(
@@ -141,6 +141,8 @@ def _save_new_backup(self, database):
 
         if self.path is None:
             self.write_to_storage(outputfile, filename)
-
+        elif self.path.startswith("s3://"):
+            # Handle S3 URIs through storage backend
+            self.write_to_storage(outputfile, self.path)
         else:
             self.write_local_file(outputfile, self.path)
diff --git a/dbbackup/management/commands/mediabackup.py b/dbbackup/management/commands/mediabackup.py
index ea6fdd54..20e1919f 100644
--- a/dbbackup/management/commands/mediabackup.py
+++ b/dbbackup/management/commands/mediabackup.py
@@ -49,7 +49,7 @@ class Command(BaseDbBackupCommand):
             "-O",
             "--output-path",
             default=None,
-            help="Specify where to store on local filesystem",
+            help="Specify where to store backup (local filesystem path or S3 URI like s3://bucket/path/)",
         ),
     )
 
@@ -122,5 +122,8 @@ def backup_mediafiles(self):
         tarball.seek(0)
         if self.path is None:
             self.write_to_storage(tarball, filename)
+        elif self.path.startswith("s3://"):
+            # Handle S3 URIs through storage backend
+            self.write_to_storage(tarball, self.path)
         else:
             self.write_local_file(tarball, self.path)


Test patch:
diff --git a/tests/commands/test_dbbackup.py b/tests/commands/test_dbbackup.py
index 5ce30529..253ba70e 100644
--- a/tests/commands/test_dbbackup.py
+++ b/tests/commands/test_dbbackup.py
@@ -64,6 +64,73 @@ def test_schema(self):
 
         self.assertIsNone(result)
 
+    @patch("dbbackup.management.commands._base.BaseDbBackupCommand.write_to_storage")
+    def test_path_s3_uri(self, mock_write_to_storage):
+        """Test that S3 URIs in output path are handled by write_to_storage instead of write_local_file."""
+        self.command.path = "s3://mybucket/backups/db.bak"
+        self.command._save_new_backup(TEST_DATABASE)
+        self.assertTrue(mock_write_to_storage.called)
+        # Verify the S3 path was passed correctly to write_to_storage
+        args, kwargs = mock_write_to_storage.call_args
+        self.assertEqual(args[1], "s3://mybucket/backups/db.bak")
+
+    @patch("dbbackup.management.commands._base.BaseDbBackupCommand.write_to_storage")
+    def test_path_s3_uri_variants(self, mock_write_to_storage):
+        """Test various S3 URI formats."""
+        test_cases = [
+            "s3://bucket/file.bak",
+            "s3://bucket/folder/file.bak", 
+            "s3://bucket-with-dashes/nested/folders/file.bak",
+            "s3://bucket/path/without/trailing/slash",
+        ]
+        
+        for s3_uri in test_cases:
+            with self.subTest(s3_uri=s3_uri):
+                mock_write_to_storage.reset_mock()
+                self.command.path = s3_uri
+                self.command._save_new_backup(TEST_DATABASE)
+                self.assertTrue(mock_write_to_storage.called)
+                args, kwargs = mock_write_to_storage.call_args
+                self.assertEqual(args[1], s3_uri)
+
+    def test_path_local_file_still_works(self):
+        """Test that regular local file paths still use write_local_file."""
+        local_tmp = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "..", "tmp")
+        os.makedirs(local_tmp, exist_ok=True)
+        
+        # Test with a real local path that should work
+        local_path = os.path.join(local_tmp, "test_local.bak")
+        self.command.path = local_path
+        self.command._save_new_backup(TEST_DATABASE)
+        
+        # Verify the file was created (meaning write_local_file was used)
+        self.assertTrue(os.path.exists(local_path))
+        
+        # Cleanup
+        os.remove(local_path)
+        
+        # Test that paths containing 's3' but not starting with 's3://' are treated as local
+        with patch("dbbackup.management.commands._base.BaseDbBackupCommand.write_local_file") as mock_write_local_file:
+            mock_write_local_file.side_effect = FileNotFoundError("Mocked error")
+            
+            test_cases = [
+                "/path/with/s3/in/name/backup.bak",
+                "s3_backup.bak",  # starts with s3 but not s3://
+                "bucket-s3-backup.bak",
+            ]
+            
+            for local_path in test_cases:
+                with self.subTest(local_path=local_path):
+                    mock_write_local_file.reset_mock()
+                    self.command.path = local_path
+                    # This should call write_local_file (and raise our mocked error)
+                    with self.assertRaises(FileNotFoundError):
+                        self.command._save_new_backup(TEST_DATABASE)
+                    # Verify write_local_file was called
+                    self.assertTrue(mock_write_local_file.called)
+                    args, kwargs = mock_write_local_file.call_args
+                    self.assertEqual(args[1], local_path)
+
     @patch("dbbackup.settings.DATABASES", ["db-from-settings"])
     def test_get_database_keys(self):
         with self.subTest("use --database from CLI"):
diff --git a/tests/commands/test_mediabackup.py b/tests/commands/test_mediabackup.py
index e4f34a46..dad6aff3 100644
--- a/tests/commands/test_mediabackup.py
+++ b/tests/commands/test_mediabackup.py
@@ -77,3 +77,19 @@ def test_output_filename(self):
         self.command.filename = "my_new_name.tar"
         self.command.backup_mediafiles()
         self.assertEqual(HANDLED_FILES["written_files"][0][0], self.command.filename)
+
+    def test_s3_uri_output_path(self):
+        """Test that S3 URIs in output path are handled correctly for mediabackup."""
+        from unittest.mock import patch
+        
+        with patch("dbbackup.management.commands._base.BaseDbBackupCommand.write_to_storage") as mock_write_to_storage:
+            self.command.path = "s3://mybucket/media/backup.tar"
+            self.command.backup_mediafiles()
+            
+            # Verify write_to_storage was called with the S3 path
+            self.assertTrue(mock_write_to_storage.called)
+            args, kwargs = mock_write_to_storage.call_args
+            self.assertEqual(args[1], "s3://mybucket/media/backup.tar")
+            
+            # Verify no files were written to local storage
+            self.assertEqual(0, len(HANDLED_FILES["written_files"]))