Repository: AgnostiqHQ/covalent
Pull request #1777 (2023-09-14 12:49:31)

Fixed rsync command to include files recursively
<!--
⚠️ Your pull request title should be short detailed and understandable for all.
⚠️ Add a note to /CHANGELOG.md summarizing the changes.
⚠️ If your pull request fixes an open issue, please link to the issue.
⚠️ Ensure that your branch is up-to-date with the base branch.
-->

- [x] I have added the tests to cover my changes.
- [x] I have updated the documentation and CHANGELOG accordingly.
- [x] I have read the CONTRIBUTING document.



Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7d6c81e3..17c8b2c7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - Formatted executor block under Qelectron job details to handle any class-type values
 - Fixed test-cases to handle latest pydantic version changes
+- Rsync command fixed to recursively copy files when using SSH
+- Removed accidentally added migrations build files
 
 ### Changed
 
diff --git a/build/lib/covalent_migrations/README.md b/build/lib/covalent_migrations/README.md
deleted file mode 100644
index d9acaf83..00000000
--- a/build/lib/covalent_migrations/README.md
+++ /dev/null
@@ -1,53 +0,0 @@
-# DB Migration Guide
-
-## Getting Started & Running Migrations
-
-Whenever merging other branches into your working branch always remember to run migrations as the database schema may have changed.
-
-In order to update the DB to reflect the most up to date DB models we run the migrations (that have not yet run) as such:
-
-```bash
-covalent db alembic upgrade head
-```
-
-To see history of which migrations we have run we can execute
-
-```bash
-covalent db alembic history
-```
-
-## Autogenerate Migrations
-
-To generate new migrations as a result of editing our DB model files, we can run the following
-
-```bash
-covalent db alembic revision --autogenerate -m "Description of DB update"
-```
-
-
-You should see a new python file generated under `alembic/versions`. Also, there should be a table name in the database called `alembic_version` which will keep track of which migrations have run corresponding to the filenames in `alembic/versions`.
-
-> The python files in `alembic/versions` contain an `upgrade()` method which will sync our database schema with our DB models `covalent/_data_store/models.py`. Furthermore, it contains a `downgrade()` command which will undo the operations performed by `upgrade()`. We do not explicitly call these methods, this is done by the alembic cli.
-
-
-## Generate Template Migration
-
-To create a migration file which will be edited manually, one can run:
-
-```bash
-covalent db alembic revision -m "my custom migration file"
-```
-
-## Undo Migrations
-
-To undo the last migration we can run
-
-```bash
-covalent db alembic downgrade -1
-```
-
-Alternatively we can specify the exact version which we want to revert to (we can use `alembic history` for this)
-
-```bash
-covalent db alembic downgrade 1ed41b6d3f3f
-```
diff --git a/build/lib/covalent_migrations/alembic.ini b/build/lib/covalent_migrations/alembic.ini
deleted file mode 100644
index 681429ed..00000000
--- a/build/lib/covalent_migrations/alembic.ini
+++ /dev/null
@@ -1,106 +0,0 @@
-# A generic, single database configuration.
-
-[alembic]
-# path to migration scripts
-script_location = covalent_migrations
-
-# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
-# Uncomment the line below if you want the files to be prepended with date and time
-# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
-# for all available tokens
-# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
-
-# sys.path path, will be prepended to sys.path if present.
-# defaults to the current working directory.
-prepend_sys_path = .
-
-# timezone to use when rendering the date within the migration file
-# as well as the filename.
-# If specified, requires the python-dateutil library that can be
-# installed by adding `alembic[tz]` to the pip requirements
-# string value is passed to dateutil.tz.gettz()
-# leave blank for localtime
-# timezone =
-
-# max length of characters to apply to the
-# "slug" field
-# truncate_slug_length = 40
-
-# set to 'true' to run the environment during
-# the 'revision' command, regardless of autogenerate
-# revision_environment = false
-
-# set to 'true' to allow .pyc and .pyo files without
-# a source .py file to be detected as revisions in the
-# versions/ directory
-# sourceless = false
-
-# version location specification; This defaults
-# to alembic/versions.  When using multiple version
-# directories, initial revisions must be specified with --version-path.
-# The path separator used here should be the separator specified by "version_path_separator" below.
-# version_locations = %(here)s/bar:%(here)s/bat:alembic/versions
-
-# version path separator; As mentioned above, this is the character used to split
-# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
-# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
-# Valid values for version_path_separator are:
-#
-# version_path_separator = :
-# version_path_separator = ;
-# version_path_separator = space
-version_path_separator = os  # Use os.pathsep. Default configuration used for new projects.
-
-# the output encoding used when revision files
-# are written from script.py.mako
-# output_encoding = utf-8
-
-# pragma: allowlist nextline secret
-# sqlalchemy.url = driver://user:pass@localhost/dbname
-
-
-[post_write_hooks]
-# post_write_hooks defines scripts or Python functions that are run
-# on newly generated revision scripts.  See the documentation for further
-# detail and examples
-
-# format using "black" - use the console_scripts runner, against the "black" entrypoint
-# hooks = black
-# black.type = console_scripts
-# black.entrypoint = black
-# black.options = -l 79 REVISION_SCRIPT_FILENAME
-
-# Logging configuration
-[loggers]
-keys = root,sqlalchemy,alembic
-
-[handlers]
-keys = console
-
-[formatters]
-keys = generic
-
-[logger_root]
-level = WARN
-handlers = console
-qualname =
-
-[logger_sqlalchemy]
-level = WARN
-handlers =
-qualname = sqlalchemy.engine
-
-[logger_alembic]
-level = INFO
-handlers =
-qualname = alembic
-
-[handler_console]
-class = StreamHandler
-args = (sys.stderr,)
-level = NOTSET
-formatter = generic
-
-[formatter_generic]
-format = %(levelname)-5.5s [%(name)s] %(message)s
-datefmt = %H:%M:%S
diff --git a/covalent/_file_transfer/strategies/rsync_strategy.py b/covalent/_file_transfer/strategies/rsync_strategy.py
index f445260c..4ff63132 100644
--- a/covalent/_file_transfer/strategies/rsync_strategy.py
+++ b/covalent/_file_transfer/strategies/rsync_strategy.py
@@ -58,9 +58,9 @@ class Rsync(FileTransferStrategy):
         remote_filepath = remote_file.filepath
         args = ["rsync"]
         if self.private_key_path:
-            args.append(f'-e "ssh -i {self.private_key_path}"')
+            args.append(f'-ae "ssh -i {self.private_key_path}"')
         else:
-            args.append("-e ssh")
+            args.append("-ae ssh")
 
         remote_source = f"{self.user}@{self.host}:{remote_filepath}"
 


Test patch:
diff --git a/tests/covalent_tests/file_transfer/strategies/rsync_strategy_test.py b/tests/covalent_tests/file_transfer/strategies/rsync_strategy_test.py
index d1cb035c..9bdd88e9 100644
--- a/tests/covalent_tests/file_transfer/strategies/rsync_strategy_test.py
+++ b/tests/covalent_tests/file_transfer/strategies/rsync_strategy_test.py
@@ -141,18 +141,18 @@ class TestRsyncStrategy:
         # commands take the form of rsync -s ssh ... [source] [destination]
         assert (
             upload_cmd_with_key
-            == f'rsync -e "ssh -i {private_key_path}" {self.MOCK_LOCAL_FILEPATH} {self.MOCK_USER}@{self.MOCK_HOST}:{self.MOCK_REMOTE_FILEPATH}'
+            == f'rsync -ae "ssh -i {private_key_path}" {self.MOCK_LOCAL_FILEPATH} {self.MOCK_USER}@{self.MOCK_HOST}:{self.MOCK_REMOTE_FILEPATH}'
         )
         assert (
             download_cmd_with_key
-            == f'rsync -e "ssh -i {private_key_path}" {self.MOCK_USER}@{self.MOCK_HOST}:{self.MOCK_REMOTE_FILEPATH} {self.MOCK_LOCAL_FILEPATH}'
+            == f'rsync -ae "ssh -i {private_key_path}" {self.MOCK_USER}@{self.MOCK_HOST}:{self.MOCK_REMOTE_FILEPATH} {self.MOCK_LOCAL_FILEPATH}'
         )
 
         assert (
             upload_cmd_without_key
-            == f"rsync -e ssh {self.MOCK_LOCAL_FILEPATH} {self.MOCK_USER}@{self.MOCK_HOST}:{self.MOCK_REMOTE_FILEPATH}"
+            == f"rsync -ae ssh {self.MOCK_LOCAL_FILEPATH} {self.MOCK_USER}@{self.MOCK_HOST}:{self.MOCK_REMOTE_FILEPATH}"
         )
         assert (
             download_cmd_without_key
-            == f"rsync -e ssh {self.MOCK_USER}@{self.MOCK_HOST}:{self.MOCK_REMOTE_FILEPATH} {self.MOCK_LOCAL_FILEPATH}"
+            == f"rsync -ae ssh {self.MOCK_USER}@{self.MOCK_HOST}:{self.MOCK_REMOTE_FILEPATH} {self.MOCK_LOCAL_FILEPATH}"
         )