Repository: ScilifelabDataCentre/dds_web
Pull request #1591: Implement Redis Queue for Async Requests & Project Deletion
Author: login: rv0lt
Labels: name: backend
name: type: feature
description: For release template
Created: 2025-02-04T10:08:36Z | Merged: 2025-03-24T11:51:54Z

## Read this before submitting the PR

1. Always create a Draft PR first
2. Go through sections 1-5 below, fill them in and check all the boxes
3. Make sure that the branch is updated; if there's an "Update branch" button at the bottom of the PR, rebase or update branch.
4. When all boxes are checked, information is filled in, and the branch is updated: mark as Ready For Review and tag reviewers (top right)
5. Once there is a submitted review, implement the suggestions (if reasonable, otherwise discuss) and request an new review.

If there is a field which you are unsure about, enter the edit mode of this description or go to the [PR template](../.github/pull_request_template.md); There are invisible comments providing descriptions which may be of help.

## 1. Description / Summary


This PR contains a first implementation for a Redis Queue to process some of the requests.
 - There is a new dashboard where you can check the status of the requests at **http://localhost:5000/rq/**
 - For tests, there is a new fixture to bypass the queue and execute the function. New tests for endpoints that use the queues can invoke it
 - The implementation of delete projects contents has been done as POC
 - For motd there is a template and suggestions to keep working, but it needs a larger change due to how rq handles the jobs


**To test the PR**:

Run the containers in full-dev mode 
`docker-compose --profile full-dev up`

- Log in the client container
Authenticate with any Unit Admin or Unit Personal.
Create a new project and upload some data
Remove all the data
`dds data rm -p someunit00001 -a`

- Check the dashboard at **http://localhost:5000/rq/** to see the status of the job, when it is finished it means it is done
- Check the minio at **http://localhost:9001/buckets** to ensure that all files have been deleted

## 2. Jira task / GitHub issue

[ Jira task ID](https://scilifelab.atlassian.net/jira/software/projects/HMS/boards/13?selectedIssue=HMS-2297&sprintStarted=true)
## 3. Type of change - Add label

**What _type of change(s)_ does the PR contain? For an explanation of the different options below, enter edit mode of this PR description template.**

_If you do not want this change to be included in release notes, add the label `skip-changelog`._

- New feature
  - Breaking --> label: `breaking` <!-- If the changes in this PR will cause existing functionality to not work as expected. E.g. with the master branch of the `dds_cli`. Add info here on how the change is breaking. -->
  - Non-breaking --> label: `feature` <!-- If the changes will not cause existing functionality to fail. "Non-breaking" is just an addition of a new feature. -->
- Database change --> label: `feature` or none at all.
  _Remember the to include a new migration version, **or** explain here why it's not needed._ <!-- If you've changed something in `models.py`. For a guide on how to add the a new migration version, look at the "Database changes" section in the README.md. -->
- Bug fix --> label: `bug` <!-- If a bug is fixed in existing functionality. If the bug fix also is a breaking change (see above), add info about that beside this check box. -->
- Security Alert fix <!-- If the PR attempts to solve a security vulnerability, e.g. reported by the "Security" tab in the repo. -->
  - Package update --> label: `dependency` <!-- If the Security alert fix consists of updating a package / dependency version -->
    - Major version update <!-- If the package / dependency version update is a major upgrade, e.g. 1.0.0 to 2.0.0 -->
- Documentation --> label can be skipped, will be included in "other changes" <!-- If the PR adds or updates documentation such as e.g. Technical Overview or a architecture decision (dds_web/doc/architecture/decisions.) -->
- Workflow --> label: `skip-changelog` <!-- If the PR includes a change in e.g. the github actions files (dds_web/.github/*) or another type of workflow change. Anything that alters our or the codes workflow. -->
- Tests **only** --> label: `skip-changelog` <!-- If the PR only contains tests, none of the other types of changes listed above. -->

## 4. Additional information

- [x] I have added an entry to the [Sprintlog](../SPRINTLOG.md) <!-- Add a row at the bottom of the SPRINTLOG.md file (not needed if PR contains only tests). Follow the format of previous rows. If the PR is the first in a new sprint, add a new sprint header row (follow the format of previous sprints). -->
- [ ] This is a PR to the `master` branch: _If checked, read [the release instructions](../doc/procedures/new_release.md)_ <!-- Check this if the PR is made to the `master` branch. Only the `dev` branch should be doing this. -->
  - [ ] I have followed steps 1-8. <!-- Should be checked if the "PR to `master` branch" box is checked AND the specified steps in the release instructions have been followed. -->

## 5. Actions / Scans

**Make sure that the following checks/actions have passed.**

- **Black**
<!--
  What: Python code formatter.
  How to fix: Run `black .` locally to execute formatting.
-->
- **Prettier**
<!--
  What: General code formatter. Our use case: MD and yaml mainly.
  How to fix: Run npx prettier --write . locally to execute formatting.
-->
- **Yamllint**
<!--
  What: Linting of yaml files.
  How to fix: Manually fix any errors locally.
-->
- **Tests**
<!--
  What: Pytest to verify that functionality works as expected.
  How to fix: Manually fix any errors locally. Follow the instructions in the "Run tests" section of the README.md to run the tests locally.
  Additional info: The PR should ALWAYS include new tests or fixed tests when there are code changes. When pytest action has finished, it will post a codecov report; Look at this report and verify the files you have changed are listed. "90% <100.00%> (+0.8%)" means "Tests cover 90% of the changed file, <100 % of this PR's code changes are tested>, and (the code changes and added tests increased the overall test coverage with 0.8%)
-->
- **CodeQL**
<!--
  What: Scan for security vulnerabilities, bugs, errors.
  How to fix: Go through the alerts and either manually fix, dismiss or ignore. Add info on ignored or dismissed alerts.
-->
- **Trivy**
<!--
  What: Security scanner.
  How to fix: Go through the alerts and either manually fix, dismiss or ignore. Add info on ignored or dismissed alerts.
-->
- **Snyk**
<!--
  What: Security scanner.
  How to fix: Go through the alerts and either manually fix, dismiss or ignore. Add info on ignored or dismissed alerts.
-->

If an action does not pass and you need help with how to solve it, enter edit mode of this PR template or go to the [PR template](../.github/pull_request_template.md).



Diff:
diff --git a/SPRINTLOG.md b/SPRINTLOG.md
index b591434a6..9575e9913 100644
--- a/SPRINTLOG.md
+++ b/SPRINTLOG.md
@@ -478,3 +478,7 @@ _Nothing merged during this sprint_
 
 - Make release template ([#1587](https://github.com/ScilifelabDataCentre/dds_web/pull/1587))
 - Fix codecov action ([#1589](https://github.com/ScilifelabDataCentre/dds_web/pull/1589))
+
+# 2024-02-03 - 2025-02-14
+
+- Implement Redis Queue to process some requests asynschronusly and avoid timeouts. Set project deletion as a background task. ([#1591](https://github.com/ScilifelabDataCentre/dds_web/pull/1591))
diff --git a/dds_web/__init__.py b/dds_web/__init__.py
index 55d7e51cb..d36eebdb5 100644
--- a/dds_web/__init__.py
+++ b/dds_web/__init__.py
@@ -9,6 +9,7 @@
 import pathlib
 import sys
 import os
+import multiprocessing
 
 # Installed
 import flask
@@ -20,7 +21,9 @@
 import flask_mail
 import flask_login
 import flask_migrate
-
+import rq_dashboard
+from redis import Redis
+from rq import Worker
 
 # import flask_qrcode
 from werkzeug.middleware.proxy_fix import ProxyFix
@@ -198,6 +201,10 @@ def create_app(testing=False, database_uri=None):
         # User config file, if e.g. using in production
         app.config.from_envvar("DDS_APP_CONFIG", silent=True)
 
+        # redis dashboard
+        app.config.from_object(rq_dashboard.default_settings)
+        rq_dashboard.web.setup_rq_connection(app)
+
         # Test related configs
         if database_uri is not None:
             app.config["SQLALCHEMY_DATABASE_URI"] = database_uri
@@ -305,6 +312,7 @@ def load_user(user_id):
             monitor_usage,
             update_unit_sto4,
             update_unit_quota,
+            restart_redis_worker,
         )
 
         # Add flask commands - general
@@ -314,6 +322,7 @@ def load_user(user_id):
         app.cli.add_command(update_unit_quota)
         app.cli.add_command(update_uploaded_file_with_log)
         app.cli.add_command(lost_files_s3_db)
+        app.cli.add_command(restart_redis_worker)
 
         # Add flask commands - cronjobs
         app.cli.add_command(set_available_to_expired)
@@ -341,6 +350,18 @@ def dds_version_filter(_):
             from dds_web.web.user import auth_blueprint
             from flask_swagger_ui import get_swaggerui_blueprint
 
+            # Redis Worker needs to run as its own process, we initialize it here.
+            # If some worker was already running, it will not be started again.
+            redis_url = app.config.get("REDIS_URL")
+            redis_connection = Redis.from_url(redis_url)
+
+            workers = Worker.all(redis_connection)
+            if not workers:
+                worker = Worker(["default"], connection=redis_connection)
+                worker.log = app.logger
+                p = multiprocessing.Process(target=worker.work, daemon=True)
+                p.start()
+
             # base url for the api documentation
             SWAGGER_URL_1 = "/api/documentation/v1"
             SWAGGER_URL_3 = "/api/documentation/v3"
@@ -374,6 +395,8 @@ def dds_version_filter(_):
             app.register_blueprint(swagger_ui_blueprint_v3, url_prefix=SWAGGER_URL_3, name="v3")
             app.register_blueprint(api_blueprint, url_prefix="/api/v1")
             app.register_blueprint(api_blueprint_v3, url_prefix="/api/v3")
+
+            app.register_blueprint(rq_dashboard.blueprint, url_prefix="/rq")
             app.register_blueprint(pages, url_prefix="")
             app.register_blueprint(auth_blueprint, url_prefix="")
 
diff --git a/dds_web/api/project.py b/dds_web/api/project.py
index 9d3530041..63487dca7 100644
--- a/dds_web/api/project.py
+++ b/dds_web/api/project.py
@@ -15,6 +15,8 @@
 import datetime
 import botocore
 import marshmallow
+from rq import Queue
+from redis import Redis
 
 # Own modules
 import dds_web.utils
@@ -479,7 +481,9 @@ def delete_project(self, project: models.Project, current_time: datetime.datetim
 
         try:
             # Deletes files (also commits session in the function - possibly refactor later)
-            RemoveContents().delete_project_contents(project=project, delete_bucket=True)
+            RemoveContents().delete_project_contents(
+                project_id=project.public_id, delete_bucket=True
+            )
             self.rm_project_user_keys(project=project)
 
             # Delete metadata from project row
@@ -519,7 +523,9 @@ def archive_project(
 
         try:
             # Deletes files (also commits session in the function - possibly refactor later)
-            RemoveContents().delete_project_contents(project=project, delete_bucket=True)
+            RemoveContents().delete_project_contents(
+                project_id=project.public_id, delete_bucket=True
+            )
             delete_message = f"\nAll files in {project.public_id} deleted"
             self.rm_project_user_keys(project=project)
 
@@ -746,8 +752,8 @@ class RemoveContents(flask_restful.Resource):
 
     @auth.login_required(role=["Unit Admin", "Unit Personnel"])
     @logging_bind_request
-    @dbsession
     @handle_validation_errors
+    @dbsession
     def delete(self):
         """Removes all project contents."""
         # Verify project ID and access
@@ -765,13 +771,26 @@ def delete(self):
             )
 
         # Delete project contents from db and cloud
-        self.delete_project_contents(project=project)
 
-        return {"removed": True}
+        # Get redis connection to add a job to delete project contents
+        redis_url = flask.current_app.config.get("REDIS_URL")
+        r = Redis.from_url(redis_url)
+        q = Queue(connection=r)
+
+        # Enqueue job to delete project contents
+        job = q.enqueue(self.delete_project_contents, project.public_id)
+
+        # TODO - return job id to client to check status of deletion
+        msg = "Data deletion has started. This might take some time. The DDS is handling this in the background."
+        return {"removed": True, "message": msg}
 
     @staticmethod
-    def delete_project_contents(project, delete_bucket=False):
+    @dbsession
+    def delete_project_contents(project_id, delete_bucket=False):
         """Remove project contents"""
+        # Get project
+        project: models.Project = models.Project.query.filter_by(public_id=project_id).one_or_none()
+
         # Delete from cloud
         with ApiS3Connector(project=project) as s3conn:
             try:
@@ -792,6 +811,7 @@ def delete_project_contents(project, delete_bucket=False):
                     models.Version.time_deleted.is_(None),
                 )
             ).update({"time_deleted": dds_web.utils.current_time()})
+
         except (
             sqlalchemy.exc.SQLAlchemyError,
             sqlalchemy.exc.OperationalError,
diff --git a/dds_web/api/superadmin_only.py b/dds_web/api/superadmin_only.py
index 3cc435079..2678d9e6c 100644
--- a/dds_web/api/superadmin_only.py
+++ b/dds_web/api/superadmin_only.py
@@ -149,6 +149,7 @@ class SendMOTD(flask_restful.Resource):
     @handle_db_error
     def post(self):
         """Send MOTD as email to users."""
+
         # Get request info
         request_json = flask.request.get_json(silent=True)  # Verified by json_required
         # Get MOTD ID
@@ -203,7 +204,8 @@ def post(self):
                         ["Content-ID", "<Logo>"],
                     ],
                 )
-                # Send email
+
+                # This funcion cannot be enqued because the connection object is not pickable
                 utils.send_email_with_retry(msg=msg, obj=conn)
 
         return_msg = f"MOTD '{motd_id}' has been "
@@ -211,7 +213,6 @@ def post(self):
             return_msg += "sent to unit personnel only."
         else:
             return_msg += "sent to all users."
-
         return {"message": return_msg}
 
 
diff --git a/dds_web/commands.py b/dds_web/commands.py
index ddca483d0..433f68fa2 100644
--- a/dds_web/commands.py
+++ b/dds_web/commands.py
@@ -18,6 +18,9 @@
 import flask_mail
 import sqlalchemy
 import botocore
+from redis import Redis
+from rq import Worker
+from rq.command import send_shutdown_command
 
 # Own
 from dds_web import db
@@ -1319,3 +1322,30 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("restart-redis-worker")
+@flask.cli.with_appcontext
+def restart_redis_worker():
+    """
+    This function restarts the redis worker intialized by the Flask application.
+    It will shutdown any existing workers and start a new one.
+    The Redis URL is specified in the Flask application's configuration.
+    The worker listens to the "default" queue and processes jobs from it.
+
+    Configuration:
+        - The Redis server URL should be specified in the Flask application's
+        configuration under the key "REDIS_URL".
+        - The worker can be further customized, see https://python-rq.org/docs/workers/
+    """
+
+    redis_url = flask.current_app.config.get("REDIS_URL")
+    redis_connection = Redis.from_url(redis_url)
+
+    workers = Worker.all(redis_connection)
+    for worker in workers:
+        send_shutdown_command(redis_connection, worker.name)  # Tells worker to shutdown
+
+    new_worker = Worker(["default"], connection=redis_connection)
+    new_worker.log = flask.current_app.logger
+    new_worker.work()
diff --git a/dds_web/config.py b/dds_web/config.py
index be2aa4254..a77ee1521 100644
--- a/dds_web/config.py
+++ b/dds_web/config.py
@@ -64,6 +64,10 @@ class Config(object):
         "RATELIMIT_STORAGE_URI", "memory://"
     )  # Use in devel only! Use Redis or memcached in prod
 
+    # Redis
+    REDIS_URL = "redis://dds_redis"  # In production the URL also has a password and it's the same as RATELIMIT_STORAGE_URI
+    RQ_DASHBOARD_REDIS_URL = REDIS_URL
+
     INVITATION_EXPIRES_IN_HOURS = 7 * 24
 
     # Argon2id settings
diff --git a/dds_web/static/swagger.yaml b/dds_web/static/swagger.yaml
index a780320a8..c368e8e3d 100644
--- a/dds_web/static/swagger.yaml
+++ b/dds_web/static/swagger.yaml
@@ -875,6 +875,14 @@ paths:
           $ref: "#/components/responses/UnauthorizedToken"
         "400":
           $ref: "#/components/responses/BadRequest"
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  message:
+                    type: string
+                    example: There are no project contents to delete.
         "500":
           $ref: "#/components/responses/InternalServerlError"
         "200":
@@ -882,8 +890,13 @@ paths:
           content:
             application/json:
               schema:
-                type: string
-                example: "{removed: true}"
+                type: object
+                properties:
+                  message:
+                    type: string
+                    example: Data deletion has started. This might take some time. The DDS is handling this in the background. # yamllint disable-line
+                  removed:
+                    type: boolean
   /proj/public:
     get:
       tags:
diff --git a/dds_web/static/swaggerv3.yaml b/dds_web/static/swaggerv3.yaml
index 8b157279b..ea11fd091 100644
--- a/dds_web/static/swaggerv3.yaml
+++ b/dds_web/static/swaggerv3.yaml
@@ -824,6 +824,14 @@ paths:
           $ref: "#/components/responses/UnauthorizedToken"
         "400":
           $ref: "#/components/responses/BadRequest"
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  message:
+                    type: string
+                    example: There are no project contents to delete.
         "500":
           $ref: "#/components/responses/InternalServerlError"
         "200":
@@ -831,8 +839,13 @@ paths:
           content:
             application/json:
               schema:
-                type: string
-                example: "{removed: true}"
+                type: object
+                properties:
+                  message:
+                    type: string
+                    example: Data deletion has started. This might take some time. The DDS is handling this in the background. # yamllint disable-line
+                  removed:
+                    type: boolean
   /proj/public:
     get:
       tags:
diff --git a/requirements.txt b/requirements.txt
index b31942161..849d0c19b 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -56,6 +56,8 @@ redis==4.5.5
 requests==2.32.0
 requests_cache==0.9.4
 rich==12.5.1
+rq==2.1.0
+rq-dashboard==0.8.2.2
 s3transfer==0.5.1
 simplejson==3.17.6
 six==1.16.0
diff --git a/tests/conftest.py b/tests/conftest.py
index 608c40032..9b373206d 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -14,6 +14,7 @@
 from requests_mock.mocker import Mocker
 import requests_cache
 import click
+from rq.queue import Queue
 
 # Own
 from dds_web.database.models import (
@@ -475,7 +476,19 @@ def add_data_to_db():
 
 
 @pytest.fixture(scope="session", autouse=True)
-def setup_database():
+def mock_redis_init():
+    """Fixture to mock the starting of Redis Queue Worker when initializing the app."""
+
+    with unittest.mock.patch("redis.client.Redis.from_url"):
+        with unittest.mock.patch("dds_web.Worker"):
+            with unittest.mock.patch("dds_web.Worker.all") as mock_get_all:
+                with unittest.mock.patch("multiprocessing.Process"):
+                    mock_get_all.return_value = None  # No previous workers
+                    yield
+
+
+@pytest.fixture(scope="session", autouse=True)
+def setup_database(mock_redis_init):
     print("setup_database is called")
     # Create database specific for tests
     if not database_exists(DATABASE_URI_BASE):
@@ -576,3 +589,43 @@ def cli_test_app():
 @pytest.fixture()
 def cli_runner(cli_test_app):
     return cli_test_app.test_cli_runner()
+
+
+@pytest.fixture(scope="session", autouse=True)
+def mock_enqueue():
+    """Fixture to mock RQ's enqueue method and bypass Redis queue."""
+
+    def _mock_enqueue(*args, **kwargs):
+
+        # Extract the function from args (RQ expects it to be the first argument)
+        f = args[0]
+        function_args = args[1:]  # Remaining args passed to function
+
+        # Call the function directly
+        f(*function_args, **kwargs)
+
+        # Return a mock job object
+        job = unittest.mock.MagicMock()
+        job.id = "mock-job-id"
+        return job
+
+    return _mock_enqueue  # Return function so it can be used in tests
+
+
+@pytest.fixture(scope="function")
+def mock_queue_redis(mock_enqueue):
+    """Fixture to mock RQ's Queue and parse_args."""
+
+    with unittest.mock.patch("redis.client.Redis.from_url") as mock_redis:
+        with unittest.mock.patch("rq.queue.Queue") as mock_queue:
+            with unittest.mock.patch.object(Queue, "enqueue") as mock_enqueue_func:
+
+                # Mock Redis and Queue objects to avoid generating a connection to Redis
+                mock_redis_instance = unittest.mock.MagicMock()
+                mock_redis.return_value = mock_redis_instance
+                mock_queue_instance = unittest.mock.MagicMock()
+                mock_queue.return_value = mock_queue_instance
+
+                # Mock the enqueue to call the function directly without actually enqueueing
+                mock_enqueue_func.side_effect = mock_enqueue
+                yield
diff --git a/tests/test_commands.py b/tests/test_commands.py
index 40855f1ef..4e53d07eb 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -39,6 +39,7 @@
     update_unit_sto4,
     update_unit_quota,
     send_usage,
+    restart_redis_worker,
 )
 from dds_web.database import models
 from dds_web import db, mail
@@ -2089,3 +2090,22 @@ def test_send_usage_error_csv(client, cli_runner, capfd: LogCaptureFixture):
             == outbox[-1].subject
         )
         assert "There was an error in the cronjob 'send-usage'" in outbox[-1].body
+
+
+## run-worker
+def test_restart_redis__worker(client, cli_runner, mock_queue_redis):
+    """Test that starts the redis workers"""
+
+    with patch("dds_web.commands.Worker") as mock_worker:
+        with patch("dds_web.commands.Worker.all") as mock_get_all:
+            with patch("dds_web.commands.send_shutdown_command") as mock_send_shutdown_command:
+
+                mock_worker_instance = MagicMock()
+                mock_worker.return_value = mock_worker_instance
+                mock_send_shutdown_command.return_value = MagicMock()
+
+                mock_get_all.return_value = [MagicMock(name="worker1"), MagicMock(name="worker2")]
+
+                cli_runner.invoke(restart_redis_worker)
+
+                mock_worker_instance.work.assert_called_once()  # work method called
diff --git a/tests/test_files_new.py b/tests/test_files_new.py
index a36d8eee5..350987ec1 100644
--- a/tests/test_files_new.py
+++ b/tests/test_files_new.py
@@ -821,7 +821,7 @@ def test_upload_move_available_delete_file(client, boto3_session):
     )
 
 
-def test_upload_and_remove_all_project_contents(client, boto3_session):
+def test_upload_and_remove_all_project_contents(client, boto3_session, mock_queue_redis):
     """Upload and then delete all project contents"""
 
     project_1 = project_row(project_id="file_testing_project")
@@ -1061,7 +1061,7 @@ def test_new_file_wrong_status(client):
     assert "Project not in right status to upload/modify files" in response.json.get("message")
 
 
-def test_delete_contents_and_upload_again(client, boto3_session):
+def test_delete_contents_and_upload_again(client, boto3_session, mock_queue_redis):
     """Upload and then delete all project contents"""
 
     project_1 = project_row(project_id="file_testing_project")
diff --git a/tests/tests_v3/test_files_new.py b/tests/tests_v3/test_files_new.py
index d101eb6be..3c36ad9ba 100644
--- a/tests/tests_v3/test_files_new.py
+++ b/tests/tests_v3/test_files_new.py
@@ -751,7 +751,7 @@ def test_upload_move_available_delete_file(client, boto3_session):
     )
 
 
-def test_upload_and_remove_all_project_contents(client, boto3_session):
+def test_upload_and_remove_all_project_contents(client, boto3_session, mock_queue_redis):
     """Upload and then delete all project contents"""
 
     project_1 = project_row(project_id="file_testing_project")
@@ -991,7 +991,7 @@ def test_new_file_wrong_status(client):
     assert "Project not in right status to upload/modify files" in response.json.get("message")
 
 
-def test_delete_contents_and_upload_again(client, boto3_session):
+def test_delete_contents_and_upload_again(client, boto3_session, mock_queue_redis):
     """Upload and then delete all project contents"""
 
     project_1 = project_row(project_id="file_testing_project")


Reviews:
user: login: reutenauer
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-07T09:03:06Z
user: login: rv0lt
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-07T09:54:02Z
user: login: valyo
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-02-10T14:31:19Z
user: login: valyo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-10T17:35:28Z
user: login: rv0lt
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-11T09:24:33Z
user: login: valyo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-11T09:34:28Z
user: login: rv0lt
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-11T09:40:57Z
user: login: i-oden
body: @rv0lt Could you remove the changes relating to the MOTD in this PR? Make a new branch with those changes if you want so that we have it when we start looking at that again
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-02-11T15:33:12Z
user: login: i-oden
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-11T15:57:16Z
user: login: valyo
body: Unless the aim of this PR is purely POC, i.e. to show that it can work with RQ by manually starting a worker, then next thing is to figure out how to start a worker in each container upon deployment 
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-02-12T11:22:33Z
user: login: i-oden
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-02-12T13:47:02Z
user: login: i-oden
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-02-12T14:31:50Z
user: login: i-oden
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-02-12T14:39:22Z
user: login: valyo
body: I tested locally and I got the following error:
```
dds_backend      | [2025-02-25 09:45:14,006] worker [INFO] default: dds_web.api.project.RemoveContents.delete_project_contents('someunit00001') (adea4af0-a2d8-4a4d-ba49-b69b9b7586bc)
dds_backend      | [2025-02-25 09:45:14,007] worker [DEBUG] Sent heartbeat to prevent worker timeout. Next one should arrive in 480 seconds.
dds_backend      | 09:45:14 [Job adea4af0-a2d8-4a4d-ba49-b69b9b7586bc]: exception raised while executing (dds_web.api.project.RemoveContents.delete_project_contents)
dds_backend      | Traceback (most recent call last):
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_collections.py", line 1008, in __call__
dds_backend      |     return self.registry[key]
dds_backend      |            ~~~~~~~~~~~~~^^^^^
dds_backend      | KeyError: <greenlet.greenlet object at 0xffff796900c0 (otid=0xffff79a8a9d0) current active started main>
dds_backend      |
dds_backend      | During handling of the above exception, another exception occurred:
dds_backend      |
dds_backend      | Traceback (most recent call last):
dds_backend      |   File "/code/dds_web/api/dds_decorators.py", line 82, in make_commit
dds_backend      |     result = func(*args, **kwargs)
dds_backend      |              ^^^^^^^^^^^^^^^^^^^^^
dds_backend      |   File "/code/dds_web/api/project.py", line 790, in delete_project_contents
dds_backend      |     project: models.Project = models.Project.query.filter_by(public_id=project_id).one_or_none()
dds_backend      |                               ^^^^^^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py", line 552, in __get__
dds_backend      |     return type.query_class(mapper, session=self.sa.session())
dds_backend      |                                             ^^^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/orm/scoping.py", line 47, in __call__
dds_backend      |     sess = self.registry()
dds_backend      |            ^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_collections.py", line 1010, in __call__
dds_backend      |     return self.registry.setdefault(key, self.createfunc())
dds_backend      |                                          ^^^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 4205, in __call__
dds_backend      |     return self.class_(**local_kw)
dds_backend      |            ^^^^^^^^^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py", line 174, in __init__
dds_backend      |     self.app = app = db.get_app()
dds_backend      |                      ^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py", line 1042, in get_app
dds_backend      |     raise RuntimeError(
dds_backend      | RuntimeError: No application found. Either work inside a view function or push an application context. See http://flask-sqlalchemy.pocoo.org/contexts/.
dds_backend      |
dds_backend      | During handling of the above exception, another exception occurred:
dds_backend      |
dds_backend      | Traceback (most recent call last):
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_collections.py", line 1008, in __call__
dds_backend      |     return self.registry[key]
dds_backend      |            ~~~~~~~~~~~~~^^^^^
dds_backend      | KeyError: <greenlet.greenlet object at 0xffff796900c0 (otid=0xffff79a8a9d0) current active started main>
dds_backend      |
dds_backend      | During handling of the above exception, another exception occurred:
dds_backend      |
dds_backend      | Traceback (most recent call last):
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/rq/worker.py", line 1633, in perform_job
dds_backend      |     return_value = job.perform()
dds_backend      |                    ^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/rq/job.py", line 1331, in perform
dds_backend      |     self._result = self._execute()
dds_backend      |                    ^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/rq/job.py", line 1365, in _execute
dds_backend      |     result = self.func(*self.args, **self.kwargs)
dds_backend      |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dds_backend      |   File "/code/dds_web/api/dds_decorators.py", line 86, in make_commit
dds_backend      |     db.session.rollback()
dds_backend      |   File "<string>", line 2, in rollback
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/orm/scoping.py", line 24, in _proxied
dds_backend      |     return self.registry()
dds_backend      |            ^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/_collections.py", line 1010, in __call__
dds_backend      |     return self.registry.setdefault(key, self.createfunc())
dds_backend      |                                          ^^^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 4205, in __call__
dds_backend      |     return self.class_(**local_kw)
dds_backend      |            ^^^^^^^^^^^^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py", line 174, in __init__
dds_backend      |     self.app = app = db.get_app()
dds_backend      |                      ^^^^^^^^^^^^
dds_backend      |   File "/usr/local/lib/python3.12/site-packages/flask_sqlalchemy/__init__.py", line 1042, in get_app
dds_backend      |     raise RuntimeError(
dds_backend      | RuntimeError: No application found. Either work inside a view function or push an application context. See http://flask-sqlalchemy.pocoo.org/contexts/.
dds_backend      |
dds_backend      | [2025-02-25 09:45:14,268] worker [DEBUG] Sent heartbeat to prevent worker timeout. Next one should arrive in 480 seconds.
dds_backend      | [2025-02-25 09:45:14,268] wo
```

state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-25T09:51:12Z
user: login: i-oden
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-26T12:46:23Z
user: login: rv0lt
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-26T13:30:36Z
user: login: valyo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T13:02:40Z
user: login: rv0lt
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T13:27:40Z
user: login: i-oden
body: This looks really good. Have tried it out, things are working. 
1. Move the job ID code to a new PR 
2. What do we need to change / add in the test/dev/prod environments to get this working?
3. The RQ dashboard -- how will this be protected? Will we (the team) be able to see queued tasks somewhere? We don't want everyone to see everything
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-06T15:13:03Z
user: login: valyo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T19:01:28Z
user: login: rv0lt
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T09:48:22Z
user: login: valyo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T10:14:29Z
user: login: i-oden
state: DISMISSED
author_association: MEMBER
submitted_at: 2025-03-07T14:14:11Z
user: login: valyo
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-21T12:42:39Z
user: login: valyo
body: I don't have any changes to suggest without clarity on my 2 questions, mainly the last one bout the command
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-21T12:45:25Z
user: login: rv0lt
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T12:53:46Z
user: login: rv0lt
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T12:56:54Z
user: login: valyo
body: Ok, I guess we should merge this one and continue from here
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-21T13:24:31Z

Review comments:
diff_hunk: @@ -1319,3 +1321,25 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("run-worker")
path: dds_web/commands.py
user: login: reutenauer
body: Since “worker” is rather a generic name, I think we should call this action something else, perhaps `run-broker`.  We don’t name to change the names workers in the code below.
created_at: 2025-02-07T09:03:05Z
author_association: MEMBER
diff_hunk: @@ -1319,3 +1321,25 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("run-worker")
path: dds_web/commands.py
user: login: rv0lt
body: Good idea, maybe run-redis-worker ? I still want to keep that name because it is how the documentation calls it, for consistency shake
created_at: 2025-02-07T09:54:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -64,6 +64,9 @@ class Config(object):
         "RATELIMIT_STORAGE_URI", "memory://"
     )  # Use in devel only! Use Redis or memcached in prod
 
+    # Redis
+    REDIS_URL = "redis://dds_redis"  # In production the URL also has a password
path: dds_web/config.py
user: login: valyo
body: The Redis SVC is `dds-redis`
```suggestion
    REDIS_URL = "redis://dds-redis"  # In production the URL also has a password
```

@rv0lt can you make this change so that we can test on dev instance
created_at: 2025-02-10T14:29:50Z
author_association: MEMBER
diff_hunk: @@ -64,6 +64,9 @@ class Config(object):
         "RATELIMIT_STORAGE_URI", "memory://"
     )  # Use in devel only! Use Redis or memcached in prod
 
+    # Redis
+    REDIS_URL = "redis://dds_redis"  # In production the URL also has a password
path: dds_web/config.py
user: login: valyo
body: Ok, I realise it shouldn't be `dds-redis`, but since we cannot have the same URL in all environments, it needs to handled via environment variable
created_at: 2025-02-10T17:35:28Z
author_association: MEMBER
diff_hunk: @@ -64,6 +64,9 @@ class Config(object):
         "RATELIMIT_STORAGE_URI", "memory://"
     )  # Use in devel only! Use Redis or memcached in prod
 
+    # Redis
+    REDIS_URL = "redis://dds_redis"  # In production the URL also has a password
path: dds_web/config.py
user: login: rv0lt
body: But for dev I think it is better to recreate the secret of the config? It has to be the same URL as RATELIMIT_STORAGE_URI uses there. And last time I checked it contains password details for the redis connection.

This one I setted it up as default due to being the one docker-compose uses, otherwise we would need to make modifications for local testing every time as well
created_at: 2025-02-11T09:24:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -64,6 +64,9 @@ class Config(object):
         "RATELIMIT_STORAGE_URI", "memory://"
     )  # Use in devel only! Use Redis or memcached in prod
 
+    # Redis
+    REDIS_URL = "redis://dds_redis"  # In production the URL also has a password
path: dds_web/config.py
user: login: valyo
body: Yes, the secret `dds-conf-file` is where it should be set. But will setting it just overwrite the value from config.py? 
Actually yes, because these are not env variables (the ones in the secret configuration), they are flask variables.
created_at: 2025-02-11T09:34:28Z
author_association: MEMBER
diff_hunk: @@ -64,6 +64,9 @@ class Config(object):
         "RATELIMIT_STORAGE_URI", "memory://"
     )  # Use in devel only! Use Redis or memcached in prod
 
+    # Redis
+    REDIS_URL = "redis://dds_redis"  # In production the URL also has a password
path: dds_web/config.py
user: login: rv0lt
body: As I am aware, the order that Flask uses (at least in our use case) is:

Env variables that start with **FLASK_**  > **Config file**  > **config.py**

So that, env variables will replace the data from config files, which replaces the default config.py
created_at: 2025-02-11T09:40:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -478,3 +478,7 @@ _Nothing merged during this sprint_
 
 - Make release template ([#1587](https://github.com/ScilifelabDataCentre/dds_web/pull/1587))
 - Fix codecov action ([#1589](https://github.com/ScilifelabDataCentre/dds_web/pull/1589))
+
+# 2024-02-03 - 2025-02-14
+
+- POC for Redis Queue ([#1591](https://github.com/ScilifelabDataCentre/dds_web/pull/1591))
path: SPRINTLOG.md
user: login: i-oden
body: This needs to be changed before merging but skip it for now. 
created_at: 2025-02-11T14:51:52Z
author_association: MEMBER
diff_hunk: @@ -14,6 +14,8 @@
 import flask
 import structlog
 import flask_mail
+from rq import Queue
+from redis import Redis
path: dds_web/api/superadmin_only.py
user: login: i-oden
body: Remove this for now then? 
created_at: 2025-02-11T15:24:20Z
author_association: MEMBER
diff_hunk: @@ -149,6 +151,12 @@ class SendMOTD(flask_restful.Resource):
     @handle_db_error
     def post(self):
         """Send MOTD as email to users."""
+
path: dds_web/api/superadmin_only.py
user: login: i-oden
body: And this too
created_at: 2025-02-11T15:24:33Z
author_association: MEMBER
diff_hunk: @@ -203,15 +211,18 @@ def post(self):
                         ["Content-ID", "<Logo>"],
                     ],
                 )
-                # Send email
+
+                # Send email in a queue to avoid blocking the API
+                # job = q.enqueue(utils.send_email_with_queue, msg)
+
+                # This funcion cannot be enqued because the connection object is not pickable
path: dds_web/api/superadmin_only.py
user: login: i-oden
body: Keep the bottom comment but remove the rest
created_at: 2025-02-11T15:24:46Z
author_association: MEMBER
diff_hunk: @@ -501,6 +501,20 @@ def send_email_with_retry(msg, times_retried=0, obj=None):
             send_email_with_retry(msg, times_retried=retry, obj=obj)
 
 
+def send_email_for_queue(msg):
path: dds_web/utils.py
user: login: i-oden
body: Remove this as well
created_at: 2025-02-11T15:32:04Z
author_association: MEMBER
diff_hunk: @@ -64,6 +64,9 @@ class Config(object):
         "RATELIMIT_STORAGE_URI", "memory://"
     )  # Use in devel only! Use Redis or memcached in prod
 
+    # Redis
+    REDIS_URL = "redis://dds_redis"  # In production the URL also has a password
path: dds_web/config.py
user: login: i-oden
body: ```
dds_backend      | /usr/local/lib/python3.12/site-packages/rq_dashboard/legacy_config.py:29: UserWarning: Configuration option REDIS_URL is deprecated and will be removed in future versions. Please use RQ_DASHBOARD_REDIS_URL instead.
```
--> We should switch to `RQ_DASHBOARD_REDIS_URL`
created_at: 2025-02-11T15:57:16Z
author_association: MEMBER
diff_hunk: @@ -323,6 +328,7 @@ def load_user(user_id):
         app.cli.add_command(send_usage)
         app.cli.add_command(collect_stats)
         app.cli.add_command(monitor_usage)
+        app.cli.add_command(run_redis_worker)
path: dds_web/__init__.py
user: login: valyo
body: this should probably not be under "Add flask commands - cronjobs" but under "Add flask commands - general"
created_at: 2025-02-12T11:09:55Z
author_association: MEMBER
diff_hunk: @@ -746,7 +752,6 @@ class RemoveContents(flask_restful.Resource):
 
     @auth.login_required(role=["Unit Admin", "Unit Personnel"])
     @logging_bind_request
-    @dbsession
path: dds_web/api/project.py
user: login: i-oden
body: This also handles any of the db errors in the `delete` endpoint,e.g. `project.current_status`, `project.has_been_available` etc, if we move this decorator from this enpoint, we also need to add a try except:

try: before `project = project_schemas.ProjectRequiredSchema [....]`
except, including everything caught in the decorator: after `if not project.files:`

created_at: 2025-02-12T13:46:26Z
author_association: MEMBER
diff_hunk: @@ -576,3 +577,43 @@ def cli_test_app():
 @pytest.fixture()
 def cli_runner(cli_test_app):
     return cli_test_app.test_cli_runner()
+
+
+@pytest.fixture(scope="session", autouse=True)
+def mock_enqueue():
path: tests/conftest.py
user: login: i-oden
body: Very nice!
created_at: 2025-02-12T14:25:31Z
author_association: MEMBER
diff_hunk: @@ -757,6 +757,36 @@ def test_send_project_access_reset_email(client):
     assert response is None
 
 
+# send email functions
+
+
+def test_send_email_with_retry(client):
+    """Send email with retry ok first try."""
+    # Get email row
+    email_row: models.Email = db.session.query(models.Email).first()
+
+    # Run function
+    with patch("dds_web.utils.mail.send"):
+        response = utils.send_email_with_retry(msg=email_row)
+    assert response is None
+
+
+def test_send_email_with_retry_exception(client):
+    """Send email with retry - exception on the 1st try."""
+
+    import smtplib
+
+    side_effect = [smtplib.SMTPException, None]  # Exception on the first try, second try is ok
+
+    # Get email row
+    email_row: models.Email = db.session.query(models.Email).first()
+
+    # Run function
+    with patch("dds_web.utils.mail.send", side_effect=side_effect):
+        response = utils.send_email_with_retry(msg=email_row)
+    assert response is None
+
+
path: tests/test_utils.py
user: login: i-oden
body: Answer to your previous question: Yes, remove this too. BUT put it in another branch, it's great that you've added these tests, we definitely want to keep them, just not in this PR.
created_at: 2025-02-12T14:31:40Z
author_association: MEMBER
diff_hunk: @@ -64,6 +64,9 @@ class Config(object):
         "RATELIMIT_STORAGE_URI", "memory://"
     )  # Use in devel only! Use Redis or memcached in prod
 
+    # Redis
+    REDIS_URL = "redis://dds_redis"  # In production the URL also has a password
path: dds_web/config.py
user: login: i-oden
body: https://github.com/ScilifelabDataCentre/dds_web/pull/1591#discussion_r1951121103
```suggestion
    RQ_DASHBOARD_REDIS_URL = "redis://dds_redis"  # In production the URL also has a password
```
created_at: 2025-02-12T14:39:14Z
author_association: MEMBER
diff_hunk: @@ -198,6 +201,10 @@ def create_app(testing=False, database_uri=None):
         # User config file, if e.g. using in production
         app.config.from_envvar("DDS_APP_CONFIG", silent=True)
 
+        # redis dashboard
+        app.config.from_object(rq_dashboard.default_settings)
+        rq_dashboard.web.setup_rq_connection(app)
path: dds_web/__init__.py
user: login: i-oden
body: What happens if you move this row and rows 300-311 to within the `with app.app_context():`? 

```suggestion
```
created_at: 2025-02-26T12:46:23Z
author_association: MEMBER
diff_hunk: @@ -198,6 +201,10 @@ def create_app(testing=False, database_uri=None):
         # User config file, if e.g. using in production
         app.config.from_envvar("DDS_APP_CONFIG", silent=True)
 
+        # redis dashboard
+        app.config.from_object(rq_dashboard.default_settings)
+        rq_dashboard.web.setup_rq_connection(app)
path: dds_web/__init__.py
user: login: rv0lt
body: Yeah, moving the ones below work. These ones only init the dashboard
created_at: 2025-02-26T13:30:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -765,13 +771,25 @@ def delete(self):
             )
 
         # Delete project contents from db and cloud
-        self.delete_project_contents(project=project)
 
+        # Get redis connection to add a job to delete project contents
+        redis_url = flask.current_app.config.get("REDIS_URL")
+        r = Redis.from_url(redis_url)
+        q = Queue(connection=r)
+
+        # Enqueue job to delete project contents
+        job = q.enqueue(self.delete_project_contents, project.public_id)
+
+        # TODO - return job id to client to check status of deletion
         return {"removed": True}
 
     @staticmethod
-    def delete_project_contents(project, delete_bucket=False):
+    @dbsession
path: dds_web/api/project.py
user: login: valyo
body: I am curious how/why this was before working without the decorator
created_at: 2025-03-06T13:02:39Z
author_association: MEMBER
diff_hunk: @@ -765,13 +771,25 @@ def delete(self):
             )
 
         # Delete project contents from db and cloud
-        self.delete_project_contents(project=project)
 
+        # Get redis connection to add a job to delete project contents
+        redis_url = flask.current_app.config.get("REDIS_URL")
+        r = Redis.from_url(redis_url)
+        q = Queue(connection=r)
+
+        # Enqueue job to delete project contents
+        job = q.enqueue(self.delete_project_contents, project.public_id)
+
+        # TODO - return job id to client to check status of deletion
         return {"removed": True}
 
     @staticmethod
-    def delete_project_contents(project, delete_bucket=False):
+    @dbsession
path: dds_web/api/project.py
user: login: rv0lt
body: The dbsession ? It doesn't commit the changes if not. So the bucket is empty but the db is not
created_at: 2025-03-06T13:27:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -478,3 +478,7 @@ _Nothing merged during this sprint_
 
 - Make release template ([#1587](https://github.com/ScilifelabDataCentre/dds_web/pull/1587))
 - Fix codecov action ([#1589](https://github.com/ScilifelabDataCentre/dds_web/pull/1589))
+
+# 2024-02-03 - 2025-02-14
+
+- Implementation of a Redis Queue to process aynschronus some requests and avoid timeouts, and enque the function to delete all the contents of a project([#1591](https://github.com/ScilifelabDataCentre/dds_web/pull/1591))
path: SPRINTLOG.md
user: login: i-oden
body: ```suggestion
- Implement Redis Queue to process some requests asynschronusly and avoid timeouts. Set project deletion as a background task. ([#1591](https://github.com/ScilifelabDataCentre/dds_web/pull/1591))
```
created_at: 2025-03-06T14:43:10Z
author_association: MEMBER
diff_hunk: @@ -765,13 +771,25 @@ def delete(self):
             )
 
         # Delete project contents from db and cloud
-        self.delete_project_contents(project=project)
 
+        # Get redis connection to add a job to delete project contents
+        redis_url = flask.current_app.config.get("REDIS_URL")
+        r = Redis.from_url(redis_url)
+        q = Queue(connection=r)
+
+        # Enqueue job to delete project contents
+        job = q.enqueue(self.delete_project_contents, project.public_id)
+
+        # TODO - return job id to client to check status of deletion
         return {"removed": True}
 
     @staticmethod
-    def delete_project_contents(project, delete_bucket=False):
+    @dbsession
path: dds_web/api/project.py
user: login: i-oden
body: @valyo How do you mean before? Do you mean before this PR (we had the decorator, it's just moved one step down) or between Alvaro removing the decorator and the readding it again? 
created_at: 2025-03-06T14:51:28Z
author_association: MEMBER
diff_hunk: @@ -1319,3 +1322,30 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("restart-redis-worker")
path: dds_web/commands.py
user: login: i-oden
body: I like this.
Question:  What happens to potential queued tasks? 
created_at: 2025-03-06T15:08:43Z
author_association: MEMBER
diff_hunk: @@ -765,13 +771,25 @@ def delete(self):
             )
 
         # Delete project contents from db and cloud
-        self.delete_project_contents(project=project)
 
+        # Get redis connection to add a job to delete project contents
+        redis_url = flask.current_app.config.get("REDIS_URL")
+        r = Redis.from_url(redis_url)
+        q = Queue(connection=r)
+
+        # Enqueue job to delete project contents
+        job = q.enqueue(self.delete_project_contents, project.public_id)
+
+        # TODO - return job id to client to check status of deletion
         return {"removed": True}
 
     @staticmethod
-    def delete_project_contents(project, delete_bucket=False):
+    @dbsession
path: dds_web/api/project.py
user: login: valyo
body: Now I see that I was confused by its removal and re-adding for the delete() method. But it doesn't seem that it was present for the delete_project_contents() before this PR. So my question remains for this method
created_at: 2025-03-06T19:01:28Z
author_association: MEMBER
diff_hunk: @@ -1319,3 +1322,30 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("restart-redis-worker")
path: dds_web/commands.py
user: login: rv0lt
body: If there is no worker, they will stay enqueded but unprocessed
created_at: 2025-03-07T09:48:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1319,3 +1322,30 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("restart-redis-worker")
path: dds_web/commands.py
user: login: valyo
body: I hope they will be picked up when new worker(s) appear. 
Question: why would we want to restart existing worker(s)?
Question: is there going to be one worker per API/backend replica?

created_at: 2025-03-07T10:14:29Z
author_association: MEMBER
diff_hunk: @@ -765,13 +771,25 @@ def delete(self):
             )
 
         # Delete project contents from db and cloud
-        self.delete_project_contents(project=project)
 
+        # Get redis connection to add a job to delete project contents
+        redis_url = flask.current_app.config.get("REDIS_URL")
+        r = Redis.from_url(redis_url)
+        q = Queue(connection=r)
+
+        # Enqueue job to delete project contents
+        job = q.enqueue(self.delete_project_contents, project.public_id)
+
+        # TODO - return job id to client to check status of deletion
         return {"removed": True}
path: dds_web/api/project.py
user: login: i-oden
body: Keep this so that the CLI doesn't break before we have a chance to fix it

Also add "message" to be returned. "Data deletion has started. This might take some time. The DDS is handling this in the background."  

created_at: 2025-03-07T14:13:58Z
author_association: MEMBER
diff_hunk: @@ -1319,3 +1322,30 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("restart-redis-worker")
path: dds_web/commands.py
user: login: valyo
body: @rv0lt I meant this question in the morning

created_at: 2025-03-21T10:02:13Z
author_association: MEMBER
diff_hunk: @@ -1319,3 +1322,30 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("restart-redis-worker")
path: dds_web/commands.py
user: login: rv0lt
body: Q1: Only locally when development I can think of, because if you make changes to the code it is not picked up unless restarted

Q2: I haven't tested in a cluster. But I am almost sure that, how it is written right now, it will only register one worker. I can agree that at least one worker per replica would be better
created_at: 2025-03-21T12:53:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1319,3 +1322,30 @@ def monitor_usage():
                 body=message,
             )
             dds_web.utils.send_email_with_retry(msg=msg)
+
+
+@click.command("restart-redis-worker")
path: dds_web/commands.py
user: login: rv0lt
body: It could also be possible to have a new variable somewhere that tells how many workers to initialize. Something like

QUEUE_WORKERS_PER_REPLICA 
created_at: 2025-03-21T12:56:54Z
author_association: CONTRIBUTOR

PR comments:
user: login: codecov[bot]
created_at: 2025-02-05T11:40:46Z
body: ## [Codecov](https://app.codecov.io/gh/ScilifelabDataCentre/dds_web/pull/1591?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ScilifelabDataCentre) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 92.61%. Comparing base [(`b5e4716`)](https://app.codecov.io/gh/ScilifelabDataCentre/dds_web/commit/b5e47163b9622a61c005a4f819a2cf6a86ce0583?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ScilifelabDataCentre) to head [(`e6d915d`)](https://app.codecov.io/gh/ScilifelabDataCentre/dds_web/commit/e6d915d52bb1bfaf3be1dcbd2b048a8ca9409ae2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ScilifelabDataCentre).
> Report is 33 commits behind head on dev.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##              dev    #1591      +/-   ##
==========================================
+ Coverage   92.55%   92.61%   +0.06%     
==========================================
  Files          29       29              
  Lines        4892     4932      +40     
==========================================
+ Hits         4528     4568      +40     
  Misses        364      364              
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/ScilifelabDataCentre/dds_web/pull/1591?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ScilifelabDataCentre).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ScilifelabDataCentre).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
author_association: NONE
user: login: valyo
created_at: 2025-02-11T10:53:35Z
body: I tested on the dev instance, and seemingly things work without errors, but the RQ job remains in Status: queued forever. 
I ran:
`dds data rm -p someunit00040 -a` 

and after ~35 min I can still see the project 

`dds ls -p someunit00040
     ︵
 ︵ (  )   ︵
(  ) ) (  (  )   SciLifeLab Data Delivery System
 ︶  (  ) ) (    https://dev.delivery.scilifelab.se/
      ︶ (  )    CLI Version 2.9.0
          ︶
Current user: unitadmin_1
INFO     Listing files for project 'someunit00040'

 Files / directories in project: someunit00040
 └── test-p8w1t8-17-fpt47.1736531492/
`

and RQ-dashboard shows this:

<img width="1910" alt="Screenshot 2025-02-11 at 11 53 23" src="https://github.com/user-attachments/assets/1bb0095b-044e-4617-97bd-fc5c6f54a67f" />

author_association: MEMBER
user: login: rv0lt
created_at: 2025-02-11T11:05:29Z
body: hmmmm interesting. Do you know if there are some errros in the terminal where you started the redis worker?

> I tested on the dev instance, and seemingly things work without errors, but the RQ job remains in Status: queued forever. I ran: `dds data rm -p someunit00040 -a`
> 
> and after ~35 min I can still see the project
> 
> `dds ls -p someunit00040 ︵ ︵ ( ) ︵ ( ) ) ( ( ) SciLifeLab Data Delivery System ︶ ( ) ) ( https://dev.delivery.scilifelab.se/ ︶ ( ) CLI Version 2.9.0 ︶ Current user: unitadmin_1 INFO Listing files for project 'someunit00040'
> 
> Files / directories in project: someunit00040 └── test-p8w1t8-17-fpt47.1736531492/ `
> 
> and RQ-dashboard shows this:
> <img alt="Screenshot 2025-02-11 at 11 53 23" width="1910" src="https://private-user-images.githubusercontent.com/582646/411934560-1bb0095b-044e-4617-97bd-fc5c6f54a67f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkyNzE1NDYsIm5iZiI6MTczOTI3MTI0NiwicGF0aCI6Ii81ODI2NDYvNDExOTM0NTYwLTFiYjAwOTViLTA0NGUtNDYxNy05N2JkLWZjNWM2ZjU0YTY3Zi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIxMVQxMDU0MDZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02ZTAyN2QzYTBmYmY5MzhlNjFjYzRhMGU3MGIxZGVmN2MyMGYyMjIyNGE3NDJhNjlkOThkY2VhNWY1MDUzMDRhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.bfkEaRitiC-jCk0DYp5g5mASRYjrqNTPa6IJH1Rd8yg">


author_association: CONTRIBUTOR
user: login: valyo
created_at: 2025-02-11T14:00:50Z
body: Now I made another test locally (docker setup, no k8s involved) and I get slightly different, but still unexpected result: 
the job in RQ finishes quite quickly:

<img width="1892" alt="Screenshot 2025-02-11 at 14 57 33" src="https://github.com/user-attachments/assets/35365c19-838e-457f-b780-be937a69b3ba" />

but the data is not really deleted:
<img width="1328" alt="Screenshot 2025-02-11 at 14 58 49" src="https://github.com/user-attachments/assets/0c5b2374-55e9-47b2-9eda-7e18cf150729" />

here's the output from `flask fun-redis-worker`:
```
/code $ flask run-redis-worker
/usr/local/lib/python3.12/site-packages/rq_dashboard/legacy_config.py:29: UserWarning: Configuration option REDIS_URL is deprecated and will be removed in future versions. Please use RQ_DASHBOARD_REDIS_URL instead.
  warnings.warn(
[2025-02-11 13:29:36,731] __init__ [INFO] Logging initiated.
13:29:37 Worker rq:worker:redis-queue started with PID 1860, version 2.1.0
13:29:37 Subscribing to channel rq:pubsub:redis-queue
13:29:37 *** Listening on default...
13:29:37 Cleaning registries for queue: default
13:43:07 Cleaning registries for queue: default
13:54:28 default: dds_web.api.project.RemoveContents.delete_project_contents('someunit00001') (d61f271a-4850-4505-a6c6-abcb5e6ef88e)
[2025-02-11 13:54:28,069] utils [INFO] Safespring location for project 'someunit00001': sto2
13:54:29 Successfully completed dds_web.api.project.RemoveContents.delete_project_contents('someunit00001') job in 0:00:01.911512s on worker redis-queue
13:54:29 default: Job OK (d61f271a-4850-4505-a6c6-abcb5e6ef88e)
13:54:29 Result is kept for 500 seconds
13:54:29 Cleaning registries for queue: default

```
author_association: MEMBER
user: login: rv0lt
created_at: 2025-02-11T14:04:47Z
body: > Now I made another test locally (docker setup, no k8s involved) and I get slightly different, but still unexpected result:
> the job in RQ finishes quite quickly:

Aha. And are the files deleted in Minio?
author_association: CONTRIBUTOR
user: login: valyo
created_at: 2025-02-11T14:11:27Z
body: hm, the bucket is actually empty, but DB is not updated 
author_association: MEMBER
user: login: rv0lt
created_at: 2025-02-11T14:12:47Z
body: > hm, the bucket is actually empty, but DB is not updated

indeed, missed that. I was only checking the buckets the whole time
author_association: CONTRIBUTOR
user: login: rv0lt
created_at: 2025-02-11T14:33:02Z
body: Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it

> > hm, the bucket is actually empty, but DB is not updated
> 
> indeed, missed that. I was only checking the buckets the whole time


author_association: CONTRIBUTOR
user: login: i-oden
created_at: 2025-02-11T15:14:55Z
body: > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> 
> > > hm, the bucket is actually empty, but DB is not updated
> > 
> > 
> > indeed, missed that. I was only checking the buckets the whole time



> Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> 
> > > hm, the bucket is actually empty, but DB is not updated
> > 
> > 
> > indeed, missed that. I was only checking the buckets the whole time

The `db_session` decorator does this 
author_association: MEMBER
user: login: i-oden
created_at: 2025-02-11T15:17:23Z
body: > > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> > > > hm, the bucket is actually empty, but DB is not updated
> > > 
> > > 
> > > indeed, missed that. I was only checking the buckets the whole time
> 
> > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> > > > hm, the bucket is actually empty, but DB is not updated
> > > 
> > > 
> > > indeed, missed that. I was only checking the buckets the whole time
> 
> The `db_session` decorator does this



> > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> > > > hm, the bucket is actually empty, but DB is not updated
> > > 
> > > 
> > > indeed, missed that. I was only checking the buckets the whole time
> 
> > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> > > > hm, the bucket is actually empty, but DB is not updated
> > > 
> > > 
> > > indeed, missed that. I was only checking the buckets the whole time
> 
> The `db_session` decorator does this

So I guess we should remove the decorator from the `RemoveContents.delete` endpoint? The point of it is to catch db exceptions 
author_association: MEMBER
user: login: rv0lt
created_at: 2025-02-12T10:28:06Z
body: Ahm, then we could probably add this decorator to the new function instead. 
Preparing a commit with this changes and the other comments

> > > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> > > > > hm, the bucket is actually empty, but DB is not updated
> > > > 
> > > > 
> > > > indeed, missed that. I was only checking the buckets the whole time
> > 
> > 
> > > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> > > > > hm, the bucket is actually empty, but DB is not updated
> > > > 
> > > > 
> > > > indeed, missed that. I was only checking the buckets the whole time
> > 
> > 
> > The `db_session` decorator does this
> 
> > > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> > > > > hm, the bucket is actually empty, but DB is not updated
> > > > 
> > > > 
> > > > indeed, missed that. I was only checking the buckets the whole time
> > 
> > 
> > > Ok, we need to add a session.commit. I guess the API call does it somewhere, but being an independent function it wasn't doing it
> > > > > hm, the bucket is actually empty, but DB is not updated
> > > > 
> > > > 
> > > > indeed, missed that. I was only checking the buckets the whole time
> > 
> > 
> > The `db_session` decorator does this
> 
> So I guess we should remove the decorator from the `RemoveContents.delete` endpoint? The point of it is to catch db exceptions


author_association: CONTRIBUTOR
user: login: rv0lt
created_at: 2025-02-12T10:33:01Z
body: > @rv0lt Could you remove the changes relating to the MOTD in this PR? Make a new branch with those changes if you want so that we have it when we start looking at that again

Yeah, I wasn't sure if I should remove it or not, because I started trying to do it with MOTD, and then I changed so kind of forgot about it.  But will move it then!!
author_association: CONTRIBUTOR
user: login: rv0lt
created_at: 2025-02-12T11:28:38Z
body: > Unless the aim of this PR is purely POC, i.e. to show that it can work with RQ by manually starting a worker, then next thing is to figure out how to start a worker in each container upon deployment

Yeah, I am also trying to think about the best way to do that
author_association: CONTRIBUTOR
user: login: rv0lt
created_at: 2025-02-12T11:29:50Z
body: > @rv0lt Could you remove the changes relating to the MOTD in this PR? Make a new branch with those changes if you want so that we have it when we start looking at that again

@i-oden I also just realized that I made two tests for **send_email_with_retry** because it was not covered, and I was touching those functions. Should I remove that also?
author_association: CONTRIBUTOR
user: login: i-oden
created_at: 2025-02-12T14:06:58Z
body: > > @rv0lt Could you remove the changes relating to the MOTD in this PR? Make a new branch with those changes if you want so that we have it when we start looking at that again
> 
> @i-oden I also just realized that I made two tests for **send_email_with_retry** because it was not covered, and I was touching those functions. Should I remove that also?

Leave it for now, I haven't looked at the tests yet 
author_association: MEMBER
user: login: i-oden
created_at: 2025-02-12T14:10:42Z
body: > Unless the aim of this PR is purely POC, i.e. to show that it can work with RQ by manually starting a worker, then next thing is to figure out how to start a worker in each container upon deployment

Imo the POC is that we can get this working in general, so personally I'd see this as a part of the POC. 
author_association: MEMBER
user: login: rv0lt
created_at: 2025-02-13T14:59:07Z
body: Okay I have been thinking, and I think the best approach to start this worker in our cluster would be:

- Add a new container in the deployment.yaml (i.e same pod) that has a readiness probe that will check that Flask has been initialized (we could even add easily some health endpoints).
After the app has been initialized, the readiness probe will pass and this new container will run the redis worker.

I think this can easily solve the issue of running this command only after the main app logic, and it won't add unnecessary complexity like the other solutions I was reading about

Thoughts: @valyo @i-oden ?
author_association: CONTRIBUTOR
user: login: rv0lt
created_at: 2025-02-13T15:12:22Z
body: Sorry, I meant liveness probe not readiness. So that if the check is not passed, the command will not be executed. My second solution would be to add the code to start the worker in the __init__.py, there is a flag that can be passed to the function to start it in background mode so it won't block the app


> Okay I have been thinking, and I think the best approach to start this worker in our cluster would be:
> 
>     * Add a new container in the deployment.yaml (i.e same pod) that has a readiness probe that will check that Flask has been initialized (we could even add easily some health endpoints).
>       After the app has been initialized, the readiness probe will pass and this new container will run the redis worker.
> 
> 
> I think this can easily solve the issue of running this command only after the main app logic, and it won't add unnecessary complexity like the other solutions I was reading about
> 
> Thoughts: @valyo @i-oden ?


author_association: CONTRIBUTOR
user: login: i-oden
created_at: 2025-02-14T11:38:24Z
body: > Sorry, I meant liveness probe not readiness. So that if the check is not passed, the command will not be executed. My second solution would be to add the code to start the worker in the **init**.py, there is a flag that can be passed to the function to start it in background mode so it won't block the app
> 
> > Okay I have been thinking, and I think the best approach to start this worker in our cluster would be:
> > ```
> > * Add a new container in the deployment.yaml (i.e same pod) that has a readiness probe that will check that Flask has been initialized (we could even add easily some health endpoints).
> >   After the app has been initialized, the readiness probe will pass and this new container will run the redis worker.
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > I think this can easily solve the issue of running this command only after the main app logic, and it won't add unnecessary complexity like the other solutions I was reading about
> > Thoughts: @valyo @i-oden ?

As discussed on Slack, we'll talk about this in a bit during the sprint review and see what the others think. Imo though, we should go for initializing the redis worker during the app initialization.
author_association: MEMBER
user: login: rv0lt
created_at: 2025-03-06T10:47:13Z
body: @i-oden I have added a new endpoint to fetch the information about the job. 

We need it for the client to be able to wait until the job has finished.
Because otherwise it will need to connect to redis directly, and I think it is better to handle it on the backend.

I can move it to a new task if you want. If you think it can stay here, I will then add instruction on how to manually test if and update the swagger documents.

But I can understand that now this PR may be too big, although this new endpoint can still be considered has part of the POC, because it can be used to check that the job sent works
author_association: CONTRIBUTOR
user: login: i-oden
created_at: 2025-03-06T13:32:56Z
body: @rv0lt I think we should have the job status changes in a new PR
author_association: MEMBER
user: login: rv0lt
created_at: 2025-03-06T13:37:48Z
body: > @rv0lt I think we should have the job status changes in a new PR

Should we create a new subtask then?
author_association: CONTRIBUTOR
user: login: i-oden
created_at: 2025-03-06T14:23:58Z
body: > > @rv0lt I think we should have the job status changes in a new PR
> 
> Should we create a new subtask then?

Yes, please do 
author_association: MEMBER
user: login: rv0lt
created_at: 2025-03-07T10:16:55Z
body: I am trying a way to have a username and password, the doc is not very complete on that but there is an option. 
> This looks really good. Have tried it out, things are working.
> 
>     1. Move the job ID code to a new PR
> 
>     2. What do we need to change / add in the test/dev/prod environments to get this working?
> 
>     3. The RQ dashboard -- how will this be protected? Will we (the team) be able to see queued tasks somewhere? We don't want everyone to see everything


author_association: CONTRIBUTOR
user: login: valyo
created_at: 2025-03-07T10:31:33Z
body:  > 3. The RQ dashboard -- how will this be protected? Will we (the team) be able to see queued tasks somewhere? We don't want everyone to see everything
I don't think it needs to be exposed to the users, I mean that it is of any interest for them provided that we handle everything in the client in an user friendly way. 
For us it will be always good to be able to take a look for monitoring/troubleshooting purposes.
The simplest way to protect it is to add IP whitelisting in the ingress we create for it. One can also set up GitHub login (OAuth2) using https://github.com/oauth2-proxy/oauth2-proxy, for example, if it doesn't feel like overkill.

author_association: MEMBER
user: login: rv0lt
created_at: 2025-03-07T10:35:30Z
body: To add to this, the library does not provide authentication apparently, they have this comment in their repo:

RQ Dashboard does not contain any built-in authentication mechanism because

    1. it is the responsbility of the wider hosting app rather than a
       particular blueprint, and

    2. there are numerous ways of adding security orthogonally.


https://github.com/Parallels/rq-dashboard/blob/master/rq_dashboard/web.py

> > 3. The RQ dashboard -- how will this be protected? Will we (the team) be able to see queued tasks somewhere? We don't want everyone to see everything
> >    I don't think it needs to be exposed to the users, I mean that it is of any interest for them provided that we handle everything in the client in an user friendly way.
> >    For us it will be always good to be able to take a look for monitoring/troubleshooting purposes.
> >    The simplest way to protect it is to add IP whitelisting in the ingress we create for it. One can also set up GitHub login (OAuth2) using https://github.com/oauth2-proxy/oauth2-proxy, for example, if it doesn't feel like overkill.


author_association: CONTRIBUTOR
user: login: rv0lt
created_at: 2025-03-07T10:38:18Z
body: > > 3. The RQ dashboard -- how will this be protected? Will we (the team) be able to see queued tasks somewhere? We don't want everyone to see everything
> >    I don't think it needs to be exposed to the users, I mean that it is of any interest for them provided that we handle everything in the client in an user friendly way.
> >    For us it will be always good to be able to take a look for monitoring/troubleshooting purposes.
> >    The simplest way to protect it is to add IP whitelisting in the ingress we create for it. One can also set up GitHub login (OAuth2) using https://github.com/oauth2-proxy/oauth2-proxy, for example, if it doesn't feel like overkill.

Is it possible to do whitelist and block for everyone even thought it lives under a subdomain of the main app?

i.e  delivery.scilifelab.se/rq
author_association: CONTRIBUTOR
user: login: valyo
created_at: 2025-03-07T11:11:30Z
body: > Is it possible to do whitelist and block for everyone even thought it lives under a subdomain of the main app?
> 
> i.e delivery.scilifelab.se/rq

I believe yes, ingress rules have the `path` element, which we have set to "/" for the main app domain:
```
...
  rules:
  - host: delivery.scilifelab.se
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: dds-backend
            port:
              number: 5000
```
author_association: MEMBER
user: login: rv0lt
created_at: 2025-03-07T11:56:06Z
body: > > Is it possible to do whitelist and block for everyone even thought it lives under a subdomain of the main app?
> > i.e delivery.scilifelab.se/rq
> 
> I believe yes, ingress rules have the `path` element, which we have set to "/" for the main app domain:
> 
> ```
> ...
>   rules:
>   - host: delivery.scilifelab.se
>     http:
>       paths:
>       - path: /
>         pathType: Prefix
>         backend:
>           service:
>             name: dds-backend
>             port:
>               number: 5000
> ```

then all good!

The changes in the cluster/deployment, as per now, are creating this two variables 

    REDIS_URL
    RQ_DASHBOARD_REDIS_URL

They need to have the same value, wich is also the same value as RATELIMIT_STORAGE_URI.  Which is the one that has the username and password of redis

author_association: CONTRIBUTOR
user: login: i-oden
created_at: 2025-04-08T09:37:45Z
body: This is not a breaking change. Changing the label.
author_association: MEMBER