Repository: pulp/pulpcore

Issue:
Need a way to force *async* execution of a DELETE task
**Version**
pulpcore-3.28.10
pulp-rpm-3.22.3

**Describe the bug**
Deleting a handful of repositories with large RPM content now takes hours instead of minutes.

**To Reproduce**
1. sync RHEL 8 baseos and appstream repos from CDN
2. try to delete them
3. since all delete calls are now synchronous, the deletion is performed in a singlethreaded fashion, instead of being distributed across available pulp workers (8 or 16, depending on the installation).  It now takes hours instead of couple of minutes.  In addition, during this time the API gunicorn worker is completely blocked and is unavailable to process other requests.
**Expected behavior**
I would like an API addition (eg. a flag ?async=true) that would force my DELETE /repositories/rpm/rpm/ call to be asynchronous and distributed to either an available worker or, failing that, put in the pending queue.
**Additional context**
Performance of a repository deletion invocation operation (not the actual time it takes to really delete the repo) is very important for RHUI ops.  Sometimes a repo needs to be deleted and re-created to get rid of the repository version history and its associated artifacts.  More importantly, this performance is vital for the RHUI automation and CI during development.


Discussion:
I am convinced now, that repository delete should always be async. There's just too much going on to be executed in the gunicorn process.

Resolving patch:
diff --git a/pulpcore/app/viewsets/base.py b/pulpcore/app/viewsets/base.py
--- a/pulpcore/app/viewsets/base.py
+++ b/pulpcore/app/viewsets/base.py
@@ -453,6 +453,8 @@ class AsyncUpdateMixin(AsyncReservedObjectMixin):
     Provides an update method that dispatches a task with reservation
     """
 
+    ALLOW_NON_BLOCKING_UPDATE = True
+
     @extend_schema(
         description="Trigger an asynchronous update task",
         responses={202: AsyncOperationResponseSerializer},
@@ -468,7 +470,7 @@ def update(self, request, pk, **kwargs):
             exclusive_resources=self.async_reserved_resources(instance),
             args=(pk, app_label, serializer.__class__.__name__),
             kwargs={"data": request.data, "partial": partial},
-            immediate=True,
+            immediate=self.ALLOW_NON_BLOCKING_UPDATE,
         )
         return OperationPostponedResponse(task, request)
 
@@ -486,6 +488,8 @@ class AsyncRemoveMixin(AsyncReservedObjectMixin):
     Provides a delete method that dispatches a task with reservation
     """
 
+    ALLOW_NON_BLOCKING_DELETE = True
+
     @extend_schema(
         description="Trigger an asynchronous delete task",
         responses={202: AsyncOperationResponseSerializer},
@@ -501,7 +505,7 @@ def destroy(self, request, pk, **kwargs):
             tasks.base.general_delete,
             exclusive_resources=self.async_reserved_resources(instance),
             args=(pk, app_label, serializer.__class__.__name__),
-            immediate=True,
+            immediate=self.ALLOW_NON_BLOCKING_DELETE,
         )
         return OperationPostponedResponse(task, request)
 
diff --git a/pulpcore/app/viewsets/exporter.py b/pulpcore/app/viewsets/exporter.py
--- a/pulpcore/app/viewsets/exporter.py
+++ b/pulpcore/app/viewsets/exporter.py
@@ -49,6 +49,9 @@ class ExporterViewSet(
     ViewSet for viewing exporters.
     """
 
+    # Too many cascaded deletes to block the gunicorn worker.
+    ALLOW_NON_BLOCKING_DELETE = False
+
     queryset = Exporter.objects.all()
     serializer_class = ExporterSerializer
     endpoint_name = "exporters"
diff --git a/pulpcore/app/viewsets/repository.py b/pulpcore/app/viewsets/repository.py
--- a/pulpcore/app/viewsets/repository.py
+++ b/pulpcore/app/viewsets/repository.py
@@ -163,6 +163,9 @@ class ImmutableRepositoryViewSet(
     mixins.CreateModelMixin,
     AsyncRemoveMixin,
 ):
+    # Too many cascaded deletes to block the gunicorn worker.
+    ALLOW_NON_BLOCKING_DELETE = False
+
     """
     An immutable repository ViewSet that does not allow the usage of the methods PATCH and PUT.
     """
@@ -370,6 +373,9 @@ class RemoteViewSet(
     AsyncUpdateMixin,
     AsyncRemoveMixin,
 ):
+    # Too many cascaded deletes to block the gunicorn worker.
+    ALLOW_NON_BLOCKING_DELETE = False
+
     endpoint_name = "remotes"
     serializer_class = RemoteSerializer
     queryset = Remote.objects.all()


Test patch: