Repository: nautobot/nautobot

Issue:
Git repository delete creates job result stuck in running state
<!--
    NOTE: IF YOUR ISSUE DOES NOT FOLLOW THIS TEMPLATE, IT WILL BE CLOSED.

    This form is only for reporting reproducible bugs. If you need assistance
    with Nautobot installation, or if you have a general question, please start a
    discussion instead: https://github.com/nautobot/nautobot/discussions

    Please describe the environment in which you are running Nautobot. Be sure
    that you are running an unmodified instance of the latest stable release
    before submitting a bug report, and that any plugins have been disabled.
-->
### Environment
* Nautobot version (Docker tag too if applicable): 2.0+
* Python version:
* Database platform, version:
* Middleware(s):

<!--
    Describe in detail the exact steps that someone else can take to reproduce
    this bug using the current stable release of Nautobot. Begin with the
    creation of any necessary database objects and call out every operation
    being performed explicitly. If reporting a bug in the REST API, be sure to
    reconstruct the raw HTTP request(s) being made: Don't rely on a client
    library such as pynautobot.
-->
### Steps to Reproduce
1. Delete a git repository 
2. Go to job results
3.

<!-- What did you expect to happen? -->
### Expected Behavior
Job result status is completed

<!-- What happened instead? -->
### Observed Behavior
Job result status stuck in running forever 

Code is here https://github.com/nautobot/nautobot/blob/ae73ab420f60e1e192209fc68c7abce324f18811/nautobot/extras/signals.py#L256-L278


Discussion:

Resolving patch:
diff --git a/nautobot/extras/signals.py b/nautobot/extras/signals.py
--- a/nautobot/extras/signals.py
+++ b/nautobot/extras/signals.py
@@ -4,6 +4,7 @@
 import os
 import secrets
 import shutil
+import traceback
 
 from db_file_storage.model_utils import delete_file
 from db_file_storage.storage import DatabaseFileStorage
@@ -20,6 +21,7 @@
 
 from nautobot.core.celery import app, import_jobs_as_celery_tasks
 from nautobot.core.utils.config import get_settings_or_config
+from nautobot.core.utils.logging import sanitize
 from nautobot.extras.choices import JobResultStatusChoices, ObjectChangeActionChoices
 from nautobot.extras.constants import CHANGELOG_MAX_CHANGE_CONTEXT_DETAIL
 from nautobot.extras.models import (
@@ -264,18 +266,31 @@ def git_repository_pre_delete(instance, **kwargs):
     # In fact, attempting to do so would cause database IntegrityErrors!
     job_result.use_job_logs_db = False
 
-    refresh_datasource_content("extras.gitrepository", instance, None, job_result, delete=True)
-
-    # In a distributed Nautobot deployment, each Django instance and/or worker instance may have its own clone
-    # of this repository; we need some way to ensure that all such clones are deleted.
-    # In the Celery worker case, we can broadcast a control message to all workers to do so:
-    app.control.broadcast("discard_git_repository", repository_slug=instance.slug)
-    # But we don't have an equivalent way to broadcast to any other Django instances.
-    # For now we just delete the one that we have locally and rely on other methods,
-    # such as the import_jobs_as_celery_tasks() signal that runs on server startup,
-    # to clean up other clones as they're encountered.
-    if os.path.isdir(instance.filesystem_path):
-        shutil.rmtree(instance.filesystem_path)
+    try:
+        refresh_datasource_content("extras.gitrepository", instance, None, job_result, delete=True)
+
+        # In a distributed Nautobot deployment, each Django instance and/or worker instance may have its own clone
+        # of this repository; we need some way to ensure that all such clones are deleted.
+        # In the Celery worker case, we can broadcast a control message to all workers to do so:
+        app.control.broadcast("discard_git_repository", repository_slug=instance.slug)
+        # But we don't have an equivalent way to broadcast to any other Django instances.
+        # For now we just delete the one that we have locally and rely on other methods,
+        # such as the import_jobs_as_celery_tasks() signal that runs on server startup,
+        # to clean up other clones as they're encountered.
+        if os.path.isdir(instance.filesystem_path):
+            shutil.rmtree(instance.filesystem_path)
+    except Exception as exc:
+        job_result.result = {
+            "exc_type": type(exc).__name__,
+            "exc_message": sanitize(str(exc)),
+        }
+        job_result.status = JobResultStatusChoices.STATUS_FAILURE
+        job_result.traceback = sanitize("".join(traceback.format_exception(type(exc), exc, exc.__traceback__)))
+    else:
+        job_result.status = JobResultStatusChoices.STATUS_SUCCESS
+    finally:
+        job_result.date_done = timezone.now()
+        job_result.save()
 
 
 #


Test patch:
diff --git a/nautobot/extras/tests/test_datasources.py b/nautobot/extras/tests/test_datasources.py
--- a/nautobot/extras/tests/test_datasources.py
+++ b/nautobot/extras/tests/test_datasources.py
@@ -737,6 +737,10 @@ def populate_repo(path, url):
                 # Now delete the GitRepository
                 self.repo.delete()
 
+                with self.subTest("Assert Deleted GitRepo do not create a never ending JobResult record"):
+                    # Bug fix test for https://github.com/nautobot/nautobot/issues/5121
+                    self.assertEqual(JobResult.objects.last().status, JobResultStatusChoices.STATUS_SUCCESS)
+
                 with self.assertRaises(ConfigContext.DoesNotExist):
                     ConfigContext.objects.get(
                         owner_object_id=self.repo.pk,