Repository: paperless-ngx/paperless-ngx

Issue:
System Status shows classifier error after update 2.6.0
### Description

In System Status, i get an Error on the Classifier Status.

### Steps to reproduce

1. Go to Settings
2. Click on System Status

### Webserver logs

```bash
[2024-03-05 06:55:47,456] [ERROR] [paperless.api] System status error loading classifier: Classifier not loaded
Traceback (most recent call last):
  File "/usr/src/paperless/src/documents/views.py", line 1625, in get
    raise Exception("Classifier not loaded")
Exception: Classifier not loaded
```


### Browser logs

_No response_

### Paperless-ngx version

2.6.0

### Host OS

Debian 12

### Installation method

Docker - official image

### Browser

Safari

### Configuration changes

_No response_

### Other

_No response_

### Please confirm the following

- [X] I believe this issue is a bug that affects all users of Paperless-ngx, not something specific to my installation.
- [X] I have already searched for relevant existing issues and discussions before opening this report.
- [X] I have updated the title field above with a concise description.


Discussion:
I mean, this is telling you theres an error with the classifier. Have you looked into whether thats accurate or not?
Im inclined to think this is doing what its supposed to, all it does is try to load the classifier:

https://github.com/paperless-ngx/paperless-ngx/blob/04af29083b660ffcfb26716b1aad83b2b2f108e5/src/documents/classifier.py#L39-L70

Feel free to clarify if otherwise
I am experiencing the same issue as described by [Flol0](https://github.com/Flol0):

![Screenshot 2024-03-05 075304](https://github.com/paperless-ngx/paperless-ngx/assets/78903224/72692efd-3bfa-4776-816d-a200162e0319)

Webserver-log:
[2024-03-05 07:21:03,381] [ERROR] [paperless.api] System status error loading classifier: Classifier not loaded
Traceback (most recent call last):
  File "/usr/src/paperless/src/documents/views.py", line 1625, in get
    raise Exception("Classifier not loaded")
Exception: Classifier not loaded

[shamoon](https://github.com/shamoon): whats the meaning of "_Im inclined to think this is doing what its supposed to, all it does is try to load the classifier_"? We can see clearly an error within the GUI and the logfile as well. I am not sure what to do now, ignore the error and wait for a fix?
Same issue here. I do agree this is a bug.

Did some investigation on my side, I think if you wait around 10 minutes, the ERROR should be gone and System Status will give you all green marks.

<img width="1140" alt="image" src="https://github.com/paperless-ngx/paperless-ngx/assets/10862845/d7853524-b683-4541-b4ec-8a526f068a11">

I think the issue here is, before 2.6.0, when Paperless gets loaded, it takes some time to train and update the classifier model. On my server, it takes around 6-7 minutes.

```
[2024-03-05 15:06:36,804] [DEBUG] [paperless.classifier] Training correspondent classifier...
[2024-03-05 15:06:52,847] [DEBUG] [paperless.classifier] Training document type classifier...
[2024-03-05 15:07:04,983] [DEBUG] [paperless.classifier] Training storage paths classifier...
[2024-03-05 15:07:10,730] [INFO] [paperless.tasks] Saving updated classifier model to /usr/src/paperless/data/classification_model.pickle...
```

Before 2.6.0, there were no user-visible exceptions about the classifier, so we won't pay attention to this. When people  need classifiers later, Paperless will call `load_classifier()` and we can get that classifier, unless it's in the first few minutes. 

However in 2.6.0, due to the period check of System Status, it calls `load_classifier()` even in the first few minutes and it can't get the classifier: https://github.com/paperless-ngx/paperless-ngx/blob/04af29083b660ffcfb26716b1aad83b2b2f108e5/src/documents/classifier.py#L40-L45

Thus it throws an error: https://github.com/paperless-ngx/paperless-ngx/blob/2b006907d536ce6e79f4c0c5ff0a9cd9e32fab66/src/documents/views.py#L1623-L1625

If you can wait a few minutes, Paperless should generate the model and the ERROR will be gone.

This is a bad experience. Probably someone could help with a fix: (1) Stop checking System Status in the first few minutes. Or, (2) To have a special condition saying the classifier is not ready and asks users to come back later.

The status page is trying to load the classifier and failing, which would indicate an actual problem with it. Thats the purpose of this feature.

Im not sure I agree this is a bug, the classifier hasn't been generated yet, which means its not there.

To me, the question is whether users who are seeing this actually have a working classifier (or some other issue that makes showing an error status inaccurate).

Resolving patch:
diff --git a/src/documents/views.py b/src/documents/views.py
--- a/src/documents/views.py
+++ b/src/documents/views.py
@@ -1581,7 +1581,9 @@ def get(self, request, format=None):
         except Exception as e:  # pragma: no cover
             applied_migrations = []
             db_status = "ERROR"
-            logger.exception(f"System status error connecting to database: {e}")
+            logger.exception(
+                f"System status detected a possible problem while connecting to the database: {e}",
+            )
             db_error = "Error connecting to database, check logs for more detail."
 
         media_stats = os.statvfs(settings.MEDIA_ROOT)
@@ -1594,7 +1596,9 @@ def get(self, request, format=None):
                 redis_status = "OK"
             except Exception as e:
                 redis_status = "ERROR"
-                logger.exception(f"System status error connecting to redis: {e}")
+                logger.exception(
+                    f"System status detected a possible problem while connecting to redis: {e}",
+                )
                 redis_error = "Error connecting to redis, check logs for more detail."
 
         try:
@@ -1615,14 +1619,40 @@ def get(self, request, format=None):
         except Exception as e:
             index_status = "ERROR"
             index_error = "Error opening index, check logs for more detail."
-            logger.exception(f"System status error opening index: {e}")
+            logger.exception(
+                f"System status detected a possible problem while opening the index: {e}",
+            )
             index_last_modified = None
 
         classifier_error = None
+        classifier_status = None
         try:
             classifier = load_classifier()
             if classifier is None:
-                raise Exception("Classifier not loaded")
+                # Make sure classifier should exist
+                docs_queryset = Document.objects.exclude(
+                    tags__is_inbox_tag=True,
+                )
+                if (
+                    docs_queryset.count() > 0
+                    and (
+                        Tag.objects.filter(matching_algorithm=Tag.MATCH_AUTO).exists()
+                        or DocumentType.objects.filter(
+                            matching_algorithm=Tag.MATCH_AUTO,
+                        ).exists()
+                        or Correspondent.objects.filter(
+                            matching_algorithm=Tag.MATCH_AUTO,
+                        ).exists()
+                        or StoragePath.objects.filter(
+                            matching_algorithm=Tag.MATCH_AUTO,
+                        ).exists()
+                    )
+                    and not os.path.isfile(settings.MODEL_FILE)
+                ):
+                    # if classifier file doesn't exist just classify as a warning
+                    classifier_error = "Classifier file does not exist (yet). Re-training may be pending."
+                    classifier_status = "WARNING"
+                    raise FileNotFoundError(classifier_error)
             classifier_status = "OK"
             task_result_model = apps.get_model("django_celery_results", "taskresult")
             result = (
@@ -1637,10 +1667,16 @@ def get(self, request, format=None):
             )
             classifier_last_trained = result.date_done if result else None
         except Exception as e:
-            classifier_status = "ERROR"
+            if classifier_status is None:
+                classifier_status = "ERROR"
             classifier_last_trained = None
-            classifier_error = "Error loading classifier, check logs for more detail."
-            logger.exception(f"System status error loading classifier: {e}")
+            if classifier_error is None:
+                classifier_error = (
+                    "Unable to load classifier, check logs for more detail."
+                )
+            logger.exception(
+                f"System status detected a possible problem while loading the classifier: {e}",
+            )
 
         return Response(
             {


Test patch:
diff --git a/src/documents/tests/test_api_status.py b/src/documents/tests/test_api_status.py
--- a/src/documents/tests/test_api_status.py
+++ b/src/documents/tests/test_api_status.py
@@ -1,4 +1,5 @@
 import os
+import tempfile
 from pathlib import Path
 from unittest import mock
 
@@ -7,8 +8,11 @@
 from rest_framework import status
 from rest_framework.test import APITestCase
 
+from documents.classifier import ClassifierModelCorruptError
 from documents.classifier import DocumentClassifier
 from documents.classifier import load_classifier
+from documents.models import Document
+from documents.models import Tag
 from paperless import version
 
 
@@ -158,7 +162,7 @@ def test_system_status_classifier_ok(self):
         WHEN:
             - The user requests the system status
         THEN:
-            - The response contains the correct classifier status
+            - The response contains an OK classifier status
         """
         load_classifier()
         test_classifier = DocumentClassifier()
@@ -169,18 +173,66 @@ def test_system_status_classifier_ok(self):
         self.assertEqual(response.data["tasks"]["classifier_status"], "OK")
         self.assertIsNone(response.data["tasks"]["classifier_error"])
 
-    def test_system_status_classifier_error(self):
+    def test_system_status_classifier_warning(self):
         """
         GIVEN:
-            - The classifier is not found
+            - The classifier does not exist yet
+            - > 0 documents and tags with auto matching exist
         WHEN:
             - The user requests the system status
         THEN:
-            - The response contains an error classifier status
+            - The response contains an WARNING classifier status
         """
         with override_settings(MODEL_FILE="does_not_exist"):
+            Document.objects.create(
+                title="Test Document",
+            )
+            Tag.objects.create(name="Test Tag", matching_algorithm=Tag.MATCH_AUTO)
             self.client.force_login(self.user)
             response = self.client.get(self.ENDPOINT)
             self.assertEqual(response.status_code, status.HTTP_200_OK)
-            self.assertEqual(response.data["tasks"]["classifier_status"], "ERROR")
+            self.assertEqual(response.data["tasks"]["classifier_status"], "WARNING")
             self.assertIsNotNone(response.data["tasks"]["classifier_error"])
+
+    def test_system_status_classifier_error(self):
+        """
+        GIVEN:
+            - The classifier does exist but is corrupt
+            - > 0 documents and tags with auto matching exist
+        WHEN:
+            - The user requests the system status
+        THEN:
+            - The response contains an ERROR classifier status
+        """
+        does_exist = tempfile.NamedTemporaryFile(
+            dir="/tmp",
+            delete=False,
+        )
+        with override_settings(MODEL_FILE=does_exist):
+            with mock.patch("documents.classifier.load_classifier") as mock_load:
+                mock_load.side_effect = ClassifierModelCorruptError()
+                Document.objects.create(
+                    title="Test Document",
+                )
+                Tag.objects.create(name="Test Tag", matching_algorithm=Tag.MATCH_AUTO)
+                self.client.force_login(self.user)
+                response = self.client.get(self.ENDPOINT)
+                self.assertEqual(response.status_code, status.HTTP_200_OK)
+                self.assertEqual(response.data["tasks"]["classifier_status"], "ERROR")
+                self.assertIsNotNone(response.data["tasks"]["classifier_error"])
+
+    def test_system_status_classifier_ok_no_objects(self):
+        """
+        GIVEN:
+            - The classifier does not exist (and should not)
+            - No documents nor objects with auto matching exist
+        WHEN:
+            - The user requests the system status
+        THEN:
+            - The response contains an OK classifier status
+        """
+        with override_settings(MODEL_FILE="does_not_exist"):
+            self.client.force_login(self.user)
+            response = self.client.get(self.ENDPOINT)
+            self.assertEqual(response.status_code, status.HTTP_200_OK)
+            self.assertEqual(response.data["tasks"]["classifier_status"], "OK")