Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Workflow with empty "filter sources" never matches anything
### Description

If the workflow has an empty list of Filter Sources, it will never match.

### Steps to reproduce

1. Create a new Workflow.
2. Add a new Trigger.
3. Set the type to *Consumption Started* (the default type).
4. Set *Filter filename* to something like `Foobar*.pdf`.
5. Click on *Filter Sources*. Add one. Add another. Then change your mind and remove all filter sources.
6. Add an action. Whatever action.
7. Save.
8. Now upload a `Foobar-xyz.pdf` document.
9. BUG! Observe how the action was NOT performed.

### Webserver logs

```
[DEBUG] [paperless.tasks] Executing plugin WorkflowTriggerPlugin
[INFO] [paperless.matching] Document did not match Workflow: FoobarWorkflow1
[DEBUG] [paperless.matching] ('Document source ApiUpload not in []',)
[INFO] [paperless.tasks] WorkflowTriggerPlugin completed with no message
[INFO] [paperless.consumer] Consuming Foobar-xyz.pdf
```

Observe `Document source ApiUpload not in []`. This means this workflow will never match anything, as no string will ever be found inside an empty list.

`<guesswork>`I'm not sure if the "step 5" above was enough to trigger this bug. I'm not sure if there is a difference between "never touched that form field, and then it's null" versus "removed everything and it's now empty but not null".`</guesswork>`

In any way, the UI is letting the user create a broken workflow, while happily showing to the user it would work fine. It's also inconsistent, as some form fields will be ignored if empty, while this specific form field is not ignored. There is no way users would have this intuition upon creating their first workflow.

![Screenshot of the workflow form](https://github.com/paperless-ngx/paperless-ngx/assets/121676/d9665bf6-c58c-4ad8-a47d-7db7822e2653)


### Paperless-ngx version

2.4.3

### Host OS

Debian

### Installation method

Docker - official image

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_

### Please confirm the following

- [X] I believe this issue is a bug that affects all users of Paperless-ngx, not something specific to my installation.
- [X] I have already searched for relevant existing issues and discussions before opening this report.
- [X] I have updated the title field above with a concise description.


Discussion:
Worth mentioning: I have not tested any of the other form fields. I have not tested any of the other trigger types (and I know those will lead to different form fields).

Whoever investigates (and fixes) this bug, please take a look at all form fields to make sure they all behave consistently.
To a certain extent, this is user error. It seems clear enough that no sources will match nothing.  We can probably add some validation to prevent empty sources though
In my humble opinion, it's not "clear enough".

Following the logic of "no sources will match nothing", we would expect that no filter path will match nothing as well. It's inconsistent among the form fields.

I believe the best option is to make "no sources" simply disable the source-checking, just like empty path also disables the path-checking.

If that is not possible, I agree with you regarding adding some validation to warn the user about invalid trigger conditions.

---

(EDIT) I did a quick research in the codebase:

* Tag filter is only active if the trigger has any tag defined: https://github.com/paperless-ngx/paperless-ngx/blob/d55900b8770ff5f9a85523aa3e8638f98a30cc2b/src/documents/matching.py#L335-L341

* Correspondent filter is only active if the correspondent was set in the trigger: https://github.com/paperless-ngx/paperless-ngx/blob/d55900b8770ff5f9a85523aa3e8638f98a30cc2b/src/documents/matching.py#L349-L352

* Document type filter is only active if the document type was set in the trigger: https://github.com/paperless-ngx/paperless-ngx/blob/d55900b8770ff5f9a85523aa3e8638f98a30cc2b/src/documents/matching.py#L359-L362

* Filename filter is only active if the filename glob was set in the trigger: https://github.com/paperless-ngx/paperless-ngx/blob/d55900b8770ff5f9a85523aa3e8638f98a30cc2b/src/documents/matching.py#L369-L377 and https://github.com/paperless-ngx/paperless-ngx/blob/d55900b8770ff5f9a85523aa3e8638f98a30cc2b/src/documents/matching.py#L281-L288

* Mailrule filter is only active if it was set in the trigger: https://github.com/paperless-ngx/paperless-ngx/blob/d55900b8770ff5f9a85523aa3e8638f98a30cc2b/src/documents/matching.py#L269-L273

* And, yet, the document source is always active, regardless if it was set in the trigger or left empty: https://github.com/paperless-ngx/paperless-ngx/blob/d55900b8770ff5f9a85523aa3e8638f98a30cc2b/src/documents/matching.py#L261

It really sounds like an oversight, and I think it leads to surprising behavior for the user.

Resolving patch:
diff --git a/src/documents/matching.py b/src/documents/matching.py
--- a/src/documents/matching.py
+++ b/src/documents/matching.py
@@ -258,7 +258,9 @@ def consumable_document_matches_workflow(
     reason = ""
 
     # Document source vs trigger source
-    if document.source not in [int(x) for x in list(trigger.sources)]:
+    if len(trigger.sources) > 0 and document.source not in [
+        int(x) for x in list(trigger.sources)
+    ]:
         reason = (
             f"Document source {document.source.name} not in"
             f" {[DocumentSource(int(x)).name for x in trigger.sources]}",
diff --git a/src/documents/serialisers.py b/src/documents/serialisers.py
--- a/src/documents/serialisers.py
+++ b/src/documents/serialisers.py
@@ -1410,9 +1410,6 @@ class Meta:
         ]
 
     def validate(self, attrs):
-        if ("filter_mailrule") in attrs and attrs["filter_mailrule"] is not None:
-            attrs["sources"] = {DocumentSource.MailFetch.value}
-
         # Empty strings treated as None to avoid unexpected behavior
         if (
             "filter_filename" in attrs


Test patch:
diff --git a/src/documents/tests/test_api_workflows.py b/src/documents/tests/test_api_workflows.py
--- a/src/documents/tests/test_api_workflows.py
+++ b/src/documents/tests/test_api_workflows.py
@@ -15,8 +15,6 @@
 from documents.models import WorkflowAction
 from documents.models import WorkflowTrigger
 from documents.tests.utils import DirectoriesMixin
-from paperless_mail.models import MailAccount
-from paperless_mail.models import MailRule
 
 
 class TestApiWorkflows(DirectoriesMixin, APITestCase):
@@ -344,56 +342,6 @@ def test_api_create_workflow_trigger_action_empty_fields(self):
         self.assertEqual(trigger2.filter_path, "*/test/*")
         self.assertIsNone(trigger2.filter_filename)
 
-    def test_api_create_workflow_trigger_with_mailrule(self):
-        """
-        GIVEN:
-            - API request to create a workflow trigger with a mail rule but no MailFetch source
-        WHEN:
-            - API is called
-        THEN:
-            - New trigger is created with MailFetch as source
-        """
-        account1 = MailAccount.objects.create(
-            name="Email1",
-            username="username1",
-            password="password1",
-            imap_server="server.example.com",
-            imap_port=443,
-            imap_security=MailAccount.ImapSecurity.SSL,
-            character_set="UTF-8",
-        )
-        rule1 = MailRule.objects.create(
-            name="Rule1",
-            account=account1,
-            folder="INBOX",
-            filter_from="from@example.com",
-            filter_to="someone@somewhere.com",
-            filter_subject="subject",
-            filter_body="body",
-            filter_attachment_filename_include="file.pdf",
-            maximum_age=30,
-            action=MailRule.MailAction.MARK_READ,
-            assign_title_from=MailRule.TitleSource.FROM_SUBJECT,
-            assign_correspondent_from=MailRule.CorrespondentSource.FROM_NOTHING,
-            order=0,
-            attachment_type=MailRule.AttachmentProcessing.ATTACHMENTS_ONLY,
-        )
-        response = self.client.post(
-            self.ENDPOINT_TRIGGERS,
-            json.dumps(
-                {
-                    "type": WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
-                    "sources": [DocumentSource.ApiUpload],
-                    "filter_mailrule": rule1.pk,
-                },
-            ),
-            content_type="application/json",
-        )
-        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
-        self.assertEqual(WorkflowTrigger.objects.count(), 2)
-        trigger = WorkflowTrigger.objects.get(id=response.data["id"])
-        self.assertEqual(trigger.sources, [int(DocumentSource.MailFetch).__str__()])
-
     def test_api_update_workflow_nested_triggers_actions(self):
         """
         GIVEN: