Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] API filtering not working with custom_fields
### Description

When accessing API endpoints (either through the browser or by script/cli), you're able to use django queries to filter the response.

For example:
- `http://localhost:port/api/correspondents/` returns all correspondents, whereas
- `http://localhost:port/api/correspondents/?name__iexact=corrname` filters the results to only find the correspond whose name matches `corrname`.

This is great and useful!

However– no queries seem to actually filter the data when accessing `http://localhost:port/api/custom_fields/` – no matter what filter is tried, the endpoint _always_ returns all custom fields.

Broadly new to Paperless, Django, REST, etc. but don't know enough after several hours of digging to figure out whether this is user error or a bug on the project's side.

### Steps to reproduce

1. Go to `http://localhost:port/api/correspondent/`
2. You'll see all of your correspondents
3. Go to `http://localhost:port/api/correspondent/?name__iexact=Client` (replace `Client` with the name of a correspondent)
4. You'll see only that one correspondent
5. Repeat steps above with `http://localhost:port/api/custom_fields/`

Expected result:
`name__iexact=` (or `data_type__iexact=` or `name__contains=` etc) properly returns only filtered records from custom_fields

Actual result:
All queries always return all custom_fields

### Webserver logs

```bash
n/a; this is viewable from a browser and doesn't leave logs
```


### Browser logs

_No response_

### Paperless-ngx version

2.2.1

### Host OS

Synology NAS / x86_64

### Installation method

Docker - official image

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_

### Please confirm the following

- [X] I believe this issue is a bug that affects all users of Paperless-ngx, not something specific to my installation.
- [X] I have already searched for relevant existing issues and discussions before opening this report.
- [X] I have updated the title field above with a concise description.


Discussion:
Duplicate of #4783
This is a little different than that one, but it's still not a bug, what youre talking about is filtering the list of custom fields, which just isnt implemented. Im curious what your use-case is for that? To be clear, this would just filter the list of available custom fields, it has nothing do with documents or values of the fields, etc. You can of course open a FR if interested.

If you open up the API in a browser you'll see some helpful buttons for filters, and note that `/api/custom_fields/` doesnt have these (whereas e.g. correspondents does).

<img width="362" alt="Screenshot 2024-01-05 at 11 03 21 AM" src="https://github.com/paperless-ngx/paperless-ngx/assets/4887959/d55cf0a6-2645-4bb0-b3b8-a5c0c53b4b5a">
@shamoon I didn't notice that button, & didn't think to check the discussion board for related FRs vs issues tracker– sorry for the triage clutter!

I'm not familiar enough to know whether this was a bug ("filtering works for everything except custom_fields, therefore something's broken") or whether filtering required specific implementation. But correct, the goal is to get a list of existing custom fields & their IDs.


## Use case

Exploring post-consumption script workflows that work with custom fields, though it's also just an inconsistency that stood out.

The goal is to, in a post-consumption script:
- Check whether a custom field exists, by name
  - If so, retrieve its ID
  - If not, create a new custom field
- Assign that custom field to the consumed document with a specified value.

Currently I've been [taking advantage of the fact that you can filter during the API requests (for other endpoints)](https://gist.github.com/zlovatt/3664bf0c01292b9ae78d272548411b9d/6438ec1f42e1cc018b8131aba8c00fb733267a18#file-post-consumption-py-L138), so that you can parse the response and know whether you've found a match– but this workflow falls apart since custom_fields doesn't allow filtering.

The workaround would be to ignore the request filtering and instead move that logic into the script itself– which is very doable! It just moves where the work is taking place from the API to the script itself.

Resolving patch:
diff --git a/src/documents/filters.py b/src/documents/filters.py
--- a/src/documents/filters.py
+++ b/src/documents/filters.py
@@ -12,6 +12,7 @@
 from rest_framework_guardian.filters import ObjectPermissionsFilter
 
 from documents.models import Correspondent
+from documents.models import CustomField
 from documents.models import Document
 from documents.models import DocumentType
 from documents.models import Log
@@ -141,6 +142,15 @@ def filter(self, qs, value):
         )
 
 
+class CustomFieldFilterSet(FilterSet):
+    class Meta:
+        model = CustomField
+        fields = {
+            "id": ID_KWARGS,
+            "name": CHAR_KWARGS,
+        }
+
+
 class CustomFieldsFilter(Filter):
     def filter(self, qs, value):
         if value:
diff --git a/src/documents/views.py b/src/documents/views.py
--- a/src/documents/views.py
+++ b/src/documents/views.py
@@ -66,6 +66,7 @@
 from documents.data_models import DocumentMetadataOverrides
 from documents.data_models import DocumentSource
 from documents.filters import CorrespondentFilterSet
+from documents.filters import CustomFieldFilterSet
 from documents.filters import DocumentFilterSet
 from documents.filters import DocumentTypeFilterSet
 from documents.filters import ObjectOwnedOrGrantedPermissionsFilter
@@ -1438,6 +1439,11 @@ class CustomFieldViewSet(ModelViewSet):
 
     serializer_class = CustomFieldSerializer
     pagination_class = StandardPagination
+    filter_backends = (
+        DjangoFilterBackend,
+        OrderingFilter,
+    )
+    filterset_class = CustomFieldFilterSet
 
     model = CustomField
 


Test patch:
diff --git a/src/documents/tests/test_api_custom_fields.py b/src/documents/tests/test_api_custom_fields.py
--- a/src/documents/tests/test_api_custom_fields.py
+++ b/src/documents/tests/test_api_custom_fields.py
@@ -662,3 +662,39 @@ def test_symmetric_doclink_fields(self):
 
         self.assertEqual(resp.status_code, status.HTTP_200_OK)
         self.assertEqual(doc5.custom_fields.first().value, [1])
+
+    def test_custom_field_filters(self):
+        custom_field_string = CustomField.objects.create(
+            name="Test Custom Field String",
+            data_type=CustomField.FieldDataType.STRING,
+        )
+        custom_field_date = CustomField.objects.create(
+            name="Test Custom Field Date",
+            data_type=CustomField.FieldDataType.DATE,
+        )
+        custom_field_int = CustomField.objects.create(
+            name="Test Custom Field Int",
+            data_type=CustomField.FieldDataType.INT,
+        )
+
+        response = self.client.get(
+            f"{self.ENDPOINT}?id={custom_field_string.id}",
+        )
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        results = response.data["results"]
+        self.assertEqual(len(results), 1)
+
+        response = self.client.get(
+            f"{self.ENDPOINT}?id__in={custom_field_string.id},{custom_field_date.id}",
+        )
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        results = response.data["results"]
+        self.assertEqual(len(results), 2)
+
+        response = self.client.get(
+            f"{self.ENDPOINT}?name__icontains=Int",
+        )
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        results = response.data["results"]
+        self.assertEqual(len(results), 1)
+        self.assertEqual(results[0]["name"], custom_field_int.name)