Repository: paperless-ngx/paperless-ngx
Issue category: Performance Issue

Reduce PAPERLESS_TASK_WORKERS and  PAPERLESS_WEBSERVER_WORKERS default values
### Description

# Problem with memory and CPU usage
In connection with #1084, reducing the `PAPERLESS_TASK_WORKERS` and  `PAPERLESS_WEBSERVER_WORKERS` default values to `1` helps not only with reducing the memory footprint (from approx. 400mb to approx. 200mb) but because the Django-Q's internals, the CPU usage goes downs as well (each worker checks in every second, which creates a Redis hit; the more workers, the more check-ins, the more Redis hits, the higher the idle CPU usage).

# Typical paperless use-case
I argue paperless is mostly used by hobbyists to store simple invoices, notes, etc. However, by and large, the instance is idling rather than running multicore, CPU heavy processes (see typical use-cases on [/r/selfhosted](https://www.reddit.com/r/selfhosted/search/?q=paperless))

# Change the defaults to match the typical use-case
If the instance is mostly idling and every once in a while processing a new document, it is more reasonable than not to limit the default number of workers to `1`. This is still configurable via env variables, so if anyone ever needs more workers, they can change it, but most users, simply starting the docker container, would benefit from the reduced memory and CPU usage.

Any thoughts, objections?

@stumpylog , I'm happy to create a PR to change the defaults and update the [documentation](https://paperless-ngx.readthedocs.io/en/latest/setup.html?highlight=PAPERLESS_TASK_WORKERS#considerations-for-less-powerful-devices) as well if we are happy with reducing the defaults to preserve memory and CPU.





# Affected code

On any modern CPU (including slow, Atom CPUs) paperless creates [unnecessary amount of task workers based on core-count](https://github.com/paperless-ngx/paperless-ngx/blob/9a1bd9637cbef0e7318a3c61cae14805c6510cea/src/paperless/settings.py#L438) and [exactly 2 web workers](https://github.com/paperless-ngx/paperless-ngx/blob/9a1bd9637cbef0e7318a3c61cae14805c6510cea/gunicorn.conf.py#L4) for historical (?) reasons

### Webserver logs

_No response_

### Paperless-ngx version

1.7.1

### Host OS

Ubuntu

### Installation method

Docker

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_


Functions to edit:
src/paperless/settings.py:default_task_workers

Patch:
diff --git a/docs/configuration.rst b/docs/configuration.rst
index a5db55927d9..c4203472c77 100644
--- a/docs/configuration.rst
+++ b/docs/configuration.rst
@@ -536,6 +536,8 @@ PAPERLESS_TASK_WORKERS=<num>
     maintain the automatic matching algorithm, check emails, consume documents,
     etc. This variable specifies how many things it will do in parallel.
 
+    Defaults to 1
+
 
 PAPERLESS_THREADS_PER_WORKER=<num>
     Furthermore, paperless uses multiple threads when consuming documents to
@@ -797,9 +799,7 @@ PAPERLESS_WEBSERVER_WORKERS=<num>
     also loads the entire application into memory separately, so increasing this value
     will increase RAM usage.
 
-    Consider configuring this to 1 on low power devices with limited amount of RAM.
-
-    Defaults to 2.
+    Defaults to 1.
 
 PAPERLESS_PORT=<port>
     The port number the webserver will listen on inside the container. There are
diff --git a/gunicorn.conf.py b/gunicorn.conf.py
index 9c0e5be7817..7193815b367 100644
--- a/gunicorn.conf.py
+++ b/gunicorn.conf.py
@@ -1,7 +1,7 @@
 import os
 
 bind = f'[::]:{os.getenv("PAPERLESS_PORT", 8000)}'
-workers = int(os.getenv("PAPERLESS_WEBSERVER_WORKERS", 2))
+workers = int(os.getenv("PAPERLESS_WEBSERVER_WORKERS", 1))
 worker_class = "paperless.workers.ConfigurableWorker"
 timeout = 120
 
diff --git a/src/paperless/settings.py b/src/paperless/settings.py
index 8c8aa8482c3..bfb9507ba04 100644
--- a/src/paperless/settings.py
+++ b/src/paperless/settings.py
@@ -425,27 +425,7 @@ def __get_float(key: str, default: float) -> float:
 # Task queue                                                                  #
 ###############################################################################
 
-
-# Sensible defaults for multitasking:
-# use a fair balance between worker processes and threads epr worker so that
-# both consuming many documents in parallel and consuming large documents is
-# reasonably fast.
-# Favors threads per worker on smaller systems and never exceeds cpu_count()
-# in total.
-
-
-def default_task_workers() -> int:
-    # always leave one core open
-    available_cores = max(multiprocessing.cpu_count(), 1)
-    try:
-        if available_cores < 4:
-            return available_cores
-        return max(math.floor(math.sqrt(available_cores)), 1)
-    except NotImplementedError:
-        return 1
-
-
-TASK_WORKERS = __get_int("PAPERLESS_TASK_WORKERS", default_task_workers())
+TASK_WORKERS = __get_int("PAPERLESS_TASK_WORKERS", 1)
 
 PAPERLESS_WORKER_TIMEOUT: Final[int] = __get_int("PAPERLESS_WORKER_TIMEOUT", 1800)
 

Test patch: diff --git a/src/documents/tests/test_settings.py b/src/documents/tests/test_settings.py
deleted file mode 100644
index 9b8edab2790..00000000000
--- a/src/documents/tests/test_settings.py
+++ /dev/null
@@ -1,35 +0,0 @@
-import logging
-from unittest import mock
-
-from django.test import TestCase
-from paperless.settings import default_task_workers
-from paperless.settings import default_threads_per_worker
-
-
-class TestSettings(TestCase):
-    @mock.patch("paperless.settings.multiprocessing.cpu_count")
-    def test_single_core(self, cpu_count):
-        cpu_count.return_value = 1
-
-        default_workers = default_task_workers()
-
-        default_threads = default_threads_per_worker(default_workers)
-
-        self.assertEqual(default_workers, 1)
-        self.assertEqual(default_threads, 1)
-
-    def test_workers_threads(self):
-        for i in range(1, 64):
-            with mock.patch(
-                "paperless.settings.multiprocessing.cpu_count",
-            ) as cpu_count:
-                cpu_count.return_value = i
-
-                default_workers = default_task_workers()
-
-                default_threads = default_threads_per_worker(default_workers)
-
-                self.assertTrue(default_workers >= 1)
-                self.assertTrue(default_threads >= 1)
-
-                self.assertTrue(default_workers * default_threads <= i, f"{i}")
diff --git a/src/paperless/tests/test_settings.py b/src/paperless/tests/test_settings.py
index 57481df5bc1..fed4079e2c8 100644
--- a/src/paperless/tests/test_settings.py
+++ b/src/paperless/tests/test_settings.py
@@ -1,7 +1,9 @@
 import datetime
+from unittest import mock
 from unittest import TestCase
 
 from paperless.settings import _parse_ignore_dates
+from paperless.settings import default_threads_per_worker
 
 
 class TestIgnoreDateParsing(TestCase):
@@ -56,3 +58,27 @@ def test_single_ignore_dates_set(self):
         ]
 
         self._parse_checker(test_cases)
+
+    def test_workers_threads(self):
+        """
+        GIVEN:
+            - Certain CPU counts
+        WHEN:
+            - Threads per worker is calculated
+        THEN:
+            - Threads per worker less than or equal to CPU count
+            - At least 1 thread per worker
+        """
+        default_workers = 1
+
+        for i in range(1, 64):
+            with mock.patch(
+                "paperless.settings.multiprocessing.cpu_count",
+            ) as cpu_count:
+                cpu_count.return_value = i
+
+                default_threads = default_threads_per_worker(default_workers)
+
+                self.assertGreaterEqual(default_threads, 1)
+
+                self.assertLessEqual(default_workers * default_threads, i)