Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] ConsumerError [Errno 1] Operation not permitted
### Description

PDF files can no longer be consumed. Neither by e-mail, nor by copying it to the inbox.

Version 1.16.1
Updated to 1.16.2

error remains.

### Steps to reproduce

- Forwarding a PDF to paperless
- Saving a PDF in the Paperless Inbox

### Webserver logs

```bash
paperless-ngx | [2023-06-21 09:02:13,872] [INFO] [paperless.management.consumer] Adding /usr/src/paperless/consume/Memoformat.pdf to the task queue.
paperless-ngx | [2023-06-21 09:02:13,927] [INFO] [celery.worker.strategy] Task documents.tasks.consume_file[a5c91aeb-9e71-4c24-8a1e-ebea1e38be22] received
paperless-ngx | [2023-06-21 09:02:14,269] [INFO] [paperless.consumer] Consuming Memoformat.pdf
paperless-ngx | [2023-06-21 09:02:16,604] [INFO] [ocrmypdf._pipeline] skipping all processing on this page
paperless-ngx | [2023-06-21 09:02:16,641] [INFO] [ocrmypdf._sync] Postprocessing...
paperless-ngx | [2023-06-21 09:02:20,485] [WARNING] [ocrmypdf._pipeline] Some input metadata could not be copied because it is not permitted in PDF/A. You may wish to examine the output PDF's XMP metadata.
paperless-ngx | [2023-06-21 09:02:20,785] [INFO] [ocrmypdf._pipeline] Optimize ratio: 1.00 savings: 0.0%
paperless-ngx | [2023-06-21 09:02:20,819] [INFO] [ocrmypdf._sync] Output file is a PDF/A-2B (as expected)
paperless-ngx | [2023-06-21 09:02:31,070] [INFO] [paperless.handlers] Assigning correspondent Jörg Mahn to 2023-06-20 Memoformat
paperless-ngx | [2023-06-21 09:02:32,630] [INFO] [paperless.handlers] Assigning document type Auftrag to 2023-06-20 Jörg Mahn Memoformat
paperless-ngx | [2023-06-21 09:02:33,390] [ERROR] [paperless.consumer] The following error occurred while consuming Memoformat.pdf: [Errno 1] Operation not permitted
paperless-ngx | Traceback (most recent call last):
paperless-ngx |   File "/usr/src/paperless/src/documents/consumer.py", line 441, in try_consume_file
paperless-ngx |     self._write(document.storage_type, self.path, document.source_path)
paperless-ngx |   File "/usr/src/paperless/src/documents/consumer.py", line 585, in _write
paperless-ngx |     shutil.copystat(source, target)
paperless-ngx |   File "/usr/local/lib/python3.9/shutil.py", line 384, in copystat
paperless-ngx |     lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns),
paperless-ngx | PermissionError: [Errno 1] Operation not permitted
paperless-ngx | [2023-06-21 09:02:33,447] [ERROR] [celery.app.trace] Task documents.tasks.consume_file[a5c91aeb-9e71-4c24-8a1e-ebea1e38be22] raised unexpected: ConsumerError('Memoformat.pdf: The following error occurred while consuming Memoformat.pdf: [Errno 1] Operation not permitted')
paperless-ngx | Traceback (most recent call last):
paperless-ngx |   File "/usr/local/lib/python3.9/site-packages/asgiref/sync.py", line 349, in main_wrap
paperless-ngx |     raise exc_info[1]
paperless-ngx |   File "/usr/src/paperless/src/documents/consumer.py", line 441, in try_consume_file
paperless-ngx |     self._write(document.storage_type, self.path, document.source_path)
paperless-ngx |   File "/usr/src/paperless/src/documents/consumer.py", line 585, in _write
paperless-ngx |     shutil.copystat(source, target)
paperless-ngx |   File "/usr/local/lib/python3.9/shutil.py", line 384, in copystat
paperless-ngx |     lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns),
paperless-ngx | PermissionError: [Errno 1] Operation not permitted
paperless-ngx |
paperless-ngx | The above exception was the direct cause of the following exception:
paperless-ngx |
paperless-ngx | Traceback (most recent call last):
paperless-ngx |   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 477, in trace_task
paperless-ngx |     R = retval = fun(*args, **kwargs)
paperless-ngx |   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 760, in __protected_call__
paperless-ngx |     return self.run(*args, **kwargs)
paperless-ngx |   File "/usr/src/paperless/src/documents/tasks.py", line 133, in consume_file
paperless-ngx |     document = Consumer().try_consume_file(
paperless-ngx |   File "/usr/src/paperless/src/documents/consumer.py", line 487, in try_consume_file
paperless-ngx |     self._fail(
paperless-ngx |   File "/usr/src/paperless/src/documents/consumer.py", line 96, in _fail
paperless-ngx |     raise ConsumerError(f"{self.filename}: {log_message or message}") from exception
paperless-ngx | documents.consumer.ConsumerError: Memoformat.pdf: The following error occurred while consuming Memoformat.pdf: [Errno 1] Operation not permitted
```


### Browser logs

_No response_

### Paperless-ngx version

1.16.2

### Host OS

Ubuntu 22.04.2 LTS

### Installation method

Docker - official image

### Browser

Edge

### Configuration changes

_No response_

### Other

_No response_


Discussion:
It seems pretty clearly a permissions issue with your setup.  Or perhaps your underlying filesystem(s) don't support some of the properties most others do.  Are you using network mounts maybe?

It don't happen for me with a regular old ext4 filesystem
Yes, I am using a network mount. But everything is fine with that.
The files to be consumed are also copied by Paperless from /mnt/paperless/inbox to /mnt/paperless/media/documents/originals/2023.

The last documents were successfully added on 06/15/2023. After that I just had the pending security patches installed on Ubuntu.

There was a (similar) problem a few months ago, right?
So your network mount isn't supporting `utime`.  It's maybe a setting in the mount options for it

Resolving patch:
diff --git a/src/documents/consumer.py b/src/documents/consumer.py
--- a/src/documents/consumer.py
+++ b/src/documents/consumer.py
@@ -582,7 +582,12 @@ def apply_overrides(self, document):
     def _write(self, storage_type, source, target):
         with open(source, "rb") as read_file, open(target, "wb") as write_file:
             write_file.write(read_file.read())
-        shutil.copystat(source, target)
+
+        # Attempt to copy file's original stats, but it's ok if we can't
+        try:
+            shutil.copystat(source, target)
+        except Exception:  # pragma: no cover
+            pass
 
     def _log_script_outputs(self, completed_process: CompletedProcess):
         """


Test patch: