Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Barcode seperator processing fails with password protected PDFs
### Description

With PAPERLESS_CONSUMER_ENABLE_BARCODES=true uploading a password protected PDF causes the file task to fail. It appears that the barcode separation stage fails with an error rather than skipping the document. The same document imports correctly with the above settings commented out.

Traceback from the failed task:
```python
/tmp/paperless/paperless-upload-0bj7vn9g: invalid password : Traceback (most recent call last):
  File "/usr/src/paperless/src/src/django-q/django_q/cluster.py", line 454, in worker
    res = f(*task["args"], **task["kwargs"])
  File "/usr/src/paperless/src/documents/tasks.py", line 99, in consume_file
    pdf_filepath, separators = barcodes.scan_file_for_separating_barcodes(path)
  File "/usr/src/paperless/src/documents/barcodes.py", line 121, in scan_file_for_separating_barcodes
    pdf = Pdf.open(pdf_filepath)
  File "/usr/local/lib/python3.9/site-packages/pikepdf/_methods.py", line 791, in open
    pdf = Pdf._open(
pikepdf._qpdf.PasswordError: /tmp/paperless/paperless-upload-0bj7vn9g: invalid password
```

### Steps to reproduce

1. Launch paperless with PAPERLESS_CONSUMER_ENABLE_BARCODES=true
2. Upload password protected PDF
3. Check the file tasks menu for failed jobs

### Webserver logs

_No response_

### Paperless-ngx version

1.9.2

### Host OS

Fedora Server 36

### Installation method

Other (please describe above)

### Browser

_No response_

### Configuration changes

_No response_

### Other

Official docker image running under root podman


Discussion:
This should be fixed by #1745, but if you can provide a sample, it should be quick to confirm.
You can now configure using the old method for barcodes, which presumably ignores passwords, assuming these files worked in 1.8
I've retested this with the :dev version of the docker container and now get an exception in both pikepdf and pdf2image when the process fails. 
I've created a simple test file (password is "test") as I can't share the actual files I'm using: [protected.pdf](https://github.com/paperless-ngx/paperless-ngx/files/9854267/protected.pdf)

```python

[2022-10-24 18:21:57,070] [INFO] [celery.worker.strategy] Task documents.tasks.consume_file[53689d80-79cb-41ec-ad58-30db27882c7a] received
[2022-10-24 18:21:57,115] [WARNING] [paperless.barcodes] Exception using pikepdf for barcodes, falling back to pdf2image: /tmp/paperless/paperless-upload-8t1qyflc: invalid password
[2022-10-24 18:21:57,149] [ERROR] [celery.app.trace] Task documents.tasks.consume_file[53689d80-79cb-41ec-ad58-30db27882c7a] raised unexpected: PDFPageCountError('Unable to get page count.\nCommand Line Error: Incorrect password\n')
Traceback (most recent call last):
  File "/usr/src/paperless/src/documents/barcodes.py", line 157, in scan_file_for_separating_barcodes
    _pikepdf_barcode_scan(pdf_filepath)
  File "/usr/src/paperless/src/documents/barcodes.py", line 117, in _pikepdf_barcode_scan
    with Pdf.open(pdf_filepath) as pdf:
  File "/usr/local/lib/python3.9/site-packages/pikepdf/_methods.py", line 803, in open
    pdf = Pdf._open(
pikepdf._qpdf.PasswordError: /tmp/paperless/paperless-upload-8t1qyflc: invalid password

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pdf2image/pdf2image.py", line 479, in pdfinfo_from_path
    raise ValueError
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/src/paperless/src/documents/tasks.py", line 104, in consume_file
    pdf_filepath, separators = barcodes.scan_file_for_separating_barcodes(path)
  File "/usr/src/paperless/src/documents/barcodes.py", line 166, in scan_file_for_separating_barcodes
    _pdf2image_barcode_scan(pdf_filepath)
  File "/usr/src/paperless/src/documents/barcodes.py", line 137, in _pdf2image_barcode_scan
    pages_from_path = convert_from_path(pdf_filepath, output_folder=path)
  File "/usr/local/lib/python3.9/site-packages/pdf2image/pdf2image.py", line 98, in convert_from_path
    page_count = pdfinfo_from_path(pdf_path, userpw, poppler_path=poppler_path)["Pages"]
  File "/usr/local/lib/python3.9/site-packages/pdf2image/pdf2image.py", line 488, in pdfinfo_from_path
    raise PDFPageCountError(
pdf2image.exceptions.PDFPageCountError: Unable to get page count.
Command Line Error: Incorrect password
```

EDIT: to be clear - I'm not expecting the barcode scanner to separate password protected PDFs, but it crashing when trying to process them means that it's impossible to upload password protected PDFs if barcode processing is enabled.
It looks to me like it would be possible to explicitly capture ```pikepdf.PasswordException``` here: https://github.com/paperless-ngx/paperless-ngx/blob/f8ce6285df44cc580319c370a9d76149012615b1/src/documents/barcodes.py#L158 and then skip trying with pdf2img but would it be better to just pass on a PDF that neither pikepdf or pdf2img can handle (by catching exceptions in the pdf2img code too) rather than failing the import?

Resolving patch:
diff --git a/src/documents/barcodes.py b/src/documents/barcodes.py
--- a/src/documents/barcodes.py
+++ b/src/documents/barcodes.py
@@ -10,9 +10,12 @@
 import magic
 from django.conf import settings
 from pdf2image import convert_from_path
+from pdf2image.exceptions import PDFPageCountError
 from pikepdf import Page
+from pikepdf import PasswordError
 from pikepdf import Pdf
 from pikepdf import PdfImage
+from pikepdf.models.image import HifiPrintImageNotTranscodableError
 from PIL import Image
 from PIL import ImageSequence
 from pyzbar import pyzbar
@@ -120,7 +123,9 @@ def _pikepdf_barcode_scan(pdf_filepath: str):
                     pdfimage = PdfImage(page.images[image_key])
 
                     if "/CCITTFaxDecode" in pdfimage.filters:
-                        raise BarcodeImageFormatError()
+                        raise BarcodeImageFormatError(
+                            "Unable to decode CCITTFaxDecode images",
+                        )
 
                     # Not all images can be transcoded to a PIL image, which
                     # is what pyzbar expects to receive
@@ -132,7 +137,7 @@ def _pikepdf_barcode_scan(pdf_filepath: str):
                         separator_page_numbers.append(page_num)
 
     def _pdf2image_barcode_scan(pdf_filepath: str):
-        # use a temporary directory in case the file os too big to handle in memory
+        # use a temporary directory in case the file is too big to handle in memory
         with tempfile.TemporaryDirectory() as path:
             pages_from_path = convert_from_path(pdf_filepath, output_folder=path)
             for current_page_number, page in enumerate(pages_from_path):
@@ -150,20 +155,42 @@ def _pdf2image_barcode_scan(pdf_filepath: str):
         if mime_type == "image/tiff":
             pdf_filepath = convert_from_tiff_to_pdf(filepath)
 
+        # Chose the scanner
         if settings.CONSUMER_USE_LEGACY_DETECTION:
-            _pdf2image_barcode_scan(pdf_filepath)
+            logger.debug("Using pdf2image for barcodes")
+            scanner_function = _pdf2image_barcode_scan
         else:
-            try:
-                _pikepdf_barcode_scan(pdf_filepath)
-            except Exception as e:
+            logger.debug("Using pikepdf for barcodes")
+            scanner_function = _pikepdf_barcode_scan
 
-                logger.warning(
-                    f"Exception using pikepdf for barcodes,"
-                    f" falling back to pdf2image: {e}",
-                )
-                # Reset this incase pikepdf got part way through
+        # Run the scanner
+        try:
+            scanner_function(pdf_filepath)
+        # Neither method can handle password protected PDFs without it being
+        # provided.  Log it and continue
+        except (PasswordError, PDFPageCountError) as e:
+            logger.warning(
+                f"File is likely password protected, not splitting: {e}",
+            )
+        # Handle pikepdf related image decoding issues with a fallback
+        except (BarcodeImageFormatError, HifiPrintImageNotTranscodableError) as e:
+            logger.warning(
+                f"Falling back to pdf2image because: {e}",
+            )
+            try:
                 separator_page_numbers = []
                 _pdf2image_barcode_scan(pdf_filepath)
+            # This file is really borked, allow the consumption to continue
+            # but it may fail further on
+            except Exception as e:  # pragma: no cover
+                logger.warning(
+                    f"Exception during barcode scanning: {e}",
+                )
+        # We're not sure what happened, but allow the consumption to continue
+        except Exception as e:  # pragma: no cover
+            logger.warning(
+                f"Exception during barcode scanning: {e}",
+            )
 
     else:
         logger.warning(


Test patch:
diff --git a/src/documents/tests/samples/password-is-test.pdf b/src/documents/tests/samples/password-is-test.pdf
new file mode 100755
Binary files /dev/null and b/src/documents/tests/samples/password-is-test.pdf differ
diff --git a/src/documents/tests/test_barcodes.py b/src/documents/tests/test_barcodes.py
--- a/src/documents/tests/test_barcodes.py
+++ b/src/documents/tests/test_barcodes.py
@@ -174,7 +174,7 @@ def test_scan_file_for_separating_barcodes(self):
         self.assertEqual(pdf_file, test_file)
         self.assertListEqual(separator_page_numbers, [0])
 
-    def test_scan_file_for_separating_barcodes2(self):
+    def test_scan_file_for_separating_barcodes_none_present(self):
         test_file = os.path.join(self.SAMPLE_DIR, "simple.pdf")
         pdf_file, separator_page_numbers = barcodes.scan_file_for_separating_barcodes(
             test_file,
@@ -585,3 +585,40 @@ def test_consume_barcode_supported_no_extension_file(self):
 
         with mock.patch("documents.tasks.async_to_sync"):
             self.assertEqual(tasks.consume_file(dst), "File successfully split")
+
+    def test_scan_file_for_separating_barcodes_password_pikepdf(self):
+        """
+        GIVEN:
+            - Password protected PDF
+            - pikepdf based scanning
+        WHEN:
+            - File is scanned for barcode
+        THEN:
+            - Scanning handle the exception without exception
+        """
+        test_file = os.path.join(self.SAMPLE_DIR, "password-is-test.pdf")
+        pdf_file, separator_page_numbers = barcodes.scan_file_for_separating_barcodes(
+            test_file,
+        )
+
+        self.assertEqual(pdf_file, test_file)
+        self.assertListEqual(separator_page_numbers, [])
+
+    @override_settings(CONSUMER_USE_LEGACY_DETECTION=True)
+    def test_scan_file_for_separating_barcodes_password_pdf2image(self):
+        """
+        GIVEN:
+            - Password protected PDF
+            - pdf2image based scanning
+        WHEN:
+            - File is scanned for barcode
+        THEN:
+            - Scanning handle the exception without exception
+        """
+        test_file = os.path.join(self.SAMPLE_DIR, "password-is-test.pdf")
+        pdf_file, separator_page_numbers = barcodes.scan_file_for_separating_barcodes(
+            test_file,
+        )
+
+        self.assertEqual(pdf_file, test_file)
+        self.assertListEqual(separator_page_numbers, [])