Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Thumbnail of PDF is not created
### Description

There is an (stale closed) issue for this: #895 

this issue still seems to persist: when uploading a PDF I have no thumbnail. The document itself looks fine.
The log indicates an issue with imagemagicks policy config, which i can't change in the docker seup

### Steps to reproduce

upload pdf
see documents

### Webserver logs

```bash
2023-08-22 17:51:30,893] [INFO] [paperless.consumer] Consuming Rechnung_91011637555_21.06.2018.pdf
[2023-08-22 17:51:30,901] [DEBUG] [paperless.consumer] Detected mime type: application/pdf
[2023-08-22 17:51:30,906] [DEBUG] [paperless.consumer] Parser: RasterisedDocumentParser
[2023-08-22 17:51:30,926] [DEBUG] [paperless.consumer] Parsing Rechnung_91011637555_21.06.2018.pdf...
[2023-08-22 17:51:31,007] [DEBUG] [paperless.parsing.tesseract] Document has text, skipping OCRmyPDF entirely.
[2023-08-22 17:51:31,008] [DEBUG] [paperless.consumer] Generating thumbnail for Rechnung_91011637555_21.06.2018.pdf...
[2023-08-22 17:51:31,028] [DEBUG] [paperless.parsing] Execute: convert -density 300 -scale 500x5000> -alpha remove -strip -auto-orient /tmp/paperless/paperless-ngx8ky7r7b4/Rechnung_91011637555_21.06.2018.pdf[0] /tmp/paperless/paperless-g_6p8tad/convert.webp
[2023-08-22 17:51:31,392] [WARNING] [paperless.parsing] Thumbnail generation with ImageMagick failed, falling back to ghostscript. Check your /etc/ImageMagick-x/policy.xml!
[2023-08-22 17:51:31,622] [DEBUG] [paperless.parsing] Execute: convert -density 300 -scale 500x5000> -alpha remove -strip -auto-orient /tmp/paperless/paperless-g_6p8tad/gs_out.png /tmp/paperless/paperless-g_6p8tad/convert_gs.webp
[2023-08-22 17:51:35,688] [DEBUG] [paperless.consumer] Saving record to database
[2023-08-22 17:51:35,689] [DEBUG] [paperless.consumer] Creation date from parse_date: 2023-06-13 00:00:00+02:00
[2023-08-22 17:51:36,343] [DEBUG] [paperless.consumer] Saving record to database
[2023-08-22 17:51:36,344] [DEBUG] [paperless.consumer] Creation date from parse_date: 2018-06-20 00:00:00+02:00
```


### Browser logs

_No response_

### Paperless-ngx version

Paperless-ngx 1.17.1

### Host OS

Raspberry pi arm64

### Installation method

Docker - official image

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_


Discussion:
What is the output of running `convert -density 300 -scale 500x5000> -alpha remove -strip -auto-orient Rechnung_91011637555_21.06.2018.pdf[0] convert.webp` using the same version of ImageMagik?
Also a fallback exists using ghostscript, which didn't fail, so this doesn't seem like a big issue.  It also doesn't look to be generally broken, I get thumbnails without issue with PDFs.  Some PDFs simply have issues or otherwise can't be processed.
Yea if you look more carefully those logs indicate paperless did successfully create the thumbnail. You haven’t provided much other info but if it’s not displaying that can be a myriad of issues, even just an issue with serving the file etc.

And as noted this isn’t widely broken. Please provide more info eg can you confirm the thumbnail isn’t there (again, logs suggest it is)? What happens when you try to load the file as it’s called from the browser?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I do see the same issue. Might be related to the PDF being encrypted.

```
[2023-09-09 15:20:07,523] [INFO] [paperless.consumer] Consuming REWE-eBon.pdf
[2023-09-09 15:20:07,524] [DEBUG] [paperless.consumer] Detected mime type: application/pdf
[2023-09-09 15:20:07,526] [DEBUG] [paperless.consumer] Parser: RasterisedDocumentParser
[2023-09-09 15:20:07,529] [DEBUG] [paperless.consumer] Parsing REWE-eBon.pdf...
[2023-09-09 15:20:07,735] [DEBUG] [paperless.parsing.tesseract] Calling OCRmyPDF with args: {'input_file': PosixPath('/tmp/paperless/paperless-ngxw3hvakai/REWE-eBon.pdf'), 'output_file': PosixPath('/tmp/paperless/paperless-tk3p70a4/archive.pdf'), 'use_threads': True, 'jobs': 4, 'language': 'deu+eng', 'output_type': 'pdfa', 'progress_bar': False, 'skip_text': True, 'clean': True, 'deskew': True, 'rotate_pages': True, 'rotate_pages_threshold': 12.0, 'sidecar': PosixPath('/tmp/paperless/paperless-tk3p70a4/sidecar.txt')}
[2023-09-09 15:20:08,105] [WARNING] [paperless.parsing.tesseract] This file is encrypted, OCR is impossible. Using any text present in the original file.
[2023-09-09 15:20:08,105] [DEBUG] [paperless.consumer] Generating thumbnail for REWE-eBon.pdf...
[2023-09-09 15:20:08,110] [DEBUG] [paperless.parsing] Execute: convert -density 300 -scale 500x5000> -alpha remove -strip -auto-orient /tmp/paperless/paperless-ngxw3hvakai/REWE-eBon.pdf[0] /tmp/paperless/paperless-tk3p70a4/convert.webp
[2023-09-09 15:20:08,254] [WARNING] [paperless.parsing] Thumbnail generation with ImageMagick failed, falling back to ghostscript. Check your /etc/ImageMagick-x/policy.xml!
[2023-09-09 15:20:08,323] [DEBUG] [paperless.parsing] Execute: convert -density 300 -scale 500x5000> -alpha remove -strip -auto-orient /tmp/paperless/paperless-tk3p70a4/gs_out.png /tmp/paperless/paperless-tk3p70a4/convert_gs.webp
[2023-09-09 15:20:09,554] [DEBUG] [paperless.consumer] Saving record to database
[2023-09-09 15:20:09,554] [DEBUG] [paperless.consumer] Creation date from parse_date: 2023-09-09 00:00:00+02:00
[2023-09-09 15:20:09,804] [INFO] [paperless.handlers] Assigning document type Nachricht to 2023-09-09 REWE eBon Dein REWE eBon vom 09.09.2023
[2023-09-09 15:20:10,913] [DEBUG] [paperless.consumer] Deleting file /tmp/paperless/paperless-ngxw3hvakai/REWE-eBon.pdf
[2023-09-09 15:20:10,921] [DEBUG] [paperless.parsing.tesseract] Deleting directory /tmp/paperless/paperless-tk3p70a4
[2023-09-09 15:20:10,924] [INFO] [paperless.consumer] Document 2023-09-09 REWE eBon Dein REWE eBon vom 09.09.2023 consumption finished
```

The resulting document is indeed missing a thumbnail.

![image](https://github.com/paperless-ngx/paperless-ngx/assets/2642714/cecae521-a002-47c9-a008-c5988a2cf057)

Viewing the document works flawlessly however.

![image](https://github.com/paperless-ngx/paperless-ngx/assets/2642714/73fb3cd5-916c-45b9-9027-e12358424d77)

I'm happy to provide the original PDF if required.
What does the browser network tab show?
200 OK for each Thumb in the documents overview

![image](https://github.com/paperless-ngx/paperless-ngx/assets/2642714/ef4d5d1d-eaa0-49ee-a3be-a06447ab89a7)

That thumbnail looks like the fallback fallback (ie convert and ghostscript both failed). It's this file: https://github.com/paperless-ngx/paperless-ngx/blob/dev/src/documents/resources/document.png

edit: looks like no log when that one fails.  Probably should correct that

Resolving patch:
diff --git a/src/documents/parsers.py b/src/documents/parsers.py
--- a/src/documents/parsers.py
+++ b/src/documents/parsers.py
@@ -166,11 +166,11 @@ def run_convert(
         raise ParseError(f"Convert failed at {args}")
 
 
-def get_default_thumbnail() -> str:
+def get_default_thumbnail() -> Path:
     """
     Returns the path to a generic thumbnail
     """
-    return os.path.join(os.path.dirname(__file__), "resources", "document.png")
+    return (Path(__file__).parent / "resources" / "document.webp").resolve()
 
 
 def make_thumbnail_from_pdf_gs_fallback(in_path, temp_dir, logging_group=None) -> str:
@@ -183,12 +183,13 @@ def make_thumbnail_from_pdf_gs_fallback(in_path, temp_dir, logging_group=None) -
         "to ghostscript. Check your /etc/ImageMagick-x/policy.xml!",
         extra={"group": logging_group},
     )
+    # Ghostscript doesn't handle WebP outputs
     gs_out_path = os.path.join(temp_dir, "gs_out.png")
     cmd = [settings.GS_BINARY, "-q", "-sDEVICE=pngalpha", "-o", gs_out_path, in_path]
     try:
         if not subprocess.Popen(cmd).wait() == 0:
             raise ParseError(f"Thumbnail (gs) failed at {cmd}")
-        # then run convert on the output from gs
+        # then run convert on the output from gs to make WebP
         run_convert(
             density=300,
             scale="500x5000>",
@@ -203,11 +204,12 @@ def make_thumbnail_from_pdf_gs_fallback(in_path, temp_dir, logging_group=None) -
 
         return out_path
 
-    except ParseError:
+    except ParseError as e:
+        logger.error(f"Unable to make thumbnail with Ghostscript: {e}")
         # The caller might expect a generated thumbnail that can be moved,
         # so we need to copy it before it gets moved.
         # https://github.com/paperless-ngx/paperless-ngx/issues/3631
-        default_thumbnail_path = os.path.join(temp_dir, "document.png")
+        default_thumbnail_path = os.path.join(temp_dir, "document.webp")
         copy_file_with_basic_stats(get_default_thumbnail(), default_thumbnail_path)
         return default_thumbnail_path
 
@@ -231,7 +233,8 @@ def make_thumbnail_from_pdf(in_path, temp_dir, logging_group=None) -> str:
             output_file=out_path,
             logging_group=logging_group,
         )
-    except ParseError:
+    except ParseError as e:
+        logger.error(f"Unable to make thumbnail with convert: {e}")
         out_path = make_thumbnail_from_pdf_gs_fallback(in_path, temp_dir, logging_group)
 
     return out_path


Test patch:
diff --git a/src/documents/tests/test_file_handling.py b/src/documents/tests/test_file_handling.py
--- a/src/documents/tests/test_file_handling.py
+++ b/src/documents/tests/test_file_handling.py
@@ -820,7 +820,7 @@ def test_archive_deleted(self):
 
     @override_settings(FILENAME_FORMAT="{title}")
     def test_archive_deleted2(self):
-        original = os.path.join(settings.ORIGINALS_DIR, "document.png")
+        original = os.path.join(settings.ORIGINALS_DIR, "document.webp")
         original2 = os.path.join(settings.ORIGINALS_DIR, "0000001.pdf")
         archive = os.path.join(settings.ARCHIVE_DIR, "0000001.pdf")
         Path(original).touch()
@@ -828,9 +828,9 @@ def test_archive_deleted2(self):
         Path(archive).touch()
 
         doc1 = Document.objects.create(
-            mime_type="image/png",
+            mime_type="image/webp",
             title="document",
-            filename="document.png",
+            filename="document.webp",
             checksum="A",
             archive_checksum="B",
             archive_filename="0000001.pdf",