Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Images with alpha layer not recognized as duplicates
### Description

If I upload the same PDF twice to paperless-ngx (either via consume folder, or via drag & drop), the file is processed only once and for the second file the message appears in File Tasks saying it's a duplicate. But if it's a PNG file (screen shot for instance), the same file can be uploaded multiple times and is not discovered as duplicate.

### Steps to reproduce

1. Upload PNG file to consume folder.
2. Wait a moment for the file to be processed and deleted from the folder.
3. Upload the same file again.
4. It is processed and deleted again. Two identical documents appear.

### Webserver logs

```bash
not checked
```


### Browser logs

_No response_

### Paperless-ngx version

2.0.1

### Host OS

Unraid (Linux)

### Installation method

Docker - official image

### Browser

Brave (Chrome)

### Configuration changes

_No response_

### Other

_No response_


Discussion:
Seems highly unlikely, as the duplicate checking is hash based.  The more likely thing is that the files are not bit for bit the same.
It’s not a duplicate if it’s a different file type
It's not a different file type. I upload the same exact PNG file twice (or more times).
Sorry I misunderstood. But as noted, I think the files are hashed to check duplicate
Can't reproduce it:
![image](https://github.com/paperless-ngx/paperless-ngx/assets/797416/394c3308-d8d2-4729-ae04-13af906bbbc9)

Again, it's bit for bit checking using MD5
Funny enough, I can
<img width="414" alt="Screenshot 2023-12-02 at 7 53 54 AM" src="https://github.com/paperless-ngx/paperless-ngx/assets/4887959/3c1d92b0-a331-4a45-b410-6c43c092cbe6">

used
<img width="264" alt="Screenshot 2023-12-01 at 10 34 16 PM" src="https://github.com/paperless-ngx/paperless-ngx/assets/4887959/e78b4ab4-c613-4557-b5a2-c4ea50311221">
Different checksums =/
<img width="666" alt="Screenshot 2023-12-02 at 8 03 05 AM" src="https://github.com/paperless-ngx/paperless-ngx/assets/4887959/9c920672-1fe9-4d25-82c0-af73cc30060c">
(and archive_checksum)
<img width="977" alt="Screenshot 2023-12-02 at 8 01 37 AM" src="https://github.com/paperless-ngx/paperless-ngx/assets/4887959/ec8706c0-3a1b-4cd3-a3d2-a5c918609a5a">

Ok, that's weird.  Can you put up a sample to play around with?
I used the one from this https://github.com/paperless-ngx/paperless-ngx/issues/4775#issuecomment-1837188591 but I can repro with most it seems, yea surprising
Just guessing - does it generate initial PDF from PNG before OCR? Maybe it takes checksum from the generated PDF, which can differ by internal timestamp?

Resolving patch:
diff --git a/src/documents/consumer.py b/src/documents/consumer.py
--- a/src/documents/consumer.py
+++ b/src/documents/consumer.py
@@ -519,7 +519,11 @@ def progress_callback(current_progress, max_progress):  # pragma: no cover
                     document.filename = generate_unique_filename(document)
                     create_source_path_directory(document.source_path)
 
-                    self._write(document.storage_type, self.path, document.source_path)
+                    self._write(
+                        document.storage_type,
+                        self.original_path,
+                        document.source_path,
+                    )
 
                     self._write(
                         document.storage_type,
@@ -711,21 +715,20 @@ def _store(
 
         storage_type = Document.STORAGE_TYPE_UNENCRYPTED
 
-        with open(self.path, "rb") as f:
-            document = Document.objects.create(
-                title=(
-                    self._parse_title_placeholders(self.override_title)
-                    if self.override_title is not None
-                    else file_info.title
-                )[:127],
-                content=text,
-                mime_type=mime_type,
-                checksum=hashlib.md5(f.read()).hexdigest(),
-                created=create_date,
-                modified=create_date,
-                storage_type=storage_type,
-                original_filename=self.filename,
-            )
+        document = Document.objects.create(
+            title=(
+                self._parse_title_placeholders(self.override_title)
+                if self.override_title is not None
+                else file_info.title
+            )[:127],
+            content=text,
+            mime_type=mime_type,
+            checksum=hashlib.md5(self.original_path.read_bytes()).hexdigest(),
+            created=create_date,
+            modified=create_date,
+            storage_type=storage_type,
+            original_filename=self.filename,
+        )
 
         self.apply_overrides(document)
 
diff --git a/src/documents/parsers.py b/src/documents/parsers.py
--- a/src/documents/parsers.py
+++ b/src/documents/parsers.py
@@ -105,7 +105,7 @@ def get_supported_file_extensions() -> set[str]:
     return extensions
 
 
-def get_parser_class_for_mime_type(mime_type: str) -> Optional["DocumentParser"]:
+def get_parser_class_for_mime_type(mime_type: str) -> Optional[type["DocumentParser"]]:
     """
     Returns the best parser (by weight) for the given mimetype or
     None if no parser exists


Test patch: