Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Paperless does not consume Word file ('BaseResponse' object has no attribute 'content')
### Description

A consumed Word file will not be passed on to Gotenberg. This error occurs after updating to version 1.16.0, with version 1.16.1 the behavior is unchanged. - A downgrade to version 1.15.1 eliminates the problem.

### Steps to reproduce

1. Docker update to Paperless NGX 1.16.0 or 1.16.1
1. Copy a Word file to consume directory
1. Result failed in File Tasks: 'BaseResponse' object has no attribute 'content'


### Webserver logs

```bash
2023-06-17T21:39:40.156450809Z [2023-06-17 23:39:40,155] [INFO] [paperless.management.consumer] Adding /usr/src/paperless/consume/yyy.doc to the task queue.
2023-06-17T21:39:40.198374101Z [2023-06-17 23:39:40,196] [INFO] [celery.worker.strategy] Task documents.tasks.consume_file[9b3796c2-e77d-403d-907d-e9d039c23e18] received
2023-06-17T21:39:40.254113883Z [2023-06-17 23:39:40,252] [WARNING] [paperless.barcodes] Unsupported file format for barcode reader: application/msword
2023-06-17T21:39:40.255892299Z [2023-06-17 23:39:40,254] [WARNING] [paperless.barcodes] Exception during barcode scanning: 'NoneType' object has no attribute 'name'
2023-06-17T21:39:40.366714534Z [2023-06-17 23:39:40,364] [INFO] [paperless.consumer] Consuming yyy.doc
2023-06-17T21:39:40.678834755Z [2023-06-17 23:39:40,677] [INFO] [paperless.parsing.tika] Sending /tmp/paperless/paperless-ngxu20164qy/yyy.doc to Tika server
2023-06-17T21:39:40.971540790Z [2023-06-17 23:39:40,969] [ERROR] [celery.app.trace] Task documents.tasks.consume_file[9b3796c2-e77d-403d-907d-e9d039c23e18] raised unexpected: AttributeError("'BaseResponse' object has no attribute 'content'")
2023-06-17T21:39:40.971808472Z Traceback (most recent call last):
2023-06-17T21:39:40.971895454Z   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 477, in trace_task
2023-06-17T21:39:40.971956328Z     R = retval = fun(*args, **kwargs)
2023-06-17T21:39:40.972011965Z   File "/usr/local/lib/python3.9/site-packages/celery/app/trace.py", line 760, in __protected_call__
2023-06-17T21:39:40.972064051Z     return self.run(*args, **kwargs)
2023-06-17T21:39:40.972111524Z   File "/usr/src/paperless/src/documents/tasks.py", line 133, in consume_file
2023-06-17T21:39:40.972163329Z     document = Consumer().try_consume_file(
2023-06-17T21:39:40.972211144Z   File "/usr/src/paperless/src/documents/consumer.py", line 382, in try_consume_file
2023-06-17T21:39:40.972262882Z     document_parser.parse(self.path, mime_type, self.filename)
2023-06-17T21:39:40.972310896Z   File "/usr/src/paperless/src/paperless_tika/parsers.py", line 61, in parse
2023-06-17T21:39:40.972359728Z     self.text = parsed.content.strip()
2023-06-17T21:39:40.972404697Z AttributeError: 'BaseResponse' object has no attribute 'content'
```


### Browser logs

_No response_

### Paperless-ngx version

1.16.0 or 1.16.1

### Host OS

Docker container on Synology NAS

### Installation method

Docker - official image

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_


Discussion:
That's Tika, not Gotenberg, but should be easy enough to fix
Having a sample file would be helpful, as our testing does consume a docx and odt document without issue
A docx document works. The attached doc document doesn't work.
[yyy.zip](https://github.com/paperless-ngx/paperless-ngx/files/11780214/yyy.zip)

Ah, that's the problem then.  I never tested an older format.

Resolving patch:
diff --git a/src/paperless_mail/parsers.py b/src/paperless_mail/parsers.py
--- a/src/paperless_mail/parsers.py
+++ b/src/paperless_mail/parsers.py
@@ -13,6 +13,7 @@
 from imap_tools import MailAttachment
 from imap_tools import MailMessage
 from tika_client import TikaClient
+from tika_client.data_models import TikaKey
 
 from documents.parsers import DocumentParser
 from documents.parsers import ParseError
@@ -172,8 +173,12 @@ def tika_parse(self, html: str):
             with TikaClient(tika_url=self.tika_server) as client:
                 parsed = client.tika.as_text.from_buffer(html, "text/html")
 
-                if "X-TIKA:content" in parsed.data:
-                    return parsed.data["X-TIKA:content"].strip()
+                if hasattr(parsed, "content") and parsed.content is not None:
+                    return parsed.content.strip()
+                elif TikaKey.Content in parsed.data:
+                    # May not be a completely handled type, but
+                    # the Tika response may still include content
+                    return parsed.data[TikaKey.Content].strip()
                 return ""
         except Exception as err:
             raise ParseError(
diff --git a/src/paperless_tika/parsers.py b/src/paperless_tika/parsers.py
--- a/src/paperless_tika/parsers.py
+++ b/src/paperless_tika/parsers.py
@@ -4,6 +4,7 @@
 import httpx
 from django.conf import settings
 from tika_client import TikaClient
+from tika_client.data_models import TikaKey
 
 from documents.parsers import DocumentParser
 from documents.parsers import ParseError
@@ -58,8 +59,15 @@ def parse(self, document_path: Path, mime_type: str, file_name=None):
                 f"{settings.TIKA_ENDPOINT}: {err}",
             ) from err
 
-        self.text = parsed.content.strip()
-        self.date = parsed.metadata.created
+        self.text = None
+        if hasattr(parsed, "content") and parsed.content is not None:
+            self.text = parsed.content.strip()
+        elif TikaKey.Content in parsed.data:
+            # May not be a completely handled type, but
+            # the Tika response may still include content
+            self.text = parsed.data[TikaKey.Content].strip()
+
+        self.date = parsed.created
         self.archive_path = self.convert_to_pdf(document_path, file_name)
 
     def convert_to_pdf(self, document_path, file_name):


Test patch:
diff --git a/src/paperless_tika/tests/samples/sample.doc b/src/paperless_tika/tests/samples/sample.doc
new file mode 100644
Binary files /dev/null and b/src/paperless_tika/tests/samples/sample.doc differ
diff --git a/src/paperless_tika/tests/test_live_tika.py b/src/paperless_tika/tests/test_live_tika.py
--- a/src/paperless_tika/tests/test_live_tika.py
+++ b/src/paperless_tika/tests/test_live_tika.py
@@ -118,3 +118,28 @@ def test_basic_parse_docx(self):
             self.assertTrue(b"PDF-" in f.read()[:10])
 
         # self.assertEqual(self.parser.date, datetime.datetime(2022, 9, 14))
+
+    def test_basic_parse_doc(self):
+        """
+        GIVEN:
+            - An input DOC format document
+        WHEN:
+            - The document is parsed
+        THEN:
+            - Document content is correct
+            - Document date is correct
+        """
+        test_file = self.SAMPLE_DIR / "sample.doc"
+
+        self.try_parse_with_wait(
+            test_file,
+            "application/msword",
+        )
+
+        self.assertIn(
+            "his is a test document, saved in the older .doc format",
+            self.parser.text,
+        )
+        self.assertIsNotNone(self.parser.archive_path)
+        with open(self.parser.archive_path, "rb") as f:
+            self.assertTrue(b"PDF-" in f.read()[:10])