Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Problem with tesseract after Bugfix: Some tesseract languages aren't detected as installed. @stumpylog (#2057)
### Description

Hi,
after Fixes [2044  ](https://github.com/paperless-ngx/paperless-ngx/issues/2044)I have problem with OCR and paperless-ngx.

Before this commit I use next ENV :

> 

      - PAPERLESS_OCR_LANGUAGE=srp_latn+srp
      - PAPERLESS_OCR_LANGUAGES=srp-latn srp script-latn 

and everything work.

After this commit if dont make any changes in ENV error is:
?: The selected ocr language srp_latn is not installed. Paperless cannot OCR your documents without it. Please fix PAPERLESS_OCR_LANGUAGE.

If i make changes in ENV, replace _ with -:

> 

      - PAPERLESS_OCR_LANGUAGE=srp-latn+srp
      - PAPERLESS_OCR_LANGUAGES=srp-latn srp script-latn
After this change system install lang and start paperless, but if I upload any document, OCR dont work, error is:

`[2022-12-04 13:05:46,369] [ERROR] [paperless.consumer] Error while consuming document apr 2022.pdf: MissingDependencyError: OCR engine does not have language data for the following requested languages:

srp-latn

**
Paperless-ngx 1.10.0 WORK
Paperless-ngx 1.10.1 DONT WORK
**
### Steps to reproduce

1. Add this ENV
  - PAPERLESS_OCR_LANGUAGE=srp-latn+srp
  - PAPERLESS_OCR_LANGUAGES=srp-latn srp script-latn

2. Upload any document

### Webserver logs

```bash
[2022-12-04 13:05:46,369] [ERROR] [paperless.consumer] Error while consuming document apr 2022.pdf: MissingDependencyError: OCR engine does not have language data for the following requested languages:

srp-latn

Note: most languages are identified by a 3-digit ISO 639-2 Code

Traceback (most recent call last):

  File "/usr/src/paperless/src/paperless_tesseract/parsers.py", line 292, in parse

    ocrmypdf.ocr(**args)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/api.py", line 331, in ocr

    check_options(options, plugin_manager)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_validation.py", line 246, in check_options

    _check_plugin_options(options, plugin_manager)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_validation.py", line 241, in _check_plugin_options

    check_options_languages(options, ocr_engine_languages)

  File "/usr/local/lib/python3.9/site-packages/ocrmypdf/_validation.py", line 70, in check_options_languages

    raise MissingDependencyError(msg)

ocrmypdf.exceptions.MissingDependencyError: OCR engine does not have language data for the following requested languages:

srp-latn

Note: most languages are identified by a 3-digit ISO 639-2 Code

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/consumer.py", line 337, in try_consume_file

    document_parser.parse(self.path, mime_type, self.filename)

  File "/usr/src/paperless/src/paperless_tesseract/parsers.py", line 346, in parse

    raise ParseError(f"{e.__class__.__name__}: {str(e)}") from e

documents.parsers.ParseError: MissingDependencyError: OCR engine does not have language data for the following requested languages:

srp-latn

Note: most languages are identified by a 3-digit ISO 639-2 Code
```


### Browser logs

_No response_

### Paperless-ngx version

1.10.1

### Host OS

Docker

### Installation method

Docker - official image

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_


Discussion:
With those languages installed (ie 1.10), what does `tesseract --list-langs` output?
Output in 1.10.0
root@paperless:/usr/src/paperless/src# tesseract --list-langs
List of available languages (9):
Latin
deu
eng
fra
ita
osd
spa
srp
srp_latn

and output in 1.10.1
List of available languages (9):
Latin
deu
eng
fra
ita
osd
spa
srp
srp_latn


Resolving patch:
diff --git a/src/paperless_tesseract/checks.py b/src/paperless_tesseract/checks.py
--- a/src/paperless_tesseract/checks.py
+++ b/src/paperless_tesseract/checks.py
@@ -16,8 +16,7 @@ def get_tesseract_langs():
     # Decode bytes to string, split on newlines, trim out the header
     proc_lines = proc.stdout.decode("utf8", errors="ignore").strip().split("\n")[1:]
 
-    # Replace _ with - to convert two part languages to the expected code
-    return [x.replace("_", "-") for x in proc_lines]
+    return [x.strip() for x in proc_lines]
 
 
 @register()


Test patch:
diff --git a/src/paperless_tesseract/tests/test_checks.py b/src/paperless_tesseract/tests/test_checks.py
--- a/src/paperless_tesseract/tests/test_checks.py
+++ b/src/paperless_tesseract/tests/test_checks.py
@@ -27,3 +27,40 @@ def test_invalid_language(self, m):
         msgs = check_default_language_available(None)
         self.assertEqual(len(msgs), 1)
         self.assertEqual(msgs[0].level, ERROR)
+
+    @override_settings(OCR_LANGUAGE="chi_sim")
+    @mock.patch("paperless_tesseract.checks.get_tesseract_langs")
+    def test_multi_part_language(self, m):
+        """
+        GIVEN:
+            - An OCR language which is multi part (ie chi-sim)
+            - The language is correctly formatted
+        WHEN:
+            - Installed packages are checked
+        THEN:
+            - No errors are reported
+        """
+        m.return_value = ["chi_sim", "eng"]
+
+        msgs = check_default_language_available(None)
+
+        self.assertEqual(len(msgs), 0)
+
+    @override_settings(OCR_LANGUAGE="chi-sim")
+    @mock.patch("paperless_tesseract.checks.get_tesseract_langs")
+    def test_multi_part_language_bad_format(self, m):
+        """
+        GIVEN:
+            - An OCR language which is multi part (ie chi-sim)
+            - The language is correctly NOT formatted
+        WHEN:
+            - Installed packages are checked
+        THEN:
+            - No errors are reported
+        """
+        m.return_value = ["chi_sim", "eng"]
+
+        msgs = check_default_language_available(None)
+
+        self.assertEqual(len(msgs), 1)
+        self.assertEqual(msgs[0].level, ERROR)