Repository: py-pdf/pypdf
Pull request: BUG: Fix error in cmap extraction
fixes #1533 and late #1091

Problem statement:
PyPDF2 throws exception during extract_text()
I'm working on a script that is parsing PDF invoices and I'm getting exception during pdf reading. This happens only with a specific type of PDF coming from a tapwater utility service provider company. However, all PDFs from them are failed to be parsed with the same error. 

## Environment

Windows 10

```
c:\>python --version
Python 3.11.1

c:\>pip show pyPdf2
Name: PyPDF2
Version: 3.0.1
Summary: A pure-python PDF library capable of splitting, merging, cropping, and transforming PDF files
Home-page:
Author:
Author-email: Mathieu Fenniak <biziqe@mathieu.fenniak.net>
License:
Location: C:\Users\lenemeth\AppData\Local\Programs\Python\Python311\Lib\site-packages
Requires:
Required-by:
````
## Code + PDF
```python
from PyPDF2 import PdfReader
reader = PdfReader(filePath)

for page in reader.pages:
     text = page.extract_text()
````

I can share the PDF in email as it contains personal data (invoice). Let me know where to send it 

## Traceback
```
Traceback (most recent call last):
  File "C:\Users\lenemeth\Documents\AOB\doc\Erőmű 8\albérlet\szamla_parser\EstateManager.py", line 63, in <module>
    em.parse_invoices()
  File "C:\Users\lenemeth\Documents\AOB\doc\Erőmű 8\albérlet\szamla_parser\EstateManager.py", line 22, in parse_invoices
    self.ip.parse_invoices(self.config['input_data']['invoices']['directory_path'])
  File "C:\Users\lenemeth\Documents\AOB\doc\Erőmű 8\albérlet\szamla_parser\InvoiceParser.py", line 47, in parse_invoices
    self.extract_pdf(os.path.join(directory, file))
  File "C:\Users\lenemeth\Documents\AOB\doc\Erőmű 8\albérlet\szamla_parser\InvoiceParser.py", line 63, in extract_pdf
    text = page.extract_text()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\lenemeth\AppData\Local\Programs\Python\Python311\Lib\site-packages\PyPDF2\_page.py", line 1851, in extract_text
    return self._extract_text(
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\lenemeth\AppData\Local\Programs\Python\Python311\Lib\site-packages\PyPDF2\_page.py", line 1342, in _extract_text
    cmaps[f] = build_char_map(f, space_width, obj)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\lenemeth\AppData\Local\Programs\Python\Python311\Lib\site-packages\PyPDF2\_cmap.py", line 28, in build_char_map
    map_dict, space_code, int_entry = parse_to_unicode(ft, space_code)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\lenemeth\AppData\Local\Programs\Python\Python311\Lib\site-packages\PyPDF2\_cmap.py", line 196, in parse_to_unicode
    process_rg, process_char, multiline_rg = process_cm_line(
                                             ^^^^^^^^^^^^^^^^
  File "C:\Users\lenemeth\AppData\Local\Programs\Python\Python311\Lib\site-packages\PyPDF2\_cmap.py", line 264, in process_cm_line
    multiline_rg = parse_bfrange(l, map_dict, int_entry, multiline_rg)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\lenemeth\AppData\Local\Programs\Python\Python311\Lib\site-packages\PyPDF2\_cmap.py", line 278, in parse_bfrange
    nbi = max(len(lst[0]), len(lst[1]))
                               ~~~^^^
IndexError: list index out of range
```

Hints from issue discussion:
At first glance, Looks like a duplicate of #1091
A PR and a fix is proposed can you try it
Thanks! I've tried this [one](https://github.com/py-pdf/pypdf/issues/1091#issuecomment-1371399505) and it seems to be working. However now there is an another issue: the returned text charset seems to be messed up a bit as Hungarian letters (iso-8859-2 / "Latin-2") are unreadable: 

**I got this:** sz♥mlakibocs♥t♦hoz t♣rt☺n☻ regisztr♥ci♦
**Should look like this:** számlakibocsátóhoz történő regisztráció

Not sure if it's because of this particular PDF type but the rest of the invoices using similar alphapet looks fine :) 


@lenemeth  can you provide your pdf please for review
> @lenemeth can you provide your pdf please for review

@pubpub-zz please provide an email address so that I can send it.  It contains personal data (invoice) so I don't want to publicly share it. Thanks for your understanding. 
@lenemeth I know that @pubpub-zz values privacy and I could imagine that he wants to keep his email address private. If you want, you can send it to me and I can forward it: info@martin-thoma.de
@MartinThoma  sent via email. Please share with @pubpub-zz privately. 
I did. Thanks for sharing :-) 
@lenemeth,
thanks for your contribution the extraction was buggy with unicode cmap where ranges were set on multiple lines.

Can you check that the PR is now good for you. I will add a test for coverage
test file for test coverage
[iss1533.pdf](https://github.com/py-pdf/pypdf/files/10376149/iss1533.pdf)

Commit message: Merge remote-tracking branch 'py-pdf/main' into indexerr_cmap_1533

Patch to review:
diff --git a/pypdf/_cmap.py b/pypdf/_cmap.py
index 8c472f87d..bc85c7b4a 100644
--- a/pypdf/_cmap.py
+++ b/pypdf/_cmap.py
@@ -280,13 +280,11 @@ def parse_bfrange(
 ) -> Union[None, Tuple[int, int]]:
     lst = [x for x in line.split(b" ") if x]
     closure_found = False
-    nbi = max(len(lst[0]), len(lst[1]))
-    map_dict[-1] = ceil(nbi / 2)
-    fmt = b"%%0%dX" % (map_dict[-1] * 2)
     if multiline_rg is not None:
+        fmt = b"%%0%dX" % (map_dict[-1] * 2)
         a = multiline_rg[0]  # a, b not in the current line
         b = multiline_rg[1]
-        for sq in lst[1:]:
+        for sq in lst[0:]:
             if sq == b"]":
                 closure_found = True
                 break
@@ -301,6 +299,9 @@ def parse_bfrange(
     else:
         a = int(lst[0], 16)
         b = int(lst[1], 16)
+        nbi = max(len(lst[0]), len(lst[1]))
+        map_dict[-1] = ceil(nbi / 2)
+        fmt = b"%%0%dX" % (map_dict[-1] * 2)
         if lst[2] == b"[":
             for sq in lst[3:]:
                 if sq == b"]":
diff --git a/tests/test_cmap.py b/tests/test_cmap.py
index 17740e167..aa2ca4987 100644
--- a/tests/test_cmap.py
+++ b/tests/test_cmap.py
@@ -3,6 +3,7 @@
 import pytest
 
 from pypdf import PdfReader
+from pypdf._cmap import build_char_map
 from pypdf.errors import PdfReadWarning
 
 from . import get_pdf_from_url
@@ -102,3 +103,11 @@ def test_iss1379():
     name = "02voc.pdf"
     reader = PdfReader(BytesIO(get_pdf_from_url(url, name=name)))
     reader.pages[2].extract_text()
+
+
+def test_iss1533():
+    url = "https://github.com/py-pdf/pypdf/files/10376149/iss1533.pdf"
+    name = "iss1533.pdf"
+    reader = PdfReader(BytesIO(get_pdf_from_url(url, name=name)))
+    reader.pages[0].extract_text()  # no error
+    assert build_char_map("/F", 200, reader.pages[0])[3]["\x01"] == "Ü"


Code review comments:
diff_hunk: @@ -102,3 +103,11 @@ def test_iss1379():
     name = "02voc.pdf"
     reader = PdfReader(BytesIO(get_pdf_from_url(url, name=name)))
     reader.pages[2].extract_text()
+
+
+def test_iss1533():
path: tests/test_cmap.py
text: @user2:
```suggestion

@user1.mark.external
def test_iss1533():
```