Repository: mindee/doctr

Issue:
TypeError: Expected Ptr<cv::UMat> for argument 'array' when using read_pdf()
Can't execute read_pdf() function.

See the pdf file sent over slack to reproduce.

python 3.6.9

```
VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  return array(a, dtype, copy=False, order=order)
Traceback (most recent call last):
  File "/home/jonathan/mindee/dev/client_tests//test_classifier/main.py", line 17, in <module>
    result = model([doc])
  File "/home/jonathan/mindee/dev/client_tests//test_classifier/doctr/doctr/models/core.py", line 51, in __call__
    boxes = self.det_predictor(pages, **kwargs)
  File "/home/jonathan/mindee/dev/client_tests//test_classifier/doctr/doctr/models/detection/core.py", line 140, in __call__
    out = [self.post_processor(batch) for batch in out]
  File "/home/jonathan/mindee/dev/client_tests//test_classifier/doctr/doctr/models/detection/core.py", line 140, in <listcomp>
    out = [self.post_processor(batch) for batch in out]
  File "/home/jonathan/mindee/dev/client_tests//test_classifier/doctr/doctr/models/detection/differentiable_binarization.py", line 173, in __call__
    boxes = self.bitmap_to_boxes(pred=p_, bitmap=bitmap_)
  File "/home/jonathan/mindee/dev/client_tests//test_classifier/doctr/doctr/models/detection/differentiable_binarization.py", line 140, in bitmap_to_boxes
    _box = self.polygon_to_box(points)
  File "/home/jonathan/mindee/dev/client_tests//test_classifier/doctr/doctr/models/detection/differentiable_binarization.py", line 106, in polygon_to_box
    x, y, w, h = cv2.boundingRect(expanded_points)  # compute a 4-points box from expanded polygon
TypeError: Expected Ptr<cv::UMat> for argument 'array'
```


Discussion:
@jonathanMindee Thanks for reporting this!

Would you mind specifying a minimal version of the script you used that raised this error please?
Sure, I used the sample code in the README.md

```python
from doctr.documents import read_pdf, read_img
from doctr.models import ocr_db_crnn

model = ocr_db_crnn(pretrained=True)
# PDF
doc = read_pdf("path/to/your/doc.pdf")
result = model([doc])
```
Strange, I'm not able to reproduce the error, even with the sample pdf.
Could you run our diagnostic script and report back the result?
```
wget https://raw.githubusercontent.com/mindee/doctr/main/scripts/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```
Collecting environment information...

DocTR version: 0.1.1a0
TensorFlow version: 2.4.1
OS: Ubuntu 18.04.5 LTS
Python version: 3.6
Is CUDA available: No
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: GeForce RTX 2080 Ti
Nvidia driver version: 455.45.01
cuDNN version: Probably one of the following:
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.0.5
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.0.5


Resolving patch:
diff --git a/doctr/models/detection/differentiable_binarization.py b/doctr/models/detection/differentiable_binarization.py
--- a/doctr/models/detection/differentiable_binarization.py
+++ b/doctr/models/detection/differentiable_binarization.py
@@ -102,7 +102,18 @@ def polygon_to_box(
         distance = poly.area * self.unclip_ratio / poly.length  # compute distance to expand polygon
         offset = pyclipper.PyclipperOffset()
         offset.AddPath(points, pyclipper.JT_ROUND, pyclipper.ET_CLOSEDPOLYGON)
-        expanded_points = np.asarray(offset.Execute(distance))  # expand polygon
+        _points = offset.Execute(distance)
+        # Take biggest stack of points
+        idx = 0
+        if len(_points) > 1:
+            max_size = 0
+            for _idx, p in enumerate(_points):
+                if len(p) > max_size:
+                    idx = _idx
+                    max_size = len(p)
+            # We ensure that _points can be correctly casted to a ndarray
+            _points = [_points[idx]]
+        expanded_points = np.asarray(_points)  # expand polygon
         if len(expanded_points) < 1:
             return None
         x, y, w, h = cv2.boundingRect(expanded_points)  # compute a 4-points box from expanded polygon


Test patch:
diff --git a/test/test_models_detection.py b/test/test_models_detection.py
--- a/test/test_models_detection.py
+++ b/test/test_models_detection.py
@@ -48,6 +48,35 @@ def test_dbpostprocessor():
     assert all(np.all(np.logical_and(sample[:4] >= 0, sample[:4] <= 1)) for sample in out)
     # Repr
     assert repr(postprocessor) == 'DBPostProcessor(box_thresh=0.1, max_candidates=1000)'
+    # Edge case when the expanded points of the polygon has two lists
+    issue_points = np.array([
+        [869, 561],
+        [923, 581],
+        [925, 595],
+        [915, 583],
+        [889, 583],
+        [905, 593],
+        [882, 601],
+        [901, 595],
+        [904, 604],
+        [876, 608],
+        [915, 614],
+        [911, 605],
+        [925, 601],
+        [930, 616],
+        [911, 617],
+        [900, 636],
+        [931, 637],
+        [904, 649],
+        [932, 649],
+        [932, 628],
+        [918, 627],
+        [934, 624],
+        [935, 573],
+        [909, 569],
+        [934, 562]], dtype=np.int32)
+    out = postprocessor.polygon_to_box(issue_points)
+    assert isinstance(out, tuple) and len(out) == 4
 
 
 def test_db_resnet50_training_mode():