Repository: open-mmlab/mmocr

Issue:
For dbnet textdect，how to produce a polygonal  prediction
I used the default dbnet with ResNet50 to do text detection，it works pretty good，but the prediction region is always a quadrangle, I would like to change the prediction format to polygon. 

I changed the default model config file ‘dbnet_r50dcnv2_fpnc_1200e_icdar2015.py’line 22- 24 
from  "bbox_head=dict(  type='DBHead',  **text_repr_type='quad',** ... "
to   "bbox_head=dict(  type='DBHead',  **text_repr_type='poly'',** ... "
and retrain the model, the loss decrease  as normal, but for inference it didn't work, there is always no result and get“UserWarning: 0 text found”.

Is there anything else to change to achieve a polygonal  prediction ？

Looking forward to hearing from you.
Thanks.




Discussion:
Your setting is correct.  Does your model converge?   at first 100 epoches,  it is normal there is no any detected text.  you'd better uploading your training log
> Your setting is correct. Does your model converge? at first 100 epoches, it is normal there is no any detected text. you'd better uploading your training log

Thanks for your quick response. Since my dataset is not big, it converges fast，and I've retried to train 110 epoch, no result again, here is my training log. Thanks again. 
[dbnet_r50dcnv2_poly_20210508_100141.log](https://github.com/open-mmlab/mmocr/files/6445113/dbnet_r50dcnv2_poly_20210508_100141.log)

> > Your setting is correct. Does your model converge? at first 100 epoches, it is normal there is no any detected text. you'd better uploading your training log
> 
> Thanks for your quick response. Since my dataset is not big, it converges fast，and I've retried to train 110 epoch, no result again, here is my training log. Thanks again.
> [dbnet_r50dcnv2_poly_20210508_100141.log](https://github.com/open-mmlab/mmocr/files/6445113/dbnet_r50dcnv2_poly_20210508_100141.log)
The log seems normal. If you change the type to quad, will it output some results via testing the latest saved checkpoint?

I can help you to debug it if you can upload one image,  the config and checkpoints.

BTW, I trained ctw, which uses the type poly. It can ouput correct results. 
I change points2boundary function in mmocr/core/mask.py and get acceptable result. I think it is a bug.

```
def points2boundary(points, text_repr_type, text_score=None, min_width=-1):
    """Convert a text mask represented by point coordinates sequence into a
    text boundary.

    Args:
        points (ndarray): Mask index of size (n, 2).
        text_repr_type (str): Text instance encoding type
            ('quad' for quadrangle or 'poly' for polygon).
        text_score (float): Text score.

    Returns:
        boundary (list[float]): The text boundary point coordinates (x, y)
            list. Return None if no text boundary found.
    """
    assert isinstance(points, np.ndarray)
    assert points.shape[1] == 2
    assert text_repr_type in ['quad', 'poly']
    assert text_score is None or 0 <= text_score <= 1

    if text_repr_type == 'quad':
        rect = cv2.minAreaRect(points)
        vertices = cv2.boxPoints(rect)
        boundary = []
        if min(rect[1]) > min_width:
            boundary = [p for p in vertices.flatten().tolist()]

    elif text_repr_type == 'poly':

        # height = np.max(points[:, 1]) + 10
        # width = np.max(points[:, 0]) + 10

        # mask = np.zeros((height, width), np.uint8)
        # mask[points[:, 1], points[:, 0]] = 255

        # contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL,
        #                                cv2.CHAIN_APPROX_SIMPLE)
        # boundary = list(contours[0].flatten().tolist())
        boundary = points.flatten().tolist()

    if text_score is not None:
        boundary = boundary + [text_score]
    if len(boundary) < 8:
        return None

    return boundary
```

Resolving patch:
diff --git a/mmocr/models/textdet/postprocess/wrapper.py b/mmocr/models/textdet/postprocess/wrapper.py
--- a/mmocr/models/textdet/postprocess/wrapper.py
+++ b/mmocr/models/textdet/postprocess/wrapper.py
@@ -223,7 +223,14 @@ def db_decode(preds,
         if len(poly) == 0 or isinstance(poly[0], list):
             continue
         poly = poly.reshape(-1, 2)
-        poly = points2boundary(poly, text_repr_type, score, min_text_width)
+
+        if text_repr_type == 'quad':
+            poly = points2boundary(poly, text_repr_type, score, min_text_width)
+        elif text_repr_type == 'poly':
+            poly = poly.flatten().tolist() + [score]
+        else:
+            raise ValueError(f'Invalid text repr type {text_repr_type}')
+
         if poly is not None:
             boundaries.append(poly)
     return boundaries


Test patch:
diff --git a/tests/test_utils/test_wrapper.py b/tests/test_utils/test_wrapper.py
--- a/tests/test_utils/test_wrapper.py
+++ b/tests/test_utils/test_wrapper.py
@@ -1,4 +1,5 @@
 import numpy as np
+import pytest
 import torch
 
 from mmocr.models.textdet.postprocess.wrapper import (comps2boundaries,
@@ -93,3 +94,22 @@ def test_textsnake_decode():
 
     results = textsnake_decode(torch.squeeze(maps))
     assert len(results) == 0
+
+
+def test_db_decode():
+    pred = torch.zeros((1, 8, 8))
+    pred[0, 2:7, 2:7] = 0.8
+    expect_result_quad = [[
+        1.0, 8.0, 1.0, 1.0, 8.0, 1.0, 8.0, 8.0, 0.800000011920929
+    ]]
+    expect_result_poly = [[
+        8, 2, 8, 6, 6, 8, 2, 8, 1, 6, 1, 2, 2, 1, 6, 1, 0.800000011920929
+    ]]
+    with pytest.raises(ValueError):
+        db_decode(preds=pred, text_repr_type='dummpy')
+    result_quad = db_decode(
+        preds=pred, text_repr_type='quad', min_text_width=1)
+    result_poly = db_decode(
+        preds=pred, text_repr_type='poly', min_text_width=1)
+    assert result_quad == expect_result_quad
+    assert result_poly == expect_result_poly