Repository: open-mmlab/mmocr

Issue:
[Bug] 数据集预处理中，对于含","的特殊文本未特殊处理
### Prerequisite

- [X] I have searched [Issues](https://github.com/open-mmlab/mmocr/issues) and [Discussions](https://github.com/open-mmlab/mmocr/discussions) but cannot get the expected help.
- [X] The bug has not been fixed in the [latest version (0.x)](https://github.com/open-mmlab/mmocr) or [latest version (1.x)](https://github.com/open-mmlab/mmocr/tree/dev-1.x).

### Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

### Branch

1.x branch https://github.com/open-mmlab/mmocr/tree/dev-1.x

### Environment

本次bug和环境无关
torch                   1.7.0
torchvision             0.8.1
mmcv                    2.0.0rc3
mmcv-full               1.7.0
mmdet                   3.0.0rc0
mmengine                0.1.0
mmocr                   1.0.0rc3 


### Reproduces the problem - code sample

相关代码在mmocr/datasets/preparers/parsers/icdar_txt_parser.py

### Reproduces the problem - command or script

python tools/dataset_converters/prepare_dataset.py icdar2015 --task textspotting

### Reproduces the problem - error message

no error message，检查发现部分图片预处理后的文本gt不对

### Additional information

1. 使用mmocr 1.x版本进行预处理icdar2015数据集为统一格式的json文件
2. 发现对文本处理时，未处理含分隔符的情况，如原始数据集的100,000文本，在统一格式中为100，即分隔符将文本分开后仅取了第一个文本
3. 具体例子如mmocr/data/icdar2015/textspotting_train.json中，在图片data/icdar2015/textdet_imgs/train/img_39.jpg的gt中，原始数据集文本标注为402,85,460,86,457,106,399,105,100,000，其中402,85,460,86,457,106,399,105为polygon标注，100,000为文本标注。但是预处理后的ocr数据集中，文本标注为100。
（因为在区分polygon标注和text标注时，通过分隔符","来区分，导致text文本被截断）



Discussion:
We recommend using English or English & Chinese for issues so that we could have broader discussion.
Thanks for the detailed information! We're looking into this issue and will get back to you asap.

Resolving patch:
diff --git a/mmocr/datasets/preparers/parsers/base.py b/mmocr/datasets/preparers/parsers/base.py
--- a/mmocr/datasets/preparers/parsers/base.py
+++ b/mmocr/datasets/preparers/parsers/base.py
@@ -79,7 +79,8 @@ def loader(self,
                separator: str = ',',
                format: str = 'x1,y1,x2,y2,x3,y3,x4,y4,trans',
                encoding='utf-8') -> Union[Dict, str]:
-        """A basic loader designed for .txt format annotation.
+        """A basic loader designed for .txt format annotation. It greedily
+        extracts information separated by separators.
 
         Args:
             file_path (str): Path to the txt file.
@@ -96,5 +97,8 @@ def loader(self,
         with open(file_path, 'r', encoding=encoding) as f:
             for line in f.readlines():
                 line = line.strip()
+                values = line.split(separator)
+                values = values[:len(keys) -
+                                1] + [separator.join(values[len(keys) - 1:])]
                 if line:
-                    yield dict(zip(keys, line.split(separator)))
+                    yield dict(zip(keys, values))


Test patch:
diff --git a/tests/test_datasets/test_preparers/test_parsers/test_icdar_txt_parsers.py b/tests/test_datasets/test_preparers/test_parsers/test_icdar_txt_parsers.py
--- a/tests/test_datasets/test_preparers/test_parsers/test_icdar_txt_parsers.py
+++ b/tests/test_datasets/test_preparers/test_parsers/test_icdar_txt_parsers.py
@@ -18,6 +18,7 @@ def _create_dummy_ic15_det(self):
             '377,117,463,117,465,130,378,130,Genaxis Theatre',
             '493,115,519,115,519,131,493,131,[06]',
             '374,155,409,155,409,170,374,170,###',
+            '374,155,409,155,409,170,374,170,100,000', ' '
         ]
         ann_file = osp.join(self.root.name, 'ic15_det.txt')
         list_to_file(ann_file, fake_anno)
@@ -25,31 +26,34 @@ def _create_dummy_ic15_det(self):
 
     def _create_dummy_ic15_recog(self):
         fake_anno = [
-            'word_1.png, "Genaxis Theatre"',
-            'word_2.png, "[06]"',
-            'word_3.png, "62-03"',
+            'word_1.png, "Genaxis Theatre"', 'word_2.png, "[06]"',
+            'word_3.png, "62-03"', 'word_4.png, "62-,03"', ''
         ]
         ann_file = osp.join(self.root.name, 'ic15_recog.txt')
         list_to_file(ann_file, fake_anno)
         return ann_file
 
     def test_textdet_parsers(self):
-        parser = ICDARTxtTextDetAnnParser()
         file = self._create_dummy_ic15_det()
+        parser = ICDARTxtTextDetAnnParser()
+
         img, instances = parser.parse_file(file, 'train')
         self.assertEqual(img, file[0])
-        self.assertEqual(len(instances), 3)
+        self.assertEqual(len(instances), 4)
         self.assertIn('poly', instances[0])
         self.assertIn('text', instances[0])
         self.assertIn('ignore', instances[0])
         self.assertEqual(instances[0]['text'], 'Genaxis Theatre')
         self.assertEqual(instances[2]['ignore'], True)
+        self.assertEqual(instances[3]['text'], '100,000')
 
     def test_textrecog_parsers(self):
         parser = ICDARTxtTextRecogAnnParser()
         file = self._create_dummy_ic15_recog()
         samples = parser.parse_files(file, 'train')
-        self.assertEqual(len(samples), 3)
+        self.assertEqual(len(samples), 4)
         img, text = samples[0]
         self.assertEqual(img, 'word_1.png')
         self.assertEqual(text, 'Genaxis Theatre')
+        img, text = samples[3]
+        self.assertEqual(text, '62-,03')