CentripetalText: An Efficient Text Instance Representation for Scene Text Detection
Authors: Tao Sheng
Jie Chen
Zhouhui Lian
Venue: NeurIPS 2021 Poster (2021)
Keywords: Scene Text Detection
Scene Text Recognition
Text Instance Representation

Abstract: Scene text detection remains a grand challenge due to the variation in text curvatures, orientations, and aspect ratios. One of the hardest problems in this task is how to represent text instances of arbitrary shapes. Although many methods have been proposed to model irregular texts in a flexible manner, most of them lose simplicity and robustness. Their complicated post-processings and the regression under Dirac delta distribution undermine the detection performance and the generalization ability. In this paper, we propose an efficient text instance representation named CentripetalText (CT), which decomposes text instances into the combination of text kernels and centripetal shifts. Specifically, we utilize the centripetal shifts to implement pixel aggregation, guiding the external text pixels to the internal text kernels. The relaxation operation is integrated into the dense regression for centripetal shifts, allowing the correct prediction in a range instead of a specific value. The convenient reconstruction of text contours and the tolerance of prediction errors in our method guarantee the high detection accuracy and the fast inference speed, respectively. Besides, we shrink our text detector into a proposal generation module, namely CentripetalText Proposal Network (CPN), replacing Segmentation Proposal Network (SPN) in Mask TextSpotter v3 and producing more accurate proposals. To validate the effectiveness of our method, we conduct experiments on several commonly used scene text benchmarks, including both curved and multi-oriented text datasets. For the task of scene text detection, our approach achieves superior or competitive performance compared to other existing methods, e.g., F-measure of 86.3% at 40.0 FPS on Total-Text, F-measure of 86.1% at 34.8 FPS on MSRA-TD500, etc. For the task of end-to-end scene text recognition, our method outperforms Mask TextSpotter v3 by 1.1% in F-measure on Total-Text.

CentripetalText: An Efficient Text Instance Representation for Scene Text Detection
Tao Sheng, Jie Chen, Zhouhui Lian'
Wangxuan Institute of Computer Technology Peking University. Beijing, China
{shengtao, jiechen01, Lianzhouhui)/pkuu edu
Abstract
Scene text detection remains a grand challenge due to the variation in text curva- tures, orientations, and aspect ratios. One of the hardest problems in this task is how to represent text instances of arbitrary shapes. Although many methods have been proposed model irregular texts in flexible manner, most of them lose simplicity and robustness. Their complicated post-processings and the regression under Dirac delta distribution undermine the detection performance and the generalization ability. In this paper, we propose an efficient text instance representation named CentripetalText (CT), which decomposes text instances into the combination of text kernels and centripetal shifts. Specifically, we utilize the centripetal shifts to implement pixel aggregation, guiding the external text pixels to the internal text kernels. The relaxation operation is integrated into the dense regression for cen- tripetal shifts, allowing the correct prediction The convenient reconstruction of text contours and the tolerance of prediction errors our method guarantee the high detection accuracy and the fast inference speed, respectively. Besides, we shrink our text detector into proposal generation module, namely CentripetallTex Proposal Network (CPN), replacing Segmentation Proposal Network (SPN) in Mask TextSpotter v3 and producing more accurate proposals. To validate the effectiveness of our method, we conduct experiments several commonly used scene text benchmarks, including both curved and multi-oriented text datascts. For the task of scene text detection, our approach achieves superior competitive performance compared other existing methods, e.g., F-measure of 86.3% at 40.0 FPS on Total-Text, F-measure 86. 1 at 34.8 FPS on MSRA-TD500, etc. For the task of end-to-end scene text recognition, method outperforms Mask TextSpotter v3 by 1.1 % F-measure on Total-Text.
range instead of specific value.
Introduction
In the past decade, scene text detection has attracted increasing interests the computer vision community, as localizing the region of each text instance in natural images with high accuracy is an essential prerequisite for many practical applications such as blind navigation, scene understanding. and text retrieval. With the rapid development of object detection
18 and segmentation P2 46. 26 50), many promising methods [51 2 3 16 38 have been proposed to solve the problem. However, scene text detection is still challenging task due to the variety of text curvatures, orientations, and aspect ratios.
29
19
How represent text instances in real imagery is one of the major challenges for scene text detection, and usually there are two strategies to solve the problem arising from this challenge. The first is to treat text instances a specific kind of object and use rotated rectangles or quadrangles for
Corresponding author
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

description. This kind of methods are typically inherited from generic object detection and often utilize manually designed anchors for better regression. Obviously, this solution ignores the geometric traits of irregular texts, which may introduce considerable background noises, and furthermore, it difficult to formulate appropriate anchors to fit the texts of various shapes. The other strategy is to decompose text instances into several conceptual or physical components, and reconstruct the steps. For example, PAN BS follows the idea of clustering and aggregates text pixels according to the distances between their embeddings. In TextSnake [23], text instances are represented by text center lines and ordered disks. Consequently, these methods are more flexible and more general than the previous ones in modeling. Nevertheless, most of them suffer from slow inference speed, due to complicated post- processing steps, essentially caused by this kind of tedious comloieemppnnnt--ee baaseeddttt- representation strategy. For another, their component prediction is modeled as a simple Dirac delta distribution, which strietly requires numerical outputs to reach the exact positions and thus weakens the ability to tolerate mistakes. The wrong component prediction will propagate errors to heuristic post-processing procedures, making the rebuilt text contours inaccurate. Based the above observations, we can find out that the implementation of a fast and accurate scene text detector heavily depends on a simple but effective text instance representation with a robust post-processing algorithm, which can tolerate ambiguity and uncertainty.
polygonal contours through a series of indispensable
To overcome these problems, we propose an ef- ficient component-base baseed representation method named CentripetalText (CT) for arbitrary- shaped texts. Enabled by the proposed CT, our scene text detector outperforms other state-of- the-art approaches and achieves the best trade- off between accuracy and speed (as shown in Fig. D. Specifically, as illustrated in Fig, 4 our method consists of two steps: i) input im- ages are fed to the convolutional neural network to predict the probability maps and centripetal shift maps. ii) pixels are grouped to form the text instances through the heuristics based on text kemels and centripetal shifts. In details, the text kernels are generated from the probabil- ity map followed by binarization and connected component search, and the centripetal shifts are predicted at each position of the centripetal shift map. Then each pixel is shifted by the amount of its centripetal shift from its original position in the centripetal shift map to the text kernel pixel or background pixel in the probability map. All pixels that can be shifted into the region of the same text kernel form a text instance. In this manner, reconstruct the final text instances fast and easily through marginal matrix operations and several calls to functions of the OpenCV library. Moreover, we develop an enhanced regression loss, namely the Relaxed L1 Loss, mainly for dense centripetal shift regression, which further improves the detection precision. Benefiting from the new loss, our method is robust to the prediction errors of centripetal shifts because the centripetal shifts which can guide pixels the region of the right text kernel are all regarded as positive. Besides, CT can be fused with CNN-based text detectors or spotters in plug-and-play manner. We replace SPN in Mask TextSpotter v3 B with our Centripetal Text Proposal Network (CPN), a proposal generation module based on CT, which produces more accurate proposals and improves the end-to-end text recognition performance further.
CT-4
Cld12
F-mensure (9)
82-
81
80
Speod (FPS)
25
75 100
Figure 1: Performance and speed of some top- performing real-time scene text detectors on the Total-Text dataset. Enabled by the proposed CT, an efficient representation of text instances, our method outperforms DB [16] and PAN BX and achieves the best trade-off between accuracy and speed. More results are shown in Tab. 3
To evaluate the effectiveness of the proposed CT and Relaxed L1 Loss, we adopt the design of network architecture in PAN [3 and train a powerful end-to-end scene text detector by replacing its text instance representation and loss function with ours. We conduct extensive experiments on the commonly used scene text benchmarks including Total-Text [2], CTW [47 and MSRA- TD500 45), demonstrating that our method achieves superior or competitive performance compared to the state of the art, e.g., F-measure of 86.3% at 40.0 FPS on Texal-Text, F-measure of 86. 1 1% at 34.8 FPS on MSRA-TE TD500, etc. For the task of end-to-end text recognition, cquipped with CPN, the
MSRA-TD500,
2

---

F-measure value of Mask TextSpotter v3 can further be boosted to 71.9% and 79.5% without and with lexicon, respectively.
Major contributions of our work can be summarized as follows:
We propose novel and efficient text instance representation method named CentripetalText. in which text instances are decomposed into the combination of text kernels and centripetal shifts. The attached post-procet post-processing algorithm is also simple and robust, making the generation of text contours fast and accurate.
To reduce the burden of model training, we develop enhanced loss function, namely the Relaxed L1 Loss, mainly for dense centripetal shift regression, which further improves the detection performance.
Equipped with the proposed CT and the Relaxed L1 Loss, our scene text detector achieves superior or competitive results compared to other existing approaches on the curved or oriented text benchmarks, and our end-to-end scene text recognizer surpasses the current state of the art.
2
Related work
Text instance representation methods can be roughly classified into two categories: component- methods and component-basee methods.
Component-free methods treat every text instance complete whole and directly regress the rotated rectangles or quadrangles for describing scene texts without any reconstruction process. These methods usually inspired by general object detectors such Faster R-CNN SSD [19]. and often utilize heuristic anchors prior knowledge. TextBoxes [5] adapted the object detection framework SSD for text detection by modifying the aspect ratios of anchors and the kernel scales of filters. TextBoxes++ (14] and EAST [51]] could predict either rotated rectangles or quadrangles for text regions with and without the prior knowledge of anchors, respectively. SPCnet [40] modified Mask R-CNN 7 by adding the semantic segmentation guidance to suppress false positives.
P and
successfully
Component-based methods prefer to model text instances from local perspectives and decompose instances into components such characters text fragments. SegLink (31] decomposed long texts into locally-detectable segments and links, and combined the segments into whole words according to the links get final detection results. MSR
4 detected scene texts by predicting dense text
boundary points. BR 30 further improved MSR by regressing the positions of boundary points from two opposite directions. PSENet (37] gradually expanded the detected areas from small kernels to complete instances via a progressive scale expansion algorithm.
3 Methodology
In this section, we first introduce our new representation (i.c., CT) for texts of arbitrary shapes. Then, we elaborate on our method and training details.
3.1
Representation
Overview An efficient scene text detector must have well-defined representation for text instances. The traditional description methods inherited from generic object detection (c.g., rotated rectangles or quadrangles) fail to encode the geometric properties of irregular texts. To guarantee the flexibility and generality, we propose a new method named CentripetalTTxxx of text kernels and centripetal shifts. As demonstrated in Fig. 2 our CT expresses text instance as cluster of the pixels which can be shifted into the region of the same text kernel through centripetal shifts. As pixels are the basic units of digital images, CT has the ability model different forms of text instances, regardless their shapes and lengths.
which text instances are composed
Mathematically, given input image 7, the ground-truth annotations are denoted as (Ti, T2, Tj ..). where T; stands for the text instances. Each text instance T; has its corresponding text kernel K;, shrunk version of the original text region. Since text kernel is subset of its text instance, which

---

Text Karnel
Text Region
/
Centripetal Shift
Figure 2: Illustration of the proposed Centripetal Text representation. Text regions (in yellow) can be decomposed into the combination of text kernels (in blue) and centripetal shifts (either in red or green). The centripetal shifts represented as green arrows start from background pixels to non-text-kerne kernel pixels, which are useless for the further generation of text contours, while other centripetal shifts in red start from text region (foreground) pixels to text kernel pixels, which contribute to define the shapes. All the pixels that can be shifted into the region of the same text kernel form text instance. For the sake of better demonstration, we only visualize the centripetal shifts over the bottom text instance.
satisfies K2 C T. we treat it as the main basic of the pixel aggregation. Different against the distance conventional methods, each centripetal shift s, which appears a position Pj of the image guides the clustering of text pixels. In this sense, the text instance T; can be easily represented with the aggregated pixels which can be shifted into the region of text kernel according to the values of their centripetal shifts:
T: = (ps (p, + +,)) E Kij.
Label generation The label generation for the probability map is inspired by PSENet 37 where the positive of the kernel (shaded areas in Fig. 3(b) generated by shrinking the annotated polygon (shaded areas in Fig. ((a)) using the Vatti clipping algorithm [35]. The offset of shrinking is computed based on the perimeter and area of the original polygon and the shrinking ratio set to 0.7 empirically, Since the annotations in the dataset may not perfectly fit the text instances well, we design training mask M to distinguish the supervision of the valid and ignoring regions. The text instance excluding the text kernel (T; K2) is the ignoring region, which means that the gradients in this area are not propagated back to the network. The training mask M can be formulated as follows:
so. if e U K;)
Mj
1, otherwise.
We simply multiply the training mask by the loss of the segmentation branch to eliminate the influence brought by wrong annotations.
In the label generation step of the regression branch, the text instance affects the centripetal shift map in three ways. First, the centripetal shift in the background region - U, T)) should prevent the background pixels from entering into any text kernel and thus we set it to (Ó, 0) intuitively. Second, the centripetal shift the text kernel region (U, K2) should keep the locations of text kernel pixels unchanged and we also set it to (0. 0) for convenience. Third, we expect that each pixel in the region of the text instance excluding the text kernel (U, K2)) can be guided to its corresponding kernel by the centripetal shift. Therefore, we continuously conduct the erosion operation over the text kernel map twice, compare these two temporary results and obtain the text kernel reference (polygons with solid lines in Fig. ((c) as the destination of the generated centripetal shift. As shown in Fig. 3(d) we build the centripetal shift between each pixel in the shaded area and its nearest text kernel reference to prevent numerical accuracy issues caused by rounding off. Note that if two instances overlap, the

---

CCO
(a)
(b)
(c)
(d)
Figure 3: Label generation. (a) Text instance; (b) Text kernel; (c) Text kernel reference; (d) Text region that contains nonzero centripetal shifts.
smaller instance has the higher priority. Specifically, the centripetal shift sy can be calculated by:
(0,0), if (I - UT:)
8) PiPj: pf E U(T; K;) (0,0), py EKK,
(3)
where p represents nearest text kernel reference to pixel pj. During training, the Smooth loss [5] is applied for supervision. Nevertheless, according to previous observation [Α]. the dense regression can be modeled as simple Dirac delta distribution, which fails to consider the ambiguity and uncertainty in the training data. To address the problem, develop regression mask R for the relaxation operation and integrate into the Smooth L1 loss to reduce the burden of model training. We extend the correct prediction from one specific value to range and any centripetal shift which moves the pixel into the right region is treated as positive during training. The regression mask can be formulated follows:
0,
ps E and (p, + + E K; or p, & UT; and (py + 8)) &UK,
R2
(4)
1. otherwise.
where 8J and sy denote the predicted centripetal shift at the position j and its ground truth, respectively, Like the segmentation loss, we multiply the regression mask by the Smooth LI loss and form a novel loss function, namely the Relaxed LI loss for dense centripetal shift prediction, to further improve the detection accuracy. The Relaxed LI loss function can be formulated as follows:
Cregression Σ (R2 Smooth (3j, 8;)),
(5)
where Smoothy: () denotes the standard Smooth L1 loss.
3.2 Scene text detection with CentripetalText
Network Architecture In order to detect texts with arbitrary shapes fast and accurately, we adopt the efficient model design in PAN [38 and equip it with our CT and Relaxed L1 loss. First, ResNet18 X used as the default backbone for fair comparison. Then, to remedy the weak repre- sentation ability of the lightweight backbone, two cascaded FPEMs KSI are utilized to continuously enhance the feature pyramid both top-down and bottom-up manners. Afterwards, the generated feature pyramids of different depths are fused by FFM 38 into single basic feature. Finally, we predict the probability map and the centripetal shift map from the basic feature for further contour generation.
The procedure text contour reconstruction is shown in Fig. 4 After feed-forwarding, the network produces the probability map and the centripetal shift map. We firstly binarize the probability map with a constant threshold (0.2) get the binary map. Then, we find the connected components (text kernels) from the binary map the clusters of pixel aggregation. Afterwards,
Inference
5

---

Probability Map
Binary Map
peod
binarization
HARBOU
CNN
HARINA
poed
grouping
Centripetal Shift Map
LIBERTY GARGOS MARINA
generation
clustering
Text Instance
Text Kernel
Figure 4: An overview of our proposed model.
assign each pixel to the corresponding cluster according to which text kernel (or background) can the pixel be shifted into by its centripetal shift. Finally, we build the text contour for each group of text pixels. Note that our post-pn pett-prcessing strategy has essential difference against PAN [38] The post-processing in PAN is an iterative process, which gradually extends the text kernel to the text region by merging its neighbor pixels iteratively. On the contrary, we conduct the aggregation one step, which means that the centripetal shifts of all pixels can be calculated in parallel by implementing one matrix operation, saving the inference time to a certain extent.
Optimization
Our loss function can be formulated as:
C segmentation + >Cregreseions
(6)
where Esegmeentation denotes the segmentation loss of text kernels, and Cregression denotes the regression loss of centripetal shifts. is a constant to balance the weights the segmentation and regression losses. We set it to 0.05 in all experiments. Specifically, the prediction of text kernels is basically a pixel-wise binary classification problem and we apply the dice loss [66 to handle this task. Equipped with the training mask M, the segmentation loss can be defined as:
Esegmentation =
Σ (M, Dice(cy, Gj)),
where Dice() denotes the dice loss function, EG and cj denote the predicted probability of text kernels the position j and ground truth, respectively. Note that we adopt Online Hard Example Mining (OHEM) 32 to address the imbalance of positives and negatives while calculating Lsegmentation Regarding the regression loss, detailed description has been provided in Sec.
CentripetalText Proposal Network Our scene text detector is shrunk to text proposal module, termed as CentripetalText Proposal Network (CPN), by transforming the polygonal outputs to the minimum area rectangles and instance masks. We follow the main design of the text detection and recognition modules Mask TextSpotter of proposal quality and recognition accuracy.
and replace SPN with our CPN for the comparison

---

Experiments
4.1 Datasets
SynthText [6] is synthetic dataset, consisting of more than 800,000 synthetic images. This dataset used pre-train our model.
Total-Text 2 is curved text dataset including 1,255 training images and 300 testing images. This dataset contains horizontal, multi--rienntdd, and curve text instances labeled at the word level.
47 another curved text dataset including 1,000 training images and 500 testing images. The text instances are annotated at text-line level with 14-polygons.
CTW1500
MSRA-TD500 ! is multi-oriented text dataset which contains 300 training images and 200 testing images with text-line level annotation. Due to its small scale, we follow the previous works 55 23 to include 400 extra training images from HUST-TR400 44
4.2
Implementation details
To make fair comparisons, we use the same training settings described below. ResNet S pre-trained on ImageNet [ is used the backbone of our method. All models are optimized by the Adam optimizer with the batch size of 16 on 4 GPUs. We train our model under two training strategies: (1) learning from scratch; (2) fine-tuning models pre-trained
the Synth'Text dataset. Whichever training strategies, we pre-train models on SenthText for 50k iterations with a fixed learning rate of 1 X 10-3 and train models on real datasets for 36k iterations with the "poly" learning rate strategy 150 where "power" is set to 0.9 and the initial learning rate is 1 x 10-3, We follow the official implementation of PAN implement data augmentation, including random scale, random horizontal flip, random rotation, and random crop. The blurred texts labeled as DO NOT CARE are ignored during training. In addition, we set the gegativee-psitivy ratio of OHEM to 3, and the shrinking rate of text kernel to 0.7. All those models are tested with a batch size of 1 on a GTX 1080Ti GPU without bells and whistles. For the task of end-to-end scene text recognition, we leave the original training and testing settings of Mask TextSpotter v3 unchanged.
4.3 Ablation study
To analyze our designs in depth, we conduct a series of ablation studies on both curve and multi- oriented text datasets (Total-Text and MSRA-TD500). In addition, all models in this subsection are trained from scratch.
Table 1:
Quantitative results of our text de- tection models with different backbones and necks. "F" means F-measure.
Table 2: Comparison between PAN 6K and our
models with different regression losses. "Rep." denotes representation and "F" means F-measure.
MSRA TDND0
MSRA TDSS0
Backhone
Neck
F
EPS
FPS
Rep.
Regression Loss
LPS
F
TPS
ResNet18
FPEM
FPN 18
$4.0
46.8
40.0
81.6
42.0
PAN 18
Smooth [.
83.5
39.6
81.0
78.9
30.2
ResNet50
TPN 18
85.1
25.5
827
21.9
CT (Ours)
Balanced L.
27
83.8
40,0
81.7
34.8
FPEM
24.0
Relaxed
On the one hand, to make full use of the capability of the proposed CT, try different backbones and necks to find the best network architecture. As shown in Tab. D although "ResNet18 + FPN" and "ResNet50+ + FPEM" are the fastest and most accurate detectors, respectively. "ResNet18 FPEM" achieves the best trade-off between accuracy and speed. Thus, we keep this combination by default in the following experiments. On the other hand, we study the validity of the Relaxed L1 loss by replacing it with others. Compared with the baseline Smooth L11 Loss 5 and the newly-released Balanced LI loss (77) the F-measure value of our method improves over 1 %% on both two datasets, which indicates the effectiveness of the Relaxed LI loss. Moreover, under the same settings of the model architecture, our method outperforms PAN by a wide extent while keeping its fast inference speed, indicating that the proposed CT is more efficient.
7

---

Table 3: Quantitative detection results on Total- Total-Text and CTW1500. "P", "R" and "F" represent the precision, recall, and F-measure, respectively. "Ext." denotes external training data. "indicctes the sulti testing is performed.
Total- Text
CTW
1500
Method
Ext.
Venue
P
R
FPS
P
R
FPS
CTPN 34
ECCV' 16
60.4
53.8
56.9
7.1
SegLink
31
CVPR 17
30.3
23.8
26.7
42.3
40.0
40.8
10.7
EAST
|51
CVPR 17
50.0
36.2
42.0
78.7
49.1
60.4
21.2
PSENet 677
CVPR 19
81.8
75.1
78.3
3.9
80.6
75.6
78.0
3.9
PAN 38
ICCV'19
88.0
79.4
83.5
39.6
84.6
77.7
81.0
39.8
CT-320
87.6
72.7
79.4
93.2
85.7
73.2
79.0
107.2
CT-512
87.9
80.8
84.2
57.0
85.2
78.4
81.7
59.8
CT-640
88.8
81.4
84.9
40.0
85.5
79.2
82.2
40.8
TextSnake
23
ECCV' 18
82.7
74.5
78.4
67.9
85.3
75.6
MSR [3]
IJCAI' 19
83.8
74.8
79.0
85.0
78.3
81.5
SegLink+
333
PR 19
82.1
80.9
81.5
82.8
79.8
81.3
PSENet 77
CVPR'19 19
84.0
78.0
80.9
3.9
84.8
79.7
82.2
3.9
SPCNet
40
19
83.0
82.8
82.9
LOMO
48
CVPR 19
87.6
79.3
83.3
85.7
76.5
80.8
CRAFT 11
CVPR 19
87.6
79.9
83.6
86.0
81.1
83.5
Boundary
366
AA
85.2
83.5
84.3
DB 16
2A
87.1
82.5
84.7
32.0
86.9
80.2
83.4
22.0
PAN KK
ICCV'
89.3
81.0
85.0
39.6
86.4
81.2
83.7
39.8
DRRG 40
CVPR
86.5
84.9
85.7
85.9
83.0
84.5
CT-320
88.0
75.4
81.2
93.2
87.7
74.7
80.7
107.2
CT-512
90.2
81.5
85.6
57.0
87.8
79.0
83,2
59.8
CT-640
90.5
82.5
86.3
40.0
88.3
79.9
83.9
40.8
4.4
Comparisons with state-of-the-art methods
Curved text detection We first evaluate our CT on the Total-Text and CTW1500 datasets to examine capability for curved text detection. During testing, we set the short side of images to different scales (320, 512, 640) and keep their aspect ratios. We compare our methods with other state-of-the-art detectors in Tab. D For Total-Text, when learning from scratch, CT-640 achieves the competitive F-measure of 84.9%, surpassing most existing methods pre-trained external text datasets. When pre-training on Syn- thText, the F-measure value of our best model CT-640 reaches 86.3%, which is 0.6% better than second-best DRRG 40 while still ensur- ing the real-time detection speed (40.0 FPS). Fig. | demonstrates the accuracy-speed trade- off of some top-performing real-time text de- tectors, from which can be observed that our CT breaks through the limitation of accuracy- speed boundary. Analogous results can also be obtained on CTW1500 With external training data, the F-measure of CT-640 is 83.9%, the second place of all methods, which is only lower than DGGR. Meanwhile, the speed can still exceed 40 FPS. In summary, proposed CT achieves superior or competitive results compared to state-of-the-art methods, indicating its superiority in modeling curved texts. We visualize our detection results in Fig. for further inspection.
Table Quantitative detection results on MSRA- TD500, "P", "R" and "F" represent the precision, recall, and F-measure, respectively. "Ext." denotes external training data.
Method
Ext.
P
F
FPS
RRPN
82.0
74.0
EAST 51
87.3
67.4
76.1
13.2
PAN 38
80.7
77.3
78.9
30.2
CT-736
87.1
79.3
83.0
34.8
SegLink 31
86.0
70.0
77.0
8.9
PixelLink 8
83.0
73.2
77.8
3.0
TextSnake A3]
83.2
73.9
78.3
1.1
RRD
77
87.0
73.0
79.0
10.0
TextField
87.4
75.9
81.3
CRAFT
88.2
78.2
82.9
8.6
MCN
88.0
79.0
83.0
PAN 38
84.4
83.8
84.1
30.2
DB
16
91.5
79.2
84.9
32.0
DRRG 49
88.1
82.3
85.1
CT-736
90.0
82.5
86.1
34.8
experiments conducted on Total-Text and CTW 1500 demonstrate that

---

WOS
DIRECT
TEA HOUSE MINE TASTINE GASTROKM
WARREN'S
WON OO BOCK
WIN Back!
FJ
org
One Card. One Entry
Figure 5: Qualitative results of the proposed method. Images in row 1-3 are sampled from Total-Teas C100500, and MSRA-TD500, respectively. Ground-truth annotations are in red and our detection results are in green.
Multi-oriented text detection We also evaluate CT on the MSRA-TD5OR dataset to test the robustness in modeling multi-oriented texts. As shown in Tab. 4 CT achieves the F-measure value of 83.0% at 34.8 FPS without external training data. Compared with PAN, our method outperforms by 4.1 %. When pre-training on SynthText, the F-measure value of our CT can further be boosted to 86.1% %. The highest performance and the fastest speed achieved by CT prove its generalization ability deal with texts with extreme aspect ratios and various orientations in complex natural scenarios.
End-to-end text recognition We simply re- place SPN in Mask TextSpotter v3 with our pro- posed CPN develop more powerful end-to- end text recognizer. We evaluate CPN-based text spotter Toxal-Text to test the proposal generation quality for the text spotting task. As shown in Tab. 5 equipped with CPN, Mask TextSpotter v3 achieves the F-measure values of 71.9% and 79.5% when the lexicon is not used and used respectively. Compared with the original version and other state-of-the-art meth- ods, our method can obtain higher performance whether the lexicon is provided or not. Thus, the quantitative results demonstrate that CPN can produce more accurate text proposals than SPN, which is beneficial for recognition and can improve the performance of end-to-end text recognition further.
Table 5: Quantitative end-to-end recognition re- sults on Texal-Text. The evaluation protocol is the same the one in Mask TextSpotter v3 [13] "None" means recognition without any lexicon. "Full" lexicon contains all words in the test set.
* indicates the multi-scale testing is performed.
Method
None
Full
TextBoxes*
15
36.3
48.9
Mask TextSpotter v1
24
52.9
71.8
Qin et al.
28
63.9
Boundary 36
65.0
76.1
Mask TextSpotter v2 12
65.3
77.4
CharNet 41
69.2
ABCNet 20
69.5
78.4
Mask TextSpotter v3
13
71.2
78.4
Mask TextSpotter v3 w/ CPN
71.9
79.5
We visualize the text proposals and the polygon masks generated by SPN and CPN for intuitive comparison. As shown in Fig. we can see that the polygon masks produced by CPN fit the text instances more tightly, which qualitatively proves the superiority of the proposed CPN in producing text proposals compared to other approaches.
5
Conclusion
To keep the simplicity and robustness of text instance representation, we proposed Centripetal Text (CT) which decomposes text instances into the combination of text kernels and centripetal shifts. Text kernels identify the skeletons of text instances while centripetal shifts guide the external text pixels to the internal text kernels. Moreover, to reduce the burden of model training, relaxation operation was integrated into the dense regression for centripetal shifts, allowing the correct prediction in

---

The 3141
The 314L
ddaa others
ahain
others
(a) Segmentation Proposal Network (SPN)
(b) CentripetalText Proposal Network (CPN)
Figure 6: Qualitative comparison of proposals obtained by SPN and CPN. The blue rectangles denote the text proposals and the green areas denote the binary polygon masks.
range. Equipped with the proposed CT, detector achieved superior comparable performance compared to other state-of-the-art methods while keeping the real-time inference speed. The source code is available at https:: bettps://github com/ htt.ssoem//taaaeeeeeeeee shengtaon6 CentripetalTexx and we hope that the proposed CT can serve as a valuable and common representation for scene texts.
Acknowledgments and Disclosure of Funding
This work was supported by Beijing Nova Program of Science and Technology (Grant No.: 1000000 19077), Project 2020BD020 supported by PKU-Baidu Fund, National Language Committee of China (Grant No.: 2D1135-130), Center For Chinese Font Design and Research, Key Laboratory of Science, Technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology), State Key Laboratory of Media Convergence Production Technology and Systems.
References
Youngmin Back, Bado Lee, Dongyoon Han, Sangdoo Yun, and Hwalsuk Lee. Character region awareness text detection. CVPR, pages 9365-9374, 2019.
[2] Chee Kheng Ch' ng and Chee Seng Chan. Total-text: A comprehensive dataset for scene text detection and recognition. In ICDAR, volume pages 935-942. IEEE, 2017.
[3] Dan Deng. Haifeng Liu, Xuelong Li, and Deng Cai. Pixellink: Detecting scene text via instance segmenta- tion. In AAAI, 2018
[4] Deng, Wei Dong, Richard Socher, Li-Jia Kai Fei-Fei. Imagenet: large-scale hierarchical image database. In CVPR, pages 248 255. Icee, 2009.
[5] Ross Girshick. Fast r-cnn. ICCV, pages 1440 1448, 2015.
[6] Ankush Gupta, Andrea Vedaldi, and Andrew Zisserman. Synthetic data for text localisation in natural images. In CVPR, pages 2315-2324, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, pages 2961-2969, 2017.
[8] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.
[9] Dimosthenis Karatzas, Lluis Gomez-Bigorda, Anguelos Nicolaou, Suman Ghosh, Andrew Bagdanov, Masakazu Iwamura, Jiri Matas, Lukas Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu, et al. ledar 2015 competition robust reading. In ICDAR, pages 1156-1160) IEEE, 2015.
[10] Dimosthenis Karatzas, Faisal Shafait, Seiichi Uchida. Masakazu Iwamura, Lluis Gomez Bigorda, Sergi Robles Mestre, Joan Mas, David Fernandez Mota, Jon Almazan Almazan, and Lluis Pere De Las Heras. Icdar 2013 robust reading competition. In ICDAR, pages 1484-1495 IEEE, 2013.
10

---

[11] Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li. Jinhui Tang, and Jian Yang. Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. In NIPS, volume 33, pages 21002-21012. Curran Associates, Inc., 2020.
[12] Minghui Liao, Pengyuan Lyu, Minghang He, Cong Yao, Wenhao Wu, and Xiang Bai. Mask textspotter: An end-to-end trainable neural network for spotting text with arbitrary shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
[13] Minghui Liao, Guan Pang, Jing Huang. Hassner, and Xiang Bai, Mask textspotter v3: Segmentation proposal network for robust scene text spotting. In ECCV, pages 706-722, Cham, 2020. Springer International Publishing.
[14] Minghui Liao, Baoguang Shi, and Xiang Bai. Texthoxes++: A single-shot oriented scene text detector. IEEE Trans. Image Prucesx. 2779);6776-360, 2018.
[15] Minghui Liao, Baoguang Shi, Xiang Bai, Xinggang Wang, and Wenyu Liu. Texthoxes: A fast text detector with single deep neural network. In AAAI, 2017.
[16] Minghui Liao, Zhaoyi Wan, Cong Yao, Kai Chen, and Xiang Bai. Real-time scene text detection with differentiable binarization. In AAAI, 2020.
[17] Minghui Liao, Zhen Zhu, Baoguang Shi. Gui-song Xia, and Xiang Bai. Rotation-sensitiee regression for oriented scene text detection. In CVPR, pages 5909-5918, 2018.
[18] Tsung- Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR, pages 2117-2125, 2017.
[19] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Yang Fu, and Alexander Berg. Ssd: Single shot multibox detector. In ECCV, pages 21-37. Springer, 2016.
[20] Yuliang Liu, Hao Chen, Chunhua Shen, Tong He, Lianwen Jin, and Liangwei Wang. Abenet: Real-time scene text spotting with adaptive bezier-curve network. In CVPR, pages 9809-9818, 18, 2020.
[21] Zichuan Liu, Guosheng Lin, Sheng Yang. Jiashi Feng, Weisi Lin, and Wang Ling Goh. Learning markov clustering networks for scene text detection. In CVPR, pages
IEEE, 2018.
[22] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmenta- tion. CVPR. pages 1341-340, 2015.
[23] Shangbang Long, Jiaqiang Ruan, Wenjie Zhang, Xin He, Wenhao Wu, and Cong Yao. Textsnake: A flexible representation for detecting text of arbitrary shapes. ECCV, pages 20-36, 2018.
[24] Pengyuan Lyu, Minghui Liao, Cong Yao, Wenhao Wa, and Xiang Bai. Mask textspotter: An end-to-end trainable neural network for spotting text with arbitrary shapes. In ECCV, pages 67-83, 2018.
[25] Jianqi Ma, Weiyuan Shao, Hao Ye, Li Wang. Hong Wang. Yingbin Zheng, and Xiangyang Xue, Arbitrary- oriented scene text detection via rotation proposals. JEEE Transactions on Multimedia, 10:113311133122, 2018.
[26] Fausto Milletari, Nassir Navab, and Scyed-Ahmal Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 3DV, pages 565-571. IEEE, 2016.
[27] Jiangmiao Pang, Kai Chen, Jianping Shi. Huajun Feng, Wanli Ouyang, and Dahua Lin. Libra r-cnn: Towards balanced learning for object detection. In CVPR, pages 821-830, 2019.
[28] Siyang Qin, Alessandro Bissacco, Michalis Raptis, Yasuhisa Fujii, and Ying Xiao. Towards unconstrained end-to-end text spotting. In ICCV, pages 4704 4714,,744422 2019.
[29] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS, pages 91-99. 2015.
[30] Tao Sheng and Zhouhui Lian. Bidirectional regression for arbitrary-shaped text detection. In ICDAR, pages 18 187-201, 2021.
[31] Baoguang Shi, Xiang Bai, and Serge Belongic. Detecting oriented text in natural images by linking segments. In CVPR, pages 2550-2558 2017.
[32] Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based based object detectors with online hard example mining. In CVPR, pages 761-769, 2016.

---

[33] Jun Tang, Zhibo Yang, Yongpan Wang, Qi Zheng. Yongchao Xu, and Xiang Bai. Seglink++: Detecting dense and arbitrary-shaper scene text by instance-aware component grouping. PR. 96:106954, 2019.
[34] Zhi Tian, Weilin Huang, Tong He, Pan He, and Yu Qiao. Detecting text in natural image with connectionist text proposal network. In ECCV, pages 56 32.72. Springer, 2016.
[35] Bala R Vatti. A generic solution to polygon clipping. Communications of the ACM, 35(7):56-63, 1992.
[36] Hao Wang, Pu Lu, Hui Zhang, Mingkun Yang, Xiang Bai, Yongchao Xu, Mengchao He, Yongpan Wang, and Wenyu Liu. All you need is houndary: Toward arhitrary-shaped text spotting. In AAAI, 2020.
[37] Wenhai Wang, Enze Xie, Xiang Li, Wenbo Hou, Tong Lu, Gang Yu, and Shuai Shao. Shape robust text detection with progressive scale expansion network. In CVPR, pages 9336-9345, 2019.
[38] Wenhai Wang, Enze Xie, Xiaoge Song, Yuhang Zang, Wenjia Wang, Tong Lu, Gang Yu, and Chunhua Shen. Efficient and accurate arbitrary-shaped text detection with pixel aggregation network. In ICCV, pages 8440-8449, 2019.
[39] Yuxin Wang, Hongtao Xie, Zheng-Jun Zha, Mengting Xing, Zilong Fu, and Yongdong Zhang. Contournet: Taking a further step toward accurate arbitrary-shaped scene text detection. In CVPR, pages 11753-11762, 2020.
[40] Enze Xie, Yuhang Zang, Shuai Shao, Gang Yu, Cong Yao, and Guangyao Li. Scene text detection with supervised pyramid context network. In AAAI, volume 33, pages 9038-9045, 2019.
[41] Linjie Xing, Zhi Tian, Weilin Huang, and Matthew R Scott. Convolutional character networks. In ICCV, pages 9126-9136. 2019.
[42] Yongehao Xu, Yukang Wang, Wei Zhou, Yongpan Wang. Zhiho Yang, and Xiang Bai. Textfield: Learning deep direction field for irregular scene text detection, IEEE Transactions on Image Processing, 28)15)55 5579, 2019.
[43] Chuhui Xue, Shijian Lu, and Wei Zhang. Msr: multi-scale shape regression for scene text detection. In IJCAI, pages 989-995. AAAI Press, 2019.
[44] Cong Yao, Xiang Bai, and Wenyu Liu. A unified framework for multioriented text detection and recognition. IEEE Transactions on Image Processing, (3(11):47774444) 2014.
[45] Cong Yao, Xiang Bai, Wenyu Liu, Ma, and Zhuowen Tu. Detecting texts of arbitrary orientations in natural images. In CVPR, pages 1083-1090. IEEE, 2012.
[46] Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. Bisenet: Bilateral segmentation network for real-time semantic segmentation. In ECCV, pages 221- 2018.
[47] Liu Yuliang, Jin Lianwen, Zhang Shuaitao, and Zhang Sheng. Detecting curve text in the wild: New dataset and new solution. arXiv preprint 1712.02/70 2017.
[48] Chengquan Zhang, Borong Liang, Zuming Huang, Mengyi En, Junyu Han, Errui Ding, and Xinghao Ding. Look more than once: An accurate detector for text of arbitrary shapes. In CVPR, pages 2019.
[49] Shi-Xue Zhang, Xiaobin Zhu, Jic-Bo Hou, Chang Liu, Chun Yang, Hongfa Wang, and Xu-Cheng Yin. Deep relational reasoning graph network for arbitrary shape text detection. In CVPR, pages 9699-9708, 2020.
[50] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiangang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, pages 2881-2890, 2017.
[51] Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang. Shuchang Zhou, Weiran He, and Jiajun Liang, East: an efficient and accurate text detector. In CVPR, pages 5551-5560, 2017.
12


Reviews:
[{"review_id": "pZ1WUyWsxXp", "reviewer": "Reviewer_qKXD", "summary": "The paper deals with the problem of scene text detection, focusing on irregular text.\nThe paper proposes a segmentation approach, in line with recent methods that deal with irregular text detection.\nThe Main contributions of the paper are:\n- A detection method base on predicting both clipped segmentation masks (as other methods do) referred to as text kernels, and shifts for the areas around the kernel that can assign nearby pixels to the kernels. This allows one to perform quick postprocessing, as opposed to some other families of methods like PAN[35] or TextSnake[21].\n- A modified loss to predict the shifts, that uses masking.\nEmpirical results on standard benchmarks show accurate results at better FPS.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality / Novelty:\nPaper build on top of recent methods that cast the text detection problem as one of finding segmentation proposals. As most other methods, the paper has a branch that predicts a clipped segmentation mask that is then thresholded into a binary mask, and where connected components are found to construct the \"core\" regions. Contrary to other methods, that then perform complex post-processing, or to methods such as Mask TextSpotter v3, that simply \"undo\" the clipping, this method proposes to learn, on a second branch, a shift vector on pixels around the text regions, that shifts them towards the text core. This seems to work well in practice (or at least slightly better), but it seems very incremental compared to the unclipping method.\nThe paper also proposes a relaxed L1 loss, but it seems to be an elementwise multiplication of the smooth L1 loss and a binary mask, similar to the loss used by many works on the segmentation branch. \nOverall, the paper is incremental, and I am not sure it reaches a minimum level of originality or novelty.\n\nQuality / clarity: The paper is, unfortunately, not very well written. The method is not well motivated, and, once challenges of current methods are explained, the paper goes straight into what are the steps of the proposed method (page 2), instead of explaining first why the proposed method would help with those problems. I had to go through the paper a couple of times before I could understand the motivation behind the described method, and how it is a smarter version of the automatic unclipping of Mask TextSpotter v3.\n\nSignificance / results: The method is evaluated on standard benchmarks and results seem to outperform state of the art methods at a good FPS. Authors mention that they will release code, which will significantly help the community. I have some questions about some specific results:\n\n- Similar to how the proposed segmentation proposal can be fed into Mask TextSpotter v3 (Table 5, with a slight increase of accuracy), one could detach the recognition branch of Mask TextSpotter v3 and use it only for detection (e.g. Table 2 of the Mask TextSpotter v3 reports results in the MSRA-TD500 dataset). \n1) Why not include the TextSpotter results in Table 4 in this paper?\n2) Compared to the TextSpotter detection shown in Table 5, there seems to be a meaningful increase in accuracy when using the proposed method. However, the proposed method is more expensive since it needs to run an additional head, creating extra activations. What is the FPS and memory requirements of the TextSpotter segmentation proposals, compared to the proposed ones?\n- Figure 3 mentions clustering. Is this just a plain assignment of pixels to the text core after applying the shift, or is there other operation involved?\n\n- Nit: Figure 7a-left in supplementary material, that's actually not an error. The pattern says NONNA GELSA. The proposed method is doing better than the authors are giving it credit for.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Supplementary material shows some limitations / failure cases. A common challenge is deciding whether two components should be connected or not.\nNo discussion about societal impact. Text detection has less obvious societal impact than other tasks such as face identification, but improving it and making it faster  can still have consequences,  e.g. license plate recognition in surveillance systems with lower quality hardware and non-stationary setups.", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "iHP77oxluor", "reviewer": "Reviewer_ZsrE", "summary": "This paper proposes a novel representation of text instances and an algorithm that detects text instances in an image. A text instance is represented as a kernel that is a shrunk version of the text instance along with surrounding pixels that belong to the kernel based on centripetal shifts. This representation simplifies the post-processing to obtain text instances from the output of neural network but yet yields accurate detection. The experimental results confirm the effectiveness of the approach on common scene text datasets containing irregular texts.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed method successfully simplifies the post-processing to obtain the final results from the output of neural network while keeping the flexibility of the pixel-level representation. The paper proposes a solution that solves the problems raised in the paper. Given the simplicity and the effectiveness, it could be one of the representations widely used for scene text recognition. It seems to have good novelty and originality.\n\nHowever, the paper seems to need more work for accept. The followings are my main concerns on the paper:\n\n\n1. The text explaining the Relaxed L1 loss seems to contain errors or very confusing. I was not able to read lines 167-178 very well. Especially, Eq. (4) and (5) did not make sense to me. I might be just wrong, but it looks like R_j is 0 everywhere according to Eq. (1). Then, Eq. (5) is just 0. I would want to be corrected if I do not understand the equations right. Some visualization might help.\n\n\n2. The comparisons with previous methods (for text representation) are conducted by only referring to the numbers reported in the original papers. Having such comparisons are still useful, but they do not necessarily help to demonstrate the superiority of the proposed method due to inevitable differences in their experimental conditions. It would be necessary to re-implement some of the methods that suffer from the problems (i.e. ones losing either simplicity or robustness) and compare the methods in a truly fair manner. If it is challenging to re-implement existing methods, it would be at least necessary to detail the differences among methods and explain the reasons why it is believed that the proposed method is superior.\n\n\n3. I know that it is unfortunately common in the domain, but I am not sure how much informative it is to compare FPS among methods where each experiment could be done in a different condition. I think it is OK to list those numbers, but I am afraid that it does not prove the superiority of the proposed method.\n\n\n4. The writing could be improved. Sections 1-3 are entangled and could be cleaned up. For example, Fig. 3 (or the overview of the method) is not explained in Section 3 but in Section 1. Section 2 is too short and important relevant work like [35] is mentioned only in Section 1.\n\n\n5. Should c_j at line 140 be s_j ?\n\n[update after author response]\n\nMy biggest concern (1) was addressed by the author response.\n\nAs also pointed out by reviewer U3is, we want to expect more detailed analyses of the proposed method. That will include more ablation studies, comparisons with locally-implemented appropriate strong baselines for the method, discussions on \"alternative (not) considered\", etc.\n\nAlthough more analyses are demanded, I see no major flaw in the paper now and it will have a certain value to the community.\n\nI'm happy to correct my rating to 6.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are not particularly discussed in the paper.\n\nNo concerns on potential negative societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "No.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "b_mWjk3nLUF", "reviewer": "Reviewer_U3is", "summary": "This paper presents a method for scene text detection with a neural network predicting both the probability of pixels belonging to text kernels (shrunk version of text bounding polygons) and pixels shifts towards or away from those kernels. The text kernels are predicted from the first head and extended by aggregating surrounding pixels which predicted shifts place them inside the kernel, to form the text regions. Experiments carried on various datasets show a better accuracy with faster predictions than the state of the art methods for text detection and recognition.", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well written, relatively clear and easy to follow. A lot of details are given regarding the model and its training and the experiments look sound and show superior performance of the proposed solution.\n\nHowever, the organisation of the paper could be improved to ease the understanding of the method. Some parts were a bit harder to follow. For example:\n  - It is difficult to understand what the authors mean by centripetal shifts in the introduction. Either give less details or simplify the description\n  - what is meant by \"pixels that can be shifted?\" in the beginning of the \"Methodology\" section?\n  - the interest of text kernel and shifts could be made clearer earlier in the text. Maybe remind the reader why predicting text kernels is preferable to predicting the text region directly.\n  - an example for the regression mask and its usefulness could be nice to better understand the motivation\n\nRegarding the related work, we expect more details and especially how this paper can put in perspective of what already exists. Maybe include or move some parts of the introduction in the related work section. Also, it could be interesting to include more reference in the text of the introduction to support the statements made regarding existing methods.\n\nRegarding the experiments, the results are good across various datasets and the experimental setup is well described. While this makes this simple method significant and useful to the community, we expect a bit more analysis of the method and choices made. In particular, the ablation study is not exactly an ablation study: that section compares network architectures and losses. The results are nonetheless interesting. More importantly, the choice of hyperparameters and the impact they have on the accuracy and speed, as well as the robustness of the method to variation of these hyperparameters could be really helpful to comprehend how the proposed method works and empirically justify the method design (e.g. interpolation factor in the loss, threshold for probability map, shrinking factor for the text kernels, ...)\n\nMisc.\n  - SPN is not defined in the abstract\n  - introduction: slow inference speed -> slow inference?\n  - introduction: fig 3 far from its first reference: maybe switch figures 2 and 3?\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors present many cases were the proposed method works well. It might be interesting to see cases where it fails along with an analysis of the possible reasons.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "SIssFjthheJ", "reviewer": "Reviewer_CRQN", "summary": "Update after discussion among reviewers: I am still strongly in favor of accepting this paper. \n\nThe paper presents a new method to represent arbitrary shaped text in images and shows how this representation can be used in the output layer of some of the recent text detection methods. \n\nThe results look qualitatively and quantitative great. \n\nThe method uses a simple representation that is intuitively understandable. \n\n", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "My biggest concern with the paper is that there appears to be a mismatch between equations 2 and 3: \n\nin eq. 2 the loss for the region \\bigcup(T_i-K_i) is masked out.\nin eq. 3 this is however the most important region. \n\nI am not sure how the method is able to learn anything about this region if the loss is fully masked. \n\n\nThe paper does provide a great comparison to many standard methods. \n\nThe method is well motivated - and I like how it can be applied in the output layer of a bunch of different networks. \n\n\nminor comments: \nabstract: \nmost intractable -> hardest \nabbreviation SPN is not explained. \n1.1% improvement (relative or absolute?)\n\ngenerally, the paper should be carefully copy read and some typos and grammatical mistakes could be fixed. \nsome examples: \nline 49: the mistakes -> mistakes\nline 57: by the CT -> by CT\nline 58: oppoonents -> methods\ncaptuion fig 2: helpless -> useless\n\nbeyond that: the introduction and the abstract feel rather repetitive.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "it's a standard domain - I feel it's an interesting question to think about the general societal impact of scene text recognition - but I wouldn't think this needs to be discussed in this paper which provides an elegant, yet incremental improvement to previous methods. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {"ethical_concerns": "it's a standard domain - I feel it's an interesting question to think about the general societal impact of scene text recognition - but I wouldn't think this needs to be discussed in this paper which provides an elegant, yet incremental improvement to previous methods. "}}]

Author rebuttal:

Meta-review: This paper presents a new method for scene text detection and recognition based on the integration of individual local responses and their centripedal shifts.

The paper has received 4 expert reviews, which were quite positive, with in particular one reviewer championing the paper. While the reviewers (and the AC) agreed, that the paper had weaknesses in presentation and writing, there was a general agreement that the paper has merits, sufficient novelty, and convincing results.

The AC concurs and proposes acceptance.
Decision: Accept (Poster)