Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a method for scene text detection with a neural network predicting both the probability of pixels belonging to text kernels (shrunk version of text bounding polygons) and pixels shifts towards or away from those kernels. The text kernels are predicted from the first head and extended by aggregating surrounding pixels which predicted shifts place them inside the kernel, to form the text regions. Experiments carried on various datasets show a better accuracy with faster predictions than the state of the art methods for text detection and recognition.
main_review: The paper is well written, relatively clear and easy to follow. A lot of details are given regarding the model and its training and the experiments look sound and show superior performance of the proposed solution.

However, the organisation of the paper could be improved to ease the understanding of the method. Some parts were a bit harder to follow. For example:
  - It is difficult to understand what the authors mean by centripetal shifts in the introduction. Either give less details or simplify the description
  - what is meant by "pixels that can be shifted?" in the beginning of the "Methodology" section?
  - the interest of text kernel and shifts could be made clearer earlier in the text. Maybe remind the reader why predicting text kernels is preferable to predicting the text region directly.
  - an example for the regression mask and its usefulness could be nice to better understand the motivation

Regarding the related work, we expect more details and especially how this paper can put in perspective of what already exists. Maybe include or move some parts of the introduction in the related work section. Also, it could be interesting to include more reference in the text of the introduction to support the statements made regarding existing methods.

Regarding the experiments, the results are good across various datasets and the experimental setup is well described. While this makes this simple method significant and useful to the community, we expect a bit more analysis of the method and choices made. In particular, the ablation study is not exactly an ablation study: that section compares network architectures and losses. The results are nonetheless interesting. More importantly, the choice of hyperparameters and the impact they have on the accuracy and speed, as well as the robustness of the method to variation of these hyperparameters could be really helpful to comprehend how the proposed method works and empirically justify the method design (e.g. interpolation factor in the loss, threshold for probability map, shrinking factor for the text kernels, ...)

Misc.
  - SPN is not defined in the abstract
  - introduction: slow inference speed -> slow inference?
  - introduction: fig 3 far from its first reference: maybe switch figures 2 and 3?


limitations_and_societal_impact: The authors present many cases were the proposed method works well. It might be interesting to see cases where it fails along with an analysis of the possible reasons.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper deals with the problem of scene text detection, focusing on irregular text.
The paper proposes a segmentation approach, in line with recent methods that deal with irregular text detection.
The Main contributions of the paper are:
- A detection method base on predicting both clipped segmentation masks (as other methods do) referred to as text kernels, and shifts for the areas around the kernel that can assign nearby pixels to the kernels. This allows one to perform quick postprocessing, as opposed to some other families of methods like PAN[35] or TextSnake[21].
- A modified loss to predict the shifts, that uses masking.
Empirical results on standard benchmarks show accurate results at better FPS.
main_review: Originality / Novelty:
Paper build on top of recent methods that cast the text detection problem as one of finding segmentation proposals. As most other methods, the paper has a branch that predicts a clipped segmentation mask that is then thresholded into a binary mask, and where connected components are found to construct the "core" regions. Contrary to other methods, that then perform complex post-processing, or to methods such as Mask TextSpotter v3, that simply "undo" the clipping, this method proposes to learn, on a second branch, a shift vector on pixels around the text regions, that shifts them towards the text core. This seems to work well in practice (or at least slightly better), but it seems very incremental compared to the unclipping method.
The paper also proposes a relaxed L1 loss, but it seems to be an elementwise multiplication of the smooth L1 loss and a binary mask, similar to the loss used by many works on the segmentation branch. 
Overall, the paper is incremental, and I am not sure it reaches a minimum level of originality or novelty.

Quality / clarity: The paper is, unfortunately, not very well written. The method is not well motivated, and, once challenges of current methods are explained, the paper goes straight into what are the steps of the proposed method (page 2), instead of explaining first why the proposed method would help with those problems. I had to go through the paper a couple of times before I could understand the motivation behind the described method, and how it is a smarter version of the automatic unclipping of Mask TextSpotter v3.

Significance / results: The method is evaluated on standard benchmarks and results seem to outperform state of the art methods at a good FPS. Authors mention that they will release code, which will significantly help the community. I have some questions about some specific results:

- Similar to how the proposed segmentation proposal can be fed into Mask TextSpotter v3 (Table 5, with a slight increase of accuracy), one could detach the recognition branch of Mask TextSpotter v3 and use it only for detection (e.g. Table 2 of the Mask TextSpotter v3 reports results in the MSRA-TD500 dataset). 
1) Why not include the TextSpotter results in Table 4 in this paper?
2) Compared to the TextSpotter detection shown in Table 5, there seems to be a meaningful increase in accuracy when using the proposed method. However, the proposed method is more expensive since it needs to run an additional head, creating extra activations. What is the FPS and memory requirements of the TextSpotter segmentation proposals, compared to the proposed ones?
- Figure 3 mentions clustering. Is this just a plain assignment of pixels to the text core after applying the shift, or is there other operation involved?

- Nit: Figure 7a-left in supplementary material, that's actually not an error. The pattern says NONNA GELSA. The proposed method is doing better than the authors are giving it credit for.
limitations_and_societal_impact: Supplementary material shows some limitations / failure cases. A common challenge is deciding whether two components should be connected or not.
No discussion about societal impact. Text detection has less obvious societal impact than other tasks such as face identification, but improving it and making it faster  can still have consequences,  e.g. license plate recognition in surveillance systems with lower quality hardware and non-stationary setups.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a novel representation of text instances and an algorithm that detects text instances in an image. A text instance is represented as a kernel that is a shrunk version of the text instance along with surrounding pixels that belong to the kernel based on centripetal shifts. This representation simplifies the post-processing to obtain text instances from the output of neural network but yet yields accurate detection. The experimental results confirm the effectiveness of the approach on common scene text datasets containing irregular texts.
main_review: The proposed method successfully simplifies the post-processing to obtain the final results from the output of neural network while keeping the flexibility of the pixel-level representation. The paper proposes a solution that solves the problems raised in the paper. Given the simplicity and the effectiveness, it could be one of the representations widely used for scene text recognition. It seems to have good novelty and originality.

However, the paper seems to need more work for accept. The followings are my main concerns on the paper:


1. The text explaining the Relaxed L1 loss seems to contain errors or very confusing. I was not able to read lines 167-178 very well. Especially, Eq. (4) and (5) did not make sense to me. I might be just wrong, but it looks like R_j is 0 everywhere according to Eq. (1). Then, Eq. (5) is just 0. I would want to be corrected if I do not understand the equations right. Some visualization might help.


2. The comparisons with previous methods (for text representation) are conducted by only referring to the numbers reported in the original papers. Having such comparisons are still useful, but they do not necessarily help to demonstrate the superiority of the proposed method due to inevitable differences in their experimental conditions. It would be necessary to re-implement some of the methods that suffer from the problems (i.e. ones losing either simplicity or robustness) and compare the methods in a truly fair manner. If it is challenging to re-implement existing methods, it would be at least necessary to detail the differences among methods and explain the reasons why it is believed that the proposed method is superior.


3. I know that it is unfortunately common in the domain, but I am not sure how much informative it is to compare FPS among methods where each experiment could be done in a different condition. I think it is OK to list those numbers, but I am afraid that it does not prove the superiority of the proposed method.


4. The writing could be improved. Sections 1-3 are entangled and could be cleaned up. For example, Fig. 3 (or the overview of the method) is not explained in Section 3 but in Section 1. Section 2 is too short and important relevant work like [35] is mentioned only in Section 1.


5. Should c_j at line 140 be s_j ?

[update after author response]

My biggest concern (1) was addressed by the author response.

As also pointed out by reviewer U3is, we want to expect more detailed analyses of the proposed method. That will include more ablation studies, comparisons with locally-implemented appropriate strong baselines for the method, discussions on "alternative (not) considered", etc.

Although more analyses are demanded, I see no major flaw in the paper now and it will have a certain value to the community.

I'm happy to correct my rating to 6.
limitations_and_societal_impact: Limitations are not particularly discussed in the paper.

No concerns on potential negative societal impact.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: Update after discussion among reviewers: I am still strongly in favor of accepting this paper. 

The paper presents a new method to represent arbitrary shaped text in images and shows how this representation can be used in the output layer of some of the recent text detection methods. 

The results look qualitatively and quantitative great. 

The method uses a simple representation that is intuitively understandable. 


main_review: My biggest concern with the paper is that there appears to be a mismatch between equations 2 and 3: 

in eq. 2 the loss for the region \bigcup(T_i-K_i) is masked out.
in eq. 3 this is however the most important region. 

I am not sure how the method is able to learn anything about this region if the loss is fully masked. 


The paper does provide a great comparison to many standard methods. 

The method is well motivated - and I like how it can be applied in the output layer of a bunch of different networks. 


minor comments: 
abstract: 
most intractable -> hardest 
abbreviation SPN is not explained. 
1.1% improvement (relative or absolute?)

generally, the paper should be carefully copy read and some typos and grammatical mistakes could be fixed. 
some examples: 
line 49: the mistakes -> mistakes
line 57: by the CT -> by CT
line 58: oppoonents -> methods
captuion fig 2: helpless -> useless

beyond that: the introduction and the abstract feel rather repetitive.
limitations_and_societal_impact: it's a standard domain - I feel it's an interesting question to think about the general societal impact of scene text recognition - but I wouldn't think this needs to be discussed in this paper which provides an elegant, yet incremental improvement to previous methods.
review_title: null

Meta-review: This paper presents a new method for scene text detection and recognition based on the integration of individual local responses and their centripedal shifts.

The paper has received 4 expert reviews, which were quite positive, with in particular one reviewer championing the paper. While the reviewers (and the AC) agreed, that the paper had weaknesses in presentation and writing, there was a general agreement that the paper has merits, sufficient novelty, and convincing results.

The AC concurs and proposes acceptance.
Decision: Accept (Poster)