DNTextSpotter: Arbitrary-Shaped Scene Text Spotting via Improved Denoising Training

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-04 09:14:06

Abstract: More and more end-to-end text spotting methods based on Transformer architecture have demonstrated superior performance. These methods utilize a bipartite graph matching algorithm to perform one-to-one optimal matching between predicted objects and actual objects. However, the instability of bipartite graph matching can lead to inconsistent optimization targets, thereby affecting the training performance of the model. Existing literature applies denoising training to solve the problem of bipartite graph matching instability in object detection tasks. Unfortunately, this denoising training method cannot be directly applied to text spotting tasks, as these tasks need to perform irregular shape detection tasks and more complex text recognition tasks than classification. To relieve the issue, in this paper, we propose a novel denoising training method (DNTextSpotter) for arbitrary-shaped text detection and recognition. Specifically, we decompose the queries of the denoising part into noised positional queries and noised content queries. We use the four Bezier control points of the Bezier center curve to generate the noised positional queries. For the noised content queries, considering that the output of the text in a fixed positional order is not conducive to aligning position with content, we employ a masked character sliding method to initialize content queries to assist the alignment of text content and position. To improve the model's perception of the background, we further utilize an additional loss function for background characters classification in the denoising training part. Although DNTextSpotter is conceptually simple, it outperforms the state-of-the-art methods on four benchmarks (Total-Text, SCUT-CTW1500, ICDAR15, and Inverse-Text), especially yielding an improvement of $11.3\%$ against the best approach in Inverse-Text.

Main paper: 
 introduction: 
 Text Spotter, as an essential foundational technology that encompasses text detection and recognition, plays a critical role in various domains [4, 29, 35, 45] such as autonomous driving, security monitoring, and social media analysis. Given that textual elements in wild scenarios are often set against complex backgrounds, presented in diverse font sizes, and subject to distortions, accurate identification of text remains a challenging and dynamic field of research. To address these challenges, traditional CNN-based text spotters [1, 20, 23, 26, 33, 36, 38] divide the text localization task into separate detection and recognition stages, following a detectthen-recognize principle.Compared to these classical spotting algorithms, building on the Deformable DETR [50] , TESTR [48] represents a significant advancement in text spotting with its dual decoder design. This novel approach streamlines the process by removing the necessity for manually designed components and eliminating intermediate steps. TESTR simplifies the complex tasks of detection and recognition by treating them as a unified set prediction problem. It leverages bipartite graph matching to concurrently assign labels for both detection and recognition, achieving a more efficient and integrated workflow. Despite the significant achievements of TESTR, the employment of two decoders significantly increases computational complexity. Moreover, initializing queries with distinct properties for detection and recognition poses challenges for model optimization. Many methods have been proposed to address these issues. For example, TTS [15] attempts to unify the detection and recognition tasks within a single decoder. DeepSolo [44] , while using a single decoder, introduces a novel query form with shared parameters, which initializes decoder queries by utilizing a series of explicit coordinates generated from the text line. However, despite further improvements in performance, these methods overlook the instability introduced by the bipartite matching employed in the DETR-like methods. In general object detection tasks, DN-DETR [16] firstly points out the instability problem of bipartite graph matching when using the DETR architecture and proposes denoising training to solve this problem. Denoising training, in simple terms, initializes noised queries using ground truth with a small amount of noise added, allowing for direct loss calculation with the ground truth after decoding, bypassing the bipartite graph matching algorithm. DINO [46] further proposes a contrastive denoising training method to further enhance the performance of denoising training. Unfortunately, for the tasks of spotting scene text, the challenge is significantly amplified due to the arbitrary shapes of the text to be detected and the need for recognition tasks that are more complex than mere classification. This complexity makes it difficult to directly apply this denoising training method. In fact, ESTextSpotter [12] directly incorporates a DINO-based denoising training method within its model architecture. In this context, as shown by the convergence curves in Fig. 1 , this denoising training starts with regular bounding boxes as queries initialization, and the results on inverse-like texts become very poor, indirectly reflecting the negative impact brought by this coarse prior.In this paper, we propose a novel denoising training method specifically designed for transformer-based text spotters that handle arbitrary shapes. Considering that the task of text spotting aims at the detection and recognition of text in any shape, and using regular boxes to initialize noised queries is coarse, we abandon the traditional approach that relies on 4D anchor boxes and classification labels. Instead, we use Bezier control points and text characters to initialize noised queries. Technically, we feed the noised queries obtained from the ground truth along with randomly initialized learnable queries into the transformer's decoder. We design the noised query using bezier control points of the bezier center curve and text scripts, thereby accomplishing the denoising of both points coordinates and text characters. In addition, considering that outputting the text characters in a fixed positional order is not conducive to aligning position with content, before initializing the text script as noised queries, we use a masked character sliding method to initialize noised content queries to assist in the alignment of content and position of the text instance.We verify the effectiveness of DNTextSpotter by using multiple public datasets. In the metrics of 'None' results with ResNet-50 backbone, compared with the current state-of-the-art methods, our method achieves 2.0% and 2.1% improved results on the Total Text and CTW1500 datasets respectively, reaching 84.5% and 67.0% respectively. On the newly released benchmark Inverse-Text dataset, our method even exceeds the state-of-the-art results by 11.3%, reaching 75.9%. When switching to the ViTAEv2-S backbone, scores for all metrics are further improved.In summary, our main contributions can be summarized as follows:• We introduce a novel denoising training method to design an end-to-end text spotting architecture. Starting from the attribute of arbitrary shapes of scene text, we utilize bezier control points as well as text characters to design this denoising training method. • Taking into account the negative impact of directly using ground truth text scripts to initialize noised queries, which leads to misalignment between the position of the characters and the content of these characters, we design a masked character sliding method to preprocess these ground truth text scripts, thereby optimizing the alignment between text position and content. • Our method achieves state-of-the-art results on multiple benchmarks. Specifically, we conduct a qualitative analysis of several text spotting architectures based on the transformer structure, including analyses of instability results and visualization of results. 
related works: 
 Early literature tends to classify end-to-end text spotting architecture into two-stage methods and one-stage methods. Recently, due to the popularity of transformer-based text spotters, we categorize these methods into CNN-based methods and transformer-based methods. Earlier comprehensive and in-depth surveys on text spotting are available in [4, 29] .The first end-to-end scene text recognition network [17] combines detection and recognition into a single system. This method is limited to recognizing regular-shaped text. Subsequent works [1, 8] improve the connection between the detector and recognizer, considering single characters or text blocks to handle irregular text more flexibly. The Mask TextSpotter series [20, 30] employs segmentation approaches to generate proposals. These methods rely on character-level annotations, significantly increasing the effort required for generating ground truth. Text Perceptron [36] and Boundary [33] utilize Thin-Plate-Spline [2] transformation to rectify features obtained from curved text. The ABCNet series [24, 26] use BezierAlign to address the problem of curved text, requiring the prediction of a small fixed number of points. While these methods achieve good performance, they require additional RoI-based [9] or TPS-based connectors, and the only shared part between the detector and recognizer is the backbone network's features, neglecting the collaborative nature of detection and recognition. [49] propose ARTS, highlighting the importance of collaborative detection and recognition in the text spotting task. Lastly, all of the aforementioned methods require complex manual operations like Non-Maximum Suppression (NMS).With the impressive success of Transformers in visual tasks [6, 7, 14, 18, 27, 28, 42, 47] , also influenced by the DETR family [3, 16, 22, 31, 46, 50] , more recent works explore Transformer-based structures for the Text Spotting. TESTR [48] employs dual decoders for detection and recognition tasks, sharing the backbone and Transformer encoder features. TTS [15] utilizes an encoder and a decoder with multiple prediction heads for performing multi-tasks. DeepSolo [44] employs an explicit points method to model decoder queries.Although these methods achieve promising results, they still exhibit certain limitations. The random initialized queries used in TESTR [48] and TTS [15] still lack clarity and fail to efficiently represent queries encompassing both positional and semantic aspects. Utilizing the encoder's output features, DeepSolo [44] generates The queries of the decoder can be divided into two parts: a matching part and a denoising part. The queries in the matching part are randomly initialized queries. The noised queries of denoising part can be found in Fig . 3 and Fig . 4 . After decoder and task-specific head, the matching part calculates loss through a bipartite graph matching algorithm, and the denoising part calculates loss directly with the ground truth.Bezier center curve proposals, which subsequently serve to generate positional queries. It effectively decouples the ambiguously defined queries into positional queries and content queries. Although these methods have achieved certain accomplishments, they, like DETR-like methods, use bipartite graph matching algorithms to obtain one-to-one matching results. The instability of bipartite graph matching has been proven to have a negative impact on detection tasks by DN-DETR [16] and DINO [46] . While ESTextSpotter [12] attempted to propose a method for denoising training, this method uses boxes as a positional prior for point prediction, failing to consider the irregular attributes of text instances and the characteristics of text scripts. Therefore, we design a denoising training approach based on Bezier control points and characters. 
methodology 3.1 preliminaries: 
 Denoising Training. The denoising training method was first proposed by DN-DETR to address the slow convergence issue of DETR. This method constructs an additional auxiliary task in the decoder section without bipartite matching, which can be used to accelerate the convergence of DETR-like methods. Technically, it additionally feeds noised ground-truth boxes and labels into the transformer decoder to reconstruct these ground-truths, and this part is updated through an additional auxiliary DN loss. DINO improved the denoising training method and further proposed a contrastive denoising training method, which adds negative queries in addition to the original noised queries to predict the background. In our method, we further extend the denoising training approach based on this foundation. Bezier Center Curve. ABCNet [24] was the first to use Bezier curves to flexibly adapt to any shape of scene text with a small number of fixed points. Subsequently, DeepSolo introduced the Bezier Center Curve, which initializes transformer decoder queries by uniformly sampling a fixed number of points on the curve. This Bezier Center Curve is obtained by calculating the average of the four Bezier control points on the top and bottom edges of each text instance. In our method, we consider utilizing Bezier center curves in denoising training.The ambiguous meaning of decoder queries in DETR is often interpreted in existing literature [22, 46] as a combination of positional queries and content queries. We also utilize this modeling approach and follow DeepSolo, using the Bezier Center Curve to initialize positional queries and then combine them with learnable content queries. In the denoising part, we represent noised queries as two parts: noised positional queries and noised content queries. Noised Positional Queries. As shown in Fig. 3 , for any text instance, the four Bezier control points of the Bezier center curve. Assuming that a picture contains 𝑁 text instances, we can represent the set of all instances as: S = {𝑆 𝑗 | 𝑗 = 1, 2, . . . , 𝑁 }, 𝑆 𝑗 is a set of the four Bezier control points of the Bezier center curve of a text σ o Ø σ Ø Ø Ø Ø σ σ σ Ø σ σ Ø Ø Ø σ Ø Ø σ Ø Ø Ø σ σ σ Ø Ø Ø Ø Ø σ Ø Noised Content Queries Generation σ m o o o s e w σ Ø Ø d Ø Ø o Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø m o o o s e w m o e w o d m o o s s e w o o o Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø m o Ø o s e w o o d Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø σ Ø Ø Ø Ø d Ø Ø o Ø σ Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø Ø m o o o s e w m o e w o dFigure 4 : This image shows the process of generating noised content queries from the ground-truth texts. "∅" indicates the characters will be masked, while "𝜎" denotes flipping the characters into any character.instance. Each instance can be represented as:EQUATIONwhere each point is composed of a pair of coordinates (𝑥 𝑖 𝑗 , 𝑦 𝑖 𝑗 ), and the index 𝑖 ranges from 0 to 3, indexing the four Bezier control points. Random noise is added to these points to obtain:EQUATIONwhere Δ𝑥 𝑖 𝑗 and Δ𝑦 𝑖 𝑗 are obtained by calculating the distance between the four Bezier control points of the center curve and the four Bezier control points on the top side and are denoted as 𝐷 𝑥 𝑖 𝑗 and 𝐷 𝑦 𝑖 𝑗 . We use elements 𝛼 𝑖 𝑗 and 𝛽 𝑖 𝑗 from sets that satisfy a 0 -1 uniform distribution to control the noise ratio for the 𝑥 and 𝑦 coordinates respectively. Therefore, we can represent the offset of the coordinates as follows:Δ𝑥 𝑖 𝑗 = (-1) 𝑚 𝛼 𝑖 𝑗 𝐷 𝑥 𝑖 𝑗 if positive; (-1) 𝑚 (𝛼 𝑖 𝑗 + 1)𝐷 𝑥 𝑖 𝑗 otherwise.(3)EQUATION)where 𝑚 and 𝑛 are used to control the direction of the offset, with "positive" indicating the coordinates of the positive part.After adding noise to the Bezier control points, we obtain a new set of control points S ′ , Using these noisy Bezier control points, we uniformly sample 𝑇 point coordinates from the resulting Bezier curves. These point coordinates form tensor Points with shape (𝑁 ,𝑇 , 2). Finally, these coordinates are processed through positional encoding (𝑃𝐸) and two layers of MLP to obtain Noised Positional Queries (𝑄 𝑁 ) with a shape of (𝑁 ,𝑇 , 256). We represent 𝑄 𝑁 as follows:EQUATIONSection 4.4 analyzes 'Why add noise to the Bezier control points?'. Noised Content Queries. We designed Mask Character Sliding (MCS), as shown in Fig. 4 , to initialize the noised content queries.For the maximum recognition length 𝑇 , which is equal to the 𝑇 mentioned above, we perform a sliding operation on the valid characters in the positive part. Specifically, we first determine the number of valid characters 𝑡, which refers to the number of characters in the input sequence that actually have meaning. Then, we calculate the number of times each valid character should be cloned by performing a division operation 𝑇 //𝑡, in order to evenly distribute the total length 𝑇 of the sequence to each valid character. Additionally, since the division of 𝑇 by 𝑡 may not be exact, there will be a remainder 𝑘 = 𝑇 -𝑇 //𝑡 , indicating that there are 𝑘 additional spaces that need to be allocated. To fairly distribute these extra spaces, we assign one additional clone to each of the first 𝑘 valid characters in the sequence, ensuring that the allocation for each character is as even as possible. Since this operation visually resembles a character sliding operation, we name it Character Sliding. After sliding the characters, we use a mask operation to control the number of consecutive characters, flipping a portion of the consecutive characters into a background label with a certain probability. After processing the positive part, we add noise to the characters in both the positive part and the negative part, causing these characters to flip to other characters with a probability of 𝜆. These characters are then transformed into noised content queries after being embedded, with all the characters initialized in the negative part being backgrounds.In addition, we use a dynamic group 𝑔 to fully utilize the performance of denoising training. Considering the computational cost, we set the maximum number of text instances 𝑁 per image to 100. When 𝐵 exceeds 100, we simply use the slicing method to take the first 100 instances. The division of 𝑔 is as follows:EQUATIONEQUATIONTo ensure that during the decoder self-attention calculation, the information in the denoising part contains ground-truth information, we need to ensure that the information in the matching part cannot see the information in the denoising part. In addition, each group should not be able to see each other. Considering that there are two parts during self-attention calculation, namely intra-relation self-attention which calculates the attention relationship between characters, and inter-relation self-attention which calculates the attention relationship between text instances. We consider whether two attention masks are needed to prevent information leakage. However, in fact, for intra-relation self-attention which calculates the attention relationship between characters, we do not need to use an attention mask, because, for any text instance (including the denoising part and the matching part), the attention calculation is within the text instance and does not interact with other text instances. So we only need to consider the design of inter-relation self-attention, we call this attention mask that only needs to be used once as Single Attention Mask, and we devise the attention mask A = a 𝑖 𝑗 (𝑔+2𝑛) × (𝑔+2𝑛) as follows:EQUATION)where 𝑔 and 𝑛 represent the number of groups and the number of text instances per image, respectively. 𝑎 𝑖 𝑗 = 1 means the 𝑖-th query is blind to the 𝑗-th query; if it's 0, they can see each other.Compared to the matching part, the denoising part uses a slightly modified loss function and an additional background computation loss function (namely, the background part is calculated twice, which we briefly refer to as BCT). The rest of the loss functions are consistent with DeepSolo [44] , including the Hungarian matching algorithm, focal loss, CTC loss [10] , and L1 loss. The details of the matching part are elaborated in the appendix. Overall Loss. We use focal loss [21] to calculate the classification of text instances. In each set of denoising queries, the positive part represents positive samples, and the negative part represents negative samples, with the focal loss being used to compute the background loss for the first time. Therefore, for the 𝜏 𝑡ℎ query in the positive part of the denoising queries, the calculation of the focal loss for text instance classification is as follows:EQUATIONwhere 1 represents the indicator function, and Im(𝜑) denotes the image of the mapping 𝜑. Concerning character classification, for the 𝜏-th denoised query in the positive part, we employ the CTC loss:EQUATIONThe cross entropy loss for the 𝜅-th denoised query in the negative part during the second background calculation:L (𝜅 )text,neg = 1 {𝜅 ∈Im(𝜑 ) } CE(𝑡 (𝜑 -1 (𝜅 ) ) , t (𝜅 ) ).Additionally, for the coordinate points of the center curve and boundaries in the positive part of the 𝜏-th denoised query, we employ the L1 loss for the computation:EQUATIONEQUATIONwhere top refers to the top curves of the boundaries, and bot refers to the bottom curves of the boundaries. The negative part does not participate in the calculation of the part. The loss function for the denoised queries consists of four aforementioned losses in the positive part and two aforementioned losses in negative part:EQUATIONEQUATIONwhere 𝜆 cls , 𝜆 text , 𝜆 coord , 𝜆 bd are hyper-parameters to balance different tasks. The final loss function of the denoising part is:EQUATION 
experiment: 
 We conduct comparisons with known Transformer-based approaches on various datasets, including Total-Text, SCUT-CTW1500, IC-DAR15, and InverseText which contain multi-directional scene text and arbitrary-shaped text instances. In addition, We choose publicly available datasets Synth150K, MLT17, IC13, and TextOCR as additional pre-training datasets.Total-Text is a widely used comprehensive scene text dataset introduced by [25] , specifically designed for arbitrary text detection. It comprises 1255 training images and 300 testing images, containing horizontal, multi-directional, and arbitrary-shaped text instances. SCUT-CTW1500 is another significant dataset for arbitrary-shaped text, published by [5] . It comprises 1500 images, consisting of 1000 training images and 500 testing images. ICDAR2015 Incidental Text (ICDAR15) [13] includes 1000 training images and 500 testing images with quadrilateral text. It contains multi-directional text instances annotated with word-level quadrilateral annotations.InverseText was manually annotated by [43] and includes 500 test images. Unlike the previous datasets, this dataset contains 40% inverse-like text instances, specifically designed to address the lack of inverse-like texts in existing test datasets.All settings are based on the ResNet-50 backbone. We employ 6 layers of encoder and 6 layers of decoder, with a hidden dimension of 256. For character classification, we predict 37 classes on the Total Text, ICDAR15 , and InverseText datasets, and 96 classes on the CTW1500 dataset. During training, we set the noise hyperparameters as follows: The probability 𝜆 of characters being flipped to other characters is set to 0.4. The learning rate scheduler utilizes an initial learning rate of 2𝑒 -5 for the backbone and 2𝑒 -4 for other parts. We train the DNTextSpotter for a total of 435𝑘 steps, and the learning rate is reduced by a factor of 0.1 at 375𝑘 steps. We use AdamW as the optimizer and train our network with a batch size of 8. The denoising part loss weights 𝜆 cls , 𝜆 coord , 𝜆 bd , 𝜆 text,pos , and 𝜆 text,neg are set to 1.0, 1.0, 0.5, 0.5, and 0.5. The focal loss parameters 𝛼 and 𝛾 are set to 0.25 and 2.0, respectively.For Arbitrarily-Shaped Scene Text Spotting: As previously indicated, the Total-Text and SCUT-CTW1500 datasets are specifically designed to emphasize text instances characterized by arbitrary shapes. In comparison to other methods on the Total-Text dataset (shown in Table 1 ), in the detection task, our approach is close to the current state-of-the-art method, ESTextSpotter, achieving 89.2%. While its detection performance is slightly lower, it has a significant advantage in recognition performance. Without a lexicon ('None' results), outperforming ESTextSpotter by 3.7%, reaching detection and recognition, with increases in F1, 'None', and 'Full' by 0.9%, 2.8%, and 2.8%, respectively. We use only ResNet-50 as the backbone, and the experimental results on various datasets reach state-of-the-art. When we switch the backbone to ViTAEv2-S, our performance greatly exceeds that of DeepSolo, which also uses ViTAEv2-S.For Arbitrarily-Oriented Scene Text Spotting: In the case of the multi-oriented benchmark ICDAR15, DNTextSpotter exhibits excellent performance when compared to other Transformer-based methods. As shown in Table 3 , DNTextSpotter achieves results of 88.7%, 84.3%, and 79.9% on the settings of "S", "W", and "G", respectively. These results surpass the current state-of-the-art method, DeepSolo, by 0.6% in "S", 0.4% in "W", and 0.4% in "G", respectively.For Inverse-like Scene Text Spotting: Besides ESTextSpotter, whose weights were measured using the publicly available weights from the paper, all other results were taken from the DeepSolo report. In the latest Inverse-Text dataset, our method achieves significant success. Compared to the current state-of-the-art method, DeepSolo, our results without a lexicon ("None") surpassed it by 11.3%, reaching 75.9%, and with a lexicon ("Full"), we exceeded it by 10.4%, achieving 81.6%. We analyze why our method performed exceptionally well on this dataset. We believe this is due to the fact that datasets for Inverse-like texts present a more complex challenge than those for arbitrarily shaped texts. Unlike conventional texts that follow a left-to-right order, these texts are ordered from right to left and are flipped downward, making it challenging for the original model to learn this pattern. Denoising training, as an auxiliary task, with its relatively simpler nature, can more easily help the model learn these uncommon or complex forms of text. Ablation experiments were conducted on the Total-Text dataset. Table 4 further demonstrates the convergence effects of the improved denoising training. Table 6 shows the impact of noise scale and mask probability. Table 7 shows the effects of adding noise to Bezier control points, using masked character sliding, and calculating an additional background loss. Instability Measurement: We utilize the analysis method for quantifying the instability(𝐼𝑆) of bipartite graph matching proposed by DN-DETR [16] . The calculation formula can be found in the appendix. For a training image in the TotalText dataset, we calculate the indices of 𝑁 proposals every 10k iterations, with every 10k iterations considered as one group. By comparing the differences in indices between the ith group and the (𝑖 + 1)th group, we obtain the results for 𝐼𝑆. We visualize the 𝐼𝑆 results as shown in Fig. As for DNTextSpotter, we achieve good performance on most of the inverse-like texts, indicating that denoising training has an increasingly positive effect on more complex tasks. More detailed visualization results can be found in the appendix.  
conclusion: 
 In this paper, we design a novel denoising training method from the perspective of the attributes of scene text. Our research has found that devising a denoising training method that aligns the positions and contents of characters is very effective for performance improvement. In the future, designing denoising training methods that align more closely with task characteristics to improve model performance is a promising research avenue. We hope our denoising training approach for text spotters provides valuable insights to other researchers.  
 
 Appendix:  


Decision: The authors did a good rebuttal. The reviewers unanimously recommend acceptance. After checking the rebuttal, the review, and the paper, the AC agrees with this assessment.

Reviews: review: paper_summary: paper_summary: This paper proposes DNTextSpotter, a modified denoising training method to  solve the problem of bipartite graph matching instability  for transformer-based text spotting. The main motivation comes from that the DETR methods use a bipartite graph matching algorithm during training, but it brings instability, which leads to slow convergence and suboptimal performance. Inspired by the denoising training method proposed by DN-DETR,  DNTextSpotter makes some improvements to the denoising training method, making it suitable for text spotting scenarios.
strength_weakness: strength_weakness: 1. DNTextSpotter extends the denoising training approach to the text spotting domain, achieving SOTA performance and accelerating the convergence speed. 
2. DNTextSpotter proposes a masked character sliding method to preprocess ground truth text scripts to align text position and content. It is reasonable.
3. The experiments are relatively sufficient. Besides, the authors conduct a qualitative analysis of transformer-based text spotting architectures, which is insightful.
limitations: limitations: 1. The Denoising training method is proposed by DNDETR,  and Bezier Curve is widely-used by previous work to model arbitrarily shaped text representation. Although the denoising method doesn't work directly in the text spotting domain, the improvements made in the paper are  somewhat small and the enhancements are limited, as shown in Table 7. I'm a little concerned about whether the novelty is enough to match MM's criteria. 

2. Lack of citations to some related works published in ACM MM. e.g. [1], [2], [3] 

[1] Textblock: Towards scene text spotting without fine-grained detection
[2] You Can even Annotate Text with Voice: Transcription-only-Supervised Text Spotting
[3] You Only Recognize Once: Towards Fast Video Text Spotting
paper_summary: paper_summary: This paper presents a new denoising training method for scene text spotting. By making a few modifications to the original denoising training, the proposed method can be used in the Transformer-based spotter DeepSolo. Experiments demonstrate the effectiveness of the proposed method. Although promising results are achieved, the proposed approach appears to be limited to working only with DeepSolo. Additionally, the comparison with previous methods may not be fair.
strength_weakness: strength_weakness: This paper is well-written and well-organized. The idea of improving performance through denoising training is interesting. The proposed method achieves state-of-the-art performance on several scene text spotting benchmarks.
limitations: limitations: 1. The denoising training proposed in DN-DETR[16] can help improve the performance under different frameworks. However, it seems like the proposed method can only be used on DeepSolo. This limits the applicability of this approach. Additionally, the proposed denoising training seems to simply modify from [16,46] to adapt it to the DeepSolo. 

2. Lack of a fair comparison with other methods. For instance, TextOCR was not used as training data in the most of previous methods. The baseline, DeepSolo, also includes the results without the use of TextOCR. It would be better to include these results. Additionally, the higher performance on Inverse-Text may be due to the use of TextOCR. 

3. The proposed method uses TextOCR as a training set but lacks results evaluated on TextOCR.

4. The ablation study is insufficient. The authors spend a section discussing the Single Attention Mask, but there is no ablation study about it.
paper_summary: paper_summary: This paper proposed a denoising training method for arbitrary-shaped text detection and recognition. It decomposed the queries of the denoising part into noised positional queries and noised contexnt queries. For the noised positional queries, the gour Bezier control points of the Bezier center curve is used. For the noised content queries, a masked character sliding method is adopted to align the text content. The proposed DNTextSPotter achieves SOTA results on standard scene text spotting benchmarks.
strength_weakness: strength_weakness: Innovative Application: The paper's primary contribution lies in the application of a denoising training strategy to the challenging task of spotting arbitrary-shaped text. This is a novel approach that addresses the need for complex positional descriptions and aligned ground truths in text recognition.


Technical Clarity: The motivation behind the proposed method is clearly articulated, and the paper provides a solid theoretical foundation for its approach. This clarity is crucial for readers to understand and potentially build upon the work.


Empirical Validation: The experimental results are compelling, showing that the proposed modules significantly enhance the performance of text spotting systems. Achieving SOTA results on standard benchmarks is a strong validation of the method's effectiveness.
limitations: limitations: Comparison Clarity: The review suggests that including the training data used by the compared methods in Table 1 would enhance the clarity of the comparisons. This information is essential for understanding the context and fairness of the comparisons made.