Free Lunch: Frame-level Contrastive Learning with Text Perceiver for Robust Scene Text Recognition in Lightweight Models

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-05 11:30:58

Abstract: Lightweight models play an important role in real-life applications, especially in the recent mobile device era. However, due to limited network scale and low-quality images, the performance of lightweight models on Scene Text Recognition (STR) tasks is still much to be improved. Recently, contrastive learning has shown its power in many areas, with promising performances without additional computational cost. Based on these observations, we propose a new efficient and effective frame-level contrastive learning (FLCL) framework for lightweight STR models. The FLCL framework consists of a backbone to extract basic features, a Text Perceiver Module (TPM) to focus on text-relevant representations, and a FLCL loss to update the network. The backbone can be any feature extraction architecture. The TPM is an innovative Mamba-based structure that is designed to suppress features irrelevant to the text content from the backbone. Unlike existing word-level contrastive learning, we look into the nature of the STR task and propose the frame-level contrastive learning loss, which can work well with the famous Connectionist Temporal Classification loss. We conduct experiments on six well-known STR benchmarks as well as a new low-quality dataset. Compared to vanilla contrastive learning and other non-parameter methods, the FLCL framework significantly outperforms others on all datasets, especially the low-quality dataset. In addition, character feature visualization demonstrates that the proposed method can yield more discriminative character features for visually similar characters, which also substantiates the efficacy of the proposed methods. Codes and the low-quality dataset will be available soon.

Main paper: 
 introduction: 
 With the advancement of deep learning, robust Scene Text Recognition (STR) has emerged as a prominent topic in both academia and industry [55] [56] [57] . Numerous remarkable models have been proposed. It is evident that the scale of STR models is rapidly increasing. Additionally, iterative decoding is gradually gaining popularity thanks to its ability to achieve higher recognition accuracy, albeit at a significantly slower pace compared to methods based on Connectionist Temporal Classification (CTC).However, text recognition serves as a fundamental module in practical document processing tasks, with limited resources allocated to this endeavor. Therefore, we need to utilize minimal resources to achieve maximal recognition performance. So we focus on the lightweight STR model in this paper.Using little or no additional costs to improve performance has consistently been a popular approach. There are mainly two ways to achieve this. One involves employing more efficient loss functions such as FocalCTC [9] , EnCTC [27] , and DCTC [61] . The other entails adopting new training approaches, such as pluggable modules [37] during training, distillation learning, and Contrastive Learning (CL). Distillation learning necessitates a large and similarlystructured high-performance model as the teacher, which limits its applicability. In contrast, contrastive learning offers a more flexible and efficient usage. Some CL-based methods [28, 58, 59 ] have demonstrated success in STR tasks. However, most existing methods perform contrastive learning at word level, overlooking the fact that text recognition is actually a frame-wise task, which may limit effectiveness.The complexity of existing features is also crucial for contrastive learning. Due to diverse image qualities, features extracted by CNNs or transformers often contain many irrelevant text features. This increases the difficulty of contrastive learning and diminishes final accuracy. Some methods utilize fully connected layers for feature projection in an attempt to mitigate the impact of irrelevant features. However, this approach is relatively direct and challenging for the purification of text-relevant information.Based on these observations, in this paper, we propose a framelevel contrastive learning framework with a text perceiver for STR tasks, as illustrated in Fig. 1 . The main difference compared to existing methods lies in conducting contrastive learning at the frame level. Traditional contrastive learning can only provide word-level statistical information, such as the number of frames containing the character 'w' or 'i' in the estimation result, but due to pooling operations, it cannot learn the exact frames. Our method addresses this issue by performing contrastive learning at each frame level without pooling, thereby achieving more accurate alignment while contrastive learning. Furthermore, in order to yield better textrelevant features, we design a bidirectional Mamba-based [13] Text Perceiver module to suppress text-irrelevant representations. We select nine well-known lightweight models and conduct experiments on six widely-recognized STR benchmarks as well as a specific low-quality dataset. All experiments demonstrate the effectiveness of the proposed method.In summary, the main contributions of this paper are as follows:(  
related works 2.1 robust scene text recognition: 
 The robustness of STR models, specifically in low-quality scenarios, e.g., blur, low resolution, and noise, is a critical issue for applications. Many previous studies have explored the probability of enhancing the robustness of models in the wild, which can be divided into two categories. One of them aims to employ additional modules for preprocessing the low-quality inputs [5, 23, 37] , where [23] proposes a text-specific hybrid dictionary for text image deblurring, [5] introduces a transformer-based text deblurring module, while [37] proposes a plugable super-resolution unit to improve the performance of the STR model faced with low-resolution text. On the other hand, with the development of language models, some work focuses on combining them with STR models to revise the incorrect prediction within low-quality contexts [8, 49, 50, 62] . These methods are effective, but they also introduce computationally heavy components, which are unaffordable for lightweight STR models.In this work, we propose a frame-level contrastive learning strategy for lightweight STR models to significantly enhance their performance in low-quality scenarios without any additional cost.Recently, [6, 12, 16] have significantly pushed the boundaries of representation learning by introducing contrastive learning. By generating positive samples via data augmentations and regarding other images as negative examples, [6, 16] pull together embeddings of positive pairs and push apart those of negative pairs. Additionally, [12] proves that merely using positive samples can also lead to a promising embedding for downstream tasks. [21] takes advantage of class labels as a criterion to separate positive and negative samples. For STR, [1] introduces a sub-word-level contrastive learning framework, in which patches from different visually augmented images are considered as positive samples. [29] proposes to view the same words in different semantic contexts as positive samples, thus deriving a word-level contrastive learning framework. [60] utilizes stroke-based partitions to help models focus on the topological structure of the stroke and learn text representations bottom-up. Existing contrastive learning-based STR methods employ linear projections to compress the contextual features, while it is still difficult for them to completely eliminate the influence caused by the text-irrelevant features. Different from them, we propose an efficient Text Perceiver instead of simple linear projections to achieve a more efficient purification of the text-relevant information in the contextual features. Additionally, we design a frame-level contrastive loss for STR models, which can improve their performance by providing more consistent supervision with the goal of the text recognition task.For efficient long-range dependency modeling, [14] T scan into the S4 layer. [10] designs a new SSM layer, i.e., H3, that nearly fills the performance gap between SSM-based and attentionbased models in language modeling. [35] builds the Gated State Space layer on S4 by introducing more gating units to improve the expressivity. Recently, [13] introduces a selection mechanism together with a specially designed hardware-aware algorithm into the SSM layer and builds a generic language model backbone, Mamba, which outperforms Transformers at various sizes on large-scale real data and enjoys linear scaling in sequence length. In this work, we explore the potential of the Mamba to purify the text-relevant features extracted by STR backbones and design a text perceiver to replace the linear projection employed in vanilla contrastive learning frameworks to improve their performance.••• Logit sequence U l S E T GT label y: -S-E-A-C-R-E-S-T- U h U l Z h Z l ••• ••• ••• ••• Share weights 
methodology 3.1 pipeline: 
 The data pipeline of our proposed frame-level contrastive learning framework is shown in Fig. 2 . Initially, high-quality inputs are subjected to generating the associated low-quality views via data augmentation, and then the high-quality inputs and their lowquality counterparts are separately fed into the backbone to extract the contextual features that are composed of task-required text features and quality-relevant image features. Subsequently, on the one hand, these contextual features are used to transcript the text via a decoder. On the other hand, we leverage a specifically designed text perceiver module to derive quality-invariant text representations from the contextual features, and then conduct the frame-level contrastive loss on the text representation space. The components and the loss function applied in the framework are detailed in sections 3.2-3.4, respectively.The backbone of STR models generally consists of two components: a feature encoder, and an optional sequence model. There are three prevailing categories of feature encoders applied in the scene text recognition (STR) model. The first is CNN-based encoders, as exemplified by [8, 25, 43] . The second refers to transformer-based encoders, as demonstrated in [3, 7, 53] . The last integrates CNN with attention mechanisms, represented by [26, 51, 52] . Due to the difficulty of CNNs capturing long-range dependencies in sequences, the STR model with a CNN-based feature encoder often utilizes an extra sequence model to process the extracted visual features for better recognition accuracy. The most widely used sequence models include RNN [43] , LSTM [11, 33] , and transformer-based models [32, 39] . They convert visual features into contextual features that are used to transcript the text predictions via the decoder. As with vanilla contrastive learning, the proposed frame-level contrastive learning framework can be compatible with various backbones with different components, thereby facilitating flexible integration and showcasing substantial potential for applications. In the Experimental section, we have executed extensive experiments with diverse backbones to substantiate this adaptability. improving their recognition performance. However, when dealing with the text instance in a varying-quality context, the backbone will inevitably extract some quality-relevent features. In order to suppress the impact of these features on the training effect, vanilla contrastive learning frameworks commonly utilize several linear projections to compress the contextual features, and then calculate the contrastive loss in a more compact feature space. However, the low-quality samples may suffer from various distortions, which makes it difficult for simple linear projections to effectively perceive the text-specific information in the contextual features from different types of low-quality samples, resulting in suboptimal performance. To address this issue, we designed a SSM-based lightweight module, i.e., Text Perceiver, to replace the widely applied linear projection for more efficient purification of the text-specific information in the contextual features.The general SSM is inspired by the continuous system that maps a 1-D function or sequence 𝑥 (𝑡) ∈ R ↦ → 𝑦 (𝑡) ∈ R through a hidden state ℎ(𝑡) ∈ R N :EQUATIONwhere A ∈ R N×N , B ∈ R N×1 , and C ∈ R 1×N are separately the discretized evolution parameter and projection parameters. After the discretization via zero-order hold (ZOH) and parallelization, the SSM can be formulated as follows:EQUATIONwhere x, y separately represents the input and output sequences.A and B are the discretized evolution parameter and projection parameter, respectively. As demonstrated by Eq. 2, the SSM exhibits the promising properties of linearly scaling in sequence length. However, it also illustrates the limitations of SSM in achieving input-dependent selection, which has been proven to be the key to the success of the attention mechanism.To address this issue, Albert Gu proposes the Selective SSM, i.e., Mamba [13] , which utilizes three linear projections combined with the discretization method to calculate the input-dependent A 𝑥 , B 𝑥 , and C 𝑥 and employs kernel fusion, parallel scan, and recomputation to improve the computational efficiency, allowing SSM to effectively yet efficiently focus on the important part of the inputs. Inspired by the input-dependent selection mechanism and linear complexity of Mamab, we consider constructing a lightweight module based on selective SSM to replace the linear projection widely applied in vanilla contrastive learning for more effective purification of the text-relevant information in the contextual features.3.3.3 Architecture. The original Mamba block is designed for 1-D sequence, which is inefficient for text recognition requiring spatialaware understanding. Inspired by some applications of SSM in the vision task [42, 54, 63] , we design the Text Perceiver, which adds an independent branch to process the reversed input for bidirectional feature extraction. The architecture of the proposed text perceiver is shown in Fig. 3 . The input contextual feature is first normalized by the normalization layer. Subsequently, the normalized feature is separately projected to the feature x and the gated weight z. For the feature x, we process it from both the forward and backward directions. For each direction, we first employ a 1-D convolution to get the feature x ′ . Inherited from Mamba, we utilize the x ′ to compute the A x ′ , B x ′ , and C x ′ . Subsequently, we compute the y forward and y backward through the SSM layer. Finally, the y forward and y backward are gated by the weight z and added together to get the output.There are two different-level loss functions in our framework, i.e., the recognition loss and the proposed frame-level contrastive loss. The former, similar to the previous works [34, 43] , is used to provide a word-level supervision for STR models, while the latter is used to provide a character-level supervision for the STR models to learn quality-invariant text representations. Before delving into them, we first clarify the notations. For better performance, STR models are generally trained with large-scale synthetic datasets that are entirely composed of high-quality samples. Hence, given a batch of data {(X 𝑖 ℎ , 𝒚 𝑖 ), 0 < 𝑖 ≤ 𝑁 } where 𝑁 is the batch size, their features are defined as {(Z 𝑖 ℎ , U 𝑖 ℎ , 0 < 𝑖 ≤ 𝑁 }, where Z 𝑖 ℎ and U 𝑖 ℎ separately represents the text representation and the logit sequence. Similarly, after data augmentation, the associated low-quality views and their features are denoted as {(X 𝑖 𝑙 , 𝒚 𝑖 , Z 𝑖 𝑙 , U 𝑖 𝑙 ), 0 < 𝑖 ≤ 𝑁 }. 3.4.1 Recognition loss. We compute recognition loss L REC on logit sequences of both the high-quality and low-quality views, which can be formulated as:EQUATIONwhere L CTC (•) denotes the CTC loss [43] widely applied in the lightweight STR models.Frame-level contrastive loss. Frame-level contrastive loss, i.e., L FLCL , aims to minimize the distance between each pair of associated frames in the projection sequence derived from the same text instance across different quality contexts, and maximize the distance between each pair of associated frames in the projection sequence derived from different text instances. Thus, giving a batch of paired projection sequences {(Z 𝑖 ℎ , Z 𝑖 𝑙 ), 0 < 𝑖 ≤ 𝑁 }, the FLCL is formulated as:EQUATIONwhere 𝒛 𝑖,𝑛 ℎ , 𝒛 𝑖,𝑛 𝑙 ∈ R 1×𝐷 are the 𝑛-th frame of the projection sequence Z 𝑖 𝑐 and Z 𝑖 𝑏 respectively. 𝜏 ∈ R + is a temperature parameter, which is set to 1 in this work. I m are the index set of all masked elements. 𝑠 (•) is the cosine similarity which can be computed as 𝑠 (𝒂, 𝒃) = 𝒂 T 𝒃/∥𝒂∥ ∥𝒃 ∥. FLCL effectively facilitates aligning the representation of clear text instances and their low-quality counterparts at the frame level while also enhancing the extraction of discriminative features, which is pivotal for learning robust text representations.Finall, the total loss takes the following form:EQUATIONwhere 𝜆 is a dynamically scaled scalar for balance between recognition loss and FLCL, which is computed as L REC /L FLCL [30] . 
experiments 4.1 datasets: 
 All models are trained on a union of two commonly used synthetic datasets, i.e., MJSynth [17, 18] and SynthText [15] , which contain about 14.4 M synthetic scene text images in total. Then, we first evaluate the models on six popular benchmarks: IIIT5K-Words (IIIT) [36] is the dataset crawled from Google image searches, which contains 3000 images for evaluation, and almost all of them are clear to recognize. ICDAR2013 (IC13) [20] contains 857 images for evaluation, of which 9.3% have low quality. CUTE80 (CT) is proposed in [41] for curved text recognition, where 288 testing images are cropped from full images by using annotated words, and about 9% of them are low-quality. Street View Text (SVT) [48] contains 647 outdoor street images collected from Google Street View, and about 14.2% of them have low-quality appearance. Street View Text-Perspective (SVTP) [40] is also cropped from Google Street View. There are 639 test images in this set, and about 20% of them are suffering from blurred or low-resolution distortion. ICDAR2015 (IC15) [19] contains 1811 images for evaluation. The images are captured by Google Glasses while under the natural movements of the wearer, resulting in about 23.5% low-quality images. In addition, we provide a task-specific benchmark for evaluation: Low-quality Text (LQT) is made up of the low-quality samples collected from the previous six datasets, which have a total of 761 images. The details of the benchmarks are shown in Table 1 .4.2.1 Data augmentation. We employ a combination of popular augmentation and visual distortions to generate low-quality views of the inputs, which can be formulated as follows:EQUATIONwhere 𝑥 and x separately denote the inputs and the associated lowquality samples. 𝑓 1 is the function for data augmentations, which includes Curve, Stretch, Shrink, AutoContrast, Fog, Snow, Frost, Rain, Shadow. 𝑓 2 is the function for visual distortions, which includes GaussianBlur, DefocusBlur, MotionBlur, GaussianNoise, JpegCompression, Pixelate. All of the operations are equal in probability and achieved by the Straug [2] library. Noteworthy, we have not conducted other augmentations on the high-quality inputs separately during the training phase. To assess the generalizability of our proposed method, we have chosen nine popular light-weight OCR models for evaluation, which include CRNN [43] , SVTR-T/S [7] , EfficientNetV2-b0/b1 [46] , EdgeViT-XXS/XS [38] , and EfficientForm-erV2-S0/S1 [24] . Notably, the EfficientNet series incorporates two Bi-LSTM layers with a hidden size of 256 for sequence modeling. Across all these models, a fully-connected layer is utilized as the decoder to transcribe contextual features into the text. In addition, the downsample ratio is set to [×32, ×4], the dimension of the output feature is set to 512, and the dimension of the contrastive learning feature is set to 192.The rectification module [31, 44] is employed for distortion correction. All the input RGB images are resized to 32 × 100, and the maximum length of prediction is set to 25. We adopt the Adam optimizer [22] with a cycle learning rate from 2e-3 to 1e-8 for training, where the weight decay is set to 1e-5.The training batch size is 256, and the training epoch is 5. Gradient clipping is used at magnitude 5. All experiments are conducted on NVIDIA RTX 4090 GPUs.We use word accuracy (ACC) to evaluate all models' performance, which is the ratio of the number of totally correct predictions over the number of test samples. Besides, we also report the number of parameters and the inference speed. Notably, only numbers and letters (case-insensitive) are evaluated.To demonstrate the effectiveness of each component in the proposed framework, we perform an ablation study in this section. Since the IIIT, IC13, and CT include a small proportion of low-quality samples, we marked them as high-quality datasets, while the SVT, SVTR, IC15, and LQT are marked as low-quality datasets. For efficiency, we adopt the EfficientFormerV2-S0 trained by vanilla contrastive learning as the baseline on all seven datasets.Components. The proposed framework has two key components, i.e., text perceiver (TP) and frame-level contrastive loss (FLCL). The TP is designed to more efficiently purify the text-relevant information in the contextual features, while the FLCL is proposed to provide a character-level link between the text instances with different qualities. We conduct ablation to validate the effectiveness of TP and FLCL, and the results are shown in Table 2 . We can observe that applying TP to replace LN can bring an improvement of 1.3% on average accuracy. On the other hand, compared with popular word-level contrastive loss, FLCL results in an average accuracy improvement of 0.8%. Finally, it is worth noting that the combination of TP and FLCL further boosts the average accuracy of about 1.2%.Perceiver. Compared with Mamba [13] , the proposed text perceiver adopts a special bidirectional strategy for more efficient feature extraction. To illustrate the effectiveness of this design, we perform an ablation on the design of text perceiver, where we consider these strategies:• None. We directly adopt the Mamba block instead of the linear projection to purify the text-relevant information in the contextual features within the forward direction. • Bidirectional Sequence. We randomly flip the contextual features during the training phase, which is like data augmentation. • Bidirectional SSM. We add an extra SSM layer for each block to process the reversed contextual features. • Bidirectional SSM + Conv. Based on Bidirectional SSM, we further add a Convolution layer before the SSM in the backward branch. (as shown in Fig. 3 ). As indicated in Table 3 , adopting the Mamba block achieves better performance than linear projection, while applying additional bidirectional strategies can further boost the averaged accuracy to varying degrees. (0.3%∼0.8%). Noteworthy, the strategy of a bidirectional SSM layer with convolution achieves the best results, which demonstrates the effectiveness of the text perceiver.The relative scale of recognition loss and the frame-level contrastive loss will be changed at different epochs of the training process. Based on this observation, we 4 . For static scalars, we can see that paying too much attention to contrastive loss will affect the recognition performance of the model faced with high-quality samples, while paying little attention to contrastive loss will make the performance of the model decline under low-quality scenarios. However, as for the dynamic scalar, it is able to provide the model with the highest recognition accuracy in both high-quality and low-quality datasets.To demonstrate the effectiveness of the proposed framework, we compare the performance of it and the CTC framework with / without contrastive learning (CL) on seven popular light-weight OCR models mentioned in Sec. 4.2, and the results are reported in Table 4 . We can clearly see that, compared with standard contrastive learning, our method can provide an average accuracy improvement of about 2% for various models with different backbones over all benchmarks without any additional cost, which profoundly verifies the effectiveness of our method at the model level. Overall, since it is difficult for vanilla contrastive learning to efficiently extract text-relevant information from the extracted contextual features, the models trained by CL are usually suffering from the unbalanced performance between the samples of different quality. However, due to the text perceiver, our framework can provide more consistent performance improvements for the models when faced with different-quality samples. To be specific, CRNN, the most classical, representative, and widely used lightweight text recognition model, obtains a 4.2% average accuracy increment via our framework. Besides, the advanced CTC-based text recognition method, the SVTR series, gains over 1% improvement in average accuracy with our method. In addition, all the rest of the text recognition models, i.e., EfficientNet, EdgeViT, and Effi-cientFormer series, also achieved about 1%∼2.5% improvement in average accuracy by our method. Furthermore, for datasets containing a large number of blurred or low-resolution samples, e.g., SVT, SVTP, IC15, and LQT, the improvement brought by our framework is more significant.State-of-the-Arts. To illustrate the superiority of our methods, we compare it to the state-of-the-art methods designed for the light-weight OCR models, e.g., FocalCTC [9] , EnCTC [27] , and DCTC [61] , with the classic OCR models. They are The results of † are reported by [4] , and the results of ‡ are reported by [7] .widely applied in real-life scenarios to enhance the performance of the light-weight OCR model without additional cost. Since these methods are not specifically designed for low-quality text images, we only report the results on the six popular benchmarks for fair comparison, which are shown in Table 5 . We can observe that in datasets containing a larger proportion of low-quality samples, i.e., SVT, SVTP, and IC15, our method can provide the models with significantly the best performance among SOTAs, illustrating its advantage in enhancing recognition performance in low-quality scenarios. Furthermore, although our method aims to enhance the recognition performance of the models in low-quality scenarios, it can also effectively enhance the model performance when faced with high-quality samples. In general, our method brings the largest increment of accuracy for not only CRNN but SVTR-T on most benchmark datasets, resulting in a 6.9%, and 1.9% improvement of the average accuracy, respectively, which is more than double the best of SOTAs.In a low-quality scenario, it is very difficult for models to identify samples with confusing characters. Our method introduces a concise yet effective text perceiver to replace the linear projection and suggests an additional frame-level distillation between high-quality samples and associated low-quality views besides the recognition supervision, which promotes the model to extract more discriminative features when faced with low-quality text images and thus improve its overall performance. To qualitatively demonstrate the effectiveness of our method, we provide a series of visualization analyses with EfficientFormerV2-S0 that is trained by vanilla contrastive learning, i.e., the baseline, and the proposed frame-level contrastive learning, respectively.To verify the effectiveness of our method, we conducted a feature visualization study with t-SNE [47] . Specifically, we select several hard example groups that are composed of characters prone to being wrongly recognized as each other. We crop some examples from the images on the test sets and separately fetch their feature embeddings from the baseline and our method. Fig. 4 shows the feature projections of two hard example groups, where different characters are marked with different colors. We can clearly observe that even when faced with low-quality samples with very similar appearances, our method can still drive the model to extract more discriminative features that are more cohesive than those extracted by the baselines. Furthermore, some predictions of high/low-quality sample pairs are given in Fig. 5 . We can find that it is easier for the model trained by our method to distinguish the confusing characters in low-quality cases and make consistent predictions between high-quality and low-quality samples. For example, the prediction of the first low-quality example in Fig. 5 is corrected from 'pleose' to 'please' by our method. 
conclusion: 
 In this paper, we propose a concise yet quite effective strategy to enhance the performance of lightweight STR models when faced with low-quality samples without additional cost, which includes a SSM-based text perceiver and a frame-level contrastive loss. By employing the text perceiver to derive the text-specific information from the contextual features extracted by the backbone and then prompting character-focused feature learning via frame-level contrastive loss, our method can help STR models learn more robust text representation, thus improving their recognition performance. The superiority of our method has been illustrated by both quantitative and qualitative analysis of several popular STR benchmarks. The proposed method not only has excellent generalization performance but also achieves the best results compared with SOTAs.  
: 
 and other non-parameter methods, the FLCL framework significantly outperforms others on all datasets, especially the low-quality dataset. In addition, character feature visualization demonstrates that the proposed method can yield more discriminative character features for visually similar characters, which also substantiates the efficacy of the proposed methods. Codes and the low-quality dataset will be available soon.• Computing methodologies → Object recognition; • Computer systems organization → Neural networks.Scene Text Recognition, Low-quality, contrastive learning, State Space Model ACM Reference Format: Anonymous Author(s). 2018. Free Lunch: Frame-level Contrastive Learning with Text Perceiver for Robust Scene Text Recognition in Lightweight Models. In Proceedings of Make sure to enter the correct conference title from your rights confirmation emai (Conference acronym 'XX). ACM, New York, NY, USA, 10 pages. https://doi.org/XXXXXXX.XXXXXXX 
 
 Appendix:  


Decision: This paper presents a new approach to enhance lightweight Scene Text Recognition (STR) models by introducing a frame-level contrastive learning (FLCL) framework combined with a Text Perceiver Module (TPM). The proposed method demonstrates improvements in low-quality scenarios, validated through experiments across multiple benchmarks. The reviewers acknowledge the innovative use of frame-level contrastive learning and the effective integration of the TPM, which contributes to the robustness of the STR models. However, some concerns were raised regarding the lack of comparison with closely related works, such as SeqCLR, and the absence of detailed parameter and inference speed analyses.

Reviews: review: paper_summary: paper_summary: The proposed frame-level contrastive learning (FLCL) framework for lightweight Scene Text Recognition (STR) models demonstrates an innovative approach by focusing on frame-level rather than word-level features. The introduction of the Text Perceiver Module (TPM) effectively enhances text-related representations, thereby improving the performance of lightweight models in low-quality scenarios without increasing computational costs. Comprehensive experiments across multiple STR benchmarks and a new low-quality dataset validate the efficacy of the FLCL framework, showing significant performance improvements over existing methods.
strength_weakness: strength_weakness: 1. Innovative Approach: The paper introduces a novel frame-level contrastive learning (FLCL) framework for Scene Text Recognition (STR), focusing on frame-level rather than word-level features. This approach is well-written and provides a new direction for improving lightweight STR models.

2. Comprehensive Experiments: 
The authors conducted extensive experiments on multiple well-known benchmarks, as well as a new low-quality dataset. The experimental results convincingly demonstrate the effectiveness of the proposed FLCL framework over existing methods. This includes a thorough ablation study and comparisons with other contrastive loss methods.
limitations: limitations: While the FLCL framework presents significant advancements in lightweight STR models, addressing the outlined concerns and inconsistencies will further solidify its contributions to the field.

1. Concerns Regarding Table 6 Experiments and Contribution: 
The comparisons with methods that only modify the loss function in Table 6 may be unfair. Integrating the State Space Model (SSM) from Vision Mamba likely contributes significantly to the observed improvements. 

2. The "Parameter" Column in Table 5 is not objective: 
The current scope of the paper focuses on improving lightweight STR model performance with little or no additional costs. However, using the state space model to replace the linear projection layer should be reflected in the parameter. Including a comparison of computational costs for models with and without the Text Perceiver Module would provide a clearer understanding of the additional overhead associated with the proposed improvements.

3. Some typos and inconsistencies: 
* Lines 56-57 contain irrelevant information.
* Lines 326-327 have inconsistent abbreviations.
* Line 408 has a typo in "Mamba."
paper_summary: paper_summary: This paper proposes a frame-level contrastive loss for the scene text recognition task. First, the authors calculate contrastive loss using each vector from the feature map. Then, a mamba-based projection module is proposed to replace linear projection in contrastive loss. As a result, the proposed loss outperforms basic contrastive loss on multiple baseline models.
strength_weakness: strength_weakness: 1. Employing contrastive learning on a fine-grained level is more suitable for character-level text recognition task.
2. Adequate experiments have been conducted on multiple lightweight STR structures, verifying the superiority of the proposed loss. 
3. The paper is well-organized and easy to follow.
limitations: limitations: 1. In line 131 and 206, it seems that references [1, 28] in the paper also use frame-level contrastive loss (such as “Frame-to-instance” in SeqCLR). Besides, “Sequence-to-Sequence Contrastive Learning for Text Recognition” (ACM MM2023) also proposes multi-level contrastive learning. Authors should further discuss with them.
2. In line 120, the proposed method cannot reduce the computation or parameters for STR models. I think FLCL has no special advantage for lightweight models. Why do authors only focus on lightweight models and CTC decoders? What is the performance of normal STR models and attention decoders?
3. Transformer layer also can generate bidirectional representation (ABINet, CVPR2021). Can attention-based Text Perceiver get similar results with the mamba layer?
4. Authors should also report their results on Union14M which contains more challenging examples.
paper_summary: paper_summary: The paper proposes a novel framework to enhance the performance of Scene Text Recognition (STR) models, especially under poor image quality conditions. It introduces a frame-level contrastive learning (FLCL) approach combined with a Text Perceiver Module (TPM) to filter text-relevant features and suppress noise. This method improves the model's ability to recognize text by generating more discriminative features. The authors demonstrate the effectiveness of their approach through experiments on several STR benchmarks and a new low-quality dataset, showing substantial performance gains.
strength_weakness: strength_weakness: (1)	Traditional contrastive learning techniques in STR operate at the word level, which can overlook the sequential nature of text data. By implementing contrastive learning at the frame level, the proposed approach aligns and discriminates features more precisely, enhancing model robustness against low-quality images.
(2)	The integration of FLCL with Connectionist Temporal Classification (CTC) loss is theoretically sound, leveraging the strengths of both techniques. CTC is widely used in sequence-to-sequence problems like STR, and combining it with frame-level contrastive learning ensures the model learns more robust and discriminative features.
limitations: limitations: (1)	While applying contrastive learning at the frame level is innovative within the context of STR, the concept of contrastive learning itself is not new. To accurately evaluate the novelty of this approach, it is essential to compare it with the current state-of-the-art methods that similarly employ contrastive learning. Such a comparison would provide a clearer perspective on the distinctiveness and potential advantages of the frame-level application within STR. 
(2)	The analysis should not be limited to comparisons with CTC-based methods, it should also encompass a broader range of state-of-the-art techniques that have been published in recent years, such as ABINet (2021 ABINet), ConCLR (2022, AAAI), Parseq(2022 ECCV), I2C2W (2023 TPAMI), SSM (2024 IJCAI) et al.
paper_summary: paper_summary: Lightweight models play an important role in real-life applications, especially in the recent mobile device era. However, due to limited network scale and low-quality images, the performance of lightweight models on Scene Text Recognition (STR) tasks is still much to be improved. The paper proposes a new efficient and effective frame-level contrastive learning (FLCL) framework for lightweight STR models. The FLCL framework consists of a backbone to extract basic features, and a Text Perceiver Module (TPM) based on a Mamba structure to focus on text-relevant representations. Experiments on six well-known STR benchmarks are conducted. Compared to vanilla contrastive learning, the proposed method can perform better on challenging low-quality datasets.
strength_weakness: strength_weakness: 1) The paper proposes to perform frame-wise contrastive learning for scene text recognition. 
2) The paper proposes to replace the linear project head with a mamba-based project head.
3) Experiments are conducted on different backbones for evaluation.
limitations: limitations: 1) Frame-level contrastive learning is not new in scene text recognition which has been explored by SeqCLR [1]. However, neither reference nor comparison is represented for SeqCLR which is one of the most related works. Another contribution is the use of a mamba-based projection head, which is quite incremental. Besides, the parameter and inference speed comparison is also absent to demonstrate the lightweight nature of TPM.
[1] Sequence-to-Sequence Contrastive Learning for Text Recognition. 
2) Lack of comparison of the most recent scene text recognition papers in MM or CVPR, which is not reasonable. 
3) Detailed issues. In L327, the reference of [25] (DB, a scene text detection) is not appropriate; In Formula (4), the scope of the negative samples is not clear;
4) Given the description in Sec. 4.2.1 and Figure 2, after the data augmentation, the frame of the low-quality and the high-quality image features may be not spatially aligned. Doesn't it influence the process of frame-level contrastive learning?