Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing
Authors: Yadong Qu
Yuxin Wang
Bangbang Zhou
Zixiao Wang
Hongtao Xie
Yongdong Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: Scene text recognition
Semi-supervised learning
Contrastive Learning
Primary area: machine_vision

Abstract: Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. The limitation lies in the insufficient exploration of character morphologies, including the monotonousness of widely used synthetic training data and the sensitivity of the model to character morphologies. To address these issues, inspired by the human learning process of viewing and summarizing, we facilitate the contrastive learning-based STR framework in a self-motivated manner by leveraging synthetic and real unlabeled data without any human cost. In the viewing process, to compensate for the simplicity of synthetic data and enrich character morphology diversity, we propose an Online Generation Strategy to generate background-free samples with diverse character styles. By excluding background noise distractions, the model is encouraged to focus on character morphology and generalize the ability to recognize complex samples when trained with only simple synthetic data. To boost the summarizing process, we theoretically demonstrate the derivation error in the previous character contrastive loss, which mistakenly causes the sparsity in the intra-class distribution and exacerbates ambiguity on challenging samples. Therefore, a new Character Unidirectional Alignment Loss is proposed to correct this error and unify the representation of the same characters in all samples by aligning the character features in the student model with the reference features in the teacher model. Extensive experiment results show that our method achieves SOTA performance (94.7\% and 70.9\% average accuracy on common benchmarks and Union14M-Benchmark). Code will be available.

Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing
Yadong Qu, Yuxin Wang, Bangbang Zhou, Zixiao Wang, Hongtao Xie, Yongdong Zhang
University of Science and Technology of China, Hefei, China
lqqqyd, bangzhou01, wzx99)@mail, ustc. ustc.edu.co edu. {wangyx58, htxie, (hyd73)@ustc.edit
Abstract
Existing scene text recognition (STR) methods struggle to recognize challenging texts, especially for artistic and severely distorted characters. The limitation lies the insufficient exploration of character morphologies, including the monotonous- ness of widely used synthetic training data and the sensitivity the model to character morphologies. To address these issues, inspired by the human learning process of viewing and summarizing, we facilitate the contrastive learning beadeing-based STR framework in self-motivated manner by leveraging synthetic and real un- labeled data without any human cost. the viewing process, to compensate for the simplicity of synthetic data and enrich character morphology diversity, we propose an Online Generation Strategy to generate background-fee free samples with diverse character styles. By excluding background noise distractions, the model is encouraged to focus on character morphology and generalize the ability to rec- ognize complex samples when trained with only simple synthetic data. To boost the summarizing process, we theoretically demonstrate the derivation error in the previous character contrastive loss, which mistakenly causes the sparsity in the intra-class distribution and exacerbates ambiguity on challenging samples. There- fore, new Character Unidirectional Alignment Loss is proposed to correct this егror and unify the representation of the same characters in all samples by align- ing the character features in the student model with the reference features in the teacher model. Extensive experiment results show that our method achieves SOTA performance (94.7% and 70.9% average accuracy on common benchmarks and Union 4M- MeM--Benhhark... Code will be available at lhttp:://githbbb.. http:g/wwww.ooogggg///oooa
Introduction
Scene text recognition (STR) aims to recognize text in cropped text images. As fundamental task, STR can provide auxiliary information for understanding natural scenes and has wide applications in financial systems, virtual reality, and autonomous driving. Because of the expensive and time- consuming annotation process, most STR methods turn to two commonly used synthetic datasets, MJSynth 14 and SynthText 112. Despite achieving satisfactory recognition results, they still struggle perform well in challenging real-world scenarios. We attribute the limitations to insufficient exploration of character morphologics. First, as shown in Fig. [a. a, b. c). synthetic data are almost simple samples, whose character morphologies distorted real-world characters. The limitations exhibited by training models using only synthetic data further amplified the challenging benchmarks WordArt (45) and Union 14M Benchmark (Union-Bi B) [15]. For example, equipped with a language model, ABINet [0] only reaches 67.49 67.4% and 46.0% accuracy WordArt and Union-B B.. Second, the model is not robust to the morphological
significantly different from the artistic and
Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

M a
Percipiente
Sacrum
YOURSELF Sender
like
was
Day
(a)
Simple synthetic samples
merry
menry
isn't Fanl
red
The
the
harana
sign
sign
girl
Jay's
jay's
(b) Different representation of character Y
(c) Similar representation
different characters
(d) ParSeq
(e) ViSu
Figure I: (a) shows some images from synthetic datasets MJSynth and SynthText. (b) and (c) show several challenging test images. (d) and (e) display the visualization of character feature distribution.
changes in characters. As shown in Fig. lb, c), due to severe distortion and artistic style, a character can visually differ from others of its category, and on the contrary, characters belonging to different categories can have similar appearances. Fig. dd) further shows the distribution of challenging
character features in ParSeq (2 from I-SNE 38 where the characters cannot be clearly distinguished.
To compensate for the diversity of character morphologies, some previous methods [ IO] introduce real unlabeled data by first training a model in the supervised way and generating pscudo-labels. However, it relies heavily on the quality of the pretrained model. The wrong pscudo-label will accumulate the errors and harm the training process. TRBA-cr (54) and Yang et al. [7 employ character- consistency regularization to train the model in a semi-supervised manner and align the sequence output from two views of unlabeled data. Nevertheless, simply assimilating more real unlabeled data and ignoring the diverse character morphologies can lead to improvement on common benchmarks but little superiority on the challenging Union-B. To enhance the robustness to character styles, some methods [4 20 synthesis the texts with unified styles and map the texts with different writing styles to the invariant features. However, they focus on handwritten mathematical expressions without the diverse orientations and affine transformations common in complicated scene text images.
In this paper, we attribute the bottleneck of the STR model to the monotonousnes: of training data and sensitivity of the model character morphologies. Inspired by the human learning process, we divide the scene text recognition into two steps: viewing and summarizing. Viewing means that when encountering new language, the first thing to collect and see as many characters with different morphologies as possible. This also conforms to the principle that deep learning models are data-driven. After seeing enough samples, we can summarize their visual feature commonalities and classify them into different characters. With small amount of label guidance, we can continuously optimize this cognitive process and distinguish some challenging characters, which referred to summarizing. Following this paradigm, propose semi-supervised scene text recognition framework by Viewing and Summarizing (ViSu).
To enrich the diversity of character glyphs in the viewing process, firstly, real unlabeled data is similar to practical application scenarios and has a variety of character morphologies. Based on contrastive learning, we adopt the Mean Teacher framework to jointly optimize the model using real unlabeled data and simple synthetic data. Secondly, to further enrich the training data and compensate for the simplicity of synthetic data, propose an Online Generation Strategy (OGS) to generate background-free samples with challenging styles based on the annotations of the corresponding synthetic data. Compared with the vanilla Mean Teacher framework that simply uses labeled data to train the student model, our method equipped with OGS allows the teacher model to generate guidance of various character styles without the interference of background noise. Thereby, the student model can learn practical features robust to character morphological diversity. Without any human annotation, the model is able to generalize the ability to recognize challenging samples while only training with simple synthetic data, which substantially raises the performance upper bound of the semi-supervised learning framework.
After viewing many characters with various shapes, in order to imitate the summarizing process, the model needs to identify the commonalities of each character and cluster the same characters. Firstly, as shown in Fig. the eight inconsistent representation forms of scene text images caused by reading order and character orientation undoubtedly aggravate the convergence burden. We unify

---

them into two primary forms based on the aspect ratio to enable the model to extract applicable and discriminative character-level features. Secondly, we theoretically prove that the previous character contrastive loss [4 confuses some character classes and incorrectly encourages the sparsity of the intra-class distribution, thus hindering the optimization of the model. Therefore, a novel Character Unidirectional Alignment (CUA) Loss is proposed to eliminate this mistake and focus on character morphological feature alignment to achieve character clustering within the same class. The images from OGS and weak augmented images, referred to as base images, make it easier to obtain noise-free and glyph-diversífied character features that help recognize complex samples. CUA Loss forces the character features under strong data augmentation to be aligned with the base image features from the teacher model, allowing the model to obtain unified features for each category of characters.
The main contributions are as follows: 1) We propose the viewing and summarizing paradigm to conduct the training of the semi-supervised model. Our model can adapt to complex scenarios such as multi-orientation and multi-morphic characters without any human annotation. 2) An Online Generation Strategy is proposed to facilitate model generalization from simple synthetic data to recognize challenging samples, which in turn improves the performance upper limit of the semi- supervised learning framework. 3) A novel Character Unidirectional Alignment Loss is proposed to theoretically correct the formula error of regarding some positive samples as negative samples and enhances the compactness of the intra-class distribution. 4) Our method achieves state-of the art performance with an average accuracy of 94.7% common benchmarks and 70.9% on the challenging Union 4M-44M44aann Benchmark.
2 Related Work
2.1 Scene Text Recognition
STR methods can be roughly divided into language-free and language-based methods, most of which are trained in a fully supervised manner on synthetic datasets. Due to the simplicity of synthetic data and the diversity of text styles in real scenes, existing methods still have difficulty
recognizing complex samples. Some language-free methods (3 22
23
46
51 55) design different
rectification modules to convert distorted irregular text into regular text to improve recognition
accuracy. CornerTransforne Transformer [45] proposes to utilize the corner points of artistic text to guide the
encoder in extracting useful features from characters. The language-based methods 22 2 10 41l [2)
[9] do not rely solely on the visual features of text images. They infer the contextual information of complex characters by introducing additional linguistic knowledge, enabling the model to correct the visual recognition results. MGP-STR (41) proposes a multi culli-gramularity (character, subword and
word) prediction strategy to implicitly integrate linguistic knowledge with the model.
2.2 Semi-Supervised Learning
As widely acknowledged, deep learning methods require substantial labeled data for effective model fitting. However, acquiring a large amount of high- labeled data is typically challenging. Therefore, semi-supervised learning aims improve model performance by leveraging large amounts of unlabeled data and limited labeled data. Yang et al. [48] comprchensively analyses
supervsde
learning methods. They can mainly be categorized into deep generative methods [6, 26) consis-
tency regularization methods (29 31 19 37 and pseudo-labeling methods 3 g 21 4. CatGAN (36
considers the mutual information between observed samples and their class distribution by modifying the original GAN loss. Temporal Ensembling [99 requires consistent predictive outputs of data under various regularization and data augmentation conditions. Mean Teacher (37) utilizes the exponential moving average (EMA) to update the parameters of the teacher model during training, which tends to generate more accurate model. Co-training 3 models posit that each sample in the dataset has two distinct and complementary views, and either view is sufficient to train a good classifier.
2.3 Semi-Suuerriied Text Recognition
Some researchers propose integrating semi-supervised learning with text recognition to enhance model recognition performance. Baek el al. ( chooses to pre-train a model only using small amount of real labeled data, and then further enhances the model performance using Pseudo Label or Mean Teacher with unlabeled data. ABINet [10] proposes ensemble self- self-training strategy to
3

---

Party
7 JOS S
TPO
AAVM
GREAT
BALFORIAA
apandard d pa ue Q 22
G
(a)
(b)
Figure 2: (a) All possible representations of English text images according to character orientation and reading order. (b) The unified representation forms of the word "standard" obtained through Online Generation Strategy. The first row with red border shows two primary forms, and the second row can be obtained by rotating them 180 degrees.
enhance model recognition performance. However, these two methods require two-stage training, which greatly reduces the training efficiency of the model. Zheng al. [54) is the first to apply a semi-supervised framework based on consistency regularization to the STR task. It designs character-level consistency regularization align the characters of the recognition sequence. Gao al. [ employs well-designed edit reward and embedding reward to assess the quality of generated sequence and optimizes the network using reinforcement learning. Seq-UPS [27], considering that generated pseudo labels may not always be correct, proposes a pscudo label generation and an uncertainty-batt based data selection framework.
3 Methodology
As illustrated in Fig. [ our framework adopts the Tean-Teecher architecture. The student model and teacher model use the transformer- based encoder and decoder. The parameters of the student model O are updated through the backpropagation algorithm, while the parameters of the teacher model O are an exponential moving average (EMA) O can be formulated as θ +- aO, + O where a E [0, I] is the smoothing factor. For the viewing process, we introduce not only augmented real unlabeled data but also the Online Generation Strategy (OGS) to obtain samples with challenging glyphs to eliminate the model's sensitivity character morphology. For the summarizing process, theoretically analyze the flaw of the existing character contrastive loss and further propose Character Unidirectional Alignment (CUA) Loss to align character features from different views, thus achieving a unified feature representation of multiple-morphic characters. Details will be introduced in the following sections.
3.1 Online Generation Strategy
Adopting the semi-supervised framework to leverage extra real unlabeled data to enrich the character morphologies during the viewing process, but modification has been made to the existing supervised branch. To address the problem that the labeled synthetic training data is still simple and monotonous, we propose OGS to unify the training process of labeled and unlabeled data and raise the performance upper bound of the semi-supervised framework.
Firstly, as shown in Fig. 2a), eight inconsistent representations brought about by the diverse character orientations and reading orders undoubtedly increase the difficulty of network convergence. Besides, scene text images with vertical reading orders often have small aspect ratios. When they resized for input into the STR model, aspect ratios change significantly, resulting in them containing little practical information. Therefore, an image with heighl/ width > r will be rotated, otherwise it remains unchanged, which referred to as Unified Representation Forms (URF). As shown in Fig. 2).), unified four representations can be further simplified into two primary forms with left-right reading order. Through URF, we drastically reduce eight representation forms to two primary ones to lessen the learning difficulty and lay the foundation for the following generation process.
Secondly, enrich the character morphology the supervised branch, according to the labels of simple synthetic training data, we propose to generate text samples in two primary forms in Fig. b with the red border. The generated samples are background-fre but with random character styles to encourage the model to concentrate on character morphology. Following the characteristics of real samples, character orientations one generated sample are the same. OGS samples serve as base images to guide the alignment of the character features in the student model. Unlike previous

---

Lrcc
"avenue"
Classifier
AVENUE BRIDGE
Strong Augmented Image
Encoder
Decoder
Projector
F
MLP
AVENUE
EMA
Labeled BRIDGE Unlabeled
Leua
Lec
pb
MLP
VENU BRIGE
Encoder
Decoder
Classifier
pb
Base Image
Student model
Teacher model
Labeled and unlabeled data flow
Only labeled data flow
Figure 3:
Our framework consists of the student and teacher model. Lree: Lecrs Lewe mean recogni-
tion loss, character consistency regularization loss and character unidirectional alignment loss. Green and orange stand for labeled and unlabeled data, respectively.
semi-supervised methods [54 47 our design is based on the following reasons. The goal of
STR model is to obtain character visual features that are not affected by background noise. Because the teacher model is gradient-fres madient-free and needs to instruct the training of the student model, OGS is expected to generate samples that are noise-free and adaptable to various character morphologies. 2) Due to the existence of word-level annotations and the fact that OGS samples only contain various character features, feeding them to the teacher model is more helpful for the model to learn valuable features for recognizing characters in artistic style severe distortion. In summary, by exploring the introduction of real unlabeled data and the extension simple synthetic data during the viewing process, the cemi-supervised model can significantly improve the recognition of challenging texts without introducing any human annotation, and break through the performance upper limit.
3.2 Character Unidirectional Alignment Loss
After viewing a large number of various characters, the model is supposed to summarize the com- monality of each character. As mentioned above, the base images input to the teacher model are more likely have practical and noise-free character features with diverse glyphs. Because the gradient of the teacher model is stopped, the character features of strong augmented images can be effectively aligned with base images to enhance the robustness the model to character morphologies and cluster the same characters. Therefore, the CUA Loss is proposed to achieve this goal. A character will calculate the similarity with all the characters in a minibatch. Characters within the same class supposed to have similar features, while the others should exhibit distinct features. Specifically, the model extracts a sequence features F = {ff f2 mented and base images, respectively, where T is the maximum recognition length in the decoder. At position f" and f.' are the character features for augmented and base images. y² is the ground truth for labeled data and the prediction from the teacher model for unlabeled data. We define the set for all position = (1,2, ... B x T). where B is the batch size. The positive set for character at position i P(t) = {p E / 1/c- v.. St > ]. where Toaa is the threshold for confidence score S.. For labeled data, S is 1. For unlabeled data, S is the confidence score for the whole predicted word Sh The positive set excluding character at position i is (i) = P(i) {p E 1 = i}. and the negative can be represented as (()) = P(i). Our CUA Loss can be formulated as
f
}.). F² = (ff.fff
ff) for aug-
exp(S 1//T) log
Lewe =
Σ IET
Σ
(1)
P())
A(i.p)
PEP(()
5

---

A(i. = erp(ff f2/T)
erp(- fT)++ p'E
E
E explfe fhe/t), n'EN())
(2)
where P(i) means the number of positive characters with and is the temperature factor.
worth noting that our CUA Loss differs from Character Contrastive (CC) Loss [45 in both motivation and operation. The formulation of CC Loss can be expressed
exp(S: Sp/T)
Lee =
Σ IE!
Σ
log
(3)
P'(1)
B(i)
(EP
B(I = E
explsi Sw/T) WEN(()
Σ
explfe fw//T;
(4)
P'EP(()
where all the symbols have the same meaning Eq. and Eq First, the motivation of CC Loss is to simply cluster the same character in a minibatch, and all images are treated equally. While our CUA Loss works two types of images: strong augmented images and base images. As stated before, due to the different roles played by the two types of images in our framework and the gradient- free characteristic of the teacher model, all positive characters in base images targeted for alignment by strong augmented images. Second, CC Loss suffers from a theoretical formula error and incorrectly sparses the distribution of characters belonging to the fi calculates the similarity with fp. other characters belonging to the same class appearing in the denominator are considered negative samples by mistake. To be specific, the gradient of the two loss functions can be expressed as
category. As shown in Eq. when
SLena Off
MI +h.Sh+Epe EpEP'N) M2 fb EnVENN Ms fh A(i.p)
Ms
Σ
(5)
T(P(i)
PEP())
dhec
NI fp + T EvEPIO) N2 fo + EnVEN()) fn B(i)
Σ
(6)
0f:
-IP'(i))
PEP'(1)
where
Mi A(i,p) exp(si f2/T),
N1 B(i),
M₂ exp(- - 1 fb /T),
N₂ -erp(s /T),
M3 = -eap(s /2/
N3 = -exp(si fw/TT.
The detailed proof is included in Appendix A As shown in Eq, and and Eq. 7 it is clear that coefficient M1, M2 > O and M3 < O. which means the gradient for S is towards the positive character features f and f in base images. f. is considered negative sample and needs to be distinguished. As shown in Eq. and Eq. 7 conditions are the same for N1 > and N3 < However, N2 < 0 indicates that the positive sample fp' is mistakenly regarded as negative sample like fw, which harmful to the learning of clustering characters in the same class.
3.3 Training objective
Given base images x", the teacher model obtains sequence of character features Fè and prediction probabilities p {pt. ..... . Correspondingly, the student model generates F and p {ri.PE. PG. } for strong augmented images x". Following [54), P is obtained with an additional projector employed in the student model ensure stable training process. We also adopt Character- level Consistency Regularization (CCR) to boost the training of our semi-supervised framework. This process can be formulated as
T
Leer = I(Sb Mear) T6 KL(PL.P)) p:):
1
Σ
where I(-) is the indicator function. th is the text length of x". neer is the threshold for confidence score 8hh which is 1 for labeled data and the cumulative product of the maximum probability sequence

---

Table 1: Comparison with SOTA methods on common benchmarks and Union-B. meaas we use publicly released checkpoints to evaluate the method. i means we reproduce the methods with the same configuration. For training data: SL MJSynth and SynthTTxt: RL. Real labeled data; RU- Union14M-U; RU Book32, TextVQA, and ST VQA; RU Places2, OpenImages, and ImageNet ILSVRC 2012. Cur, M-O, Art, Ctl, Sal, M-W, and Gen represent Curve, Multi-Orientee Artistic, Contextless, Salient, Multi- Words, and General. P(M) means the model size.
Donasets
SVT
ICT3
Cammon Renchmarks
RC13
SVTP
CUITE
WAYG
Cir
Art
Union RM--Rechhakkkk
Sal
Cenn
PMO
CRNN
SL
52.9
81.6
91.9
69.4
70.0
65.5
78.6
7.5
0.9
20.7
25.5
13.9
25.5
32.0
18.0
8.3
Robeatcanner
50
SIL
95.3
88.1
94.8
77.1
79.5
90.3
38.4
43.6
7.9
41.2
42.6
44.9
46.9
39.5
38.1
SRN
95.5
82.7
85.1
67.8
63.4
56.3
26.7
46.3
SL+Wiki
97.4
89.3
89.2
59.5
50.5
VisonLAN
1N
SL.
95.8
91.7
95.7
R3.7
86.0
85.5
91.2
57.7
14.2
47.8
48.0
64.0
47.9
52.1
47.4
32.8
MATRN
S1.
96.6
95.0
97.9
85.6
90.6
93.5
93.5
63.1
13.4
43.8
41.9
65-4
53.2
57.0
48.4
44.2
SVTR 2
96.0
91.5
97.1
85.2
89.9
91.7
92.3
63.0
32.1
$7.9
44.2
67.5
49.1
528
49.5
24.6
ParSegt
93.6
88.9
92.3
58.2
67.7
23.8
MGP* +1)
96.4
94.7
97.3
87.2
91.0
90.3
93.3
55.2
52.8
48
65.1
48.
50.0
141.1
CLIPAOCK
S3.
97.3
94.7
97.7
87.2
89.0
93.1
93.8
99.4
15.9
57.6
59.2
69.2
62.5
62.3
55.2
31.1
LPV [3
SR.
97.3
94.6
97.6
87.5
90.9
94.33
94.0
68.3
21.0
59.5
65.3
76.2
63.6
62.0
59.4
35.1
LISTER
93.8
89.6
90.6
93.5
62.6
49.9
CRNN- pm
RL-RU
74.1
85.1
TRBA p
BL-KU*
94.9
92.4
94.2
84.3
82.8
67.9
90.7
62.9
12.5
64.7
74.3
51.2
68.1
52.0
56.5
49.9
ABINe:
10
SL+ SL.WWii-Uhhr
96.9
94.7
97.0
85.9
89.0
69.9
95.0
60.9
14.2
47.9
16.6
65.6
49.0
57.6
48.8
36.7
TRBA-cr
SL-RUP
96.5
96.3
98.3
89.3
93.3
93.4
94.5
77.5
30.4
66.0
59.8
76.3
79.5
64.6
59.2
49.0
TRBA-er
SR.4RII
91.3
CRNNI
SL.
58.8
82.7
90.3
69.5
69.3
76.7
$1.4
21.1
63.0
26.3
25.0
20.1
25.0
38.6
31.3
5.6
CRNN--VSU
SLIRU
64.77
901.9
73.6
77.4
24.0
27.6
TRBA1
92.7
96.3
93.4
85.6
87.9
91.2
63.8
49.1
70.2
51.8
49.9
TRDA ViSa
SL+RU
96.4
96.1
97.2
86.2
87.8
89.9
92.9
71.8
83.9
63.4
52.3
73.2
57.5
54.4
66.6
69.9
Baneline
SL
96.1
96.4
96.5
86.0
88.2
88.5
92.5
60.5
R2.7
53.6
54.0
70.6
54.5
62.0
62.6
24.4
ViSu
Si. ARU
97.6
96.1
98.3
89.3
91.3
92.4
94.7
71.6
85.8
66.8
57.6
80.3
62.6
71.2
76.9
24.4
pf for unlabeled data. To extract the commonalities of each character and cluster the character features belonging to the same class, we align the character features Fb and F using CUA Loss, which is described in Sec. 3.2) To equip the model with basic recognition ability, both x² and x" for labeled data are input into the student model to obtain the predicted sequence ÿd and ya. We use character-level cross-entropy loss as the recognition loss, which can be expressed as follows,
1 T
Lree --ec=--2 Σ log 1og())))?"
where ýr means the prediction at position for 3dh and 3r. yg is the corresponding label. The overall loss function is
L= Lree + Leer + XLeua
(10)
where A is a hyper-parameter to balance the numeric values and is set to 0.
4 Experiment
4.1
Datasets
Without any laborious annotation process, method only needs Synthetic Labeled (SL) data and Real Unlabeled (RU) data. SL includes two widely used synthetic datasets MJSynth [1] and SynthText [12), which contain 9M and 7M synthetic images. For real data without annotations,
we adopt Union 1MMU [15) with a total of 10M refined images from Book32 [13, CC [32], and OpenImages [8] We evaluate STR models on six common benchmarks, including IIIT 24
SVT 40) 1C13 17, IC15 16). SVTP 28). and CUTE [30] Following the previous methods [54
use IC13-857 and IC15-1811 without images containing non-alphanumeric characters. To further evaluate the performance on difficult text images, we introduce several challenging benchmarks, including Union A Benchhacharrk (Union-B) [5). WordArt [45), ArT (6). COCO-Text (COCO) (39), and Uber-Text (Uber) [53]. Union-B comprises Curve, Multi-Oriented, Artistic, Contextless, Salient, Molti-Words, and General with 2,426, 1,369, 900, 779, 1, 1,585, 829, and 400,000 images, respectively.
4.2 Evaluation Metrics
The word accuracy is used to evaluate scene text recognition models, which means that all predicted characters in word must be identical to the label to be considered correct. We further calculate the
7

---

Table 2: Comparison with SOTA methods on several challenging benchmarks, All symbols have the same meaning as Table. [
Method
Datasets
WordArt
ArT
COCO
Uber
CLIP-OCR
SL
73.9
70.5
66.5
42.4
CornerTransforinn
45
SL
70.8
ParSeq
SL.
70.7
64.0
42,0
LISTER
SI.
69.8
70.1
65.8
49.0
MGP
SL
72.4
69.0
65.4
40.7
CRNN-pr*
RL+RU1
53.4
58.6
56.8
45.2
TRBA-pr*
RL+RU1
64.3
66.8
67.1
54.2
*BINet*
SL+Wiki+Uber
71.3
68.3
63.1
39.6
TRBA-cr*
SL+RU2
80.2
73.3
70.3
39.3
CRBA-cri
SL+RU
76.7
72.0
69.3
40.0
CRNN
SL
54.0
58.8
46.2
42.4
CRNN-ViSI ViSu
SL+RU
56.3
61.7
47.1
45.9
TRBA
SL
69.5
69.9
63.3
54.7
TRBA- ViSu
SL+RU
77.7
72.8
69.8
58.4
Baseline
SI.
72.3
74.7
67.6
60.3
Baseline- ViSu
SL+RU
80.5
78.8
74.9
67.2
weighted average score (WAVG) for six common benchmarks. Because the number of General in Union-B much larger than the others, we report their average score (AVG).
4.3 Comparison with SOTA
In Table. [ we compare our method with previous state-of-the-ar methods on common benchmarks and Union-B. Table. dissplass the performance on several challenging datasets. Our method achieves higher accuracy with smaller parameters all benchmarks than fully-supervised methods. Compared with the semi- sumi--sperviiedd methods, because the samples in common benchmarks are relatively easy, our method has an accuracy of 0.2% higher than TRBA-cr [54]. When evaluated on challenging benchmarks, our method shows significant performance superiority, namely 1 11.7%, 0.3%, 5.5%, 4.6%, and 27. 27.9% accuracy increase on Union-B, WordArt, ArT, COCO, and Uber, respectively. Because the released checkpoint of TRBA-cr 54) uses the different real unlabeled datasets that are not publicly available, for a fair comparison, we reproduce the official code and train with the same real unlabeled datasets as ours. Benefiting from Unified Representation Forms (URF), our method surpasses them by a large margin on the benchmarks containing enormous multi-directional texts (85.8% 23. 3.1%% on Multi- Caiti-Oriieted and 67. .2% vs 40.0% Uber). For other benchmarks with challenging texts, benefiting from OGS and CUA Loss, our method shows significant improvement in Artistic and WordArt.
We further apply our proposed framework other models. As shown the last six lines in Table. and Table. 2 to make the model robust to multi-directional text, the methods without ViSu and Baseline also adopt URF Therefore, they perform way better than other supervised and semi- supervised methods least 1.6% on common benchmarks, 3.5% on Union-B, 2.3% on WordArt, 2.9% on ArT, 0.9% on COCO, and 3.5% Uber, proving the effectiveness of our proposed framework.
Multi-Oriented and Uber. Equipped with ViSu, all the listed models gain at
4.4 Ablation Study
4.4.1 Online Generation Strategy
We generate 1.6M samples using OGS to replace MJSynth [1] and SynthText [12] datasets. Because MJSynth and SynthText contain approximately 16M images total, we utilize only 10% of them for fair comparison. As shown in Table. 3 ViSu trained with 10% synthetic data achieves an average accuracy of 58.2%. Upon incorporating OGS and employing the corresponding losses, we observe an improvement of 2.1 1,,, underscoring the effectiveness of proposed OGS and CUA

---

Table 3: Ablation experiments with different training data.
Method
Datasets
Cur
M-O
Art
Con
Sal
M-W
Gen
AVG
ParSeq
10% (MJ ST) +GG
54.3
15.7
52.3
53.2
67.3
55.9
58.8
51.1
MGP 41
10% (MJ + ST) + OGS
46.8
10.5
49.9
33.0
55.1
26.7
55.8
39.7
CLIPOCR 43
10% (MJ + ST) OGS
57.1
13.0
57.1
49.2
65.5
60.2
59.8
51.7
LPV [52
10% (MJ ST) OGS
58.6
12.9
53.3
53.3
67.4
59.3
56.9
51.7
LISTER
10% (MJ ST) + OGS
52.0
13.7
48.9
54.4
59.8
54.5
61.0
49.2
TRBA-cT
54
10% (MJ ST) + OGS
67.1
17.4
58.6
51.1
67.7
33.5
57.4
50.4
ViSu
10% (MJ + ST) OGS 10% (MJ ST) OGS
57.5
79.6
49.8
44.4
66.8
50.4
59.2
58.2
ViSu
1.7
25.1
5.8
4.9
3.0
6,9
10.2
8.2
ViSu
60.8
80.8
52.4
47.1
69.1
51.8
59.9
60.3
Table 4: Ablation experiments with different configurations. URF means unified representation forms. OGS represents the online generation strategy. RU indicates whether to use real unlabeled data. CC! means only aligning with other characters in strong augmented images. CC2 means the alignment between strong augmented images and base images.
Consistency loss
Alignment loss
URF
RU
Cur
M-O
Con
Sal
M-W
CR
58.0
17.0
52.9
55.4
64.9
54.8
61.3
$2.0
CE
CUA Loss
71.2
86.2
65.8
58.5
79.4
63.4
70.9
70.8
KL-div
4
69.9
83.9
64.1
$4.9
76.0
60.8
67.6
68.2
KL-div
CC Loss
69.4
84.7
64.8
56.5
76.1
62.2
69.8
KL-div
CC2 Loss
70.6
84.7
79.4
61.3
KL-div
CUA Loss
v
65.9
34.6
64.6
60.7
75.2
62.7
66.2
61.4
KL-div
CUA Loss
64.4
83.9
56.0
55.5
73.2
55.9
62.6
64.5
KI.div
CUA Loss
v
68.2
84.5
62.0
58.5
75.8
62.5
69.7
68.8
KL-div
CUA Loss
V
71.6
85.8
66.8
$7.6
80.3
62.6
71.2
70.9
Loss. However, the model trained solely on OGS samples struggles to recognize texts. We attribute the poor performance to the diverse styles of characters, domain gap with Union-B, and varied orientations of text images generated by OGS. Models trained from scratch with such challenging data struggle to extract practical character information and are susceptible to background noise, hindering the acquisition of basic text recognition ability. By incorporating simple synthetic data to establish a foundation for recognition abilities and boosting it with our proposed OGS and CUA Loss targeted for challenging texts, the model is able achieve promising performance. Furthermore, we add OGS samples to the training data of other methods to ensure the consistency of the dataset. The experimental results strongly demonstrate that the superiority does not come from the use of additional data but from the promotion of the semi-supervised learning framework by OGS.
4.4.2 Discussion about loss functions
We evaluate the performance of the model when using cross entropy and KL-divergence as consistency loss, respectively. As shown Table. 4 they have similar performance. For the alignment loss, both CC Loss and CUA Loss can bring performance improvement. However, as stated in Sec. 3.2 the original CC Loss only clusters the character features the minibatch, referred to as CC², Adopting framework and aligning the character features in strong augmented images the base images (CC?) achieve 1.0% higher performance. Moreover, because some positive samples are mistakenly treated negatives, adopting CUA Loss brings an accuracy gain of 0.8%, demonstrating the effectiveness of our CUA Loss.
4.4.3 Ablation on training setting
As shown in Table. because Curve and Multi ceiti-OOriened have many samples with multiple reading orders and character orientations, the model without URF has significantly lower accuracy on these benchmarks. When the model is not trained with real unlabeled data, it only reaches 64.5% accuracy. Compared to the baseline Table. T the proposed OGS, along with the consistency and alignment losses, brings a 1.9% accuracy improvement, which proves that OGS and CUA Loss improve the

---

recognition performance of the model on challenging text images when trained using only simple synthetic data. The experiments the last two lines Table. fllso prove the superiority of OGS.
Conclusion
In this paper, attribute the bottleneck of STR models to the insufficient exploration of character morphologies. Therefore, considering the human cognitive process, we develop a semi-supervised framework by viewing and summarizing. In the viewing process, we first adopt the Mcan-Teacher Teacher framework to introduce real unlabeled data. Secondly, an Online Generation Strategy is proposed to explore the potential of synthetic data and raise the performance upper bound of the semi-supervised framework. In the summarizing process, novel Character Unidirectional Alignment Loss is proposed to theoretically correct the previous formula error of mistakenly treating some positive samples with negative ones. Extensive experiments show that our method achieves state-of-the-ar performance on all benchmarks, with particularly signifieant superiority the challenging Union-B.
10

---

Acknowledgments
This work is supported by the National Key Research and Development Program of China (2022YFB3104700), the National Nature Science Foundation of China (U23B2028, 62121002, 62102384). We acknowledge the support GPU cluster built by MCC Lab of Information Science and Technology Institution, USTC. We also thank the USTC supercomputing center for providing computational resources for this project.
References
[1] Jeonghun Back, Yusuke Matsui, and Kiyoharu Aizawa. What if only real datasets for scene text recognition? toward scene text recognition with fewer labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 31-3-1222, 2021.
[2] Darwin Bautista and Rowel Atienza. Scene text recognition with permuted autoregressive sequence models. In European Conference on Computer Vision, pages 178-196. Springer, 2022.
[3] Avrim Blum and Tom Mitchell. Combining labeled and unlabeled data with co-training, In Proceedings of the eleventh annual conference on Computational learning theory, pages 92-100, 1998.
[4] Ting Chen, Simon Korbllith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi suni-supervieed learners. Advances neural information processing systems, 01222243-222555 2020.
[5] Changxu Cheng, Peng Wang, Cheng Da, Qi Zheng, and Cong Yao. Lister: Neighbor decoding for length-insensitive scene text recognition. Proceedings of the IEEE/CVF International Conference Compuler Vision, pages 19541-19551, 2023.
[6] Chee Kheng Chng, Yuliang Liu, Yipeng Sun, Chun Chet Ng, Canjie Luo, Zihan Ni, ChuanMing Fang, Shuaitao Zhang, Junyu Han, Errui Ding, et al. lcdar2019 robust reading challenge on arbitrary-shaped text-rre-art. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 1571-1577 IEEE, 2019.
[7] Emily Denton, Sam Gross, and Rob Fergus. Semi-supervised learning with context-conditional generative adversarial networks. arXiv preprint 1x1xv:6611.0644 2016.
[8] W Dong-DongChen and ZH WeiGao. Tri-net for semi-supervised deep learning. In Proceedings of twenty-seventh international joint conference on artificial intelligence, pages 2014-2020, 2018.
[9] Yongkun Du, Zhineng Chen, Caiyan Jia, Xiaoting Yin, Tianlun Zheng, Chenxia Li, Yuning Du, and Yu-Gang Jiang. Svtr: Scene text recognition with a single visual model. arXiv preprint arXiv:2205.001599 2022.
[10] Shancheng Fang. Hongtao Xie, Yuxin Wang, Zhendong Mao, and Yongdong Zhang. Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 7098-7101 2021.
[11] Yunze Gao, Yingying Chen, Jinqiao Wang, and Hanqing Lu. Semi- sumi-supervised scene text recognition. IEEE Transactions on Image Processing. 30:3005-3015 2021.
[12] Ankush Gupta, Andrea Vedaldi, and Andrew Zisserman. Synthetic data for text localisation in natural images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2315 23244, 2016.
[13] Brian Kenji Iwana, Syed Tahseen Raza Rizvi, Sheraz Ahmed, Andreas Dengel, and Seicchi Uchida. Judging book by its cover. arXiv preprint arXiv: 2016.
[14] Max Jaderberg, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Synthetic data and artificial neural networks for natural scene text recognition. arXiv preprint arXiv: 2227,
2014.
11

---

[15] Qing Jiang. Jiapeng Wang, Dezhi Peng, Chongyu Liu, and Lianwen Jin. Revisiting scene text recognition: A data perspective. In Proceedings of the IEEE/CVF international conference on computer vision, pages
2023.
[16]
Dimosthenis Karatzas, Lluis nomez-Bigordan Anguelos Nicolaou, Suman Ghosh, Andrew Bagdanov, Masakazu Iwamura, Jiri Matas, Lukas Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu, et al. ledar 2015 competition on robust reading. In 2015 13th international conference on document analysis and recognition (ICDAR), pages 1 1156-1155 1160. IEEE, 2015.
[17] Dimosthenis Karatzas, Faisal Shafait, Seiichi Uchida, Masakazu Iwamura, Lluis Gomez i Big- orda, Sergi Robles Mestre, Joan Mas, David Fernandez Mota, Jon Almazan Almazan, and Lluis Pere De Las Heras. Icdar 2013 robust reading competition. In 2013 12th international conference document analysis and recognition, pages 1484-1493, IEEE, 2013.
[18] Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International Journal of Computer Vision, 128(7):1956-999) 2020.
[19] Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242, 2016.
[20] Anh Duc Le. Recognizing handwritten mathematical expressions via paired dual loss attention network and printed mathematical expressions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshops, pages 566-567, 2020.
[21] Dong-Hymn Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896. Atlanta, 2013.
[22] Wei Liu, Chaofeng Chen, and Kwan-Yee Wong. Char-net: A character-aware neural network for distorted scene text recognition. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
[23] Canjie Luo, Lianwen Jin, and Zenghui Sun. Moran: A multi-object rectified attention network for scene text recognition. Puttern Recognition, 10:109-118, 2019.
[24] Anand Mishra, Karteek Alahari, and CV Jawahar. Scene text recognition using higher order language priors. In BMVC-British machine vision conference. BMVA, 2012.
[25] Byeonghu Na. Yoonsik Kim, and Sungrae Park. Multi-modal text recognition networks: Interactive enhancements between visual and semantic features. European Conference on Computer Vision, pages 446-463. Springer, 2022.
[26] Augustus Odena. Semi-supervised learning with generative adversarial networks. arXiv preprint arXiv:1606.00 013,, 2016
[27] Gaurav Patel, Jan P Allebach, and Qiang Qiu. Seq-ups: Sequential uncertainty-awan pseudo- label selection for semi-supervised text recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 61 6180-6190, 2023.
[28] Trung Quy Phan, Palaiahnakote Shivakumara, Shangxuan Tian, and Chew Lim Tan. Recognizing text with perspective distortion natural scenes. In Proceedings of the IEEE International Conference Computer Vision, pages 569-576, 2013.
[29] Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, and Tapani Raiko. Semi- supervised learning with ladder networks. Advances in neural information processing systems, 28, 2015.
[30] Anhar Risnumawan, Palaiahankote Shivakumara, Chee Seng Chan, and Chew Lim Tan. A robust arbitrary text detection system for natural scene images. Expert Systems with Applications, 81(188)8027-8848, 2014.
12

---

[31] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic transfor- mations and perturbations for deep semi-supervised learning. Advances in neural information processing systems, 29, 2016.
[32] Piyush Sharma, Nan Ding. Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 2556-2565, 2018.
[33] Baoguang Shi, Xiang Bai, and Cong Yao. An end-to-end trainable neural network for image- based sequence recognition and its application to scene text recognition. JEEE transactions on pattern analysis and machine intelligence, 39(1 1):2298 -3004, 2016.
[34] Baoguang Shi, Mingkun Yang. Xinggang Wang, Pengyuan Lyu, Cong Yao, and Xiang Bai. Aster: attentional scene text recognizer with flexible rectification. IEEE transactions on pattern analysis and machine intelligence, 41(9):2035 2048, 2018.
[35] Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi- operations applications, volume 1 10066 pages 369-386. SPIE, 2019.
[36] Jost Tobias Springenberg. Unsupervised and semi-supervised learning with categorical genera- tive adversarial networks. arXiv preprint arXiv:1511.06390 2015.
[37] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged con- sistency targets improve supervuper deep learning results. Advances neural information processing systems, 30, 2017.
[38] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, (11),, 2008.
[39] Andreas Veit, Tomas Matera, Lukas Neumann, Jiri Matas, and Serge Belongie. Coco-text: Dataset and benchmark for text detection and recognition in natural images. arXiv preprint arXiv:1601.077 07,1400,,0 2016.
[40] Kai Wang, Boris Babenko, and Serge Belongie. End-to-end scene text recognition. In 2011 International conference on computer vision, pages 1457-1464. IEEE, 2011.
[41] Peng Wang, Cheng Da, and Cong Yao. Multi-granularity prediction for scene text recognition. In European Conference on Computer Vision, pages 339-355. Springer, 2022.
[42] Yuxin Wang, Hongtao Xie, Shancheng Fang, Jing Wang, Shenggao Zhu, and Yongdong Zhang. From two to one: A new scene text recognizer with visual language modeling network. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14194 14203, 2021.
[43] Zixiao Wang, Hongtao Xie, Yuxin Wang, Jianjun Xu, Boqiang Zhang, and Yongdong Zhang. Symmetrical linguistic feature distillation with clip for scene text recognition. In Proceedings of the 31st ACM International Conference on Multimedia, pages 509-518, 2023.
[44] Jin- Wen Wu, Fei Yin, Yan-Ming Zhang, Yao Zhang, and Cheng-Lin Liu. Handwritten mathematical expression recognition via paired adversarial learning. International Journal of Computer Vision, 128:2386-2401, 2020.
[45] Xudong Xie, Ling Fu, Zhifei Zhang, Zhaowen Wang, and Xiang Bai. Toward understanding wordart: Corner-guided transformer for scene text recognition. In European Conference on Computer Vision, pages 303-321. Springer, 2022.
[46] Mingkun Yang, Yushuo Guan, Minghui Liao, Xin He, Kaigui Bian, Song Bai, Cong Yao, and Xiang Bai. Symmeery-constrained rectification network for scene text recognition. In Proceedings of the JEFE/CVF international conference on computer vision, pages 9147-9156. 2019.
13

---

[47] Mingkun Yang, Biao Yang, Minghui Liao, Yingying Zhu, and Xiang Bai. Sequential visual and semantic consistency for semi- -upervisue text recognition. Pattern Recognition Letters, 178:1744 178:174-180, 2024.
[48] Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. A survey deep semi-supervised learning. IEEE Transactions on Knowledge and Data Engineering, 2022.
[49] Deli Yu, Xuan Li, Chengquan Zhang, Tao Liu, Junyu Han, Jingtuo Liu, and Errui Ding. Towards accurate scene text recognition with semantic reasoning networks. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pages 12113-12122, 2020.
[50] Xiaoyu Yue, Zhanghui Kuang, Chenhao Lin, Hongbin Sun, and Wayne Zhang, Robustscanner: Dynamically enhancing positional clues for robust text recognition. In European Conference on Computer Vision, pages 135-155 151. Springer, 2020.
[51] Fangneng Zhan and Shijian Lu. Esir: End-to-end scene text recognition via iterative image rectification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2059-2068, 2019.
[52] Boqiang Zhang, Hongtao Xie, Yuxin Wang, Jianjun Xu, and Yongdong Zhang. Linguistic more: Taking a further step toward efficient and accurate scene text recognition. arXiv preprint arXiv:2305.05 140, 2023.
[53] Ying Zhang, Lionel Gueguen, Ilya Zharkov, Peter Zhang, Keith Seifert, and Ben Kadlec. Uber- text: A large-scale dataset for optical character recognition from street-level imagery. In SUNw: Scene Understanding Workshop-CVVe CVCR, volume 2017, page 2017.
[54] Caiyuan Zheng, Hui Li, Seon-Min Rhee, Seungju Han, Jae-Joon Han, and Peng Wang. Pushing the performance limit of scene text recognizer without human annotation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14116-14125, 2022.
[55] Tianlun Zheng, Zhineng Chen, Jinfeng Bai, Hongtao Xie, and Yu- Gang Jiang. Tps++: Attention- enhanced thin-plate spline for scene text recognition, arXiv preprint arXiv:2305.0533 05322, 2023.
14

---

A
Grandient of Loss Functions
For Character Unidirectional Alignment (CUA) Loss, have
exp(S; fp/T)
Lowa
log
(1I)
A(i.p)
iE!
PEP(1)
A(i,p) - exp(p) +
E
exp(-
12/7)+
E
exp(sf f2//T).
(12)
TEP((P
w'EN(1)
The derivation of the gradient for CUA loss:
AA(i.p) p)
-lexplff fp/T){;-
1
Σ exp(-fi -//T))ff)
05T
P'P((m)
(13)
+ E
explfi f2
x'EN())
dLeua
O 1//T)
Gloy( A(i,p)
Σ
0f.
P(i)
Off
0ff
PEP
1
Σ
f
Olog(A(i.p) A(i., p)) 0A(i.p)
PP()]
dA(ipp)
Of
(14)
PEr(()
M1 f + +E (EP(())) M2 f4. + EnVEN(
M3 fh
Σ
TP((
A(i.p)
PEP())
where
M1 A(i, explst 1D//);
M2 exp( - frr ffy f//TT),
(15)
M3 -exp(fe f
And the Character Contrastive (CC) Loss 45 can be expressed as
exp(Se. Sp/T)
Lec = Σ
Σ log
(16)
P'(1))
B(i)
iEl
(EP
B(i = Σ
exp(Si fp//T)
Σ
exp(Je Sw//T)
(17)
P'E((()
wEN(()
The derivation of the gradient for CC loss:
O
Σ
expls: fw/T)St +
Σ
exp(s: fw /T)ss,
(18)
0f.
('P
m'EN()
dhm
off. fn/T) Olog((B(i)
Σ
Of:
PP)
Of
0f.
pEP
Σ
fp
(B())( AB(i)
PP(ii) PEP
T
BB(i)
0f.
(19)
N fo Epeerr N₂ fo + EnvENtY No far
Σ
B(i)
(EP
15

---

where
Ni B()),
N₂ eeplfi fp/T),
(20)
N3 = -eppffe fie/T)
B Experiment
B.1
Implementation Details
In the model configuration, our model consists of a transformer-basee encoder with 12 transformer blocks and 6 attention heads and transformer-baa attention heads. All images are resized to 100 x 32, and the patch size is 8 x 4. The maximum length T is set to 25. The character set size 36, including 10 digits and 26 alphabets.
decoder with transformer block and 12
For training settings, the network is trained in an end-to-end manner without pre-training, We adopt AdamW optimizer and one-cycle 35 learning rate scheduler with a maximum learning rate of 6e-4. The batchsize is 384 for both synthetic data and real unlabeled data. We set the EMA smoothing factor a 0.999, aspect ratio thresh r 1.3, confidence threshold neer 0.5, Meaa 0.7, and temperature factor 7 = 0.1. Due to the Online Generation Strategy (OGS) and Unified Representation Forms (URF), all images are rotated 1 180 degrees with a probability of 0.5 during the training stage. Following (2). the student model takes strong augmented images for labeled and unlabeled data. The unlabeled real images fed into the teacher model are weakly augmented, where only color jitter used, including brightness, contrast, saturation, and hue. ViSu is trained on 4 NVIDIA RTX 4090 GPUs.
Table Comparison with state-of-the-ar methods on common benchmarks and Union-B. means we use publicly released checkpoints to evaluate the model. f means we reproduce the methods with the same configuration. For training data: RL 14M-L RL Combination of 11 real labeled datasets; RU Union 14M-U; RU Book32, Tex and ST-VQA. Cur, M-O, Art, Ctl, Sal, M-W, and Gen represent Curve, Oniti-Orieettd, Artistic, Contextless, Salient, Words, and General. P(M) means the model size.
Mahhod
Thrasets
TITT
SVT
ICI3
Comman Benchmirks
IC15
SVTP
CUTE
WAYG
Cur
M-O
Art
Union Union Lam---MM-nnhhtakk
Con
Sal
M.W
Gen
AVG
P(M)
80.9
RobustScunner
50
96.8
92.4
95.1
85.4
83.9
93.8
92.3
56.2
54.2
61.4
72.7
60.1
74.2
75.7
66.4
SRN [g
RL
95.5
$9.5
94.7
79.1
83.9
91.3
89.3
49.7
20.0
50.7
61.0
43.9
51.5
62.7
48.5
$1.7
AHINei
L0
95.7
97.2
94.4
69.2
Visionl
96.3
91.3
95.1
85.4
92.4
91.2
$7.2
56.7
67.6
47.3
74.2
62.5
32.8
MATRN
RL
98.2
96.9
97.9
88.2
94,1
97.9
95.0
80.5
64.7
71.1
74.8
79.4
67.6
77.9
74.6
46.2
SVTR
RT.
95.9
92.4
95.5
83.9
85.7
93.1 1
91.3
72.4
68.2
54.1
68.0
71.4
67.7
77.0
68.4
24.6
Parsey
91.7
96.5
76.2
82.5
MGPP
RL.
97.2
91.7
96.8
N7.2
95.6
94.8
94.1
75.8
74.3
67.7
68.7
75.7
600.1
30.1
72.2
141.1
LPVI +
RT.
98.77
98.6
97.9
88.7
93.9
96.2
95.4
85.2
75.9
74.8
80.5
83.3
82.2
62.8
80.7
35.1
LISTER
CLIP-OCRI
98.3
98.2
8R.8
94.3
97.6
95.5
84.6
83.1
76.3
80.0
81.3
81.8
83.9
81.6
CRNN-gc"
11
RL XU
90.2
56.1
91.5
71.8
74.1
$1.6
85.1
32.3
2.9
40.4
52.6
21.3
38.8
49.8
34,0
8.5
TRDA pr
8
RL'+
92.4
94.2
86.8
82.8
$7.9
62.9
12.5
64.7
51.2
68.1
62.0
56,5
49.9
TRBA-cr
8L4RU
5885
58.3
97.7
94.3
966.5
95.6
53.4
81.7
74.8
83.6
79.1
90.5
49.9
MAFRae-S
RI+
98.0
96.8
97.6
87.1
93.2
97.
94.5
81
71.4
72.0
82.0
78.5
82.4
$2.5
78.6
34.0
MAERec
RL+RU
98.5
97.8
98.1
89.5
96.4
98.6
95.6
85.8
83.9
80.0
83.5
84.9
87.5
85.8
85.2
135.5
Baseline ViSu
RL
98.6
98.0
96.3
89.4
3
96.5
95.7
85.8
95.8
78.1
83.4
86.3
83.6
$2.7
85.5
24.4
XL
96.3
97.6
96.1
79.4
B.2 Training with Real Labeled Data
Our method aims to boost the scene text recognition (STR) model without any human costs. However, to demonstrate the effectiveness our method when training with real labeled data, we replace the synthetic labeled data with Union14M- [15), which consists of 3.2M refined images from 14 publicly available datasets. For the convenience comparison and following [5] IC13-1015 and IC15-2077 evaluate all the STR methods.
we employ
As shown Table. 5 and Table. 6 our method achieves higher accuracy than all fully supervised methods. Compared to the ttate-of-the-art pretraining method MAERee [15), with approximately 1/5
16

---

Table 6: Comparison with state-of-the-ar methods on several challenging benchmarks. All symbols have the same meaning in Table.
Method
Datasets
WordArt
ArT
COCO
Uber
ParSeqi
f
RL
83.1
83.1
77.8
87.4
MGP
RL
80.5
79.6
74.1
80.7
LPVf [52
RL
82.5
82.4
76.8
81.8
LISTER
[5
RL
74.8
78.3
72.7
81.3
CLIP-OCRI
43
RL
83.5
82.7
78.3
86.3
CRNN-pr*
RL'+RU'
53.4
58.6
56.8
45.2
TRBA-pr*
RL
64.3
66.8
67.1
54.2
TRBA-cri
154
RL+RU
82.7
81.9
77.6
77.7
ViSu
RL+RU
85.9
84.9
80.4
91.6
Table 7: Recognition efficiency of different methods. WAVG means weighted average accuracy on six common benchmarks. AVG means average accuracy on Union-B. All methods are trained with real data.
Methods
(msii())/img)
Params(M)
WAVG
AVG
ABINet
10
17.3
36.7
93.9
69.2
ParSeq
12.0
23.8
95.3
82.3
MGP 41
9.0
141.1
94.1
72.2
LPV [22
20.0
35.1
95.4
80.7
LISTER
24.9
49.9
94.1
69.0
CLIP-OCR
43
20.2
31.1
95.5
81.6
TRBA-cr [54]
15.3
49.9
95.6
80.5
MAERec-S [5
282.2
34.0
94.5
78.6
MAERec-B 11
308.1
135.5
95.6
85.2
ViSu
17.7
24.4
96.0
86.9
of the parameters, our method outperforms it by 0.4% on common benchmarks and by 1.7% on Union- [5]. Furthermore, for fair comparison, we reproduce TRBA-cr [54] using the same datasets as ours. The experimental results show that ViSu performs better on all benchmarks, especially on challenging benchmarks (86.9% vs 80.5% on Union-B [15,, 65.9% vs 82.7% on WordArt [45), 84.9% vs 81.9% ArT [6], 80.4% 77.6% on COCO 33 and 91.6% 77.7% on Uber [53
We attribute this superior performance the increased diversity in character morphology from Online Generation Strategy and the clustering of similar character features facilitated by the CUA
Loss.
B.3
Efficiency
Table. 7 presents the efficiency metrics. Speed is calculated by averaging the inference time over 4,000 images. All experiments are conducted on an NVIDIA RTX 4090 GPU with a batch size The reported methods are trained using real data. In comparison to MAERec-B [15), our method achieves higher accuracy (86.9% vs 85.2%) with a significantly lighter model, i.e., approximately 1/5 parameter size and 1/17 inference time. Compared to models of similar parameter sizes and inference times, ViSu outperforms them by at least 4.6%, demonstrating the effectiveness of our approach.
B.4 Diseuss about configuration settings for OGS
Table, showw the performance with different configuration of OGS. The first rows indicate that applying independent random font and unified random character orientation to each character in
17

---

Table 8: Performance on Union-B with different configuration settings for OGS. The first row represents the baseline model without OGS. Character-level means that all characters sample have independent random font or orientation. Instance-level means that means that all characters in sample have a unified font or orientation, but different samples are independent.
random
random orientation
background
color
Cur
Con
M-W
Gen
AVG
font
68.2
84.5
62.0
58.5
75.8
62.5
69.7
68.8
character-level
lnstance-level
71.6
85.8
66.8
57.6
80.3
62.6
71.2
70.9
instance-level
instance-level
56.7
63.4
70.7
character-level
character-level
71.1
85.1
66.7
56.4
80.8
63.1
70.9
70.6
instance-lever character-level
70.3
86.4
65.4
56,9
79.9
62.5
70.4
70.3
character-level instance-level
70.3
84.3
64.9
57.5
78.7
61.7
68.8
69.5
character-level
instance-level
71.2
85.3
66.3
58.3
79.4
63.2
70.9
70.7
a sample leads to better performance. This is because random fonts have more diversity in character glyphs, making the model robust to character morphology. All characters in real scene text images tend to have the same orientation, thus adopting a uniform orientation in one sample is a better solution. Moreover, adding random background and random color to the text both bring slight performance degradation. Because OGS samples act as base images in the teacher model to guide the character feature alignment in the student model, they are expected to have noise-free features to enhance the robustness of the model to character morphology. Therefore, introducing extra backgrounds and colors redundant. In general, changing the configuration settings of OGS brings some slight performance fluctuations, but all are better than not using OGS, which proves the effectiveness of our approach.
B.5 The influence of hyper-parameters in CUA Loss
We conduct several experiments to demonstrate the influence of X, Meaa. and т. is the weight of CUA Loss. When À is set O, indicating that the model is trained without CUA Loss, the accuracy can only reach 68.2%. A small À implies minimal clustering of characters into the same class and makes the model focus the characters involved in the CUA Loss. low threshold introduces many uncertain characters, harm the alignment. Conversely, high threshold reduces the number
the basic recognition loss. The confidence threshold YJcua determines
where incorrect predictions of characters participating in the alignment and clustering, similar to small X. The temperature factor adjusts the smoothness character features. A small focuses more challenging characters but sharpens the feature distribution. As shown in Table. [ changes in these parameters do not significantly affect model performance. To achieve relatively better performance, we set 0.1,
Neua 0.7, and = 1 as our default configuration.
B.6 Qualitative Analysis
Fig. 4a) displays several challenging examples. The baseline correctly recognize simple images with multiple directions, benefiting from URF, However, it fails to recognize complex characters due lack of adaptation character morphology. The supervised method ParSeq and semi-supervised method TRBA-cr have difficulty recognizing these challenging text images. Fig. 4T)) and (c) show the distribution of character features from t-SNE [38]. Compared with CC Loss, CUA Loss better clusters characters in the same class and makes character features in different classes more distinguishable.
B.7 Limitations
Our method has difficulty recognizing scene text images with extreme aspect ratios. As shown in 5 when the text in a scene image is very long, it usually has a large aspect ratio. However, all images resized changes in the aspect ratio of long text images. The information loss caused by resizing the images impairs the extraction of character features. This is a common problem in scene text recognition task and can be alleviated by splitting long texts into multiple segments for recognition and then concatenating them together.
fixed size before being fed into the text recognizer, which results in drastic
18

---

Table 9: Performance on Union-B with different hyper-parameters in CUA Loss. The model is trained on synthetic datasets and Union14-U.
Hyperparameters
AVG
A 0
68.2
À 0.01, ecaa 0.7, 0.1
69.0
À = 0.1, Mua
0.7, = 0.1
70.9
À 1, Mou
0.7, 0.1
68.6
À 0.1, Meua
0.5, 0.1
70.5
À 0.1, Mea
0.9, 0.1
70.6
À 0.1, 1, neaa
0.7, 0.05
70.6
0.1, Nuaa
0.7, 0.2
70.3
Mecca
mecca mecca 2229 date
ARGEN
ARGENTIC
argentina argentina apr
CENIUS
genius
dots
genius
dots
sunctor
5200
the
sender
ey
2100
tfboys
panda -ndd pares ponds
christm
cola
lakers
TF
tifboys
h
-h-isty
gola
LAKERS
jakers
tifboys
-h-istm
gold
jakers
teboys
-h-istm
gola
takers
(a)
(b)
(c)
Figure 4: (a) shows several challenging examples. The four lines from top to bottom represent the recognition results from ViSu, Baseline, ParSeq. and TRBA-cr. The first shows examples with multiple directions. The second row displays examples with artistic or distorted characters. (b) and (c) are the visualizations of character features for CC Loss and CUA Loss, respectively.
HUANGDONG MANTOR BANK
LANNEIL
tiantian Art Space
MANSENG SAUVIGNON
semi skimme semiskimmed semiskimme
guangdongnanyueban guangdongnanyuzbank.
qinguuluanrenlaji ngwulve renglas
tlantianartspace tiantianartspact
mansengsauvignon haanseggsauvignon
Figure 5: Failure cases. The first line is the ground-truth, and the second line is the recognition results.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in Appendix B.7
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
20

---

Justification:
The full set of assumptions and a complete proof for the theoretical result is shown in Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The information needed to reproduce the main experimental results is shown in Sec. and Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
21

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will release the code after the paper is accepted, which is also claimed in
Abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
Policy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.cc/publi
SroodeSubiissiii ssiiiissiiiiiyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specify all the training and test details, which can be found in Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification:
The paper does not include error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
22

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provide sufficient information on the computer resources Ap- pendix B.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttp:://neurip::: http://pp.........p..cccccccccea
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The paper does not involve societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
23

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The paper cite all the original papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
24

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code and its corresponding documentation will be released together after the paper is accepted.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
25


Reviews:
[{"review_id": "JkJxvFjbJP", "reviewer": "Reviewer_9AqC", "summary": "This paper addresses the problem of insufficient exploration of character morphology in scene text recognition and proposes a new framework comprising an Online Generation Strategy (OGS) and Character Unidirectional Alignment (CUA) Loss to enable the model to learn from unlabeled real data. OGS mitigates the issue of data scarcity, while CUA aids in clustering characters. Comprehensive experiments demonstrate the effectiveness of the proposed method.", "questions": "The proposed method struggles with handling images with extreme aspect ratios. The authors attempt to address this by splitting the image, recognizing each part, and then concatenating them. However, this splitting operation may interfere with character recognition and degrade the overall recognition quality.", "limitations": "N.A", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The proposed method effectively addresses the issue of excessive representation methods for scene images through Unified Representation Forms.\n2. The errors in the previous Character Contrastive Loss method are corrected.\n3. The idea of utilizing character morphology is novel.", "weaknesses": "1. The writing quality needs improvement.\n2. There is a lack of detailed explanation regarding the Online Generation Strategy (OGS) and Unified Representation Forms (URF).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive feedback provided. Below, we address the identified weaknesses and questions.\n\n> Q1: The writing quality needs improvement.\n\nA1: We appreciate your observation regarding the writing quality. We will refine and enhance the clarity of the writing.\n\n> Q2: There is a lack of detailed explanation regarding the Online Generation Strategy (OGS) and Unified Representation Forms (URF).\n\nA2: Thanks for highlighting this issue. We will revise these sections in the subsequent version.\n\nUnified Representation Forms: Different from typical visual objects, text images are characterized by serialized information, necessitating a specific reading order dictated by linguistic rules. Moreover, scene text images frequently exhibit rotation, resulting in varied character orientations. As illustrated in Fig. 2, the inconsistent representation forms caused by multiple combinations of reading orders and character orientations undoubtedly increase the difficulty of network convergence. Besides, because scene text images with vertical reading orders often have small aspect ratios, leading to significant deformation of visual information when resized, rendering the characters unrecognizable. To accommodate vertical and horizontal reading orders, we intuitively rotate them clockwise to unify the reading order to horizontal, which is referred to as Unified Representation Forms (URF). Specifically, an image with a height-to-width ratio exceeding a certain threshold will be rotated, otherwise it remains unchanged. It is easy to notice that the existing four representations can be further distilled into two primary forms with left-right reading order and their counterparts rotated by 180 degrees. This simplification and unification alleviate the issue of inconsistent representation forms and lay the foundation for the following Online Generation Strategy (OGS) process. \n\nOnline Generation Strategy: The problem of diverse character morphology primarily stems from the simplistic and uniform training samples in synthetic datasets. Despite the abundance of samples in synthetic datasets, the homogeneity in character styles restricts the model's ability to acquire useful information for recognizing challenging texts in real scenarios. To address this problem, we propose to generate text samples in two primary forms and their 180-degree rotations based on their labels. As depicted in Fig. 2, these samples are background-free but with diverse character styles. Following the characteristics of real samples, all character orientations within a generated sample are consistent. Random selection of font, position, and character spacing enhances the diversity of character morphologies. Both synthetic training images and their corresponding online-generated samples are concurrently fed into our semi-supervised framework. By eliminating background noise and enriching the diversity of character styles, these samples encourage the model to concentrate on character morphology and generalize the ability to recognize complex texts from simple synthetic training data.\n\n> Q3: The proposed method struggles with handling images with extreme aspect ratios. The authors attempt to address this by splitting the image, recognizing each part, and then concatenating them. However, this splitting operation may interfere with character recognition and degrade the overall recognition quality.\n\nA3: (1). In practical application scenarios, text recognizers are typically attached to text detectors, working collaboratively to complete the OCR process. Therefore, employing a word-level text detector instead of a line-level detector can mitigate this issue.\n\n(2). When encountering long texts with extreme aspect ratios, resizing them to a preset aspect ratio leads to a loss of contextual information, which is a common problem among text recognizers. Although slicing long texts for recognition may indeed cause some degradation, but it still yields better results than attempting to recognize the entire text in one pass, which is a limitation of the current model. Future specialized improvements and optimizations can be developed to enhance the recognition of long texts.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ddszMR7kYv", "reviewer": "Reviewer_Smtd", "summary": "Existing scene text recognition (STR) methods struggle to recognize challenging texts, which originates from the insufficient exploration of character morphologies. To address the issues, the paper proposes to facilitate the contrastive learning-based STR framework in a self-motivated manner by leveraging synthetic and real unlabeled data without any human cost. An Online Generation Strategy is proposed to enrich the diversity of characters in training data. Besides, a new Character Unidirectional Alignment Loss is proposed for aligning the character features in the student model with the reference features in the teacher model. Extensive experiment results show the effectiveness of the proposed method.", "questions": "Please refer to the weakness part.", "limitations": "Yes. The limiation is well descirbed in the paper.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1) The paper proposes to improve the recognition for challenging texts, especially for artistic and severely distorted characters within a mean-teacher framework without the requirement for real labeled data.\n2) An Online Generation Strategy is proposed to enrich the diversity of characters in training data without need of psuedo labeling real data explicitly. \n3) A Character Unidirectional Alignment Loss is proposed to improve the existing Character Contrastive (CC) Los [45] in character representation learning. \n4) Extensive experiments on Common Benchmarks, Union14M-Benchmark, and other challenging benchmarks demonstrate the effectiveness of the proposed method.", "weaknesses": "1) The overall framework follows a mean teacher framework, which is not new for semi-supervised scene text recognition.\n2) In Table 1, the baseline model without RU data has achieved a high performance than existing SOTA. Is there any special design in the baseline model?\n3) Why not simply remove the second item in equation, i.e., exclude the positives in the denominator?\n4) Absense of comparison between the proposed method and other methods in Sec. 2.3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's constructive feedback and address each concern below.\n\n> Q1: The overall framework follows a mean teacher framework, which is not new for semi-supervised scene text recognition.\n\nA1: While certain methods, such as Zheng et al.[54], also employ the mean teacher framework for semi-supervised OCR, they do not take the character morphologies into consideration, making them suboptimal. We adopt the mean teacher framework to incorporate real unlabeled data, with our primary contribution being the proposal of a novel training paradigm through viewing and summarizing. Overall, we make two significant improvements to the mean teacher framework:\n\n(1). The conventional mean teacher framework usually only utilizes information from unlabeled data without additional exploration of labeled data. However, our method proposes the Online Generation Strategy (OGS) to further exploit labeled data. With the equipment of OGS, the viewing process enriches the diversity of character morphologies, facilitating the model to generalize the ability to recognize challenging samples using only simple synthetic training data.\n\n(2). The ordinary mean teacher framework aligns the features of two perspectives of one unlabeled sample without optimization for text-specific characteristics. Our method enhances the robustness of the model to character morphology during the summarizing process. On the one hand, Unified Representation Forms (URF) is used to unify text image reading order and character orientation. On the other hand, Character Unidirectional Alignment (CUA) loss is adopted to theoretically correct the previous formula error and obtain unified features for each character category. \n\nBenefiting from the above exploration of the monotonousness of training data and sensitivity of the model to character morphologies, our method achieves SOTA performance on all benchmarks, with particularly notable superiority on the challenging Union14M-Benchmark.\n\n> Q2: In Table 1, the baseline model without RU data has achieved a high performance than existing SOTA. Is there any special design in the baseline model?\n\nA2: Our baseline model incorporates URF. As described in lines 269-270, the same applies to CRNN and TRBA baselines. As shown in Table 1, our baseline achieves higher accuracy only on the Multi-Oriented (M-O) of Union14M-Benchmark. This is because URF is particularly effective at handling text with vertical and reverse reading orders, which are prevalent in M-O.  In the following, we further present the performance of the baseline model without URF. The experiments indicate that our ViSu equipped with URF significantly excels in recognizing challenging texts. We will also release the code.\n\n| Method | Datasets | IIIT | SVT | IC13 | IC15 | SVTP | CUTE | WAVG | Cur | M-O | Art | Con | Sal | M-W | Gen | AVG\n|  ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  \n| Baseline without URF | SL | 96.1 | 94.0 | 95.9 | 85.8 | 87.8 | 89.9 | 92.3 | 58.0 | 17.0 | 52.9 | 55.4 | 64.9 | 54.8 | 61.3 | 52.0 \n| Baseline | SL | 96.1 | 94.4 | 96.5 | 86.0 | 88.2 | 88.5 | 92.5 | 60.5 | 82.7 | 53.6 | 54.0 | 70.6 | 54.5 | 62.0 | 62.6 \n| ViSu | SL+RU | **97.6** | **96.1** | **98.3** | **89.3** | **91.3** | **92.4** | **94.7** | **71.6** | **85.8** | **66.8** | **57.6** | **80.3** | **62.6** | **71.2** | **70.9** \n\n> Q3: Why not simply remove the second item in equation, i.e., exclude the positives in the denominator?\n\nA3: The second term in the denominator represents the feature similarity between the character and all identical characters in the mini batch except for itself. When optimizing CUA loss, the feature distances of all characters within the same category are minimized, i.e., clustering the same characters. As illustrated in Fig. 1(b), the same character can have very different representation forms. This optimization process plays a crucial role in extracting the visual feature commonalities, which can help the network enhance its robustness to character morphologies. Experimental results supporting this are presented below.\n\n| Method | Cur | M-O | Art | Con | Sal | M-W | Gen | AVG\n|  ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----\n| without CUA Loss | 69.9 | 83.9 | 64.1 | 54.9 | 76.0 | 60.8 | 67.6 | 68.2\n| CUA Loss without second term | 71.1 | 85.0 | 65.3 | **57.9** | 79.5 | 62.4 | 70.6 | 70.3\n| CUA Loss | **71.6** | **85.8** | **66.8** | 57.6 | **80.3** | **62.6** | **71.2** | **70.9**\n\n> Q4: Absense of comparison between the proposed method and other methods in Sec. 2.3.\n\nA4: In Sec. 2.3, we reference Baek et al.[1], ABINet[10], Zheng et al.[54], Gao et al.[11], and Seq-UPS[27]. We compare ViSu with [1][10][54] in Sec. 4. However, [11] and [27] did not release their code or checkpoints, preventing detailed comparisons on metrics such as performance on the Union14M Benchmark, parameters, and speed. In the following, we compare the metrics reported in their papers with our method. The experimental results indicate that our ViSu significantly outperforms these methods on several common benchmarks.\n\n| Method | IIIT | SVT | IC13 | IC15 | SVTP | CUTE \n|  ----  | ----  | ----  | ----  | ----  | ----  | ----\n| Gao et al.[11] | 74.8 | 78.1 | 81.2 | 54.7 | - | -\n| Gao et al.[11] (ensemble) | 76.8 | 80.8 | 84.5 | 57.6 | - | -\n| Seq-UPS[27] | 92.7 | 88.6 | 92.2 | 76.9 | 78.8 | 84.4 \n| Seq-UPS[27] w/ SeqCLR (All-to-instance) | 92.3 | 87.2 | 91.8 | 77.9 | 78.9 | 85.4\n| Seq-UPS[27] w/ SeqCLR (Frame-to-instance) | 92.8 | 86.7 | 92.6 | 77.4 | 79.2 | 86.1\n| Seq-UPS[27] w/ SeqCLR (Window-to-instance) | 93.1 | 86.7 | 91.7 | 76.8 | 81.4 | 85.9\n| ViSu | **97.6** | **96.1** | **98.3** | **89.3** | **91.3** | **92.4**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uUDNs5nFuO", "reviewer": "Reviewer_S6t1", "summary": "This paper focuses on the character morphologies and proposes to boost the scene text recognizer through viewing and summarizing paradigm. In the viewing process, the Mean Teacher framework is used to train with unlabeled data. In the summarizing process, the proposed method theoretically proves the mistakes in the previous method and proposes a new loss function.\n\nI have read the author responses and the comments of other reviewers, I would recommend the 'accept' score.", "questions": "See Weaknesses.", "limitations": "The authors adequately addressed the limitations and potential negative societal impact", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method achieves good results, especially on challenging benchmarks.\n2. The paper is well written. It clearly points out the problem in the previous method and provides a detailed explanation. The proposed OGS and Mean Teacher framework effectively increase the diversity of character glyphs. The CUA loss corrects the previous error and better achieves clustering of identical characters.\n3. There are lots of experiments to prove the effectiveness of the proposed method.", "weaknesses": "1. There is a lack of clear description in the paper on how CRNN-ViSu and TRBA-ViSu in Table 1 are set up and trained.\n2. Although the proposed method achieves high average accuracy on common benchmark and Union14B-Benchmark, it is not SOTA on certain benchmarks, such as SVT, SVTP, CUTE, Curve, Contextless, MultiWords.\n3. When the proposed method recognizes text with vertical reading order, how to decide whether to rotate it 90 degrees clockwise or counterclockwise.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful and constructive review. We hope these responses will address your concerns appropriately.\n\n> Q1: There is a lack of clear description in the paper on how CRNN-ViSu and TRBA-ViSu in Table 1 are set up and trained.\n\nA1: For a fair comparison, CRNN-ViSu and TRBA-ViSu use the same training data as ViSu, namely MJSynth, SynthText, and Union14M-U. As detailed in Appendix B.1, they both incorporate OGS, CUA loss, URF, and a 0.5 probability of rotation by 180 degrees. The batchsize is set to 384 for both synthetic data and real unlabeled data.\n\n> Q2: Although the proposed method achieves high average accuracy on common benchmark and Union14B-Benchmark, it is not SOTA on certain benchmarks, such as SVT, SVTP, CUTE, Curve, Contextless, MultiWords.\n\nA2: As shown in Table. 1, the SOTA method for SVT, SVTP and Curve is TRBA-cr*[54], which uses different traning data. When we reimplement TRBA-cr$\\dagger$ with the same training data, our ViSu surpasses it by 1.5%, 1.2%, and 1.1% on SVT, SVTP, and Curve, respectively.  A similar situation occurs with the SOTA method TRBA-pr[1] for Contextless and Multi-Words, which uses real labeled and unlabeled data for training. For the CUTE benchmark, with its small sample size of only 288, our method is 1.1% (3 samples) lower than MATRN [25]. Small sample sizes can easily lead to error fluctuations and instability. However, MATRN is 0.7% to 2.7% lower than our method on other common benchmarks and Union14M-Benchmark, demonstrating the stability of our method.\n\n> Q3: When the proposed method recognizes text with vertical reading order, how to decide whether to rotate it 90 degrees clockwise or counterclockwise.\n\nA3: For text images with a vertical reading order, the model will recognize the results after rotating the image by both 90 and 270 degrees and will select the one with the higher confidence as the final recognition result.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gfhpLzKnWz", "reviewer": "Reviewer_qi8n", "summary": "This paper proposes a ViSu framework to enhance text recognition by viewing and summarizing. For the viewing process, the authors generate various text images to lead the model to focus on different text styles. For the summarizing process, they analyze the drawback in existing character consistency loss and propose a CUA loss to cluster the character under different styles. As a result, the proposed framework outperforms other existing methods on multiple datasets.", "questions": "1. As the semi-supervised framework can use large-scale unlabeled real data for training, why does ViSu require OGS to generate synthetic data?\n2. To verify the fairness, do other semi-supervised methods in Tab. 1 and 5 have the same implementation details as ViSu?", "limitations": "Please refer to Weaknesses.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The design of OGS can obtain promising improvements without additional human cost.\n2. The authors point out that previous consistency loss mistakenly regards some positive samples as negative samples, which is an important theoretical drawback in existing metric learning methods. And the proposed CUA loss provides a solution for this issue.\n3. The proposed method gets SOTA performance on both common and challenging datasets.\n4. The proposed framework can be applied on multiple text recognition models seamlessly.", "weaknesses": "1. To further verify the effectiveness of OGS. Authors should compare the performance of replacing OGS with normal synthetic data. \n2. Tab.5 lacks the comparison with the baseline model. \n3. In Tab.5, it seems that MAERec-B performs better on the Union-14m dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks for the feedback. In the following we address the weaknesses and questions pointed out by the reviewer.\n\n> Q1: To further verify the effectiveness of OGS. Authors should compare the performance of replacing OGS with normal synthetic data.\n\nA1: Replacing OGS with normal synthetic data means adding a background to the synthetic samples, incorporating color to the text, and using the same fonts and orientations for the characters within each sample. We have validated these elements separately in Appendix. B.4. Real scene text images often have characters with consistent orientations, thus generating characters with the same orientation in one sample is advantageous. Because OGS samples serve as base images and aim to enrich character morphologies without noises, introducing extra backgrounds and colors is redundant. However, applying randomized fonts to each character is shown to be effective for enhancing the robustness of the model.\n\n> Q2: Tab.5 lacks the comparison with the baseline model.\n\nA2: Thanks for pointing this out. Our method aims to elevate the performance upper bound of model recognition without incurring additional human costs, relying on simple synthetic labeled data and real unlabeled data. Table. 5 is to demonstrate that our method still performs well when introducing manual annotations, so there is no obvious definition of baseline. We show the results of training with only real labeled data below. From the experimental metrics, we can see that our ViSu can still yield further improvement with the introduction of manual annotations.\n\n| Method | Datasets | IIIT | SVT | IC13 | IC15 | SVTP | CUTE | WAVG | Cur | M-O | Art | Con | Sal | M-W | Gen | AVG\n|  ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  | ----  \n| Baseline | RL | **98.6** | 98.0 | **98.3** | 89.4 | 94.3 | 96.5 | 95.7 | 88.8 | 95.8 | 78.1 | 83.4 | 86.3 | 83.6 | 82.7 | 85.5\n| ViSu | RL+RU | 98.5 | **98.3** | 97.8 | **90.4** | **96.3** | **97.6** | **96.0** | **90.7** | **96.1** | **79.4** | **85.4** | **87.1** | **86.3** | **83.1** | **86.9**\n\n\n> Q3: In Tab.5, it seems that MAERec-B performs better on the Union-14m dataset.\n\nA3: MAERec-B performs better accuracy than ours on the some benchmarks of Union14M-Benchmark. However, our method achieves 1.7% higher average accuracy. It is worth noting that, as shown in Table. 7, MAERec-B has approximately 5 times the number of model parameters and about 17 times the inference time. Our ability to achieve higher accuracy with a much lighter model underscores the effectiveness of our method. Compared to MAERec-S, which has 1.4 times the number of parameters, the inference time is about 16 times longer than ours, and the accuracy is significantly lower than ours (78.6% vs 86.9%).\n\n> Q4: As the semi-supervised framework can use large-scale unlabeled real data for training, why does ViSu require OGS to generate synthetic data?\n\nA4: Both real unlabeled data and labeled data are essential for a semi-supervised framework. Semi-supervised scene text recognizers require annotated data to provide the model with basic character recognition capabilities. Our OGS is designed to enrich the character morphology and maximize the potential of simple data. Therefore, using both real unlabeled data and OGS samples simultaneously is complementary, as both aim to improve the performance from different perspectives.\n\n> Q5: To verify the fairness, do other semi-supervised methods in Tab. 1 and 5 have the same implementation details as ViSu?\n\nA5: The methods marked with * in Table. 1 and Table. 5 mean that we evaluate the officially released checkpoints. The absence of any mark means the metrics reported in the paper. The above methods follow the same evaluation metrics as ours. For the methods marked with $\\dagger$, such as TRBA-cr, we reimplement it with the same training data, but the training configuration follows the official. CRNN-ViSu and TRBA-ViSu use the same training data as ViSu. As detailed in Appendix B.1, they both adopt OGS, CUA loss, URF, and a 0.5 probability of rotation by 180 degrees. The batchsize is set to 384 for both synthetic data and real unlabeled data.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: After discussion, all the reviewers provided positive feedback on this submission, therefore we recommend accepting it.
Decision: Accept (poster)