GLOMA: Global Video Text Spotting with Morphological Association
Authors: Han Wang
Yanjie Wang
Yang Li
Can Huang
Venue: ICLR 2025 Poster (2025)
Keywords: Text Spotting
Text Detection
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Video Text Spotting (VTS) is a fundamental visual task that aims to predict the trajectories and content of texts in a video. Previous works usually conduct local associations and apply IoU-based distance and complex post-processing procedures to boost performance, ignoring the abundant temporal information and the morphological characteristics in VTS. In this paper, we propose \model{} to model the tracking problem as global associations and utilize the Gaussian Wasserstein distance to guide the morphological correlation between frames. Our main contributions can be summarized as three folds. 1). We propose a Transformer-based global tracking method \model{} for VTS and associate multiple frames simultaneously. 2). We introduce a Wasserstein distance-based method to conduct positional associations between frames. 3). We conduct extensive experiments on public datasets. On the ICDAR2015 video dataset, \model{} achieves \textbf{56.0} MOTA with \textbf{4.6} absolute improvement compared with the previous SOTA method and outperforms the previous Transformer-based method by a significant \textbf{8.3} MOTA.

Published as a conference paper at ICLR 2025
GLOMA: GLOBAL VIDEO TEXT SPOTTING WITH MORPHOLOGICAL ASSOCIATION
Han Wang Bytedance
Yanjie Wang Bytedance
Yang Li Bytedance
Can Huang Bytedance
ABSTRACT
Video Text Spotting (VTS) is fundamental visual task that aims to predict the tra- jectories and content of texts in a video. Previous works usually conduct local as- sociations and apply loU-based distance and complex post-processing procedures to boost performance, igmoring the abundant temporal information and the mor- phological characteristics in VTS. In this paper, propose GLOMA to model the tracking problem as global associations and utilize the Gaussian Wasserstein distance guide the morphological correlation between frames. Our main con- tributions can be summarized as three folds. 1). We propose a Transformer-baan based global tracking method GLOMA for VTS and associate multiple frames simulta- neously. 2). We introduce a Wasserstein distance- sitional associations between frames. 3). We conduct extensive experiments on public datasets. On the ICDAR2015 video dataset, GLOMA achieves 56.0 MOTA with 4.6 absolute improvement compared with the previous SOTA method and outperforms the previous Transformer-bent method by significant 8.3 MOTA.
method to conduct po-
INTRODUCTION
Video Text Spotting is essential topic in computer vision, which facilitates video understanding, video retrieval, and video captioning. By simultaneously carrying out detection, recognition, and tracking, VTS can locate and recognize the texts in each frame and build trajectories through time.
In Fig. [1 the expansion of the Multi-Object Tracking (MOT) framework, as seen in studies like
Wu et al. 2022b); Koo & Kim (2013); Tian al.] (2016); Cheng et al.] (2020); Gao al. (2021);
Feng et al. (2021); Yu et al. (2021). involves a bilateral matching approach across two consecutive
frames. These studies leverage appearance and positional relations, with notable emphasis on the IoU score for text matching and trajectory construction. In case of trajectory interruptions, they retain information solely from the final frame where the text is detected, in line with MOT practices. We note that text characteristics in videos differ from those of pedestrians, cars, other common MOT objects. Text transformations evolve more slowly than those in MOT, with texts experiencing minimal deformation from limb or pose changes, suggesting more gradual visual progression. This stability over time often results in more consistent text appearances, enabling the use of global features to counter issues like blur. Additionally, text frequently undergoes swift translations due to camera movements, leading to weaker positional associations compared to MOT objects. Nonetheless, text shapes usually remain more stable than MOT objects, given the static nature of texts themselves. These observations suggest the importance of relying on global information rather than solely depending on features from individual frames, and highlight the need to consider morphological details over positional relationships. Thus, how to explicitly utilize temporal information and properly conduct morphological correlation text scenarios remains question.
In this paper, we propose novel model GLOMA with global associations explicitly use temporal information and a shape-aware distance to measure morphological similarity. We modify the detector YOLOX Ge et al. (2021) to detect texts as polygons in each frame. The tracking embeddings are extracted by Rotated RolAlign Liu et al. (2018) and supervised by recognition loss to obtain semantic awareness. To utilize temporal information, a global embedding pool is maintained during the whole inference process to hold the historical tracking embeddings and trajectory information. Then a Transformer-bar based architecture is proposed to access long- range temporal associations by conducting associations between texts in the current frame and texts in

---

Published as a conference paper at ICLR 2025
the global embedding pool for each frame. We also introduce Wasserstein distance-basee based Yang et al. (2021) method as the positional measurement, which takes both location and morphology into account.
To prove the effectiveness of the proposed method, we conduct extensive experiments on several datasets and achieve state-of-the-art performance. On ICDAR2015 video Karatzas et al. (2015)
dataset, our GLOMA obtains 56.0 MOTA on the test split, with 4.6 absolute improvement compared with the previous SOTA method Wu al. (2022b), and outperforms the previous Transformer-based method Wu et al. (2022a) by 8.3 MOTA. On the ICDAR2013 Karatzas et al. (2013) video and Minetto Minetto et al. (201T) datasets, our GLOMA also reaches leading performance. Our GLOMA can run at around 20 FPS and the global association procedure takes 3.6 ms per frame on a single Tesla V100 GPU.
RELATED WORK
2.1 SCENE TEXT DETECTION
Different from object detection, Scene Text Detection aims to detect arbitrarily shaped texts in images. Benefit- ing from the development of object detection, Tian et al. (016):: Zhong et al.] succeed in horizontal text detection by adopting similar methods. Based on FCN Long et al. (2015). EAST /Zoou et al. (2017) is proposed to detect texts with different angles. PSENet Wang et al. (2019a) and PAN/PAN+ Wang et al. (2019b) 2021) adopt kernel- based methods and apply post-processing procedures produce the final detecting results.
Sae Nakkon
Previous methork.
FE-R
Re-2
RY. Peritt d0
Caar Maodhe)
2.2 MULTI-OBJECT TRACKING
Fe-1 R:
e Global Umbedding Pool Frrbwding
Multi-Object Trackin Bewley et al. (2016); Wojke et al. (2017; Zhou al. (020):: Zhang et al. (2021): Wang et al. (2020): Zhou et al. (222b)) ams to predict the coor- dinates of each object each frame. Most existing meth-
b)
Figure 1: Motivation. Previous works usually conduct local associations and easily fail in scenes with interference (e.g., identical texts). To solve the prob- lems, we introduce global associations utilize temporal information make our method more robust towards such scenes.
ods Bewley et al. (2016) 6) Wojke et al. (2017) 7); Zhou et al. (020); Zhang et al. (202 ); Wang et al. (2020) model the tracking task as bilateral matching problem between in- stances in two adjacent frames. Wojke et al. (2017) adopts separate detection and tracking networks and tracking- by-detection pipeline, with an loU-based positional dis- tance and cascaded matching procedures. To simplify the pipeline, Wang et al. (2020); Zhang et al. (202)) introduce siint-deeection-ad--reeee
protocol, which combines both detection and tracking in single network. How- ever, they highly rely on complex post-processing procedures and require many handcrafted hyper- parameters. Recently, some works Sun et al. (2020): ZZng et al. (202T) model tracking as query problem, treating different trajectories as different queries and decoding the corresponding coor- dinates with a Transformer-based architecture. Though more precise pipeline, these approaches usually fail in crowded scenes due to the absence of explicit positional awareness.
2.3 VIDEO OBJECT DETECTION
Video Object Detection (VOD) aims to boost the detection performance by aggregating context features. Attention blocks are widely used in Chen et al. (2020); Wu et al. (2019); Deng et al. } Shhvets et al. (2019); Zhou et al. 2022a): Wang et al. (2022); ): Zhu et al. (2017 7 to conduct correlations between reference images and the current image, achieving awareness of long- range
temporal information. Based on two-stage detector, CCen et al. (2020): Wu et al. (2019): ;; Deng
et al. (2019); Shvets et al. (2019) Zhu et al. (017) aggregate features after Rols Girshick 2015
2

---

Published as a conference paper at ICLR 2025
Extectio
di Host
Donectoon
Distano
W
Dotectio Danetoo
Comptaly
Trounist
Decader
Comptot
Tokd
Embedding
Matrix product
Globol Emberlding Pool
Figure 2: Overview. A global embedding pool is maintained to store historical tracking embeddings and trajectory information and is updated after each frame. With a shallow Transformer layer, we conduct associations between embeddings of the current frame and embeddings the global embed- ding pool to obtain the global association score. Furthermore, Wasserstein distance-based method applied to measure the positional similarity between texts in frames. Some detailed architectures are ignored for clarity.
He et al. (017)) to gain enhanced features. In particular, Chen et al. (2020) designs hierarchical structure to aggregate local and global features. Based on Transformer, Wang et al. (2022): Zhou ct al. 2022a) aggregate queries through time, also achieving temporal awareness.
2.4 VIDEO TEXT TRACKING AND VIDEO TEXT SPOTTING
Given a video clip. Video Text Tracking (VTT) aims to predict the coordinates of each text each frame and Video Text Spotting further requires recognition results. Existing methods et al. (202T); Wu al. (2022b a 2021); Cheng et al. (2020) Feng et al. (202I) succeed in most common scenes by conducting local associations. For example, typical structure consists of backbone, detector and an Rol to extract instance-level features. The appearance similarity between instances is measured by pairwise distance (e.g., cosine distance). The positional association score is calculated by the ToU between instances in frames. cascaded post-processing is applied to fully use the appearance similarity and positional association. Motivated by ZZng et al. (2021) Sun al. (2020), some models Wu et al. (2022a) 2021) directly apply Transformer-baseee architectures to VTS, with an extra network for recognition. However, lacking loU-based post-processing procedures and utilization of temporal information, these methods struggle in many difficult VTS situations (e.g., crowded scenes, fast movement, lighting change).
3 METHODS
3.1 OVERVIEW
GLOMA is an end-to-end framework for Video Text Spotting, which conducts global associations and adopts morphology-awan measurements. The whole framework can be seen in Fig. [2] There are three parallel heads: detection head, recognition head, and tracking head. Given a video, we first detect the potential objects as 4-point coordinates in Frame F,, and then extract corresponding tracking embeddings as ef for each object. We maintain global embedding pool as G2 and the concatenated features of all the embeddings in G are represented as G. E M, where M,
Ei= N₂ stands for the number of embeddings in G, N, is the number of texts in frame Fü d is the dimension of each embedding, and L is the sliding window size. Our tracking head calculates association scores between objects in frame F, and objects in G, generating association matrix
3

---

Published as a conference paper at ICLR 2025
Came
Case
Case
0,06
60
0.03
055
0.01
Polegons in Prame Fc-3
Polygons Prome
W: W Wasserstatn distance
Figure 3: Three cases demonstrate the effectiveness of Wasserstein distance. IoU-based distance and Wasserstein distance both succeed in Case 1. But in Case 2 and Case 3, the fast movements result in the poor performance of loU-based distance, where Wasserstein distance produces more steady results by considering both location and morphology.
Asss E IR NeNx and then PAs Asso turned into tracklet-level association matrix represented as P Pracklet E RR, x,, where K, is the number of tracklets in the global embedding pool G2 Besides, we also calculate the morphological similarities between the two adjacent frames Fr-1 and Fr and output a distance matrix W, IR N. The two score matrices P Prrnckeet and W are united by a simple max operation without any other post-processing and output the final scores depicted as And Hungarian algorithm is applied to assign IDs.
S. E gg, N. in Fig. 2
3.2 GLOBAL TRACKING
We adopt Transformer- bhased network for global tracking similar to Zhou al.] (2022b). All the previous embeddings in the global embedding pool are encoded by Transformer encoder global memories. With an extraordinary long-range temporal modeling ability, Transformer is able to capture global information. Transformer decoder inputs encoded historical information as memory, with embeddings in the current frame as queries to calculate the similarity scores between the current instances and trajectories. The whole procedure can be written
H Encoder (.1,
(1)
P Asso Decoder( Q H.)
(2)
where H, E IR Mixd denotes the encoded historical temporal memory. P E N is the output association matrix. Q, N. refers to the query embeddings (i.e., embeddings of the current frame).
During training, associate all the embeddings with themselves (iee,, Q, G2), generating an association matrix RMMx where and B represents the number of images in batch, which is fixed as 16 in experiments. For each text in each timestamp t, we have a vector a E IR N:+1 which indicates the association scores between query embedding q, and embeddings in frame Note that the extra dimension in refers to the empty association (i.e., the query has no matched target in this frame, usually indicating an occlusion the end of the trajectory). A softmax function is used to transform the score into the possibility P Assoi
Σド= N, the number of all texts within a batch
exp (a,)
P Asso (q:, eg)
(3)
Eje exp (a,)
Thus, we learn the scores by minimizing the likelikeldoood of each possibility. For each tracklet set Te, = 1,,,..., ...,K K, we assume there are N₂ embeddings Tr and for each query embedding e; in the global embedding pool G, we calculate the loss only when e; Tи. The loss writes as:
Ni
ltracklet (Tk, = -
1 ET, log P AAsO (q.+ ej),
(4)
K
M
Etrack =
Σ
ltrakket (Tk, q,),
k=
4

---

Published as a conference paper at ICLR 2025
where 1 ET, is if the query embedding belongs to the tracklet set Ta- Semantic embeddings. To increase the discrimination of embeddings, we also introduce semantic information to boost the performance of tracking. In detail, the embeddings fed into Transformer are extracted by Rotated RolAlign with a shallow convolutional layer and an LSTM Hochreiter & (chhiidhuber (1997) layer, followed by fully connected layer to project features into the classes of words. The architecture writes as:
e, (stm(conv((-roo((x))))
(5)
O, fc(et).
where X is the backbone feature map in frame Fr, e, the embedding fed into Transformer for associations, and o, is the recognition output supervised by Connectionist Temporal Classification (raC)s Graves al. (0066 loss.
3.3 WASSERSTEIN DISTANCES IN CORRELATION
Concerning morphological information, we apply the Wasserstein distance to model both location similarity and shape similarity. Previous methods usually measure the location similarity between two adjacent frames by a pairwise calculation of loU of each pair of bounding boxes and ignore the shape similarity, just the same as the methods in MOT. However, considering the differences from the scenes MOT, the shapes of the texts are much more steady in a small time window, which can also be a strong feature for tracking.
As demonstrated in Fig. 3 we use three cases to exhibit the advantage of Wasserstein distance over the IoU. Case all texts are moving at low speed, so the location information is enough for associations, and both IoU and Wasserstein distance can conduct a correct match. In Case 2, the fast movement of the camera leads to the fast drift of texts. this situation, the IoU scores give false positional association clue, leading to a false matching. However, the morphological differences are obvious so that the Wasserstein distance can capture the incoherence. In Case 3, the loU scores are both 0 due to the fast movement of cars, while the Wasserstein distance can still perform the correct positional association.
To obtain the awareness of both the locations and shapes, model the polygons in different frames as Gaussian distributions and measure the similarity via distribution distance. For each predicted 4-point coordinate b in two adjacent frames, calculate pairwise Wasserstein distances between the corresponding convex hull rotated bounding boxes b(x, y, w, h, 0). The first step is to convert the rotated box b into Gaussian distribution N ())
μ (z, v),
RSR
0
) cos 0
sin 0
)
cos 0
sin 0 cos 0
(6)
sin 0
0
sin
w cos" 0 sin? w-h cos sin 0
10=h cos 0 sin 0 cos
sin" 0 h
where R is the rotated matrix and S is the diagonal matrix. The Wasserstein distance between two Gaussian distributions is represented as:
d = 1 1211 Tr (0) 1 -2- -0 (20) 1/2
With proper consideration of both the angles and the coordinates, Wasserstein distance can cap- ture both location similarity and morphological similarity. Finally, to convert the distance into an applicable positional score, we have:
W (b1, b2) = W (W (111 ( 02)),
ad
(8)
f(a),02))
where hyper-parameter, and is function normalize the distance. We f(a1, 02) (Tr(0102))2//4

---

Published as a conference paper at ICLR 2025
3.4
LOSS FUNCTIONS
There are three tasks and three corresponding losses. For the detection head, we adopt LI loss regress the 4-point polygons and other losses the same losses in YOLOX Ge et al. 102TT. For the recognition head, we adopt Connectionist Temporal Classification (CTC) Graves al. (006g loss for texts. We also apply multi-task learning losses. The whole losses are written as:
t= e + tree + e te + 01 02 + 03.
where 11, 02, dg are learnable parameters.
3.5 INFERENCE
During inference, we iteratively build the trajectories. For the initial frame Fo. we regard each text the start of trajectory. For frame F,, we have global embedding pool G, which stores all the previous emheddings in a sliding window, and the corresponding embedding matrix writes as G, (i.e., the concatenated embeddings). Assume there has N₂ embeddings. We regard embeddings in the current frame Fe as Q, and the corresponding embedding matrix as Q. The association score A, calculated between Qe and Gt, and converted to possibility matrix PAsSO by a softmax function. Then a tracklet-wiss, sum is applied to calculate the possibility of each tracklet. For each q; E Q2 we have:
K, trajectories in G,, and each trajectory
N₂ P Asso qs eÿ).
Praakklet (qi, Gi)
Σ j=1
(10)
where ej is the embedding in each trajectory. After calculating all the embeddings in Qr we can finally obtain matrix Prrackeet E RNNcK,, which refers to the possibility that the embedding belongs to the tracklet. Also, Wasserstein distance between frame Fr and F, is calculated as positional and morphological similarity score W., 1, and the final output is max Tracklets W e). A Hungarian algorithm is applied to ensure the ID assignment is unique for each text.
4
EXPERIMENTS
4.1
IMPLEMENTED DETAILS
We adopt ResNet-500 He et al. (2016) with FPN Lin al. (2017) layers our backbone and use the checkpoint pretrained on ImageNet Deng et al. 2209); Russakovsky et al. (2015). The architecture of the detection head is borrowed from YOLOX Ge et al. (2021) with an extra branch regress the polygons. The tracking head is lightweight architecture with only one-layer Transformer. All ex- periments are conducted on Tesla V100 GPUs We first pretrain the model on COCOText Veit al. (2016) and apply Random Crop, Random Resize, Random Color Jittering, and Pseudo Track Zhoou et al. (2020) for data augmentation. Then it fine-tuned on other datasets. The batch size is fixed as 16 when training and random sampling within clip is applied to make sure images in a batch from the same video clip. During inference, we resize the images with the shorter side fixed and the ratio of images kept.
4.2 DATASETS AND METRICS
Following previous protocols, we evaluate our methods on several different datasets.
ICDAR2015 video and ICDAR2013 video. ICDAR2015 video contains 25 clips for training and 24 clips for testing. Most scenes are street views with tens of texts in one image. ICDAR2013 video a sub-dataset of ICDAR2015 video.
Minetto. Minetto is small dataset that contains 5 videos harvested outdoors. Without training split, is used as a test dataset in previous methods.
Metrics. Following previous protocols Ristani et al. (2016), we adopt the metrics inherited from MOT. Different from MOT, metrics in Video Text Tracking adopt the loU between polygons to measure the similarity of two instances. Three metrics, MOTA, MOTP, and IDF1, are mainly used to evaluate performance. MOTA measures a comprehensive performance both the detection and tracking performance, and MOTP mainly concerns the ability to fit the bounding boxes. IDFI only

---

Published as a conference paper at ICLR 2025
Table 1: Video Text Tracking different datasets. Our proposed method outperforms previous methods by a large margin.
Dataset
Methods
MOTAT MOTPT
IDFI1
MM1
MI
AJOU Koo & Kim
(0013)
16.4
72.7
36.1
14.1
62.0
Free Cheng al. (200
43.2
76.7
57.9
36.6
44.4
SAVTD Feng et al. (2021)
44.1
75.2
58.2
44.8
29.0
ICDAR2015
SVRep Li
2021
49.5
73.9
66.1
44.9
27.1
video
CoText Wu et al (002bb)
51.4
73.6
68.6
49.6
23.5
TransV'I Wu al.
(202)
44.1
75.8
57.3
34.3
33.7
TransDETR Wu et al.
(2022a)
47.7
74.1
65.5
42.0
32.1
Ours
56.0
77.4
70.5
49.7
27.3
YORO Cheng et al. (2019)
47.3
73.7
62.5
33.1
45.3
SVRep Li ct al. 2021
53,2
76.7
65.1
38.2
33.2
ICDAR2013
Co'Text Wu et al
2022b)
55.8
76.4
68.1
44.6
28.7
video
TransDETR Wu al.
2002a
54.7
76.6
67.2
43.5
33.2
Ours
56.3
78.7
68.6
46.0
28.6
SAVTD Feng et al. (2021)
83.5
76.8
SVRep Li al.
202T
86.3
81.0
83.9
96.4
0
CoTTex Wu et al. (2022b)
86.9
80.6
83.9
87.7
0
Minetto
Trans Spoottrr Wu al. (2021)
84.1
77.6
74.7
TransDETR Wu al
(2022a
84.1
57.9
76.7
36.6
44.4
Ours
87.1
80.6
84.2
89.3
3.6
Table 2: Video Text Spotting on ICDAR2015 video dataset. performance.
Our GLOMA also achieves leading
Methods
MOTA MOTP*
IDF11
MM1 ML.
Free Cheng al.
2020
53.0
74.9
61.9
45.5
35.9
CoTTxx Wu et al
(2022b)
59.0
74.5
72.0
48.6
26.4
Trans TSpotter Wu a.. 2021
53.2
74.9
61.5
TransDETR Wu et al. (202 2a
58.4
75.2
70.4
32.0
20.8
TransDETR (aug) Wu et 2022a)
60.9
74.6
72.8
33.6
20.8
OURS
62.5
78.2
74.2
51.0
22.0
75
25
Wasserstein
70
20
Data Distribution (%)
155
IDR
50
[0.0. 0.1) [0.1. 0.2) 10.2, 0.3) [0.3, 0.4) [0.4, 0.5) [0.5, 0.6) [0.6, 0.7) [0.7, 0.8) [0.8, 0.9) [0.9, 1.0]
Motion loU Ranges
Figure Motion-aware evaluations on the ICDARI3 video dataset

---

Published as a conference paper at ICLR 2025
Table 3: Ablation study the sliding window size. As the window size goes larger, the tracking performance tends to improve.
Window size
MOTA
MOTP*
IDFIt
MM1
MI
Asso. Tmm))
2
55.1
77.5
63.0
46.8
30.1
3.1
4
55.9
77.4
68.3
49.4
28.2
3.2
8
56.0
77.4
70.5
49.7
27.3
3.6
16
55.7
77.4
71.0
49.1
27.1
3.9
Table 4: Experiments on positional distance (Row 1-3), self attention layer in decoder (Row 4-5), max operation (Row 6-8), and semantic embeddings (Row 9-10).
Methods
MOTA
MOTP*
IDF1
MM1
MI.
w/o. distance
55.5
77.4
70.0
47.3
28.1
W. IoU Wasserstein
55.8
77.4
70.6
49.0
27.1
56.0
77.4
70.5
49.7
27.3
w. self-attn
53.2
77.1
69.8
52.8
25.4
w/o. self-attn
56.0
77.4
70.5
49.7
27.3
P Tracklet
55.5
77.4
70.0
47.3
28.1
W W.
43.2
77.8
50.2
33.0
40.2
maz P. W)
56.0
77.4
70.5
49.7
27.3
w/o. semantic
53.9
77.4
66.9
47.7
28.0
w. semantic
56.0
77.4
70.5
49.7
27.3
measures the ability of tracking. Besides, we also adopt Mostly-Matched (MM) and Mostly-Lost (ML) to evaluate the completeness of trajectories. In Video Text Spotting, we also use the same metries to measure the performance, but the similarity between instances calculated by the edit distance between texts.
4.3 STATE-OF-THE-ART COMPARISONS
To verify the effectiveness of the proposed GLOMA, compare its performance with several SOTA methods.
Video Text Tracking. Video Text Tracking core task measure the performance of all methods. Thus, we evaluate our method in VTT and compare with other methods on several public datasets. On the most commonly used dataset ICDAR2015 video, we outperform previous works by large margin. We obtain 4.6 absolute improvement over the previous state-of-ther offie-----hearr art method and 8.3 absolute improvement over the previous Transformer-ben based method, which proves the effectiveness of the proposed GLOMA. Besides, our method also achieves leading performance on MOTP and IDF1. On ICDAR2013 video, our GLOMA achieves top performance on all metrics. On a smaller dataset Minetto, also achieve SOTA results.
Video Text Spotting. Video Text Spotting concerns the tracking performance and the recognition results. As shown in Tab. compared with the previous SOTA method TransDETR (aug), Wu al. (2022a) our GLOMA obtains 1.6 absolute improvement on MOTA, 3.6 absolute improvement on MОTP and 1.4 absolute improvement IDFI. Therefore, with a simple and shallow recognition head, our GLOMA can also achieve great performance.
4.4 ABLATION STUDIES
To verify the effectiveness of the components of GLOMA, conduct several ablation studies ICDAR2015 video, as shown in Tab. 3
Sliding window size. We study how the size of the sliding window impacts the final results during
8

---

Published as a conference paper at ICLR 2025
the inference stage. As shown in Tab. B] up when the window length increases. When the window length is 2, the situation only involves associations between the two adjacent frames (i.e. local associations), and the performance is much worse than when using a larger window size. Note that the metric MOTA concerns both the detection performance and tracking performance, and IDF1 only focuses on the tracking performance. Besides the temporal awareness obtained in training process, a longer window during inference stage also brings longer range of temporal information, leading to a sharp increase in IDFI, which proves the contribution of global embeddings to the final tracking performance. Therefore, we adopt 8 as the default sliding window size.
see a trend that the tracking performance goes
Distance measurement. We adopt Wasserstein distance for the measurement of positional association instead of loU. As mentioned above, Wasserstein distance pays attention to both the location and morphological features so that it outperforms IoU when applied as a positional distance mcasurement in VTS. As shown in Tab. 4 (Row 1-3), positional distance can improve the performance, and with Wasserstein distance, the proposed GLOMA can achieve better results than with loU. We also conduct motion-aware evaluation similar to that in Video Object Detection (VOD) using motion IoU Zhuu et al.) (2077) to indicate the motion speed of texts. We evaluate GloTSFormer on the ICDARI3 video dataset, as the test labels of ICDARI5 video are on the webdriver which are unavailable. As we evaluate the performance on data segments with different motion IoUs, we use ID Recall (IDR) Ristani et al. (2016) instead of other metrics involving false positives because TPs of motion IoU range of [0.3, 0.4) can be taken as FPs of [0.4, 0.5). As shown in Fig. 4 (Overall IDR: 62.5% v.s. 62.9%). where W. refers to Wasserstein distance, when the motion loU decreases (indicating faster motion), Wasserstein distance brings out greater improvements.
Attention layer. The encoder layer is one multi-head attention layer and the decoder layer includes one cross-attention layer without a self-attention layer. As shown in Tab. 4 (Row 4-5), we do not observe obvious improvement of the overall performance when introducing self-attention layer and remove it.
Max operation. We show extra experiments in Tab. (Row 8-10). Both Prracktet and W contribute the performance.
Semantic embeddings. We use the embeddings before the final fully-connected layer for as- sociations. The embeddings we select carry semantic information and can provide some prior information and boost tracking performance. To verify the effectiveness, we only feed Transformer the features after Rotated RolAlign to explore the influence of semantic information. As shown in Tab. 4 (Row 11 11-12), the ablation studies show the effectiveness of the proposed GLOMA.
Speed analysis. Our GLOMA runs at around 20 FPS on single Tesla V000 GPU. The backbone, FPN layers, detection head, and recognition head take about 48 ms per frame, and the tracking procedure takes about 3.6 ms per frame. By maintaining global embedding pool, we do not have to repeatedly extract the embeddings from each frame. As shown in Tab. 3 Asso. T refers to the time cost by the tracking procedure (i.e., the inference of Transformer and associations). When the window size goes larger, we only observe slight increment in the time consumed.
4.5 VISUALIZATION
As demonstrated in Fig. 5 we select two videos to show the advantage of our GLOMA. From an overall perspective, our GLOMA performs fewer false assignments in tracking and fewer FNs in de- tection. As presented in video (a), when facing crowded scenes with motion blur, TransVTSpotter and TransDETR fail to perform the right assignments due to excessive interfering texts. We could notice that the polygon colors of the texts (e.g., "CV", "de", "super", "Tel") are changed over time, indicating many ID) switches. On the contrary, our GLOMA has much more steady performance without any ID switch. As presented in video (b), the architecture of TransDETR and Trans Trans VTSpot- ter have some limitations in detection, resulting in False Negatives in some frames. Without global view, Trans T VTSpoteer and TransDETR tend to conduct incorrect assigmments especially when the fractures of trajectories take place. Our detection performance is relatively independent of tracking, and our GLOMA has a global view of historical information, leading to better performance on both detection and tracking.

---

Published as a conference paper at ICLR 2025
Time
Traus TraeVISpoii ISpotte
A LiONN
VENDRE
TrarDEER
VENTRE
MENORE
Oues
A NEEWOS
VENDRE
TrancV ISpaltes
BOOM
TranDea
BORKEAD
PODMERAMA
OuiE
TOOMERAA TELEIES
Figure 5: We demonstrate the results of previous Transformer- methods Wu et al. (2021 2022a) and our GLOMA. Different IDs are represented in different colors. Some of the false sults (e.g., FNs, ID switches, and IDFs) are marked with dotted red circle and pointed out by red arrow. Apparently, our GLOMA performs better than previous Transformer-he based methods, espe- cially in crowded scenes.
LIMITATIONS AND CONCLUSION
Limitations. Though GLOMA achieves great performance on VTS, we find the detector fails in detecting texts when facing severe motion blur and extreme sizes, which leads fractures of tra- jectories. Our detector is less robust towards deterioration compared with our tracking method. We would consider how apply trajectory information to improve detection performance our future work.
Conclusion. this work, we propose a Transformer-based global text spotting method GLOMA. We explore how fully exploit temporal information and morphological information with global association module and Wasserstein distance, respectively. We also conduct extensive experiments on the public datasets to verify the effectiveness of our method. We hope work can shed light future research on the utilization of temporal information and morphological information Video Text Spotting.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Uperoft. Simple online and realtime tracking. 2016 IEEE international conference on image processing (ICIP), pp. 3464 3468. IEEE, 2016.
Yihong Chen, Yue Cao, Han Hu, and Liwei Wang. Memory enhanced global-local aggregation for video object detection. In Proceedings of the IEEE/CVF conference computer vision and pattem recognition, pp. 10337-10346, 2020.
Zhanzhan Cheng. Jing Lu, Yi Niu, Shiliang Pu, Fei Wu, and Shuigeng Zhou. You only recognize once: Towards fast video text spotting. Proceedings of the 27th ACM International Conference Multimedia, pp. 855 855-863, 2019.
Zhanzhan Cheng, Jing Lu, Baorui Zou, Liang Qiao, Yunlu Xu, Shiliang Pu, Yi Niu, Fei Wu, and Shuigeng Zhou, Free: A fast and robust end-to-end video text spotter. IEEE Transactions on Image Processing, 30:822-837, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 245.255. Ieee, 2009.
Jiajun Deng, Yingwei Pan, Ting Yao, Wengang Zhou, Houqiang Li, and Tao Mei. Relation distilla- tion networks for video object detection. In ICCV, pp. 7023-7032, 2019.
Wei Feng, Fei Yin, Ya-Yao Zhang, and Cheng-Lin Liu. Semantic-aware video text detection. In Pro- ceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pp- 1695- 1705, 2021.
Yuzhe Gao, Xing Li, Jiajian Zhang, Yu Zhou, Dian Jin, Jing Wang, Shenggao Zhu, and Xiang Bai. Video text tracking with spatio-temporal complementary model. IEEE Transactions Image Processing, 30:9321-9331, 2021.
Zheng Ge, Songtao Liu, Feng Wang. Zeming Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 2021.
Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference computer vision, pp. 1440-1448, 2015.
Alex Graves, Santiago Fernández, Paustino Gomez, and Jürgen Schmidhuber. Connectionist tem- poral classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pp. 369-376, 2006.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference computer vision, pp. 2961-2969, 2017.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 98):
1735 1735-1780, 1997.
Dimosthenis Karatzas, Faisal Shafait, Seiichi Uchida, Masakazu Iwamura, Lluis Gomez Bigorda, Sergi Robles Mestre, Joan Mas, David Fernandez Mota, Jon Almazan Almazan, and Lluis Pere Las Heras. Icdar 2013 robust reading competition. 2013 12th international conference on document analysis and recognition, pp. 1484- 1493. IEEE, 2013.
Dimosthenis Karatzas, Lluis Gomez Bigorda, Anguelos Nicolaou, Suman Ghosh, Andrew Bag- danov, Masakazu Iwamura, Jiri Matas, Lukas Neumann. Vijay Ramaseshan Chandrasekhar, Shi- jian Lu, et al. Iedar 2015 competition on robust reading. In 2015 13th international conference document analysis and recognition (ICDAR), pp. 1556-1160. IEEF, 2015.

---

Published as a conference paper at ICLR 2025
Hyung Il Koo and Duck Hoon Kim. Scene text detection via connected component clustering and nontext filtering. IEEE transactions on image processing, 22(6)) 2013.
Zhuang Li, Weijia Wu, Mike Zheng Shou, Jiahong Li, Size Li, Zhongyuan Wang, and Hong Zhou. Contrastive learning of semantic and visual representations for text tracking. arXiv preprint arXiv:2112.149766 2021.
Tsung-Y Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference com- puter vision and pattern recognition, pp. 2117-2125, 2017.
Xuebo Liu, Ding Liang, Shi Yan, Dagui Chen, Yu Qiao, and Junjie Yan. Fots: Fast oriented text spotting with a unified network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5676-5685. 2018.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-34444 2015.
Rodrigo Minetto, Nicolas Thome, Matthieu Cord, Neucimar J Leite, and Jorge Stolfi. Snoopertrack: Text detection and tracking for outdoor videos. In 2011 18th IEEE international conference on image processing, pp. 505-508. IEEE, 2011.
Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. Performance mea- sures and data set for multi-target, multi-camera tracking. In European conference computer vision, pp. 17-35. Springer, 2016.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Scan Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenct large scale visual recognition challenge. International journal of computer vision, 115())211-252 2015.
Mykhailo Shvets, Wei Liu, and Alexander C Berg. Leveraging long-range temporal relationships between proposals for video object detection. In ICCV, pp. 9756-97644 2019.
Peize Sun, Jinkun Cao, Yi Jiang, Rufeng Zhang, Enze Xie, Zchuan Yuan, Changhu Wang, and Ping Luo. Transtrack: Multiple object tracking with transformer. arXiv preprint urXiv:2012.15400 2020.
Zhi Tian, Weilin Huang, Tong He, Pan He, and Yu Qiao. Detecting text in natural image with connectionist text proposal network. In European conference on computer vision, pp. 56-72. Springer, 2016.
Andreas Veit, Tomas Matera, Lukas Neumann, Jiri Matas, and Serge Belongie. Coco-text: Dataset and benchmark for text detection and recognition in natural images. arXiv preprint 1611v01601 2016.
Han Wang, Jun Tang, Xiaodong Liu, Shanyan Guan, Rong Xie, and Li Song. Ptseformer: Pro- gressive semporal-spatial enhanced transformer towards video object detection. arXiv preprint arXiv:2209.02242. 2022.
Wenhai Wang, Enze Xie, Xiang Li, Wenbo Hou, Tong Lu, Gang Yu, and Shuai Shao. Shape ro- bust text detection with progressive scale expansion network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9336-9345, 2019a.
Wenhai Wang, Enze Xie, Xiaoge Song, Yuhang Zang, Wenjia Wang, Tong Lu, Gang Yu, and Chun- hua Shen. Efficient and accurate arbitrary-shaper text detection with pixel aggregation network. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 8449, 2019b.
Wenhai Wang, Enze Xie, Xiang Li, Xuebo Liu, Ding Liang, Zhibo Yang, Tong Lu, and Chunhua Shen. Pan++: Towards efficient and accurate end-to-end spotting of urbitrarily-shape text. JEEE Transactions Pattern Analysis and Machine Intelligence, (4(9):5349-5366 2021.
12

---

Published as a conference paper at ICLR 2025
Zhongdao Wang, Liang Zheng, Yixuan Liu, Yali Li, and Shengjin Wang. Towards real-time multi- object tracking. In European Conference on Computer Vision, pp. 107-122. Springer, 2020.
Nicolai Wojke, Alex Bewley, and Dietrich Paulus. Simple online and realtime tracking with a deep association metric. In 2017 IEEE international conference on image processing (ICIP), pp. 3645- 3649. IEEE, 2017.
Haiping Wu, Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang. Sequence level semantics aggre- gation for video object detection. In ICCV, pp. 1117-9225, 2019.
Weijia Wu, Yuanqiang Cai, Debing Zhang, Sibo Wang, Zhuang Li, Jiahong Li, Yejun Tang, and Hong Zhou. A bilingual, openworld video text dataset and end-to-end video text spotter with transformer. arXiv preprint 2 12.04888, 2021.
Weijia Wu, Debing Zhang, Ying Fu, Chunhua Shen, Hong Zhou, Yuanqiang Cai, and Ping Luo. End-to-end video text spotting with transformer. arXiv preprint arXiv:2203.105:. 2022a.
Wejia Wu, Zhuang Li, Jiahong Li, Chunhua Shen, Hong Zhou, Size Li, Zhongyuan Wang, and Ping Luo. Real-time end-to-end video text spotter with contrastive representation learning. arXiv preprint arXiv:2207.084 084 17, 2022b.
Xue Yang, Junchi Yan, Qi Ming, Wentao Wang. Xiaopeng Zhang, and Qi Tian. Rethinking rotated object detection with gaussian wasserstein distance loss. In International Conference on Machine Learming, pp. 11830-11841. PMIR, 2021.
Hongyuan Yu, Yan Huang, Lihong Pi, Chengquan Zhang, Xuan Li, and Liang Wang. End-to-end video text detection with online tracking. Pattern Recognition, 113:107791, 2021.
Fangao Zeng, Bin Dong, Tiancai Wang, Xiangyu Zhang, and Yichen Wei. Motr: End-to-end multiple-object tracking with transformer. arXiv preprint arXiv:2105.03244 2021.
Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu. Fairmot: On the
fairness of detection and re-identificattin in multiple object tracking. International Journal of Computer Vision, 129(1 1):3069-3087 2021.
Z Zhong, Jin, Zhang, and Feng. Deeptext: A unified framework for text proposal generation and text detection in natural images. arxiv 2016. arXiv preprint arXiv:1605.07314
Qianyu Zhou, Xiangtai Li, Lu He, Yibo Yang, Guangliang Cheng, Yunhai Tong, Lizhuang Ma. and Dacheng Tao. Transvod: End-to-end video object detection with spatial-temporal transformers. arXiv preprint arXiv:2201.0504 2022a.
Xingyi Zhou, Vladlen Koltun, and Philipp Krähenbühl. Tracking objects as points. In European Conference on Computer Vision, pp. 474-490. Springer, 2020.
Xingyi Zhou, Tianwei Yin, Vladlen Koltun, and Philipp Krähenbühl. Global tracking transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 8771-8780, 2022b.
Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang, East: an efficient and accurate scene text detector. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 5551-5560, 2017.
Xizhou Zhu, Yujie Wang, Jifeng Dai, Lu Yuan, and Yichen Wei. Flow-guided feature aggregation for video object detection. In ICCV, pp. 408-417, 2017.
13


Reviews:
[{"review_id": "zDx6wfDOfZ", "reviewer": "Reviewer_Qrc3", "summary": "This article proposes a more effective tracking method for video text based on the characteristics of the text. In specific, The paper introduce GLOMA to model the tracking problem as global associations and utilize the Gaussian Wasserstein distance to guide the morphological correlation between frames. GLOMA achieves state-of-the-art performance on multiple video text recognition benchmarks.", "questions": "See weaknesses", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The article is clearly written and easy to understand. \n\n2. The motivation is strong, Wasserstein distance is indeed a better measurement for video text association. \n\n3. Extensive experiments demonstrate the effectiveness of the proposed new association method.\n\n4. The proposed method demonstrates improved performance compared to several SOTA methods across multiple benchmarks.", "weaknesses": "1. Compared to TransDETR, which uses learnable query for object tracking, what is the advantage of the proposed association method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mppnUhyq4H", "reviewer": "Reviewer_qTr3", "summary": "This manuscript introduces GLOMA, a Transformer-based global tracking approach that frames the tracking challenge as a problem of global associations. It employs the Gaussian Wasserstein distance to facilitate morphological correlation across frames. The efficacy of GLOMA is demonstrated through experimental results on renowned benchmarks, including the ICDAR 2015 video dataset. The contributions of this manuscript are:\n1. A Transformer-based global tracking method for VTS that associates frames globally.\n2. A Wasserstein distance-based method for measuring morphological similarity.\n3. Extensive experiments on public datasets, demonstrating state-of-the-art performance.", "questions": "Please give the responses to the concerns in the Weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Global Association Approach**: The paper introduces a Transformer-based global tracking method that simultaneously associates multiple frames, effectively utilizing global context information in video sequences.\n\n**Gaussian Wasserstein Distance**:\nThe use of Wasserstein distance for positional association considers both location and morphological features, enhancing the accuracy of matching, especially in scenarios with fast movement or interference.\n\n**Performance Improvements**:\nAchieving a 56.0 MOTA on the ICDAR2015 video dataset, with a 4.6 absolute improvement over the previous state-of-the-art method and an 8.3 MOTA improvement over the previous Transformer-based method, demonstrating significant performance gains.\n\n**Extensive Experimental Validation**:\nThe paper conducts extensive experiments on public datasets, validating the effectiveness of the GLOMA method, which proves its practicality and efficacy in the field of video text recognition.", "weaknesses": "**The Choice of the Text Detector/Spotter**:\nThe paper discusses the use of a 4-point coordinate prediction head for text detection within the GLOMA framework. However, it does not incorporate state-of-the-art text detectors or text spotters capable of detecting text of arbitrary shapes, such as DBNet++ and Mask TextSpotter v3. It would be beneficial to understand why these advanced detectors or spotters were not utilized in the proposed method. Additionally, the paper should ideally include a discussion on existing scene text spotting methods as part of its related work, given that these are foundational to the field of video text spotting.\n\n**The Use of the Synthetic Data**: \nRegarding the use of synthetic data for training, as demonstrated in TransDETR, synthetic data can significantly enhance model performance when used as pretraining data. For instance, TransDETR achieved a 51.8 MOTA score on the ICDAR 2015 video dataset with synthetic data, compared to a 47.7 MOTA score without it. It would be insightful to know why the proposed GLOMA method did not leverage synthetic data for training purposes. Addressing these points could provide a more comprehensive understanding of the design choices made in developing the GLOMA framework and how it compares to other leading approaches in the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pdy0r6SRfi", "reviewer": "Reviewer_JtLg", "summary": "Summary\nThe article presents a novel method for video text spotting (VTS) that leverages global associations and morphological information, called GLOMA. This approach primarily addresses the limitations of current VTS methods, which typically rely on local associations and IoU-based distance metrics, often ignoring rich temporal and morphological cues. The main contributions of GLOMA include: a Transformer-based global tracking method, a position association method based on the Wasserstein distance, and extensive experiments demonstrating state-of-the-art performance across multiple benchmark datasets.", "questions": "Please refer to my previous comments.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Strengths\n1. GLOMA makes a significant contribution to the field by utilizing global associations through the Transformer architecture and integrating morphological information with the Gaussian Wasserstein distance. This effectively addresses the limitations of previous methods that primarily relied on local pairwise comparisons.\n2. The paper demonstrates significant performance improvements across multiple datasets. For example, the MOTA on the ICDAR2015 video dataset improved by 4.6 absolute points, and compared to the previous best Transformer-based method, MOTA improved by 8.3 points. These results strongly validate the effectiveness of the GLOMA method. \n3. The authors conducted an in-depth ablation study to examine the impact of key components such as sliding window size, distance measurement, attention layers, and semantic embeddings. This provides valuable insights for model design.\n4. The paper is well-written, with clear explanations of the method, including architecture, training process, and inference process. Figures and tables effectively illustrate key concepts and results.", "weaknesses": "Weaknesses\nAlthough the experimental results are encouraging, there are still several key issues that need further clarification: \n1. In the abstract, the sentence “ignoring the abundant temporal information and the morphological characteristics in VTS” conveys the idea of neglecting temporal information and morphological features, but it does not sufficiently emphasize the importance of these features for the video text tracking (VTS) task. It is suggested to more explicitly highlight the core role of temporal information and morphological features in VTS to enhance reader understanding.\n2. Although the abstract mentions the \"Gaussian Wasserstein distance,\" it does not explain its specific advantages and role in VTS in detail. Readers may want more information on why this method is more effective than traditional IoU distance. Similarly, the innovations of GLOMA could be slightly expanded to show its contributions to solving existing problems.\n3. The transitions between different sections in the introduction are not very smooth. For example, the description of existing research methods directly jumps to the introduction of the new model GLOMA, lacking transitional paragraphs to clarify why a new model is needed. It is suggested to add some transition sentences to better guide readers from existing work to the proposal of the new model.\n4. Figure 1 does not clearly indicate the specific types of failure situations (such as target loss or target error) before directly introducing global associations to reduce the probability of failure. Perhaps the phrase \"To solve the problems\" could be removed, and the role of global associations could be directly stated. Additionally, other types of failure situations could be added to enhance persuasiveness and comprehensiveness of the explanation.\n5. In section 3.1 Overview, it is mentioned that GLOMA has three parallel modules: detection module, recognition module, and tracking module. However, these modules are not clearly represented in Figure 2, and the aesthetic and logical clarity of the illustration is lacking. It is recommended to use more distinct dividing lines to separate the modules and add text descriptions in the figure, especially in the right-side process section, to enhance overall visualization.\n6. The statement \"Wasserstein distance can capture both location similarity and morphological similarity\" theoretically makes sense, but it lacks detailed explanation on how it specifically captures morphological similarity. It is suggested to add an explanation of how the Wasserstein distance is used to measure morphological features, helping readers understand its advantages over traditional IoU.\n7. In equation (8), it is mentioned that α is a hyperparameter, but the text does not detail how to choose this hyperparameter and its impact on model performance. Moreover, the form and role of the function f are not thoroughly discussed. It is recommended to provide details on the hyperparameter selection process, especially the normalization process of f and its impact on the results.\n8. Although the text provides a general framework for embedding pool and trajectory association, it lacks detailed explanations about the embedding computation and updating mechanism. For example, how are embeddings updated? How is the length of the sliding window selected? Is there a trajectory drift problem? It is suggested to further elaborate on the embedding update strategy and trajectory construction details during inference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wi4Wedpm8t", "reviewer": "Reviewer_1EQ7", "summary": "This paper studies the video text spotting problem by introducing a GLOMA method as a new solution. The proposed GLOMA formulates a global association task to tackle the VTS problem, in which a Transformer tracking method and a Wasserstein distance method facilitates the tracking.", "questions": "(1) After reading through the paper, it remains unclear to me why the authors need to devise a specific Transformer and specific association mechanism to fulfill the VTS. To my understanding, there are many SOTA multi-object tracking algorithms and single object tracking algorithms that seem perfectly fit the proposed task. The current SOTA methods do not simply rely on the IoU matching, despite that the VTS fields may have many IoU-based approaches. I am not saying that devising new approaches for VTS is not important, but at least the paper requires a much clearer justification of why existing SOTA multi-/single-object tracking algorithms cannot tackle this task effectively. The potentially related SOTA tracking methods include but not limited to [1,2,3,4]. The comparison (or at least discussion) about more SOTA tracking methods helps better justify why the proposed technique is superior over existing multi-/single-object tracking algorithms. \n\n(2) The presentation is also not very easy-to-understand. In particular, the colorful circles are not easy to interpret. Both figure 1 and 2 lacks sufficient clarifications about the patterns used in the figure. What do the colorful circles exactly refer to? Text spotting results in bounding boxes or masks? What are associated? The current presentation style requires better clarification. \n\n(3) It is not clear to me what new insights are contributed in this paper. All the employed technologies are well-known techniques. In particular, I suggest the authors to explicitly state the novel aspects of their approach and how it differs from simply combining existing techniques. How does this ensemble of existing techniques introduce new design ideas in the community? In addition, I do not quite buy the claim of \"global\" in this paper, please better clarify what they mean by \"global\" and how their approach achieves this, given the limited number of frames considered. Lastly, Please provide a more comprehensive and informative justification for using Wasserstein distances specifically, perhaps by comparing it to other distance metrics they considered.\n\n(4) Lastly, the experiments seem not very convincing as well. The compared methods look kind of old. Not many more recent approaches are discussed and compared in the results. This limits the contribution of this paper.  For the compared methods, more recent related literature deserves a comparison or at least discussion. This includes but not limited to [5][6]. \n\nI would like to see the authors' response before making my final decision.\n\n[1] Xu, Yuanyou, Zongxin Yang, and Yi Yang. \"Integrating boxes and masks: A multi-object framework for unified visual tracking and segmentation.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n[2] Shim, Kyujin, et al. \"Adaptrack: Adaptive Thresholding-Based Matching for Multi-Object Tracking.\" 2024 IEEE International Conference on Image Processing (ICIP). IEEE, 2024.\n[3] Cao, J., et al. \"Observation-centric sort: Rethinking sort for robust multi-object tracking. arXiv 2022.\" arXiv preprint arXiv:2203.14360.\n[4] Cheng, Ho Kei, and Alexander G. Schwing. \"Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n[5] Wu, Weijia, et al. \"End-to-end video text spotting with transformer.\" International Journal of Computer Vision 132.9 (2024): 4019-4035.\n[6] Wu, Weijia, et al. \"DSText V2: A comprehensive video text spotting dataset for dense and small text.\" Pattern Recognition 149 (2024): 110177.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "Writing looks good. Results are promising.", "weaknesses": "I have many concerns about the demonstration of novelty. The contributions of this paper also need better clarification. See the questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)