Lattice Boltzmann Model for Learning Real-World Pixel Dynamicity
Authors: Guangze Zheng
Shijie Lin
Haobo Zuo
Si Si
Ming-Shan Wang
Changhong Fu
Jia Pan
Venue: NeurIPS 2025 poster ()
Keywords: physics-inspired
point tracking
object tracking
online
real-time
dynamicity
lattice Boltzmann model
collision
streaming
Primary area: applications
TL;DR: This work presents a lattice Boltzmann model, a novel physics-inspired framework for real-time online pixel tracking and object tracking.

Abstract: This work proposes the Lattice Boltzmann Model (LBM) to learn real-world pixel dynamicity for visual tracking.
LBM decomposes visual representations into dynamic pixel lattices and solves pixel motion states through collision-streaming processes. 
Specifically, the high-dimensional distribution of the target pixels is acquired through a multilayer predict-update network to estimate the pixel positions and visibility. The predict stage formulates lattice collisions among the spatial neighborhood of target pixels and develops lattice streaming within the temporal visual context. The update stage rectifies the pixel distributions with online visual representations. Compared with existing methods, LBM demonstrates practical applicability in an online and real-time manner, which can efficiently adapt to real-world visual tracking tasks. Comprehensive evaluations of real-world point tracking benchmarks such as TAP-Vid and RoboTAP validate LBM's efficiency. A general evaluation of large-scale open-world object tracking benchmarks such as TAO, BFT, and OVT-B further demonstrates LBM's real-world practicality.

Lattice Boltzmann Model for Learning Real-World Pixel Dynamicity
Guangze Zheng', Shijic Lin',
Haobo Zuo', Si Si
Ming-Shan Wang"
Changhong Fu', Jia P.n
'HKU
Institute of Zoology, CAS TTongii University
*LimX Dynamics
guangze@connect. hku. jpan@cs haunecs.hku.b. hku.hk
Project page:
https://george: http://ggorrezzuuagg github. io/lbm
time
0.0 Dasislale pinds
H(+1 2
Visible poids
world
Object dynamiicity
#C
Collining
Streamingg
Rt+1
streaming Eo pulliston
Yarget
heighbor
Each point
Other pissl
Proposed lattiie Boltzmann model (LBM)
dynamicity in video
colligon
#t1
Figure 1: Real-world object dynamicity pixel dynamicity fluid dynamicity. The object dynamicity the open world manifests through deformation and self-ncclusion, as exemplified by the bird in the figure. From a visual perspective, such object dynamicity can be decomposed into pixel dynamicity. The pixels are subsequently modeled fluid lattices that simulate hydrodynamic streaming and collision processes, and the pixel motion states efficiently addressed with the proposed lattice Boltzmann model (LBM).
Abstract
This work proposes the Lattice Boltzmann Model (LBM) to learn real-world pixel dynamicity for visual tracking. LBM decomposes visual representations into dynamic pixel lattices and solves pixel motion states through collision-streaming processes. Specifically, the high-dimensional distribution the target pixels is acquired through a multilayer predict- updatee network to estimate the pixel positions and visibility. The predict stage formulates lattice collisions among the spatial neighborhood of target pixels and develops lattice streaming within the temporal visual context. The update stage rectifies the pixel distributions with online visual representations. Comprehensive evaluations of well-wordd point tracking benchmarks such as TAP-Vid and RoboTTAP validate LBM's efficiency. A general evaluation of large-scale open-world object tracking benchmarks such TAO, BFT, and OVT-B further demonstrates LBM's real-world practicality.
Introduction
Online and real-time pixel tracking is designed to achieve continuous localization of any specified pixel for real-world applications, such as embodied manipulation Wen et al. (2024); Zhang al. (2024) and medical vision Schmidt et al. (2024). However, existing solutions are predominantly
"Corresponinng author: jpan @ hs.hku.hk
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

offline Doersch et al. (2022); Cho et al. (2024) or semi comi-online Harley et al. (2022); Karaev et al. (2024b); Li et al. (2024b), leading to sigmificant practical limitations: 1) high resource consumption from full video time window buffering causes excessive memory usage, which is unsuitable for edge-device deployment in embodied systems; 2) inevitable latency due to the integrity of video or window input, preventing real-time inference; 3) inadequate dynamic responsiveness, lacking the ability to adapt to newly emerging pixels in videos immediately; 4) privacy and storage concerns, as storing full video data poses risks of privacy breaches and imposes substantial storage costs.
The limitations of offline and semi-online methods primarily stem from the inherent reliance on spa- tiotemporal integrity, which manifests in: 1) tem- poral bi di-directionalty, where point trajectories can be optimized through bawwadd-ackward analysis within the complete temporal context, 2) iterative refinement, involving multiple cost volume updates through identical networks to minimize estimation errors. Such reliance on spatiotemporal integrity compromises the real-time performance during in- ference. For instance, LocoTrack Cho et al. (2024)
Track-On (49M)
LBM (ours, 18M)
Average Jaccard (AJ)
64
TAPTRv2
CoTracker3 (semi-online)
TAPTR
62
CoTtacker2
Run
falled
60
Online BootsTAPIR
demonstrates over 60% reduction of throughput from a single iteration to 4 iterations.
12 Frames Per Second (FPS)
Before introducing the proposed LBM, we system- atically outline its theoretical foundation the lat- tice Boltzmann method Mohamad (201) 1) for fluid simulations. Fundamentally, the lattice Boltzmann method discretizes fluids into lattices where the distribution functions undergo collision and streaming operations governed by the Boltzmann trans- port equation. The inherent locality of collision and explicit time-stepping streaming contribute to computational efficiency.
Efficiency comparison on TAP-Vid DAVIS benehmark with an NVIDIA Jetson Orin NX super. LBM shows efficiency with higher inference speed and smaller model size. The size of the circles corresponds to the number of parameters.
Figure 2:
Analogous to the lattice Boltzmann method, LBM discretizes the video into individual pixel lattices and estimates the motion states by characterizing the high-dimensional distributions of the lattices, as shown in Figure 1. Specifically, LBM employs multi-layer predict-update network to estimate the distribution of specified pixel lattices. During the predict stage at each layer, the LBM employs collision and streaming operations to estimate the current target particle distribution. The collision step primarily accounts for the neighborhood distribution of lattices, modeling local lattice interactions. The streaming step governs the temporal evolution of the distribution function by propagating particles to adjacent lattices. In the update phase, the predicted lattice distribution is refined using online visual features. The position and visibility of target pixels at the current timestep are derived from the updated lattice distribution, processed through dedicated tracking and visibility heads. Compared with existing methods, LBM exhibits distinct efficiency advantage, as illustrated in Figure 2.
To further address the dynamicity of real- weal-world objects, LBM decomposes targets as ensembles of fine-grained pixels and establishes object associations through pixel tracking. Specifically, LBM decomposes object motion into more robust multi-pixel motion patterns, thereby enabling enhanced stability in resolving object kinematic states and stronger robustness against detection failures. In contrast to the method Zheng et al. (2024) that adopts window-based tracking, LBM dynamically prunes outlier pixels (e.g., background and drifted pixels) and incorporates new inliers, thereby improving tracking responsiveness for highly dynamic objects.
2 Related Work
2.1 Tracking any point
Offline and semi-online methods The predominant frameworks for point tracking encompass offline methods that process the entire video and semi-online methods that rely on a multi-frame sliding window. PIPs Harley et al. (2022) and TAP-Net Doersch et al. (2022) establishes the point tracking baseline. TAPIR Doersch et al. (2023) and LocoTrack Cho et al. (2024) provide efficient solutions for cost volume computation. CoTracker series Karaev et al. (2024b,a) introduce proxy tokens to reduce computational cost. TAPTR Li et al. (2024b) and TAPTRv2 Li et al. (2024a) employ an architectural framework analogous to DETR Carion al. (2020) and tracking points by detection.

---

Despite marked advancements in model performance, prevailing methods are still constrained to offline or window-based online paradigms that incur substantial systemic latency.
Online methods Driven by the pragmatic demands of real-world applications, online methods have witnessed burgeoning emergence. MFT Neoral et al. (2024) extends the optical flow framework to multi-frame contexts. TAPIR Doersch et al. (2023)-relate) models achieve online adaptation through temporally causal masking. DynOMo Seidenschwarz et al. (2025) achieves online point tracking through dynamic 3D Gaussian reconstruction. Track-On Aydemir et al. (2025) further enhances online performance through spatiotemporal memory components. Compared to these works, the LBM places emphasis on tracking efficiency, particularly in real-time tracking under aesource-constraing edge computing conditions, to meet the requirements of practical tracking applications.
2.2 Tracking dynamic objects
Traditional object tracking methods Multi-object tracking (MOT) predominantly focus on targets with limited dynamic characteristics in constrained scenarios, such pedestrians Dendorfer et al. (2020) and vehicles Yu et al. (2020). Methods like Trans Sun et al. (2020), TrackFormer Mein- hardt et al. (2022), and TransCenter Xu et al. (2022) adopt DETR Carion et al. (2020)-based architectures that model targets as learnable queries. However, these solutions typically represent targets holistic entities and vulnerable performance degradation when handling highly dynamic targets. Such limitations become particularly pronounced during target deformation, partial occlusion, and fast motion.
Open-world world object tracking methods Recent advancements have extended MOT to diverse scenar- ios and arbitrary object categories. extend diverse scenarios and arbitrary targets. OVTrackk Li et al. (2023) and MASA Li et al. (2024c) integrate text encoder to specify tracking targets. UNINEXT Yan et al. (2023) and GLEE Wu et al. (2024) adapt open-world detection architectures tracking tasks through fine-tuning on videos. Motion modeling methods from conventional trackers like SORT Bew- ley et al. (2016) can achieve open- world tracking through integration with open-vocabular detectors. NetTrack Zheng et al. (2024) addresses dynamic targets through decomposition of holistic objects into nets, enabling fine-grained tracking. Considering the vulnerability of most methods to catas- trophic detection failures in applications, LBM adopts fine-grained pixel tracking and high-responsive updates to ensure robust applicability across diverse dynamic scenarios.
3 Method
3.1 Preliminary: lattice Boltzmann method
The lattice Boltzmann method solves the discrete velocity of the fluid on the lattice with streaming and collision processes. Given position and time I, the discrete distribution function f is as follows:
f(x, t) = I
At) + 22 c, t (+)),
(I)
where At and c denote the time step and discrete velocity in the i-th direction, respectively. Ω describes the collision of lattices on each node, which describes the relaxation of the distribution function towards the equilibrium distribution. By solving for cecaming-coolision processes, the density p and discrete velocity of the fluid are obtained as:
p(x,t) Σ fi(x,t), pu(x,t) = cc.((xxt).
(2)
3.2 Lattice Boltzmann model for point tracking
Given the input image I E H3 and N query points q E Nx2, lattice Boltzmann model (LBM) estimates the positions p E Nx2 visibility E R.V of these points in subsequent image streams in the real-time and online manner. This process is achieved by treating the query points as fluid particles and solving their d-dimensional distribution f E IR Nxà in dynamic scenes through streaming and collision processes. As shown in Figure 3, the specific steps include the following:
Visual encoding To model visual representations, LBM employs the first three layers of ResNet18 He et al. (2016) model pre-trained on ImageNet Deng et al. (2009) as the visual en- coder. In contrast to previous methods that process multi-level features separately, we follow Xie et al.
3

---

l2
Projection
Strident
Res block
MHA 1 MHA
Flattened feature
Skndees
Res block
x2
Conv layers
MIlA
Saropie
Backbone
Finit
ScrMt-16
Ros block
Prame
distribution
Visual encoder (6.01 Mj
LBM predict (1.58 M)
LBM update (1.56
LBM update
Projection
fanic
Backbone
Track
Frume
Reference points
Updote distribution
Collision distributiin
LBM predict
LBM update
Projection
Attn
Attn
Streaming attention
Collision
Backbone
ottention
Track head
Prome it
Track results
Figure 3: LBM framework for point tracking. illustrating 1) the distribution initialization process in LBM, 2) the LBM update step that incorporates online visual features to update the pixel distribution, 3) derivation of streaming and collision distributions, and 4) the LBM predict process that utilizes both streaming and collision distrihutions to predict current pixel distribution. Lightweight architectures have heen implemented for I.BM modules to accommodate real-world deployment requirements.
(2021) by upsampling all feature maps to a stride of and concatenating them after projections, and acquire the output visual representations o E Rdx 4 x 4 The design of the visual encoder primarily considers efficiency.
Distribution initialization It is essential to initialize the distribution function when formulating query points as fluid particles. LBM accomplishes this by sampling the visual representations corresponding to the query points q, i.e., finit BilinearSample(o q) E Nxd
Distribution prediction Corresponding to Equation 1, within new time step, LBM learns from the previous distribution functions and predicts the distribution functions at the current moment through the streaming and collision processes. Differently. LBM consolidates the distribution functions from multiple directions into a single d-dimensional distribution. contrast to Equation 1, LBM does not employ fixed neighboring pixels collision elements. Instead, it computes the interaction between the pixel distribution and a learnable neighborhood 6, thereby ensuring adaptability to dynamic scenes. At this stage, the prediction step can be formulated follows.
f(x, t(ō) f(x, () 1(x, (t)(),
where the collision operator implemented as the deformable attention Zhu et al. (2020). For stronger robustness, the temporal context is further extended from a single historical time step to N.. consisting of streaming distributions E Nx,,N and collision distributions E N Ng To ensure the stability of pixel distributions, we initialize the distribution with fint at each time step and facilitate its interaction with and f. via cross cross-attention attention modules Φ. Equation is reformulated as:
Le(0, (friit fa), f, = (f) f = (A(f, o, p.(o.))
The details of the collision process are discussed in the Appendix A.1.
Distribution update In
time step t and its corresponding image L,, pixels should dynamically update their positions p and distribution functions LBM first computes the correlation map between the pixel distribution f and visual representations The top-k response values from this map then selected as reference points r E R xxxx to update the pixel distribution function via deformable attention module v (f,, r). The update stage primarily refines the distribution function by integrating visual representations from multiple latent potential positions. The adoption of deformable attention remains instrumental in enhancing computational efficiency.
Multi-layer predict-update Transformer Different from approaches employing multiple iterations, LBM employs multi-layer Transformer architecture, which enhances inference efficiency while

---

Detected
sample
points
point
sample
tracking
outlier
+ 22
Adi points
Remove outliers
points
outler
d
Tracked
Frame
Frame
Correspondence
Frame 2
Figure 4: LBM framework for object tracking. LBM is initialized by sampling multiple fine- grained pixels within the target boxes, matches with pixels, and dynamically prunes outliers while replenishing with newly sampled inliers.
maintaining high tracking accuracy. Each Transformer layer comprises a prediction step and an update step discussed earlier. As the depth of the Transformer layers increases, the number of reference points in the update stage progressively decreases layer by layer, and is ultimately reduced one in the last layer, serving as the definitive reference point Flast E IR Nx2.
The final output distribution functions are fed into the track head and visibility head to predict the point position p and visibility v, respectively. Conresponding Equation 2, the track head predicts the offset Ap E RNNx of tracked points from the final reference points rjast- Following previous work Karaev al. (2024a), the confidence P E I and visibility R are predicted through coupled head. These processes are as follows:
Ap Herank (f, O, Plast),
vp.v Hois (f, o, riast)
Both heads Herack and Hais consist of a deformable attention module and an MLP layer.
Loss The composition of the loss in LBM is as follows:
L ActeCols + Creg + Luis + Leon.
(6)
The cross-entropy loss is employed to the correlation map at each layer of the Transformer as Lais = CE(c, Vg)) The offset Åp is supervised by LI loss as Creg = LI(Ap. Apyt Vgr). Only visible points are considered in the above two losses. The visibility loss and confidence loss both adopt cross-entropy losses as: as:L.se = CE(o(v), Vgt), Lcons = (E(or(), ||] Poell < 8). Please refer to Appendix A. for details.
3.3 Lattice Boltzmann model for object tracking
In object tracking. LBM takes image NxW and corresponding M detection boxes b E Mx4 as input, and associates objects in the subsequent image streams.
Matching Associating instances across consecutive time steps is realized by point-based matching, as shown in Figure 4. Compared with Zheng et al. (2024), LBM demonstrates higher efficiency by eliminating prior coarse matching within temporal windows. Specifically, upon receiving new instance, N inlier pixels are randomly sampled from the bounding box as fine-grained pixels and initialized. As a new frame arrives, the LBM predicts both positional coordinates and visibility states of the pixels. Instance association is subsequently achieved by evaluating the spatial correspondence between the predicted pixels and the inlier pixels within the bounding boxes of instances in the new frame. Please refer to Appendix A.2 for implementation details.
Update LBM'' real-time responsiveness further benefits object tracking eliminating background pixels, thereby enhancing robustness under noise. Specifically, LBM implements dynamic point management: pixels persistently residing outside target bounding boxes for consecutive frames are systematically climinated during each update cycle. Concurrently, novel inlier pixels are replenished within the current bounding box. The update mechanism effectively maintains instance represen- tation integrity while accommodating challenging target dynamicity, including partial occlusions, deformations, and background changes that usually cause tracking failures.
Experiments
4.1 Experimental setup
The critical details of the experimental setup are discussed as follows, Please refer Appendix B for more training and evaluation details.
5

---

Table 1: Real-world point tracking performance on TAP-Vid DAVIS, TAP-Vid Kinetics, and RoboTAP datasets. * denotes training on additional data. LBM reaches SOTA online performance with fewer parameters and higher efficiency, even compared with offline methods.
TAP-Vid DAVIS
TAP- Vid Kinetics
RoboTTP
Model
Params
AJ1
Bogg
OAT
AJT
OAT
AJT
OA1
I) Offiine
TAPIR Doersch et (2023)
31M
56.2
70.0
86.5
49.6
64.2
85.0
59.6
73.4
87.0
LocoTrack Cho et al. (2024)
62.9
75.3
87.2
52.9
66.8
85.3
62.3
76.2
RooISTAPIR Doensch al. (2024)
78M
61.4
73.6
88.7
54.6
68.4
86.5
64.9
80.1
86.3
CoTracker3 Karaey et al. (2024a)
25M
63.3
76.2
88.0
53.5
66.5
86.4
59.9
73.4
87.1
CoTracker3" Karaev al. (2024a)
25M
64.4
76.9
91.2
54.7
67.8
87.4
64.7
78.8
90.8
2) Window-bused Online
PIPs Harley et al. (2022)
29M
42.2
64.8
77.7
31.7
53.7
72.9
PIPs++ Zheng al. (2023)
73.7
63.5
63.0
CoTracker Karaey et (2024b)
45M
61.8
76.1
88.3
49.6
64.3
83.3
58.6
73.4
87.0
TAPTR et al. (2024b)
42M
63.0
76.1
91.1
49.0
64.4
85.2
60.1
75.3
86.9
TAPTRv2 (2024a)
63.5
75.9
91.4
49.7
64.2
85.7
60.9
74.6
87.7
SpatiaTrracker Xiao al. (2024)
34M
61.1
76.3
89.5
50.1
65.9
86.9
Co'racker3 Karaev et (2024a)
25M
64.5
76.7
89.7
54.1
66.6
87.1
60.8
73.7
87.1
CoTracker33 Karaev al. (2024a)
25M
63.8
76.3
90.2
55.8
68.5
88.3
64.7
78.0
89.4
3) Online
DynOMo Seidenschwarz al. (2025)
45.8
63.1
81.1
MFT Neoral (2024)
47.3
66.8
77.8
60.4
12.7
Online TAPIR Doersch al. (2023)
31M
56.7
70.2
85.7
51.5
64.4
85.2
59.1
DOT Le Moing et al. (2024)
53.5
67.8
85.4
45.3
58.0
81.4
51.9
62.9
79.9
Track-On Aydemir (2025)
78.0
90.8
76.4
LBM (ours)
18M
65.1
77.5
89.5
53.4
66.9
86.1
61.4
75.8
87.4
Training Consistent with previous works Doersch et (2022); Karaev et al. (2024b), LBM uses TAP-Vid Kubric Greff et al. (2022) dataset for training, which contains 11k video sequences of 24 frames each. The training process encompasses 150 epochs (approximately 100k iterations) using NVIDIA H800 GPUs with total batch size of 16 and FP16 mixed- -iexeddprecisio..
Evaluation datasets Three real-world point tracking benchmarks employed, including TAP-Vid DAVIS, TAP-Vid Kinetics, and RoboTAP Vecerik et al. (2024). Open-world object tracking datasets include: TAO Dave al. (2020) validation set, BFT Zheng et al. (2024) test set, and OVT-B Liang and Han (2024).
Evaluation metrics For point tracking, evaluation adheres TAP-Vid benchmark Doersch et al. (2022), comprising average Jaccard (AJ), &a and occlusion accuracy (OA). The evaluation metrics for open-world object tracking include TETA Li et al. (2022) and OWTA Liu et al. (2022). TETA is a comprehensive metric assessing association accuracy (AssA), localization accuracy (LocA), and classification accuracy (CIsA). The object categories are divided into novel and base.
4.2
Main results
Point tracking performance evaluation summarized in Table 1. State-of-the-art (SOTA) methods categorized into three classes: offline, window-based online, and online. Offline methods ingest full video sequences as input, window-based approaches process temporal segments of or 16 frames Karaev et al. (2024b), while online methods exclusively utilize the current frame with per-frame inference, achieving optimal responsiveness and demonstrating strong practicality for real-world deployment. LBM achieves SOTA performance with exceptionally lean parameter configuration (18 M). surpassing most existing window-based online and offline methods. Notably, LBM reaches the SOTA performance with only 37% parameters compared with Track-On. As quantitatively validated Figure 2, L.BM demonstrates real-time operational capability at 14.3 FPS on the NVIDIA Jetson Orin NX Super edge platform, with 3.9x speed advantage over Track-On, thereby showing computational efficiency and practicality in real-world environments.
Object tracking performance of LBM and other SOTA methods is systematically compared in Table 2 for TAO, Table for OVT-B, and Table for BFT. The evaluated models are categorized into two paradigms based training strategies: additional training with trackers fine-tuned with

---

Table 2: Real-world object tracking performance TAO validation dataset. Without training on domain-specific data of object tracking, LBM demonstrates state-of-the-ar performance.
All
Now!
TETA!
LocAt
AssocAT
ClsAt
AssocA
1) AdAdditional training
Trackirre Dergmann et (2019)
28.0
69.0
22.8
12.1
23.3
47.4
20.5
17.0
22.7
46.7
2.2
DeepSCIKE Woyke et (2017)
26.0
48.4
12.5
12.1
26.9
47.1
15.8
17.7
21.1
464
14.7
2.3
UNINEXT Yan at. (2023)
31.9
43.4
35.5
17.1
ADA Da (2021)
25.3
23.4
30.6
21.9
QDTxcc Pung (2021)
30.0
90.5
27.4
27.1
45.6
24.7
11.0
22.5
42.7
26,4
0.4
TEler (2022)
(2023)
27.8
GIIFF-PPu Wi (2024)
MASA Li (202
46.3
63.8
44.1
28.9
47.0
66.0
44.5
30.5
40.8
64.4
17.0
SLAck ei al. (2024t)
41.1
56.3
41.8
25.
OVTrack Liung and Han (2024)
38.4
57.5
40.8
16.9
39.2
57.5
41.0
18.9
32.5
57.0
38.7
1.8
7) Training-free
(2016)
14.3
Rengmann
ByneTrack Zhang (2022)
OC-SORT Cao et al. (2023)
28.5
49.7
21.8
28.9
51.4
19.8
13.4
23.7
49.6
20,4
Netllnck Zheng et al. (2024)
33.0
45.7
28.6
24.8
32.5
51.3
33.0
13.3
LBM (ours)
45.3
70.0
32.4
33.4
46.5
69.9
33.2
36.4
36.1
70.8
26.2
11.4
Table Real- world object tracking performance on OVT-B dataset. Without training on domain- specific data of object tracking, LBM demonstrates state-of-the-ard performance.
All
Base
Novel
Model
TRTAi
LocAf
AssocAT
CBAT
TETA
LocAt
AssocAT
C&AT
TETA
LocA
AswaA11
C&AT
1) Additional
OVTrack (2023)
134
66.1
OVTrnck+ lang Han (2034)
13.2
2) Praining-free free
ByteTrack Zhang at (2022)
20.6
35.6
12.7
13.4
19.5
36.6
12.0
10.3
20.1
36.1
12.4
11.9
OC-SORT Can al. (2023)
16.5
31.0
4.4
14.3
15.4
31.4
4.3
10.3
16.0
31.2
4.3
12.1
StongSOKT Du
31.4
12.2
supplementary data and tracking an- notations, e.g., TAO training set, YTVIS Yang et al. (2019)). and training-free with models operating without leveraging domain-specific tracking supervision. LBB estab- lishes SOTA performance and demon- strates methodological universality, outperforming both non-finetuned ap- proaches and domain- specifice finetun- ing strategies. On the TAO bench- mark, LBM achieves comparable per- formance to MASA. Notably, when processing identical detection inputs as GLEFE-Plus, LBM delivers statisti- cally significant +4.2 gains TETA. LBM achieves best performance on the OVT-B benchmark with +9.2 TETA gain over SOTA OVTrack+. These cross-domain advancements substantiate LBM's zero-shot generalization capacity without dataset-specific adaptation. Further validating operational robustness, LBM attains 74.5
Table 4: Real-world dynamic object tracking performance on BFT dataset.
Model
OWTAT
D. Re:t
A. Acc."
1) Finetuned on BFT train set
CenterTrack Zhou et al. (2020)
61.6
70.5
$4.0
FairMOT Zhang et al. (2021)
40.2
57.5
28.2
TransTrack
(2020)
TrackFormer Meinhardt
(2022)
67.4
74.5
61.1
TransCenter Xu et al. (2022)
63.5
73.2
55.3
Zero-shot setting
StrongSORT Du et al. (2023)
43.2
54.7
34.2
SORT Bewley et al. (2016)
59.9
63.9
56.2
1OUTracker Bochinski et (2017)
70.9
77.4
65.0
Byte Track Zhang
(2022)
60.5
OC-SORT Cao et al. (2023)
69.0
70.9
67.2
NerTrack Zheng et al. (2024)
72.5
80.7
65.2
(ours)
OWTA on the BFT benchmark track highly dynamic objects, surpassing NetTrack by +2.0 OWTA gain, demonstrating the ability to track highly dynamic objects.
4.3 Ablation study
Collision and streaming Module ablation is shown in Table 5, which shows 1) removing the streaming module eliminated historical distribution, thereby erasing temporal context and making it susceptible to abrupt changes in dynamic pixel distributions, resulting in a 0.9 AJ degradation; 2) disabling the collision module deprived pixels of neighborhood distribution, causing locality constraints. Compared to streaming module removal, this incurred 1.2 OA reduction, indicating heightened vulnerability occlusions; 3) both modules contributed to performance gains the cost of increased parameters and approximately 1l ms latency the NVIDIA Jetson Orin NX.

---

Table 5: Module ablation of LBM on TAP-Vid DAVIS benchmark. The speeds are tested on an NVIDIA Jetson Orin NX super.
Modules
Streaming
Collision
Params
Speedt
AJ1
Oang"
OA1
64pe9
6
6480
8 1
17.8 M
14.3 FPS
65.1
77.5
89.5
46.8
70.2
84.9
91.3
94.6
15.4 M
17.1 FPS
64.2
77.0
89.2
46.0
69.7
84.7
90.7
94.1
14.7
17.8 FPS
63.6
76.9
88.3
46.4
69.7
83.9
90.6
93.9
X 1244
51.8
66.2
77.0
59.5
71.3
82.6
Table 6:
Ablation on number of predict- update layers. 3 predict-update layers achieve better efficiency on TAP-Vid DAVIS.
Table 7:
Ablation on the visual encoder.
ResNet18 obtains better efficiency compared
with Swin-T on TAP-Vid DAVIS.
Naayer
Purams
Speed
Alf
OA1
Encoder
Params
Speedt
AJf
OAt
2 14.7
18.5 FPS
64.2
77.3
89.3
3 17.8 M
14.3 FPS
65.1
77.5
89.5
Swin-T
27.3 M
8.9 FPS
63.0
76.6
88.7
4
21.0
11.2 FPS
64.9
77.3
89.6
ResNer18 17.8
14.3 FPS
65.1
77.5
89.5
Transformer layer The number of predict-update layers in the Transformer is discussed in Table 6. Each predict-update layer contains approximately 3.1 M parameters and introduces a computational latency of 18 ms. Compared to the 2-layer architecture, the 3-layer model demonstrates a performance gain of +0.9 AJ metric. However, the 4-layer configuration shows negligible improvement over its 3- layer counterpart, indicating the existence of performance saturation in deeper network configurations for point tracking. Therefore, LBM adopts the 3-layer configuration as the default architectural setting, achieving optimal balance between computational efficiency and model performance.
Visual encoder As shown in Table 7, we substituted ResNet18 with Swin-T Liu et al. (2021) while maintaining identical implementation protocols: utilizing ImageNet pre-trained weights, extracting hierarchical features from blocks with stride configurations of [4, 8, 16], and spatially aligning these multi-scale representations through convolutional projection layers to stride=4 followed by channel-wil -wiee concatenation. Despite introducing 9.5 additional parameters and incurring 42 computational overhead, the architectural substitution demonstrated 2.1 AJ metric degradation compared to the ResNet18 8 baseline, highlighting the non-trivial performance trade-offs. The observed performance discrepancy could be attributed to ResNet's superior capability in preserving spatial integrity, particularly through enhanced spatial alignment during hierarchical
Spatial awareness CoTracker proposes expanding the number of queries enhance the model's spatial awareness. Therefore, Track-On, CoTracker3, and CoTracker employ initialized K K grid points extended queries. However, increasing query number typically compromises the model's inference speed, especially in real-world applications. In contrast, LBM benefits from learning the pixel collision process and inherently possesses stronger spatial perception capabilities, illustrated in Figure 5. Without additional extended queries, the performance improvement of LBM less pronounced compared other methods (+0.1 against -1.7, -1.6, and -1.8 on AJ metric), demonstrating its better spatial awareness.
Dynamic object tracking BFT benchmark is shown in Figure 6. The tolerance timeout denotes the maximum frames allowed for allocated pixels to be outliers. higher timcout slows pixel updates, reduces fine-grained dynamicity, and hence degrades tracking performance. A timeout of 1 frame induces excessively low tolerance and influences the stability fine-grained pixels. Therefore,
OWTTA
6
OA
Tel timenut.
0.3
extend
3
5
7
9
11
13
16
17
10
76
+B.1
-03
OWTA
AJ
extend
5
64
6
Allocated
LBM (ours)
Track-On
CoTracker3
CoTracker
Figure 5: Ablation on extended queries. LBM benefits from spatial awareness and reduces de- pendency on extended queries for efficiency.
Figure 6: Ablation on LBM for object track- ing. The tolerance timeout and number of allo- cated pixels are taken into account.
8

---

Tracked pixel
Deform
Advazice
Figure 7: Dynamic neighbor visualization. A single pixel is tracked, and its neighbors are visualized. The tracked pixels are marked by red circles.
Detection success
Detection success
Detection success
Detection failure
Detection recovered
Finc grained association
Remove outliers Add new points
Rnvironment change
Fine-groined ussociation
1D:0
ID:0
ID:0
10:0
Initialized
Tracked
Tracked
Kept
Tracked
Figure 8: Object tracking visualization. LBM achieves robust object tracking by learning the dynamic pixel trajectories of the object, effectively mitigating the issue of detection failure.
timeout of frames is set by default. Furthermore, tracking performance improves with increased pixel allocation per object, but plateaus beyond 16 pixels.
4.4 Efficiency
Efficiency comparison NVIDIA Jetson Orin NX super (16 GB) is shown in Figure 2. The speed of online and semi-online models is evaluated on TAP- Vid DAVIS, and the runs of TAPTRv2, TAPTR, and CoTracker2 fail due insufficient resources. LBM demonstrates higher efficiency due to its lightweight architecture and inference speed. Although CoTracker3 adopts a window-band online structure capable of processing 16 images in single pass, its inference speed remains significantly slower than LBM under resource constraints of edge devices, while still suffering from semi-online latency. None of the models employs extended queries in this comparison for efficiency.
TensorRT quantization To further mitigate deployment complexity in real-world applications, LBM compiled into an ONNX format to enhance cross-platform deployment compatibility. Furthermore, FP16 quantization via TensorRT was implemented to accelerate inference on widely adopted embed- ded systems such the NVIDIA Jetson series. After quantization, the quantized model realizes x3.5 acceleration with 49 FPS.
4.5 Visualization
Dynamic neighbors Benefiting from multi-scale deformable attention, the tracked pixels learn from dynamic neighboring regions during collision process, thereby enhancing spatial perception capabilities. The dynamic neighbors are visualized in Figure 7. This enables LBM to maintain robust target tracking even when dynamic objects undergo deformation and fast motion. As the camera advances toward the target, the enlarged target scale stabilizes the appearance of tracked pixels, while observable dynamic neighbors demonstrate enhanced spatial aggregation characteristics and concentrate adjacent spatial regions.
Tracking against detection failure As shown in Figure LBM demonstrates tracking robustness against detection failures. During initialization, the object detection yields favorable results, and fine-grained pixels are sampled from the bounding box. Over the subsequent frames, the fine-grained pixels are tracked, with their trajectories utilized for association. During this process, pixels that extend beyond the bounding box are removed, while new pixels are sampled within the bounding box. When detection fails due to environmental changes, the tracking method typically also fails, which is detrimental to real-world applications. In contrast, LBM can persistently track the fine-grained pixels of the object, maintaining robust tracking once detection recovers. The detection results are provided by YOLOE- 1-L Wang et al. (2025).

---

#
# 20
30
40
Start
#
#
30
Stort point
Stort point
Figure 9: LBM real-world applications: behavioral analysis of zebrafish. Given two inde- pendent multi-view videos, LBM enables dicensiinnsional trajectory reconstruction of zebrafish, facilitating quantitative behavioral analysis.
4.6 Real-world application
Figure 9 shows the behavioral analysis of zebrafish with target gene knockout. Utilizing two orthogonal perspectives (top and lateral views). the LBM enables researchers to reconstruct three- dimensional trajectories of zebrafish swimming behavior induced by pipette transfer into a container. The quantitative analysis revealed that with the specific gene knockout, zebrafish exhibited pronounced rotational swimming patterns, thereby demonstrating LBM's practical utility in quantifying complex biomechanical phenotypes.
Limitations and Future Work
While LBM achieves efficient learning of real-world pixel dynamicity and demonstrates effective- in both point tracking and object tracking tasks, certain limitations persist. In point tracking applications, the collision-streaming processes remain constrained by inherent locality, leading to discontinuity issues in long-term tracking. Regarding object tracking. the current random sampling within bounding boxes exhibits vulnerability to background interference, which could potentially be mitigated by employing instance segmentation masks instead of conventional detection frameworks in future implementations. From a practical perspective, LBM shows promising potential for integration with embodied tracking tasks, where its computational efficiency and practical applicability could be further exploited through synergistic system development. Future research directions should prioritize addressing these identified constraints while exploring novel application domains.
6
Conclusion
This work presents the LBM, a novel framework for real-time pixel tracking. By decomposing visual objects into dynamic pixel lattices and solving motion states through collision-streamiin processes, LBM achieves efficient, iteration-free tracking with the multi-layer predict-update architecture. Comprchensive evaluations point tracking and open-world object tracking benchmarks demonstrate SOTA performance in both accuracy and efficiency. Notably, the fine-grained pixel tracking LBM alleviates detection failure challenges inherent in object tracking applications. The lightweight design of LBM establishes new possibilities for real-world deployment in animal behavior analysis and future embodied tracking systems. LBM extends the paradigm of physics-inspireds visual tracking, offering practical utility in dynamic real-world perception.
Acknowledgments
This work is supported by GRF 17201025, GRF 17200924, NSFC-RGC Joint Research Scheme HKU705/24, and the Natural Science Foundation of China (62461 160309). This work is also supported in part by the National Natural Science Foundation of China under Grant 62173249 and the Natural Science Foundation of Shanghai under Grant 20ZR 1460100.
10

---

References
Aydemir. G.. Cai X.,, Xie. W., & Güney. F. (2025) Track-on: Transformer-based online point tracking with memory. In International Conference Learning Representations pages 1-23.
Bergmann P., Meinhardt. T., & Leal-Taixe, L. (2019) Tracking without hells and whistles. In Proceedings the IEEE/CVF International Conference on Computer Vision pages 941-951.
Bewley. A., Ge Z.. Ou, L., Ramos, F., & Uperoft, (2016) Simple online and realtime tracking. In IEEE International Conference on Image Processing pages 3464-3468. 3. 2,
Bochinski, E.,, Fiselein V., & Sikora T. (2017) High-speed bocking-by-deteeee without using image information. IEEE International Conference on Advanced Video and Signal Based Surveillance pages 1-6.
Cao J., Pang J. Weng, X., Khirodkar, R., & Kitani K. (2023) Observation-centie sort: Rethinking sort for robust multi-object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pages 9686-9696. 7,24
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov A., & Zagoruyko, S. (2020) End-to-end object detection with transformers. In European conference computer vision pages 213-229. Springer. 2,
Cho S., Huang. J.. Nam J., An H., Kim S., & Lee J.-Y. (2024) Local all all-pair correspondence for point tracking. In European Conference Computer Vision pages 306 325, Springer. 2, 6
Dave, A., Khurana, T., Tokmakov P., Schmid, C., & Ramanan D. (2020) Tao: A large-scale benchmark for tracking any object. European Conference on Computer Vision pages 436-454. Springer. 22
Dendorfer, P., Rezatofighi, , H., Milan A., Shi J., Cremers D., Reid, I., Roth S., Schindler K., & Leal-Taixé, L. (2020) Mor20: A benchmark for multi object tracking in crowded scenes. arXiv preprint arXiv:2003.0900
Deng J., Dong. W., Socher, L. K.,, Fei-Fei, L. (2009) Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pages
248-255. IEEE. 3
Doersch, C., Gupta A., Markeeva, L., Recasens A., Smaira, L., Aytar, Y., Carreira J., Zisserman, A., Yang. Y. (2022) Tap-vid: A benchmark for tracking any point in a video. Advances in Neural Information
Processing Systems 18:1100011666. 2,
Doersch, C., Yang. Y., Vecerik, M., Gokay, D., Gupta, A., Aytar Y., Carreira J.,, & 7isserman, A. (2023) Tracking any point with per-frame initialization and temporal refinement. In Proceedings of the IEEE/CVF International Conference on Computer Vision pages 10061-10072. 2,3,
Tapir:
Doersch, Luc, P., Yang Y., Gokay D., Koppula, S., Gupta , A., Heyward J., Rocco, I., Goroshin. R.. Carreira, J., others (2024) Bootstap: Bootstrapped training for tracking-any-poin. Asion Conference on Computer Vision pages 3257-3274. 6
Du F., Xu, B., Tang. J., Zhang Wang F. & Li, H. (2021) 1st place solution eccv-tao-2020: Detect and represent any object for tracking. arXiv preprint arXiv:2101.0000 7
Du Y.. Zhao Z., Song Y., Zhao Y.. Su, F., Gong, T., & Meng (2023) Strongsort: Make deepsort great again. IEEE Transactions on Multimedia 7
Greff, K.. Belletti F., Beyer, L., Doersch, C., Du Y., Duckworth, D., Fleet D. J., Gnanapragasam D.,
Golemo, F., Herrmann, C., & others (2022) Kubric: A scalable dataset generator: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pages 7749-3761. 6
Gupta, A., Dollar P., & Girshick, R. (2019) Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition pages 5356-5364, 22
Harley. A. W., Fang Z., & Fragkiadaki, K. (2022) Particle video revisited: Tracking through occlusions using point trajectories. In European Conference Computer Vision pages 59-75. Springer. 2,
He K., Zhang, X., Ren S., & Sun J. (2016) Deep residual learning for image recognition. In Proceedings of the IEFE Conference Computer Vision and Pattern Recognition pages 770-778.
Karaev N., Makarov I., Wang J., Neverova, N., Vedaldi, A., & Rupprecht, C. (2024. a) Cotracker3; Simpler and better point tracking by baeudo-labelling real videos. arXiv preprint 2rX.v:2410.11831 5,
11

---

Karaev, N.. Rocco, I., Graham, B., Neverova N., Vedaldi A., & Rupprecht, C. (2024. b) Cotracker: It is better to track together. European Conference on Computer Vision pages 18-35. Springer. 22
Le Moing. G., Ponce, J., & Schmid, C. (2024) Dense optical tracking: Connecting the dots. In Proceedings the IEEF/CVF Conference Computer Vision and Pattern Recognition pages 19187-19197.
Li H., Zhang H., Liu, S., Zeng Z., Li F., Li, B., Ren T., & Zhang L. (2024. a) Taptrv2: Attention- based position update improves tracking any point. Advances Neural Information Prucessing Systems 37: 101074-101095. 2,6
Li H., Zhang H., Liu S., Zeng Ren, T., Li, & Zhang L. (2024. b) Taptr: Tracking any point with transformers as detection. In European Conference on Computer Vision pages 57-75. Springer. 2,
Li S., Danelljan, M., Ding, H., Huang, T. E.E & Yu, F. (2022) Tracking every thing in the wild. In European Conference on Compuder Vision pages 498-515. Springer. 6, 2,
Li S., Fischer T. Ke L., object tracking. In Proceedings the IEFE/CVF Conference Computer Vision and Pattern Recognition pages 5567-5577. 3,7
L.,, Ding, H.. Danelljan M. & Yu, F. (2023) Ovtrack: vocabuacay multiple
Li S., Ke. L., Danelljan M., Piccinelli Segu M., Van Gool, L.,, & Yu, F. (2024. Matching anything by segmenting anything. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pages 18963-18972 3,7
Li, S., Ke L., Yang, Y.H.., Piccinelli, L., Segü, M., Danclljan, M., & Gool, L. V. (2024. d) Slack: Semantic, location, and appearance aware open-vocabulary tracking. In European Conference on Computer Vision pages 1-18. Springer.
Liang, H. & Han, R. (2024) Ovt-b: A new large-scale benchmark for open- vocabuccyylarry multi-object tracking. Advances Neural Information Processing Systems 37:14849 14863. 6, 7,
Liu, Y., Zulfikar, B., Luiten J., Dave A., Ramanan, D., Leibe B., OSep A., & Leal-Taixé L. (2022) Opening up open world tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition pages 19045-19055. 6,22
Z.. Lin Y., Cao Y., Hu H., Wei Zhang Z.. Lin S., Guo B. (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference computer vision pages 10012-1003 8
Meinhardt, T., Kirillov A., Leal-Taixe L., & Feichtenhofer, C. (2022) Trackformer: Multi-object tracking with transformers. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition pages 8844-8854. 3,7
Mohamad A. (2011) Lattice boltzmann method, 70. 70: Springer. 2
Neoral M., Serych, & Matas J. (2024) Mft: Long-term tracking of every pixel. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision pages 6837-6844 6847. 3,
Pang. J., Qiu, I.., Li, X., Chen I., Q., Darrell, T., Yu F. (2021) Quasi-dense similarity learning for multiple object tracking. In Proceedings of the IEEF/CVF Conference on Computer Vision and Patter Recognition pages 164-173.
Schmidt, A., Mohareri, O., DiMaio S., Yip, M. C., & Salcudean, S. E. (2024) Tracking and mapping in medical computer vision: A review. Medical Image Analysis page 103131.
Seidenschwarz. J., Zhou Q. Duisterhof, B. P., Ramanan, D., & Laii- L.. (2025) DynOMo: Online point tracking by dynamic online monocular gaussian reconstruction. In Proceedings the International Conference on 3D Vision pages 1-17.
Sun P.,. Cao J., Jiang Y., Zhang, Xie E.. Yuan Z.. Wang C., Luo (2020) Transtrack: Multiple object tracking with transformer. arXiv preprint arXiv: 15460 3,7
Vecerik, , M., Doersch, C., Yang. Y., Davehev T,, Aytar, Y., Zhou, G., Hadsell R., Agapito, L., & Scholz, J. (2024) Robotap: Tracking arbitrary points for few-shot visual imitation. In IEEE International Conference on Robotics and Automation pages 5397-5403, IEEE, 6, 22
Wang A., Liu L.. Chen H., Lin, Z., Han J., & Ding G. (2025) Yoloe: Real- file-time seeing anything. arXiv preprint ar 2774452533.074
12

---

Wen, C., Lin X., So J., Chen K., Dou, Q. Cao Y., & Abbeel P.. (2024) Any-point trajectory modeling for policy learning. In Proceedings of Robotics: Science and Systems
Wojke. N., Bewley. A., & Paulus, D. (2017) Simple online and realtime tracking with deep association metric. In IEEE International Conference Image Processing pages 3645-3649) 7
Wu. J., Jiang, Y., Liu, Q., Yuan, Z., Bai X.. & Bai S. (2024) General object foundation model for images and videos at scale. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition pages 3783 3795. 3,7
Xiao Y.. Wang. Q.. Zhang S., Xue ..., Peng S... Shen Y., Zhou, X. (2024) Spatialtracker: Tracking any 2d pixels 3d space, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pages 20406-20417.
Xie E., Wang W. Z. Anandkumar, A., Alvarez M., Luo P. (2021) Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems 34:12077-1200
Xu, Y., Ban, Y., Delorme, G., Gan C., Rus, D., & Alameda-Pineda. X. (2022) Transcenter: Transformers with dense representations for multiple-object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 7,7
Yan B., Jiang.
Y.. Wu J., Wang D., Luo, P., Yuan Z., & Lu, H. (2023) Universal instance perception as object discovery and retrieval. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition pages 15325-15336. 7
Yang. L.. Fan, Y. & Xu N. (2019) Video instance segmentation. In Proceedings the IEEE/CVF International Conference on Computer Vision pages 5188 5197.
Yu, F., Chen H., Wang X., Xian W., Chen Y., Liu F. Madhavan, V., & Darrell, T. (2020) Bdd100k: A diverse driving dataset for heterogeneous multitask learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pages 2636-2645.3 3
Zhang L., Gao J., Xiao Z., Fan, (2023) Animaltrack: benchmark for multi-animal tracking in the wild. International Journal of Computer Vision 131(2):496-11.. 24
Zhang Y., Wang C., Wang X., Zeng W., Liu W. (2021) Fairmot: On faimess detection and
re- re-identification in multiple object tracking. International Journal of Computer Vision 129:3069-3088 7
Zhang Y.. Sun, P., Jiang. Y.. D., Weng F., Yuan Luo P., Liu, W., Wang (2022) Bytetrack:
Multi-object tracking by associating every detection box. European Conference on Computer Vision pages 1-21. 7, 24
Zhang Z., Zheng G., Ji X., Chen G., Jia R., Chen W., Chen G., Zhang L., & Pan J. (2024) Understanding particles from video: Property estimation of granular materials via visuo-haptic learning. IEEE Robotics and Automation Letters
Zheng. G., Lin S., Zuo H.,, Fu, & Pan J. (2024) Nettrack: Tracking highly dynamic objects with net. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition pages 19145-19155 3, 5, 7,22
Zheng. Y., Harley A. W., Shen B., Wetzstein. G., & Guibas L. J. (2023) Pointodyssey: A large-scale synthetic dataset for long-term point tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision pages 19855-19865. 6
Zhou, X., Koltun V., & Krähenbühl, (2020) Tracking objects points. In European Conference on Computer Vision pages 474-490.
Zhu X., Su, W., Lu end-to-end object detection. In International Conference on Learning Representations
Wang X., & Dai J. (2020) Deformable detr: Deformable transformers for
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Abstract and introduction claim that the lattice Boltzmann model (LBM) is proposed for learning to track real-world dynamic pixels.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

The paper proposes an efficient pixel tracking model and does not include theoretical results.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The implementation details are provided in Section 1. The code for training and evaluation is provided. The data for training and evaluation is public.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code for training and evaluation is provided. The links public datasets are also provided.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
oniioonooolloooo icy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. cctpublic//aeccccccccccccccc /gwpi/gg///////eeee// /Ce///eeeubbiiic wttsSuubbssiion.. ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The code for training and evaluation is provided. The implementation details is provided in Section 4.1.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Section
The results to validate the efficiency of this work is provided mainly in
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The details of used computation resources are provided Section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: //tpaar////////s/////e cc/
http///trtp..ccpplliiil////c///t c/Ethi MaGuuideliness"
Answer:
[Yes]
Justification: This research complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The broader impacts are discussed in the Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper provides a pixel tracking model, and safeguards are not applicable.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The use of assets is mentioned and properly respected.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswi wwwode.hcoe...c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
18

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: There is document for demo, data preparation, training, and evaluation alongside the provided code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
19

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The method does not involve LLMs any important, original, non- standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standarr components.
Please refer to LLM policy (https: //tpuriee..p///e/ee ww/Confeeeeces////2//eeee /LLM) for what should or should not be described.
20

---

A
Detailed architecture of LBM
A.1 LBM for Point tracking
ResBlock 2
[R T2""T6"
Conv
RR"16"
Upx 4
d. h.w
ResBlock 1
Conv
Upx 2
3xxbow
ResBlock 0
B
A.A.
Cony
d. R 2
Conv
E T
a) Backbone
NxMgod
| kv
kv
Multi-head attention
Multi-head
uxd
attention
b) LBM predict in multi-layer Transformer
f R Bxd
E Rxa
19 Multiscale deform attn
Multi-head attention
BRxdd
f ig Mkd
Correlation
TopK
E R
E
RAXKXZ
MLP
Py BNNx
LBM update in multi-layer Transformer
f go 9
R Vod
O R 7
Finst R
W.w In
Mx2
Multiscale deform atin
MLP
Ap gxxxz
Finst E D2
W.W. Iy 2
Multiscale deform atin
MLP
{p. v) g
d) Track head
Visibility head
E BoX
gg 3 bv
Multiscale
ft(f. o. p(5) E &xd
Flast + Ap = N22
deform atin
f Collision distribution update
Figure Detailed architecture of the proposed LBM.
Architecture configuration of the LBM framework is illustrated in Figure 10. Specifically, in a) backbone module, given image the feature dimension d of the output o 256. b) LBM predict module, finit denotes the initial sampled distribution functions. N refers to the number of points. N, denotes the number of memorized streaming distributions f, and collision distributions f.- f is the predicted query distribution. For c) LBM Update module, K is the number of reference points for a query point, which decreases progressively as the predict-update layers deepen, with K = in the first layer, K = 1 in the final layer, and K = 4 the intermediate layers. Pr quantifies the positional uncertainty of reference points, enforcing the spatial confinement of reference points to neighborhoods of tracked pixels. In d) track head and e) visibility head modules, the final distribution f and reference points Flast derived from the last predict-updee predicted positional offset the pixel coordinates relative to Tlast, while v and p are visibility and uncertainty. The streaming distributions f, and collision distributions f, are initialized as zero and updated as f, {f and f. {ft(,, o,, p:
layer. Here, Ap represents the
Training loss In Section 3.2, discussed the fundamental loss components in LBM. Here, we provide further supplementation. The classification loss Let = CE(c, Cg is applied at each layer of the Transformer to supervise the correlation at each level. Specifically, Cut represents the index in the correlation map corresponding to the point's 's ground-trute position. The classification loss ensures that the correlation value the ground-truth position the highest.
regression loss freg = L1(Ap, Appt Vgt). Apgt Pst Flasts where Plast is the reference point from the last Transformer layer and Pgt denotes the ground-truth point location. In the confidence loss Lcon/ CE(r(p), 1[lpp Porll < 8), the ground-truth confidence is if the mean square error
21

---

between the predicted and ground -ouun points is within a threshold of 8 and the ground ground-truth points are not occluded; otherwise, it is O. In addition to computing the confidence of final output points, we also supervise reference points at each layer as an auxiliary constraint to regularize their positions, Le., feonf.ref = ()(() I ||| - Pot < 8), where r the reference point.
A.2 Object tracking
The similarity metric for association is derived from NetTrack Zheng et al. (2024), with the cross- temporal correspondence between the i-th tracked instance in historical observations and the j-th candidate instance in current detections being formally expressed through the following formulation:
A Aj
Nin
St min 11,
(7)
Nin Nout
where A, represents the area corresponding to the bounding box of the i-th instance. The scaling factor min A,) penalizes current detection instances with larger bounding areas, as expansive regions exhibit higher probabilities of containing more tracked pixels. In LBM, the similarity metric undergoes reweighting through multiplicative integration of detection scores and categorical labels. Specifically, the detection score sj and label 1, of current j-th instances is considered:
A A₃"
Nin
St = 83 (0.5 + 0,5 min
(8)
Nin + Nout
where da,b is the Kronecker delta function, defined such that St.,4, = 1 1, and lj are equal, and dt.d, otherwise. In this context, when the label 1, differs from 4, a penalty weight of 0.5 is imposed. This mechanism effectively suppresses detections with ambiguous class assignments and low confidence scores by applying multiplicative attenuation inconsistent label predictions. The final matching correspondence is determined through the normalized aggregation of the cross-frame similarity matrix IR N computed as the arithmetic mean of bidirectional softmax-normalized distributions along both spatial dimensions (row-wise and column-wisen M and N respectively represent the number of tracked and newly detected instances.
B Implementation details
Training details We employ the AdamW optimizer with a peak learning rate of 5 10-4 and weight decay of x 10-5, implementing cosine decay schedule with 5% linear warm-up initialization. The whole training process takes over 2 days on 4 NVIDIA H800 GPUs with batches each. LBM adopts identical data augmentation strategies as CoTracker Karaev et al. (2024b), processing input images at 384x 512 resolution while sampling 256 points per batch.
Evaluation datasets Tip-Vid DAVIS comprises 30 real-world videos sourced from the DAVIS dataset; TAP-Vid Kinetics contains 1,184 challenging real-world videos; and RoboTAP Vecerik et al. (2024) consists of 265 real-world robotic videos. Open-world object tracking datasets include: TAO Dave al. (2020) validation set, containing 988 videos spanning 330 object categories annotated at 1 frame per second; BFT Zheng et al. (2024) test set, comprising 36 videos featuring highly dynamic avian objects: and OVT-B Liang and Han (2024), a large-scale open-wort world object tracking benchmark encompassing 1,973 videos with 1,048 object categories.
Evaluation metrics AJ serves predicted positions and the accuracy of visibility predictions. K%. evaluates the position precision of visible points by calculating the average proportion of predicted positions falling within specified thresholds (1, 2, 4, 8, 16 pixels) relative to ground-truth positions. OA specifically quantifies the accuracy of visibility predictions for occluded states. The evaluation metries for open-world object tracking include TETA Li et al. (2022) under the open- open-vocabylary setting, a comprehensive metric assessing association accuracy (AssA), localization accuracy (LocA), and classification accuracy (CIsA), with rare categories in the LVIS Gupta et al. (2019) dataset designated as novel and the remaining categories categorized as base; OWTA Liu et al. (2022), holistic evaluation metric integrating open-world object detection recall (D. Re.) and association accuracy (A. Acc.). Specifically, TETA is used for validation on TAO and OVT-B benchmarks, and OWTA is evaluated on BFT benchmark.
comprehensive metric quantifying both position precision of
22

---

LIM (ours
80
Tradk-On
Co CoTrodier3
02.3 10.2
61.8
773 75.5
74.7
87.9 09.1
67.0
165
168
173
262
AJ
82ng
OA
Figure 12: Ablation on tracking manipulation of deformable objects. A subset of RoboTAP for manipulating deformable objects was selected with video IDs. In the comparative analysis, LBM demonstrated superior tracking capabilities.
Table 8: Ablation on the fine-grained similarity on TAO validation benchmark. The combination of the label penalty term and detection score weight achieves the best performance.
All
Base
Novel
Label
Score
TETA
LocA
AssocA
ClsA
TETA
LocA
AssocA
ClsA
TETA
LocA
AssocA
ClsA
45.3
70.0
32.4
31.4
46.5
69.9
33.2
36.4
36.
70.8
26.2
11.4
45.2
33.3
33.4
36.3
35.9
70.8
25.5
11.4
44.3
69.6
29.8
33.4
45.5
69.5
30.7
36.3
34,9
70.1
23.3
11.3
x
33.4
36.4
11.4
C Detailed ablation study
Ablation the number of active dynamic neighbors TAP-Vid DAVIS is illus- trated in Figure 11. Enhanced active neighbor participation improves the accuracy of col- lision process, thereby optimizing LBM's spatial localization precision for target pixels. This enhancement manifests through three compu- tational phases: 1) collision distribution acquisition, 2) collision attention, and 3) inference of tracking and visibility heads. Empirical observations indi- cate negligible GPU memory overhead and infer- ence speed degradation with increased neighbor counts. Given the plateau effect observed in preci- sion gains beyond 7 neighbors, LBM establishes neighbors the optimal configuration.
Average Jaccard (AI)
6g
64.4
64.7
62
5 rescaled
40
47.6
20
3
7
Active neighbors
Alation tracking deformable objects in ma- nipulation is shown in Figure 12. A subset com- prising data entries is extracted from RoboTAP dataset, specifically focusing on manipulation tasks involving highly deformable objects such as ropes and fabrics. This selection explicitly excluded mod- erately deformable objects, including toys and shoes. A comparative analysis is conducted on this subset across three models: LBM (18 M), Track-On (49 M), and CoTrrackrr (a semi-online method based on 16-frame window processing). For fairness, queries are not extended globally, but local extension is still performed in CoTracker3. Experimental results demonstrated that LBM achieved superior performance o: exhibiting enhanced localization accuracy attributed its strengthened spatial perception capability regarding pixel-level neighborhood relationships in deformable object manipulation. This empirical investigation reveals promising potential of LBM for applications involving deformable objects.
Figure 11: Ablation on the number of active neighbors. For enhanced visualization clarity, the segment of the Y-axis below 60 has been rescaled with a factor of 0.1. More active neighbors bring better performance.
C.1 Ablation on object tracking
Ablation on fine-grained similarity on TAO validation benchmark shown in Table 8. The label penalty term and detection score weight term in Equation 8 are comprehensively considered. As evidenced by experimental results, the label penalty term contributes + +1.4 performance gain in overall association metrics while moderately improving localization accuracy. Although isolated introduction of the detection score weight marginally enhances localization capability, it concurrently induces deterioration in association performance. Significantly, simultancous incorporation of both
23

---

Table 9: Ablation on AnimalTrack subset of OVT-B. LBM shows better tracking performance against real-world object dynamiiityy Best results shown in bold.
Novel
Model
TETA
LocA
AssocA
ClsA
TETA
LocA
AssocA
ClsA
TETA
LocA
AssocA
ClsA
SORT Bewley (2016)
18.3
22.0
ByncTrrak Zhang et al. (2022)
61.3
72.0
-00.6
71.3
60.7
68.1
37.3
76.9
62.1
71.8
45.5
62.9
SC-SORT Cao (2023)
62.1
71.9
43.3
71.2
61.4
67.9
39.4
76.9
63.3
77.8
49.1
62.9
(our))
51.6
43.6
Figure 13: Visualization of point tracking for dynamic object manipulation. LBM shows robustness against the dynamicity of deformable objects.
label penalty and detection score weight terms synergistically elevates both LocA and AssocA. This combined approach demonstrates particularly pronounced performance improvements in novel categories, which can be attributed to the complementary mechanisms between category- label penalty and detection confidence weighting that effectively address both semantic alignment and spatial correspondence challenges.
Ablation on tracking animals is shown in Table 9. Animals typically represent highly dynamic tracking targets. In addition to demonstrating the effectiveness of LBM compared SOTA trackers in tracking highly dynamic avian objects, as shown in Table 4, we further validated LBM's capability for animal tracking on the AnimalTrack Zhang al. (2023) subset of OVT-B. To ensure fairness, identical detection results from the GLEE-plus detector were employed in all experiments. The results indicated that LBM achieved optimal performance the AssocA metric compared other trackers, with particularly notable improvements in novel class tracking, demonstrating a +14.5 gain over OC-SORT. The results substantiate the effectiveness and practical utility of LBM for real-world dynamic object tracking scenarios.
D
Comprehensive visualization
D.1 Visualization of point tracking
The visualization results of the point tracking task are presented in distinet components: the robotic manipulation scenario involving deformable objects from RoboTAP illustrated in Figure 13, while dynamic scenes from TAP-Vid Kinetics are demonstrated in Figure 14. Benefiting from the learned collision and streaming processes, LBM maintains robust tracking performance even for
Figure 14: Visualization of point tracking in dynamic scenes. LBM demonstrates adaptability to dynamic environments, including scenarios involving rapid motion and viewpoint transformations.
24

---

Iarget bire
Target: dolphin
Figure 15: Visualization of object tracking for dynamic animals. LBM exhibits robustness against object dynamicity, such as rapid motion, deformation, similar objects, and occlusion.
highly deformable flexible objects. In dynamic environments characterized by rapid motion, such as first-person skiing scenarios with intense move- ment or situations involving repeated viewpoint variations, LBM exhibits remarkable adaptability. Notably, when the viewpoint temporarily loses and subsequently reacquires the target, LBM can precisely relocate tracked pixels through memo- rized streaming and collision distribution patterns, demonstrating exceptional robustness.
Figure 16: Failure cases of LBM. Tracking fail- ures can be attributed to the uniformity in target ap- pearance characteristics and discontinuous fragments vidcos.
D.2 Visualization of object tracking
The visualization results of the target tracking task, shown in Figure 15, are derived from the BFT and OVT-B datasets, respectively. Birds and dolphins, as highly dynamic targets, pose challenges including rapid motion, deformation, similar targets, and occlusion. Benefiting from the dynamic pixel management mechanism, the LBM exhibits robustness in tracking real-world dynamic objects.
E
Failure cases and potential solutions
Failure cases Figure 16 illustrates failure cases of LBM in point tracking tasks. In the first video sample, point tracking failures the desktop surface occur due its uniform appearance in both color and texture, revealing the localized nature of collision and streaming processes. In the second scenario, discontinuities arising from the composition of multiple spliced video segments result in observable point drift phenomena. For visual object tracking, the primary limitations persist in two aspects: 1) the random sampling within target bounding boxes exhibits inherent susceptibility to background interference contamination; 2) despite maintaining fine-grad grained pixel-level tracking fidelity during detection failure scenarios, the framework lacks effective mechanisms for holistic tracking state recovery at the object level, as shown in Figure 8.
Potential solutions In response to the limitations inherent in LBM, several potential solutions have been enumerated as follows:
Explicit temporal continuity mechanisms. While streaming distribution of pixels preserves temporal feature learning, the correlation for reference point acquisition in online tracking scenarios exhibits inadequate exploitation of historical positional contexts. Therefore, ex- plicit modeling of pixel trajectory persistence emerges critical enhancement opportunity,
Global semantic context augmentation. For discontinuous video sequence tracking, the semantic state coherence of associasse objects inherently governs motion pattern interpretability. Developing hierarchical architectures to extract and propagate object-level semantic embeddings could substantially improve pixel motion comprehension.
Depth-aware constraint integration. Incorporating depth-aware constraints or semantic segmentation priors could mitigate background interference during target-specific tracking, particularly through spatial attention mechanisms that discriminatively weight foreground- background sampling probabilities.
25

---

Holistic motion decomposition modules. Implementing motion composition layers that ag- gregate pixel-wise displacements into interpretable object kinematics would benefit tracking state estimation and downstream applications requiring macroscopic motion understanding.
F
Broader impacts
The proposed LBM for real-tine real-time and online pixel tracking presents both positive and negative socictal implications that merit careful consideration.
F1 Positive societal impacts
Enhanced efficiency practical applications LBM's lightweight design and edge-device com- patibilityy enable real-time tracking in resource-consstrinee scenarios. This could benefit fields like robotics and autonomous systems.
Scientific research advancement As demonstrated in zebrafish behavioral analysis. LBM's ability reconstruct 3D trajectories of dynamic objects supports quantitative studies in biomechanics and ecology. This may accelerate discoveries in genetic research or environmental monitoring.
Robustness against system failures By decomposing objects into fine-grained pixels and dynam- ically pruning outliers, LBM reduces reliance on detection results. This improves reliability in safety-critical applications like surveillance disaster response.
F.2 Negative Societal Impacts
Privacy concerns The technology's s capacity for persistent pixel-level tracking raises risks misuse in unauthorized surveillance. For instance, malicious actors could exploit LBM to track individuals across video feeds without consent.
Data bias If LBM is finetuned on data that lacks diversity, the performance could degrade for specific demographics scenarios, exacerbating fairness issues in deployed systems.
F.3 Mitigation Strategies
Strict Ethical Guidelines Deployment in sensitive domains, e.g., public surveillance, should require transparency audits and opt-in consent mechanisms.
Bias mitigation Actively curate diverse training data spanning varied motion dynamics and environ- mental contexts to minimize performance disparities.
26


Reviews:
[{"review_id": "zWKAgeAc4n", "reviewer": "Reviewer_6HeP", "summary": "This paper proposes a new method for point tracking. The proposed method, LBM, utilizes ideas from deformable attention to construct a more efficient neural network compared to prior works. Extensive experiments are provided to highlight the SOTA performance of the proposed LBM compared to online tracking methods. LBM seems particularly effective in resource constrained settings.", "questions": "Please see the weaknesses section for questions.", "limitations": "Yes.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\nThe proposed method provides promising efficiency gains over prior works, and therefore can be a valuable practical system for tracking. The paper also provides several interesting ablation studies that reflect the importance of some of its design choices.\n\n**Weaknesses:**\n\nI have the following concerns and questions about the paper:\n\n1- The paper is not easy to understand, and lacks in mathematical rigor when presenting key components such as the collision and streaming distributions (lines 144-147). It also moves the loss computation to the appendix, which limits the clarity of the main paper. Even the explanation in the appendix lacks exact mathematical definitions for various operations.\n\n2- The connection to the lattice Boltzmann method is not clear to me. In particular, I don’t understand what is the value (and relevance) of Equations 1-4 to the actual proposed method. Are these equations providing some performance guarantees, if so, how and what?\n\n3- The description of the method remains on a very high level in my opinion, not discussing some important design choices such as the selection of reference points, and collision and streaming distributions.", "was_revised": true, "final_justification": "I appreciate the clarifications provided by the authors, and I do think the paper has novelty and technical merit. However, I choose to keep my score because I think the paper needs to undergo a major revision to improve its clarity and mathematical rigor.", "rebuttal": "We appreciate your recognition of the efficiency and ablation studies, as well as your detailed comments that will help improve this work. To answer your questions more systematically, we have rearranged the questions as follows.\n\n> ### **Q1** What’s the connection between the lattice Boltzmann method and the proposed LBM? In particular, what’s the contribution of Equations 1\\~4 to LBM?\n\n**A1.1** Thank you for pointing out that the clarity in the theoretical support for LBM needs improvement. Equations 3\\~5 are the core pipeline of LBM, which are derived from Equations 1\\~2, the discrete distribution function of the lattice Boltzmann method.\n\n**A1.2** The core inspiration from the lattice Boltzmann method lies in its high computational efficiency. During streaming and collision processes, it circumvents the inherent complexity of modeling long-range particle interactions by computing interactions only between a point and its neighbors within a defined local vicinity. The proposed LBM adopts this core computational logic, but further merges computations in the neighborhood and makes temporal extension.\n\n**A1.3** Specifically, as shown in Equation 1, to update the distribution function over time, the Lattice Boltzmann Method computes the directional distribution functions of fluid particles within a single time step through streaming and collision processes. For example, in a two-dimensional scenario, an individual lattice interacts with 9 fixed neighbors, with its state represented by the distribution functions denoted as $\\mathbf{f}=\\lbrace{f_i}\\rbrace_{i=0}^{8} \\in \\mathbb{R}^9$. In the $i$-th direction, the streaming term is the distribution function of the previous time step, and the collision term describes the collision between the particle and its neighbors.\n\n**A1.4** In this work, we first represent the multi-direction distribution functions as a $d$-dimensional visual embedding $\\mathbf{f} \\in \\mathbb{R}^d$, as formulated in Equation 3. The collision process in multiple directions is combined into a **dynamic neighborhood** $\\delta$ for efficiency. Furthermore, LBM extends the **temporal context** of the lattice Boltzmann method, extending the evolution of a single time step to learning the streaming and collision processes of the past $N_s$ and $N_c$ time steps. This extension integrates the temporal evolution of distribution functions over multiple time steps, ​thereby enabling the robust modeling of long-range temporal dependencies inherent in complex dynamic scenes. In this process, the distribution functions of the previous $N_s$ time steps constitute the streaming distribution $\\mathbf{f}_s \\in \\mathbb{R}^{N_s \\times d}$, and the collision results of the previous $N_c$ time steps establish the collision distribution, denoted as $\\mathbf{f}_c \\in \\mathbb{R}^{N_c \\times d}$. We implement two distinct cross-attention modules to interact between the current distribution function and both the streaming distribution $\\mathbf{f}_s$ and collision distribution $\\mathbf{f}_c$. \n\n**A1.5** In the revised manuscript, based on the single-time-step form in Equation 3, we will further present the LBM formulation with extended temporal context, including learning from streaming distributions $\\mathbf{f}_s$ and collision distributions $\\mathbf{f}_c$ as: \n\n$\\hat{\\mathbf{f}} = \\phi_{c} (\\phi_{s} (\\mathbf{f}, \\mathbf{f}_s), \\mathbf{f}_c) \\quad ,$\n\nwhere $\\hat{\\mathbf{f}}$ is the predicted distribution function, $\\phi_{c}$ and $\\phi_{s}$ are cross-attention modules. The acquisition of $\\mathbf{f}_s$ and $\\mathbf{f}_c$ will be discussed in Q2\\&A2.\n\n> ### **Q2** How to understand the key components, such as the collision and streaming distributions?\n\n**A2.1** We acknowledge the need for enhanced mathematical rigor in our derivations, as mentioned in your feedback. The streaming and collision distribution is obtained as follows: we compose the final distribution function output by the Transformer in the previous $N_s$ time steps into the streaming distribution $\\mathbf{f}_s$:\n\n$\\lbrace\\hat{\\mathbf{f_i}}\\rbrace_{\\mathbf{i}=t-N_s}^{t-1}\\quad,$ \n\nwhere $\\hat{\\mathbf{f}}$ denotes the final distribution function acquired from the Transformer, and $t$ refers to the current time step. We compose the interaction result in the dynamic neighbors into the collision distribution $\\mathbf{f}_c$: \n\n$\\lbrace\\Omega(\\hat{\\mathbf{f_i}}, \\mathbf{o_i}, \\mathbf{p_i} | \\delta_\\mathbf{i})\\rbrace_{\\mathbf{i}=t-N_c}^{t-1}\\quad,$ \n\nwhere $\\Omega$ is the collision operator in Equation 3, implemented as multi-scale deformable attention [1]. $\\mathbf{o}$ denotes the image feature and $\\mathbf{p}$ is the estimated position of tracked points, which serves as reference points in deformable attention. $\\delta$ is the learned dynamic neighborhood. \n\n**A2.2** The distinction from the lattice Boltzmann method in Equation 1 lies in our employment of multi-time-step streaming terms and collision terms. This temporal extension facilitates the establishment of longer-term temporal dependencies, thereby enabling the robust handling of challenging dynamic scene variations​, such as deformations and illumination changes, ultimately enhancing temporal robustness.​​\n\n> ### **Q3** How to explain the mathematical definitions of operations in LBM?\n\n**A3.1**  We are grateful for your recommendations on refining the mathematical definitions of the LBM operations. The detailed explanation of LBM operations is outlined as follows:\n- Visual encoding: $\\mathbf{o} = \\mathcal{E}(\\mathbf{I})$, where $\\mathbf{o}\\in\\mathbb{R}^{h\\times{w}\\times{d}}$ is the visual features with $d$ dimensions. The detailed architecture of visual coder $\\mathcal{E}$ is shown in Figure 10-a, which includes 3 ResBlocks from ResNet18 and 3 projection layers. $\\mathbf{I} \\in \\mathbb{R}^{3\\times{H}\\times{W}}$ is the input image. The stride of the output feature $\\mathbf{o}$ is 4, *i.e.*, $h=\\frac{H}{4}$ and $w=\\frac{W}{4}$.\n- Initial sampling of pixel distribution: given $N$ point queries $\\mathbf{q}\\in\\mathbb{R}^{N\\times2}$, the initial pixel distributions of the queries is acquired by bilinear sampling from the visual features $\\mathbf{o}$, which is formulated as: $\\mathbf{f}_{init}=\\text{BilinearSample}(\\mathbf{o}, \\mathbf{q}) \\in \\mathbb{R}^{N\\times d}$.\n- Predict current distribution functions: the pixel distribution function evolves temporally, with the current time step being predicted from temporal context with streaming distributions $\\mathbf{f}_s \\in \\mathbb{R}^{N \\times N_s\\times d}$ and collision distributions $\\mathbf{f}_c \\in \\mathbb{R}^{N \\times N_c\\times d}$, which are discussed in A1.3 and A2.1 in detail. \n\n  This process can be mathematically formalized as follows: $\\hat{\\mathbf{f}} = \\phi_{c}(\\phi_{s}(\\mathbf{f}, \\mathbf{f}_s), \\mathbf{f}_c)$, where $\\mathbf{f}$ is the input distribution functions. \n- Update distribution functions with image: as illustrated in Figure 3, the predicted distributions undergo correlation operations with current image features $\\mathbf{o}_t$ at time step $t$, subsequently triggering the selection of top-$K$ response points as reference points $\\mathbf{r}\\in\\mathbb{R}^{N\\times K \\times 2}$. These points then guide the updating of distribution functions through deformable attention modules utilizing image features. This process can be formulated as: ${\\mathbf{r}}= \\text{TopK}(\\text{SoftArgmax}(\\langle {\\mathbf{f}}, \\mathbf{o} \\rangle))$, where $\\langle,\\rangle$ denotes correlation. And the updated distribution functions are: $\\mathbf{f} = \\text{MSDeformAttn}(\\hat{\\mathbf{f}}, \\mathbf{o}, {\\mathbf{r}})$.\n- Multi-layer predict-update Transformer: we employ a multi-layer Transformer architecture, where each layer implements sequential prediction and update processes. As discussed in Section 3.3, the parameter $K$ in the top-$K$ operations diminishes as the layers get deeper, and $K=1$ in the last layer denotes the final reference points $\\mathbf{r}_{last}\\in\\mathbb{R}^{N\\times2}$. The final output distribution functions $\\mathbf{f}$ are fed into the track head and visibility head to predict the point position and visibility. \n\n  Both the track head and visibility head consist of a deformable attention module and an MLP layer. The track head predicts the offset of tracked points from the final reference points $\\mathbf{r}_{last}$ as:\n\n   $\\Delta \\mathbf{p}=\\mathcal{H_{track}}(\\mathbf{f}, \\mathbf{o}, \\mathbf{r}_{last}) \\in \\mathbb{R}^{N\\times2}$. \n\n  And the visibility head estimates the visibility and confidence of tracked points as: \n\n  ${\\rho}=\\mathcal{H_{vis}}(\\mathbf{f}, \\mathbf{o}, \\mathbf{r}_{last}) \\in \\mathbb{R}^{N\\times2}$. \n  \n  The current estimated position $\\mathbf{p}=\\Delta {\\mathbf{p}}+\\mathbf{r}_{last}$.\n\n- Acquisition of streaming distributions $\\mathbf{f}_s$ and collision distributions $\\mathbf{f}_c$ are discussed in A2.1 in detail.\n\n**A3.2** We thank your advice and will systematically formalize the LBM workflow with mathematical rigor in the main text to enhance readability. \n\n> ### **Q4**  Why is loss computation moved to the Appendix?\n\n**A4.1**  We sincerely appreciate your feedback regarding the explanation of the loss function. We discuss the loss function in the Appendix primarily due to space constraints in the original manuscript. We will incorporate essential technical elaborations on the loss functions within the revised body text for better clarity.\n\n> ### **Summary**\n\nThank you again for helping us improve the paper. We hope our response can resolve your concerns. Please let us know if you have any further questions. We will be actively available during the rebuttal and discussion period. If you feel your concerns are addressed, please consider reevaluating our work. Looking forward to hearing from you!\n\n```\n[1] Zhu, Xizhou, et al. \"Deformable DETR: Deformable Transformers for End-to-End Object Detection.\" International Conference on Learning Representations. 2021.\n```"}, {"review_id": "8sPOuRz574", "reviewer": "Reviewer_pfjb", "summary": "This work introduces a novel Lattice Boltzmann Model (LBM) designed for real-time visual tracking of dynamic objects. The model decomposes visual information into dynamic pixel lattices and utilizes collision-streaming processes to analyze pixel motion effectively. It employs a multilayer predict-update network to estimate distributions of target pixels, refining estimations by managing lattice interactions during prediction and leveraging online visual features in the updating phase. The model’s performance is evaluated using benchmarks like TAP-Vid and RoboTAP, showing superior efficiency and practicality compared to traditional offline methods. LBM addresses significant challenges such as high resource consumption and latency, enhancing robustness in tracking scenarios involving occlusion and deformation.", "questions": "SEE WEAKNESS", "limitations": "SEE WEAKNESS", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengthens：\n1. The use of the Lattice Boltzmann Model to decompose visual information into dynamic pixel lattices is a novel approach that offers a new perspective on pixel tracking.\n2.The paper effectively demonstrates LBM’s ability to perform real-time tracking tasks, which is crucial for applications in resource-constrained environments.\n3. The work outlines a theoretical framework rooted in the lattice Boltzmann method for fluid simulations, bridging established physics principles with modern computational techniques, thereby enriching the understanding of pixel tracking dynamics\n4.The paper presents extensive evaluations on widely recognized benchmarks like TAP-Vid and RoboTAP, demonstrating LBM’s efficiency and practicality in both point tracking and large-scale open-world object tracking tasks\n\nWeakness\n1.The performance improvements could raise concerns regarding overfitting, especially if the model is highly tailored to the benchmarks used for evaluation. More diverse datasets would provide a clearer indication of generalizability\n2.The proposed model’s complexity may pose challenges in practical deployment, particularly in less controlled environments where computational resources are limited. Simplifying the architecture or providing detailed guidelines for implementation could enhance its adoption.\n3.Although the model is designed for resource-constrained environments, its computational requirements could still be a barrier in extremely low-resource scenarios.", "was_revised": true, "final_justification": "ACCEPTED", "rebuttal": "We sincerely appreciate your recognition of the novelty, computational efficiency, and physics-inspired principles of the LBM methodology, as well as the comprehensiveness of evaluation. We are also grateful for your insightful comments regarding generalizability, deployment complexity, and performance in low-resource scenarios. We address your questions as follows.\n\n> ### **Q1**. Does the model exhibit potential overfitting risks, and what is its generalization capability on broader datasets?\n\n**A1.1** We appreciate your concerns regarding the model's generalization capability. We will address these issues from the perspective of the heterogeneity of training and evaluation data.\n\n**A1.2** The training dataset for the LBM comprises the Kubric Movi-F benchmark generated by the Kubric [1] simulator, following CoTracker [2] and TAPTR [3]. Specifically, the dataset demonstrates a 3D rigid body falling, colliding with the ground, and rebounding. The diversity of scenarios and objects in the scalable Kubric simulator empowers LBM to achieve robust generalization across extensive real-world applications.\n\n**A1.3** The evaluation for LBM encompasses benchmarks for point tracking tasks like TAPVid-DAVIS, TAPVid-Kinetics, and RoboTAP, comprising 1,479 real-world scenarios, alongside the data for object tracking tasks, including 2,997 real-world scenarios in TAO, BFT, and OVT-B. These datasets cover diverse indoor/outdoor environments and targets, including animal activities, autonomous driving, embodied manipulation, and crowd behaviors. More importantly, as the evaluation data originates from a different domain than the simulated training data, LBM's performance on the evaluation sets ​demonstrates its strong zero-shot generalization capability.\n\n**A1.4** Following your suggestion, LBM was further evaluated on the longer-term point tracking benchmarks, *i.e.*, ​Dynamic Replica​ and ​PointOdyssey. Dynamic Replica is a benchmark designed for 3D reconstruction and comprises 20 sequences of 300 frames each. ​PointOdyssey​ features 12 videos containing thousands of frames (up to 4,325). Consistent with the TAP-Vid benchmark, position precision $\\delta_{avg}^{vis}$ of visible ​points​ is adopted as the evaluation metric. The results are shown as follows, where LBM reaches the state-of-the-art performance, especially on the long-term PointOdyssey benchmark.\n\n| Model        | Dynamic Replica | PointOdyssey |\n|--------------|:-------------------------------:|:-----------------------------:|\n| PIPs++       | 64.0                           | 32.4                          |\n| TAPIR        | 66.1                           |          -                     |\n| CoTracker    | 68.9                           | 32.7                          |\n| TAPTR        | 69.5                           |               -                |\n| LocoTrack    | 71.4                           |             -                  |\n| BootsTAPIR   | 69.0                           |             -                  |\n| CoTracker3   | 73.3                           |               -                |\n| Track-On     | **73.6**                       | 38.1                          |\n| LBM (Ours)   | 71.5                           | **41.9**                      |\n\n**A1.5** We are actively conducting experiments to evaluate the generalization capabilities of LBM across more benchmarks and tracking-related tasks. We shall further mitigates overfitting risks through training on more extensive and diversified datasets.\n\n> ### **Q2.** How to mitigate model complexity and deployment challenges in practical scenarios?\n\n**A2.1** We appreciate your insightful comments on model deployment challenges and valuable suggestions. To address your concerns, we will implement solutions from two aspects, focusing on edge device adaptation and comprehensive documentation.\n\n**A2.2** To further mitigate deployment complexity, we eliminated the dependency on CUDA-compiled multi-scale deformable attention from the MMCV library, substituting it with native PyTorch source code. Subsequently, the model was compiled into an ONNX format to enhance cross-platform deployment compatibility. Furthermore, FP16 quantization via TensorRT was implemented to accelerate inference on widely adopted embedded systems such as the NVIDIA Jetson series, as shown in the table below.\n\n| Platform | Model | FPS |\n| --- | --- | --- |\n| Orin NX | MMCV | 14  |\n| Orin NX | PyTorch | 7   |\n| Orin NX | TensorRT | 49  |\n\nCompared with the reported inference speed, the quantized model realizes a $\\times3.5$ acceleration with 49 frames per second (FPS).\n\n**A2.3** Complementing our quantization models, the open-source repository implements comprehensive documentation for LBM inference and edge deployment. This encompasses:\n\n- Dependency installation across diverse operating systems\n  \n- Conversion of open-source weights to ONNX models\n  \n- Transformation into TensorRT engines\n  \n- Inference demos for both ONNX and TensorRT formats\n  \n- Benchmarking velocity tests across heterogeneous models and devices\n  \n- Troubleshooting guides for common issues.\n  \n\nAll associated code, documentation artifacts, and deployment methodologies will be fully open-sourced.\n\n**A2.4** We thank your advice and will further streamline model inference and deployment pipelines to improve implementation in real-world application scenarios.\n\n> ### **Q3.** How can LBM be effectively deployed on edge devices with more constrained computational and memory resources?\n\n**A3.1** We appreciate your insightful comment on the more resource-constrained deployment. LBM benefits from its lightweight model architecture and online inference manner, exhibiting low GPU memory consumption that scales approximately linearly with increasing number of points, as shown in the table below. $N$ is the number of tracked points, the maximum consumed GPU memory is monitored, and the average memory consumption of a single point compared with $N=1$ is reported. The test platform is an NVIDIA Jetson Orin NX.\n\n| $N$ | 1   | 2   | 5   | 10  | 20  | 50  | 100 |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Max memory (MB) | 159 | 160 | 164 | 169 | 181 | 214 | 271 |\n| Average point memory (MB) | -   | 1.00 | 1.25 | 1.11 | 1.16 | 1.12 | 1.13 |\n\n**A3.2** We further implemented LBM with FP16 quantization via TensorRT for higher inference efficiency, as discussed in A2.2.\n\n**A3.3** We will test the efficiency of LBM on edge devices with more limited resources, *e.g*., NVIDIA Jetson Nano, to further realize broader practicality.\n\n> ### **Summary**\n\nWe appreciate your valuable comments regarding the model's generalizability and deployment considerations. We hope our responses have addressed your concerns. During the Rebuttal and Discussion period, we will promptly address any further questions you may have. Should your concerns be resolved, please consider reassessing the review of our work. We respectfully look forward to your reply!\n\n\n```\n[1] Greff, Klaus, et al. \"Kubric: A Scalable Dataset Generator.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.\n\n[2] Karaev, Nikita, et al. \"Cotracker: It is Better to Track Together.\" European conference on computer vision. 2024.\n\n[3] Li, Hongyang, et al. \"TAPTR: Tracking Any Point with Transformers as Detection.\" European Conference on Computer Vision. 2024.\n```"}, {"review_id": "IRdiMtufPM", "reviewer": "Reviewer_Ycni", "summary": "This paper proposes using a Lattice Boltzmann Model to model the dynamics of pixels in videos, thereby enabling point tracking and object tracking tasks. Specifically, the authors use a predict-update network to estimate the positions and visibility of pixels. The predict stage simulates particle collisions, while the update stage refines the estimation using online features. Experimental results demonstrate the effectiveness of the proposed method, and the authors conduct extensive ablation studies to validate the contribution of each module.", "questions": "How does the proposed method perform on traditional object tracking datasets such as GOT10k, LaSOT, and TrackingNet? Is its performance consistent with that on TAO, OVT-B, and BFT?", "limitations": "The authors have already clearly listed the limitations in the main paper.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths \n1. The use of a Lattice Boltzmann Model to model the dynamics of pixels in videos proposed by the authors is novel and effective.\n2. The proposed method has significant advantages over current SOTA methods in terms of the number of parameters and running speed.\n3. The proposed method can perform both point tracking and object tracking, and its generalization performance on object tracking is significantly higher than that of other methods.\n### Weaknesses\n1. The authors use a relatively small network (ResNet18) for feature extraction. Have they tried using deeper or larger networks? Besides the increased computational cost, would this also improve performance?\n2. What is the motivation for using a transformer architecture in the Predictor and Updater modules? What would happen if CNNs were used instead?", "was_revised": true, "final_justification": "The idea of using Lattice Boltzmann Model to model the dynamics of pixels in videos is interesting and novel. The experimental results support the effective of the proposed method. The authors carefully addressed my concerns on the motivation of using a shallow CNN for feature extraction and a Transformer for the Predictor and Updater modules. Therefore, I recommend accepting it.", "rebuttal": "We sincerely appreciate your positive feedback regarding the novelty, efficiency, and generalization capabilities of the LBM. We also thank you for raising inquiries concerning the use of larger models, Transformer architectures, and the performance on traditional object tracking benchmarks. Your concerns are addressed as follows, with structured responses provided accordingly.\n\n> ### **Q1.** Will a deeper and larger feature extraction network improve performance?\n\n**A1.1** We appreciate your attention to the relationship between model parameter count and performance, along with your valuable insights.\n\n**A1.2** Theoretically, increasing parameters in the feature extraction network should enhance performance. To verify this, we replaced the ResNet18 backbone with Swin-T [1], increasing parameters from 17.8M to 27.3M. However, this modification did not yielded significant performance difference, as detailed in the table below.\n\n| Encoder | Params | Speed↑ | AJ↑ | $\\delta^x_{avg}$↑ | OA↑ |\n| --- | --- | --- | --- | --- | --- |\n| Swin-T | 27.3 M | 8.9 FPS | 63.0 | 76.6 | 88.7 |\n| ResNet18 | 17.8 M | 14.3 FPS | 65.1 | 77.5 | 89.5 |\n\n**A1.3** Here is our analysis: Point tracking task relies more heavily on shallow spatial features than on deep semantic representations. This occurs because the task primarily learns spatiotemporal consistency of geometric patterns and local appearance similarity within a point's neighborhood, while remaining agnostic to object-level semantics.\n\n  We posit that such a characteristic confers strong zero-shot generalization capabilities, yet concurrently introduces a critical limitation: the lack of semantic comprehension regarding instance identity in scenes inevitably hinders long-term tracking robustness.\n\n  In future work, we will explore strategies for integrating deep semantic cues to enhance long-range point association, thereby improving the scalability of point tracking models.\n\n> ### **Q2.** What would happen if the Transformer were replaced by a CNN?\n\n**A2.1** Thank you for your inquiry regarding the architectural design of the model. Our selection of the Transformer framework was motivated principally by its two key advantages over CNNs, the capability of processing sequential information and efficiency in the proposed collision operation.\n\n**A2.2** Building upon the single-time-step streaming and collision processes of the lattice Boltzmann method, we extend the framework to multiple time steps to enhance the robustness of the particle distribution function. This extension is achieved through cross attention between the current estimated distribution $\\mathbf{f}\\in\\mathbb{R}^{N\\times{d}}$ and historical streaming distributions $\\mathbf{f}_s \\in \\mathbb{R}^{N\\times N_s \\times d}$ and collision distributions $\\mathbf{f}_c \\in \\mathbb{R}^{N\\times N_c \\times d}$ as:\n\n$\\hat{\\mathbf{f}} = \\phi_{c}(\\phi_{s}(\\mathbf{f}, \\mathbf{f}_s), \\mathbf{f}_c)\\quad,$\n\nwhere $\\phi_{c}$ and $\\phi_{s}$ are cross-attention modules. $\\hat{\\mathbf{f}}$ denotes the predicted distribution. $N$ is the number of tracked points and $d$ is the feature dimension. $N_c$ and $N_s$ respectively denote the number of time steps over which the streaming and collision distributions are recorded. Furthermore, $N_c$ and $N_s$ can be dynamically configured according to application requirements and edge device computational capabilities. However, the extended and dynamic temporal modeling remains inherently constrained by the architectural limitations of CNNs.\n\n**A2.3** Within the proposed collision operation, the interaction between the tracked points and the dynamic neighborhoods is implemented through deformable attention [2], and the collision distribution $\\mathbf{f}_c$ is obtained as:\n\n$\\lbrace\\Omega(\\hat{\\mathbf{f_i}}, \\mathbf{o_i}, \\mathbf{p_i} | \\delta_\\mathbf{i})\\rbrace_{\\mathbf{i}=t-N_c}^{t-1}\\quad,$ \n\nwhere $\\Omega$ refers to the deformable attention. $\\mathbf{o}$ is the image feature, and ${\\mathbf{p}}$ is the estimated positions of tracked points, which serve as reference points in the attention. $t$ is the current time step. Here are two possible solutions to use CNN to replace Transformer:\n\n- Directly using CNN to replace deformable attention, *i.e.*, the collision distribution $\\mathbf{f}_c$ is formulated as:\n\n  $\\lbrace\\text{CNN}(\\text{Sample}(\\mathbf{o_i}, \\mathbf{p_i})\\rbrace_{\\mathbf{i}=t-N_c}^{t-1},$\n  \n  where sparse image features are first sampled by point positions, followed by convolution on the sampled results to generate a collision distribution. However, the size of the sampled sparse features is fixed, which hinders dynamic neighborhood perception as in Figure 7.\n  \n- Alternatively, applying deformable convolution to the entire feature map before sampling to obtain the collision distribution enables dynamic neighborhoods:\n  \n  $\\lbrace\\text{Sample}(\\text{DeformCNN}(\\mathbf{o_i}), \\mathbf{p_i})\\rbrace_{\\mathbf{i}=t-N_c}^{t-1}.$\n  \n  However, this approach incurs a computational complexity of $O(hwK^2d^2)$, where $h$ and $w$ are the spatial dimensions of the feature map, yielding a total of $hw$ points, $K$ is the kernel size, and $d$ is the feature dimension. This complexity is significantly higher than that of deformable attention, which operates at $O(NMd^2)$, since $N \\ll hw$, where $N$ is the number of tracked points and $M$ is the number of neighbors per point.\n  \n\n**A2.4** Our primary motivation for employing the Transformer lies in its capabilities for temporal modeling and enhanced computational efficiency for the collision process. For the backbone, we deliberately selected a CNN (ResNet18) over Transformer-based alternatives due to its superior efficiency in extracting shallow visual features, as discussed in Section A1.3. We appreciate your inquiry regarding the model architecture and will conduct further quantitative analysis to evaluate the feasibility of substituting CNNs for Transformers.\n\n> ### **Q3.** How does LBM perform on traditional object tracking datasets such as GOT10k, LaSOT, and TrackingNet?\n\n**A3.1** We appreciate your insights regarding the performance of LBM on traditional single-object tracking datasets. Your concerns will be addressed through two primary dimensions.\n\n**A3.2** LBM is fundamentally a pixel-tracking model devoid of object priors. Evaluations on multi-object tracking datasets (TAO, OVT-B, and BFT) follow established protocols like SORT [3] and ByteTrack [4]. which employ public detections (bounding boxes and class labels) from off-the-shelf detectors.\nHowever, in classical single-object tracking tasks, the bounding box of the target is directly regressed based on initial target information. Consequently, LBM requires additional object priors for assistance on datasets such as LaSOT.\n\n**A3.3** Considering that open-world multiple object tracking encompasses challenges in single object tracking across diverse scenarios and extensive categories, such as occlusions, similar objects, and deformations. Moreover, the evaluation of open-world multiple object tracking tasks includes larger-scale evaluation datasets, with totally 2,997 videos in TAO, BFT, and OVT-B. The performance of LBM in these benchmarks confirms its robustness for real-world object tracking.\n\n**A3.4** We sincerely appreciate your valuable feedback and are actively exploring the application of LBM on object tracking datasets such as LaSOT. However, due to time constraints during the rebuttal period, we will implement these tests to the fullest extent possible.\n\n> ### **Summary**\n\nWe thank you again for your constructive feedback and will implement revisions accordingly. We hope these responses can address your concerns. Should further questions arise, please do not hesitate to contact us. We will be providing comprehensive responses to all inquiries during both the Rebuttal and Discussion periods.\n\n```\n[1] Liu, Ze, et al. \"Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows.\" Proceedings of the IEEE/CVF international conference on computer vision. 2021.\n\n[2] Zhu, Xizhou, et al. \"Deformable DETR: Deformable Transformers for End-to-End Object Detection.\" International Conference on Learning Representations. 2021.\n\n[3] Bewley, Alex, et al. \"Simple Online and Realtime Tracking.\" IEEE International Conference on Image Processing. 2016.\n\n[4] Zhang, Yifu, et al. \"ByteTrack: Multi-Object Tracking by Associating Every Detection Box.\" European conference on computer vision. 2022.\n```"}, {"review_id": "n92v07BYYq", "reviewer": "Reviewer_TFHM", "summary": "This paper proposes a Lattice Boltzmann Model for real-time pixel tracking in videos. By modeling pixel dynamics through fluid-inspired collision-streaming processes, LBM estimates pixel positions and visibility using a lightweight predict-update network. The proposed method demonstrates computational efficiency by utilizing lightweight architectures, and experimental results on point tracking and object tracking benchmarks show its competitive accuracy and speed compared to both offline and online baselines. Its fine-grained pixel modeling also enables robust tracking under occlusion and detection failures, demonstrating strong potential for real-world applications.", "questions": "- Can the authors provide more empirical or theoretical evidence that the proposed collision-streaming mechanism, inspired by the lattice Boltzmann method, offers advantages in spatiotemporal modeling compared to standard attention-based architectures?\n- For the object tracking benchmarks on TAO and OVT-B (Tables 2 and 3), do all compared methods, including LBM, use the same detection or segmentation backbone? If not, how do variations in detection quality influence the reported tracking performance?\n- The method samples inlier pixels randomly within bounding boxes during object tracking. Have the authors considered or tested more robust strategies, such as using instance segmentation masks or uncertainty-guided sampling? This could potentially reduce the impact of background noise.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths\n- The paper addresses critical practical challenges of existing point and object tracking methods, particularly for real-time and low-latency applications on edge devices. The proposed approach introduces a novel physics-inspired framework by adapting the lattice Boltzmann method to pixel-level tracking, offering a novel perspective on spatiotemporal modeling in vision tasks.\n- The architecture is thoughtfully designed for efficiency, using a multi-layer predict-update mechanism and lightweight visual encoders. The reported hardware deployment and runtime evaluations on edge platforms further validate its suitability for real-world scenarios.\n- The experimental evaluation is comprehensive, covering point and object tracking benchmarks, ablation studies, and hardware tests. The comparisons against both online and offline baselines are thorough and convincingly support the claims of robustness, efficiency, and accuracy.\n- The paper provides a clear discussion of its limitations, including issues related to the locality of collision-streaming processes and sensitivity to background pixels in random sampling. It also reflects on failure cases and potential solutions in the supplementary, demonstrating a comprehensive presentation.\n\nWeakness:\n- While the model is inspired by the lattice Boltzmann method, the paper provides only a loose analogy to fluid dynamics without formal theoretical justification. It lacks deeper analysis on convergence properties, stability, or potential benefits over purely attention-based models in terms of tracking accuracy or robustness.\n- Although the proposed method achieves strong efficiency, its improvements in tracking accuracy over existing online methods (e.g., Track-On) are marginal (e.g., only a 0.1 increase in AJ on TAP-Vid).\n- While the method is evaluated on standard datasets, its limitations under prolonged occlusions or long-horizon tracking are only briefly mentioned. Additional experiments or analysis on more challenging scenarios could strengthen the claims of robustness.", "was_revised": true, "final_justification": "The authors' rebuttal effectively addresses my initial concerns, particularly regarding the computational motivation, generalization ability, and robustness in long-term tracking. They also demonstrate a clear understanding of the current limitations and outline promising directions for future improvement.\n\nOverall, I remain positive about the paper’s contributions and believe it presents a novel and efficient approach for real-world object tracking.", "rebuttal": "Thank you for your positive comments regarding the computational efficiency, novelty, structural rationality of the LBM model, as well as its practical deployment and evaluation. We also appreciate your questions and suggestions to make this work better. To systematically address your concerns, we have compiled the following responses:\n\n> ### **Q1.** Can you provide more theoretical evidence that the collision-streaming mechanism offers advantages compared to standard attention-based architectures?\n\n**A1.1** Thank you for your advice on explaining the architectural advantages of the collision-streaming mechanism. The primary motivation behind LBM lies in the computational **efficiency**, especially in the collision-streaming mechanism compared to pure attention architectures.\n\n**A1.2** Taking the collision process in the lattice Boltzmann method as an instance, our approach considers modeling relationships between pixels and their dynamic local neighborhoods, rather than performing global cross-attention directly with aggregated visual embeddings.\n\n**A1.3** The collision procedure of the lattice Boltzmann method can be formally represented as: \n\n$f_{c,i} = \\Omega_i(\\mathbf{x} - \\mathbf{c}_i\\Delta{t}, t-\\Delta t)\\quad,$\n\nwhere $f_{c,i}$ means the collision term caused by the neighbor from $i$-th direction, $\\Omega$ denotes the collision operator, $\\mathbf{x}$ denotes the position, $\\mathbf{c}$ is the direction, $t$ is the current time, and $\\Delta t$ is the time interval.\nWithin the LBM framework, we employ deformable attention [1] mechanisms as $\\Omega$ to construct learnable dynamic neighborhoods for tracked points, with the standard visualization of this process provided in Figure 7. The collision distributions $\\mathbf{f}_c$ can be mathematically formulated as: \n\n$\\lbrace\\Omega(\\hat{\\mathbf{f_i}}, \\mathbf{o_i}, \\mathbf{p_i} | \\delta_\\mathbf{i})\\rbrace_{i=t-N_c}^{t-1}\\quad,$\n\nwhere we extend the collision process from a single time step to $N_c$ steps, and multiple direction collision is merged. $\\mathbf{o}$ denotes visual embeddings, $\\mathbf{p}$ denotes the predicted point positions, $\\delta$ is a learned dynamic neighborhood. Specifically, this procedure initially predicts $M$ candidate positional offsets to identify valuable neighboring pixels, samples corresponding neighbor embeddings from visual embeddings $\\mathbf{o} \\in \\mathbb{R}^{h\\times w \\times d}$, and subsequently applies attention weighting between the pixel distribution $\\hat{\\mathbf{f}}$ and sampled neighbor embeddings to derive the collision distribution. The space complexity characterizing this computational procedure is formulated as $O(NMd^2)$, where $N$ denotes the number of tracked points, and $d$ corresponds to the feature dimensionality.\n\n**A1.4** If we performed global cross-attention directly between particle distributions and aggregated visual features as:\n\n$\\lbrace\\text{CrossAttn}(\\mathbf{p_i}, \\mathbf{o_i}) \\rbrace_{i=t-N_c}^{t-1}\\quad,$\n\nthe space complexity is $O(Nhwd^2)$. Since the sampled neighbor count $M$ is significantly smaller than total $hw$ spatial positions within visual embeddings, the computational complexity of pure attention mechanisms is substantially higher than that achieved in the LBM framework.\n\n**A1.5** We sincerely appreciate your valuable suggestions and will ensure that the theoretical comparison of frameworks is presented with enhanced conceptual clarity in the revised manuscript.\n\n> ### **Q2.** How do variations in detection quality influence the reported object tracking performance?\n\n**A2.1** We appreciate your comments on the details of object tracking evaluation. Given the differing approaches among object tracking schemes, the detection results employed are typically non-uniform across methodologies. Consequently, all comparative evaluation results were derived from their respective official implementations. Considering that GLEE [2] serves as a general image and video foundation model, GLEE-generated detections were used in evaluating LBM across our benchmark assessments.\n\n**A2.2** Higher-quality detection results typically yield superior tracking performance, especially in localization accuracy (LocA). This is particularly relevant for open-world object tracking, which faces increased complexity due to challenging object categories. This phenomenon has driven tracking approaches to employ stronger detectors. For instance, MASA [3] has demonstrated the advantage of utilizing Detic [4] over alternative methods like SAM [5] and Grounding DINO [6].\n\n**A2.3** Nevertheless, the robustness of the association method in tracking remains critically important, particularly concerning the improvement of association accuracy (AssocA). Without training on domain-specific data on object tracking, the proposed LBM demonstrates the strongest overall AssocA among training-free methods on the TAO and OVT-B benchmarks, confirming its strong generalizability for real-world object tracking deployment scenarios.\n\n**A2.4** We appreciate your recommendation to clarify the impact of detection quality. We shall provide a more detailed discussion addressing this point in the revised manuscript.\n\n> ### **Q3.** Will segmentation masks or uncertainty-guided sampling benefit tracking performance?\n\n**A3.1** We appreciate your suggestion regarding the implementation of more effective point sampling schemes for object tracking. As you rightly noted, random sampling can indeed introduce background noise compared to alternative strategies like segmentation masks or uncertainty-guided sampling. To address this limitation, we have introduced a dynamic point management mechanism: LBM systematically prunes tracking points that persistently fall outside the target bounding box for a defined duration. Concurrently, it replenishes the depleted point set by sampling a corresponding number of new points within the current bounding box.\n\n**A3.2** The adoption of random sampling is motivated by our intention to employ a relaxed target prior (*e.g.*, bounding box), thereby validating the generalizability and robustness of LBM for real-world object tracking tasks.\n\n**A3.3** We acknowledge your suggestion and will incorporate more effective sampling strategies in subsequent generalization experiments to mitigate background interference during tracking, thereby further enhancing tracking performance gains.\n\n> ### **Q4.** How does LBM perform on longer-term benchmarks, which are more challenging?\n\n**A4.1** Thank you for your valuable suggestions on evaluating LBM on longer-term point tracking benchmarks. Following this suggestion, we evaluated LBM on the longer-term point tracking benchmarks, *i.e.*, ​Dynamic Replica​ and ​PointOdyssey. Dynamic Replica is a benchmark designed for 3D reconstruction and comprises 20 sequences of 300 frames each. ​PointOdyssey​ features 12 videos containing thousands of frames (up to 4,325). Consistent with the TAP-Vid benchmark, position precision $\\delta_{avg}^{vis}$ of visible ​points​ is adopted as the evaluation metric. The results are shown as follows, where LBM reaches the state-of-the-art performance, especially on the long-term PointOdyssey benchmark.\n\n| Model        | Dynamic Replica | PointOdyssey |\n|--------------|:-------------------------------:|:-----------------------------:|\n| PIPs++       | 64.0                           | 32.4                          |\n| TAPIR        | 66.1                           |          -                     |\n| CoTracker    | 68.9                           | 32.7                          |\n| TAPTR        | 69.5                           |               -                |\n| LocoTrack    | 71.4                           |             -                  |\n| BootsTAPIR   | 69.0                           |             -                  |\n| CoTracker3   | 73.3                           |               -                |\n| Track-On     | **73.6**                       | 38.1                          |\n| LBM (Ours)   | 71.5                           | **41.9**                      |\n\n**A4.2** Your constructive suggestions have contributed to significant refinements in this work. In the subsequent phase, we shall conduct an in-depth investigation at both the model architecture and dataset levels to enhance robustness in long-term tracking tasks.\n\n> ### **Q5.** Why are the improvements of LBM in tracking accuracy over existing online methods marginal?\n\n**A5.1** We appreciate your insights​ regarding the performance-speed tradeoff. While LBM exhibits ​marginal improvements​ compared to recent SOTA works (*e.g.*, Track-On), it demonstrates ​superior computational efficiency​ as you noted. This aligns with our fundamental motivation: ​achieving real-time online tracking through fluid dynamics-inspired computation.\n\n**A5.2** We embrace your constructive suggestion​ and will ​diligently investigate​ strategies for ​enhancing model precision. The next effort will specifically focus on performance improvement in complex dynamic environments.\n\n> ### **Summary**\n\nWe once again express our gratitude for your recognition and shall refine our work following your valuable feedback. We hope that this response adequately addresses your concerns. Should any further questions arise, we stand ready to actively engage in discussion throughout the rebuttal and discussion period.\n\n```\n[1] Zhu, Xizhou, et al. \"Deformable DETR: Deformable Transformers for End-to-End Object Detection.\" ICLR. 2021.\n\n[2] Wu, Junfeng, et al. \"General Object Foundation Model for Images and Videos at Scale.\" CVPR. 2024.\n\n[3] Li, Siyuan, et al. \"Matching anything by segmenting anything.\" CVPR. 2024.\n\n[4] Zhou, Xingyi, et al. \"Detecting Twenty-Thousand Classes Using Image-Level Supervision.\" ECCV. 2022.\n\n[5] Kirillov, Alexander, et al. \"Segment Anything.\" ICCV. 2023.\n\n[6] Liu, Shilong, et al. \"Grounding DINO: Marrying DNO with Grounded Pre-Training for Open-Set Object Detection.\" ECCV. 2024.\n```"}]

Author rebuttal:

Meta-review: The paper was generally well-received, obtaining three accepts and one borderline reject, with reviewers highlighting its novelty, efficiency, and practical relevance. Nonetheless, Reviewer #6HeP raised significant concerns regarding clarity and mathematical rigor, citing the lack of precise definitions for the collision and streaming processes, the unclear connection to lattice Boltzmann theory, and omission of important design choices. Reviewer #pfjb acknowledged the contributions but noted risks of overfitting, deployment complexity, and limitations in very low-resource environments. Reviewer #TFHM emphasized weak theoretical grounding, marginal accuracy improvements over baselines, and insufficient evaluation of long-term tracking robustness.

In the rebuttal, the authors provided clarifications on the theoretical grounding of LBM (addressing concerns from #6HeP and #TFHM), added results on Dynamic Replica and PointOdyssey to evaluate long-term tracking (#TFHM, #pfjb), demonstrated optimized deployment with FP16 quantization on edge devices (#pfjb), and included analyses of deeper backbones and Transformer justification (#Ycni). Reviewers #pfjb, #Ycni, and #TFHM confirmed that their concerns were resolved. Reviewer #6HeP acknowledged the technical merits but maintained that the paper requires substantial rewriting to improve clarity and rigor.

After considering the paper, rebuttal, and reviewer feedback, the AC recognizes the novelty and practical value of the work and supports the majority opinion, recommending acceptance. The AC strongly encourages the authors to revise the manuscript thoroughly for the camera-ready version, particularly addressing the clarity and rigor issues emphasized by Reviewer #6HeP.
Decision: Accept (poster)