PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching
Authors: Yun Wang
Junjie Hu
Qiaole Dong
Yongjian Zhang
Yanwei Fu
Tin Lun Lam
Dapeng Wu
Venue: NeurIPS 2025 poster ()
Keywords: video stereo matching; 3D Vision
Primary area: applications
TL;DR: we introduce the use of the memory buffer into the video stereo matching task, aiming to construct a memory buffer that is both compact and informative.

Abstract: Temporally consistent depth estimation from stereo video is critical for real-world applications such as augmented reality, where inconsistent depth estimation disrupts the immersion of users.
Despite its importance, this task remains challenging due to the difficulty in modeling long-term temporal consistency in a computationally efficient manner.
Previous methods attempt to address this by aggregating spatio-temporal information but face a fundamental trade-off: limited temporal modeling provides only modest gains, whereas capturing long-range dependencies significantly increases computational cost.
To address this limitation, we introduce a memory buffer for modeling long-range spatio-temporal consistency while achieving efficient dynamic stereo matching.
Inspired by the two-stage decision-making process in humans, we propose a Pick-and-Play Memory (PPM) construction module for dynamic Stereo matching, dubbed as PPMStereo. PPM consists of a pick process that identifies the most relevant frames and a play process that weights the selected frames adaptively for spatio-temporal aggregation.
This two-stage collaborative process maintains a compact yet highly informative memory buffer while achieving temporally consistent information aggregation.
Extensive experiments validate the effectiveness of PPMStereo, demonstrating state-of-the-art performance in both accuracy and temporal consistency.Codes are available at \textcolor{blue}{https://github.com/cocowy1/PPMStereo}.

PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching
Yun Wang', Junjie Hu²; Qiaole Dong"; Yongjian Zhang Yanwei Fu", Tin Lun Lam², Dapeng Wu! City University of Hong Kong. 'The Chinese University of Hong Kong, Shenzhen 3 Fudan University,* Shenzhen Campus, Sun Yat-sen University ywang3875-cumy cityu. hk, dpwudieee.org {qldogg18, yanseiful&fudan edu. cn zhangyj j85mma112..... attemmmll.syuuuul wyw..edu.uuuco.o..oo en, (hujunj ie, t1llall1m@@mlm @cuhk. edu. cn
Abstract
Temporally consistent depth estimation from stereo video is critical for real-world applications such as augmented reality, where inconsistent depth estimation dis- rupts the immersion of users. Despite its importance, this task remains challenging due to the difficulty in modeling long-term temporal consistency in a computation- ally efficient manner. Previous methods attempt to address this by aggregating spatio-temporal information but face a fundamental trade-off: limited temporal modeling provides only modest gains, whereas capturing long-range dependen- cies significantly increases computational cost. To address this limitation, we introduce a memory buffer for modeling long-range spatio-temporal consistency while achieving efficient dynamic stereo matching. Inspired by the two-stage decision-making process in humans, propose Pick-and-Play Memory (PPM) construction module for dynamic Stereo matching, dubbed as PPMStereo. PPM consists of a 'pick' process that identifies the most relevant frames and a 'play' process that weights the selected frames adaptively for This two-stage collaborative process maintains compact yet highly informative memory buffer while achieving temporally consistent information aggregation. Extensive experiments validate the effectiveness of PPMStereo, demonstrating state-of-the-art performance in both accuracy and temporal consistency. Codes are available at
aggregation.
L/PMMSoroo
1 Introduction
Stereo matching refers to binocular disparity estimation, which is fundamental computer vision task focused on estimating the disparity between a pair of rectified stereo images [53, 21, 34J. Deep learning-basee stereo matching methods have achieved remarkable progress in terms of accuracy [53, 58, 50, 11], efficiency [46, 59, 52, 2]. and robustness [42, 64, 63, 55]. Despite impressive performance for static scenes, these methods exhibit severe temporal inconsistencies when applied to dynamic scenes [24]. This manifests itself as flickering artifacts and blurred disparity maps due to the absence of effective inter-frame temporal information integration. Therefore, the algorithm deployment in dynamic scenarios such as autonomous driving, robotics, and augmented reality platforms is limited, which requires temporally consistent disparity maps.
"Corresponding author.
i Project Leader
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Read
Mema Sull
Upplaal
(s) Esteiig
(b) The proposed
Figure Comparison between prior methods (a) and our method (b). For the t-th frame, prior works process video sequences using small temporal sliding windows with attention or optical flow, restricting cost information propagation. Our method captures long-range spatio-tempont relationships across the input sequence by constructing and updating compact memory buffer.
To address the task of dynamic stereo matching, recent approaches start incorporate temporal cues from two main perspectives to achieve temporally consistent estimation. Some methods [30, 61, 12] refine the current disparity with disparity or motion of previous neighbor frame, while achieving limited improvements in temporal consistency due to the narrow temporal context. Secondly, other approaches [24, 22] (Fig. 1 (a)) expand the temporal receptive field by using attention mechanisms to model spatio-temporal relationships [24] within a sliding window while treating all frames equally, which overlooks variations in frame reliability. BiDAStereo [22] further depends on optical flow priors for aligmment, may incurring errors from flow inaccuracies and high computational cost. Overall, video-based methods face a core trade-off: narrow context yields marginal improvements. whereas naively using all frames drives up computation without reliability awareness.
Naturally, these considerations lead to key question: How can we design model that effectively models long-range temporal relationships while maintaining computational efficiency? To answer this question, we draw inspiration from recent advances in sequence processing and bring memory buffer into the dynamic stereo matching task. We present Pick-and-Plas Memory for dynamic Stereo matching named PPMStereo which enables effective and efficient utilization of reference frames for Long-range spatio-temporal modeling by dynamically reducing redundant frames while selectively retaining and leveraging the most valuable frames throughout the video sequence to ensure accuracy and efficiency, as illustrated in Fig. 1 (b).
Specifically, our method draws inspiration from human decision-making in complex scenarios, which typically involves the 'pick' process that identifies the most essential elements from a set of candidates and the "play' process that meticulously balances and leverages the identified elements [5, 18, 38]. In this paper, we propose a novel Pick-and-Play Memory construction method for video stereo matching. Specifically, the 'pick' process identifies the most relevant K frames from T reference frames for the current frame. To facilitate this process, we introduce a novel Quality Assessment Module (QAM), which evaluates each frame's contribution by jointly evaluating confidence, redundancy, and similarity of reference frames. Upon identifying the most relevant K frames, the 'play' process adaptively weights the importance of the features extracted from those K selected frames via a dynamic memory modulation mechanism. Subsequently, we utilize an attention- that queries the high-quality memory buffer using the current frame's contextual feature, yielding temporally and spatially aggregated cost features. By combining this aggregated cost feature with the current cost and context features, we can use GRU modules to regress the residual disparities.
memory read-out mechanism
Extensive experiments show that our method achieves state-of-the-art temporal consistency and accuracy. Specifically, on both the clean and final pass of the Sintel [6] dataset, our model achieves temporal end-of-point error (TEPF) of 0.62 and 1.11 pixels, with 3-pixel error rates of 5. 19% and 7.64%, respectively. Compared to the previous SoTA method, BiDAStereo [22], this represents a 17.3% and 9.02% reduction in TEPE and a 9.74% and 10.32% improvement in 3-pixel error rate, while enjoying lower computational costs. Overall, the contributions of our work can be summarized as follows: (1) We introduce PPMStereo, the first work that successfully builds a memory buffer to tackle dynamic stereo matching, allowing for long-range spatio-temporal modeling in a computationally memory buffer construction method that first identifies the key subset of reference frames with the pick process and then effectively aggregates them with play process, enabling highly accurate and temporally consistent disparity estimation. (3) Extensive experiments demonstrate that PPMStereo achieves ofate-of-the-art performance across multiple dynamic stereo matching benchmarks.
efficient way. (2) We propose a novel 'Pick-and-Pie

---

2
Related Work
Deep Stereo Matching, Existing deep stereo matching methods [47] primarily focus on cost volume aggregation for network and representation design. These approaches are generally categorized into regression-baas based [34, 25, 59, 62, 42, 56, 53] and iterative-based methods [32, 27, 54, 58, 50, 64]. Regression- based methods typically regress a probability volume to estimate disparity maps, which can be further divided into 2D [34, 31, 59, 54] and 3D cost aggregation approaches [19, 62, 42, 33, 43, 53, 52]. These methods either directly regress disparity across a predefined global range [25, 62, 53] or employ a noarse-to-fing refinement strategy to improve accuracy [42, 43, 33J. Recently, iterative- based methods [57, 32, 4, 54, 28, 50, 55, 51, 26] have emerged as the dominant paradigm in stereo matching. These methods leverage multi-level GRU or LSTM modules to iteratively refine disparity maps through recurrent cost volume retrieval, achieving state-of tf-the-arr performance. However, despite their remarkable results, these approaches infer disparities independently for each frame, ignoring temporal correlations across video sequences. As a result, they often suffer from poor temporal consistency, which manifests as flickering artifacts in the disparity outputs.
Dynamic Stereo Matching. few methods in stereo matching have focused on leveraging tem- poral cues from dynamic scenes to enhance disparity consistency. These methods can be mainly categorized into two paradigms: (i) Adjacent-frame Integration, which propagates disparity or motion fields from the immediately preceding frame to maintain local temporal smoothness. These works [30, 65, 12, 61] typically employ warped disparity or motion estimates for robust initialization, thereby enhancing the temporal consistency. However, these methods are limited by their reliance on only the most recent frame, resulting in a narrow temporal receptive field. (ii) Multi-frame Integration, which employs sliding window aggregation across extended temporal contexts to force temporal consistency through attention mechanisms (DynamicStereo) [24] or optical flow priors (BerDAttereo) [22]. Despite their strengths, attention-based methods treat all frames equally without assessing the reliability of reference frames and suffer from high computational costs with a large window. Additionally, flow-based methods sensitive optical flow estimation errors and introduce extra computational overheads. In contrast, our method effectively aggregates long-range spatio-temporal information from a compact yet high-quality memory buffer. Thanks to our 'pick' process, PPMStereo remains computationally efficient, even with the enlarged temporal window.
sliding-window
Memory Cues for Video Tasks. Prior works have explored memory model [45] across various video tasks, including optical flow [15], segmentation [37, 66, 9, 10]. tracking [60, 17], and video understanding [44, 20], demonstrating its significant effectiveness for video-related tasks. Among them, XMem [9] consolidates memory by selecting prototypes and evicting obsolete features via least-frequentl--ssee
policy, while RMem [66] improves the segmentation accuracy by using a fixed frame memory bank [1]. Prior works have explored memory model [45] across various video tasks, including optical flow [15], segmentation [37, 66, 9, 10], and video understanding [44, 20]. demonstrating its significant effectiveness for video-related tasks. Among them, XMem [9] consolidates memory by selecting prototypes and evicting obsolete features via least-frequently-used policy, while RMem [66] improves the segmentation accuracy by using fixed frame memory bank []]. The closest related work is MemFlow [15], which develops an adjacent- fraame memory buffer framework to aggregate spatio-temporal motion for optical flow estimation. While effective for optical flow, MemFlow yields limited gains when directly applied dynamic stereo matching, as it only retains the immediate adjacent frame. Expanding its temporal scope without reliability assessment introduces redundant and noisy cues. contrast, our method adaptively updates and modulates the most valuable memory cues across the entire sequence, enabling robust long-range spatio-tempont modeling while filtering out inferior ones, leading to significant performance improvements.
3 Methodology
3.1 Overview
Dynamic stereo matching seeks to recover a sequence of temporally consistent disparity maps (d') he(..)) E #W from stereo video frames {I/, f h E / x3 where T' is the number of frames, H and W are the height and width dimensions. However, prior approaches struggle to capture long-range temporal dependencies without incurring prohibitive cost. To address this, we introduce PPMStereo, which augments the DynamicSterco backbone [24] with Pick-
3

---

Featur Enosder
Fi Fi
Cost Encodar
Concatonation
F
Velue Buffer J.
Memory Pick Process
Memory Play Process
Frame
Memory Key
Vanilla Memory
Memory
QAM Module
Top
Memory Modulation
Dynami
Desoder
Dynomic Memoory Butfer Jul
Read oul
Frama
Key
Query
4
Memory Key
Vemory Value
120 Astropatee Cost
Disparty
Context Features
An overview of PPMStereo. The gray part is the memory 'pick' process, and the blue part is the memory play process. Our PPMSteréo employs a dynamic memory buffer for modeling long-range spatio-temporal relationships while maintaining computational efficiency.
Figure 2:
and-Play Memory (PPM) module that selectively aggregates high-quality references into compact, query-adaptive buffer, thereby strengthening spatio-temporal modeling while remaining efficient. As illustrated in Fig. 2, the overall pipeline proceeds as follows: (I) Feature Extraction; shared encoder extracts multi-scale features {F ( E R xaW xC at scales E {1/16, 1/8, 1/4), with C channels. These pyramidal representations provide both receptive- field diversity and a convenient substrate for multi-scale matching. (2) Cost Volume Construction: each time step we construct 3D correlation volume from {Fi, ()) and pass through a lightweight cost encoder to obtain
matching costs F* Foost: subsequently projected to value embedding et (3) Context Encoding: A context encoder operating on the left view produces Ft, which are linearly projected to a query qr and ky. (4) Memory Buffer Initialization and Update: To expose the model long-range spatio- temporal correlations, we initialize a vanilla memory M = km E L2C Um E R RLXC that stores km. (kI...., kr) and Um {Ui,....Uy, VT} with T sll x sW. This naive memory buffer stores all reference-frame features, making per-iteration queries prohibitively expensive. To retain accuracy without sacrificing efficiency, we introduce the Pick-and-Plan Memory (PPM): driven by Quality Assessment module (omitting the iteration index for brevity), PPM first picks the most informative references to construct a compact, dynamic buffer M¹ = K,, E RL'xC, v' e R4'xC) } with L' K x sHl sW and K << T, and then plays by adaptively weighting these entries to produce aggregated cost features that balance contributions across selected frames. (6) Iterative Refinement: following RAFT-style iterative scheme [32], we alternate GRU-based updates of disparity estimates with PPM-based memory updates, progressively refining {de} while preserving temporal consistency and computational efficiency.
3.2 Memory Pick Process
Naive heuristic strategies, such as random selection or solely keeping the latest frame, are unreliable. Since the former neglects frame reliability and relevance, while the latter suffers from limited temporal context and knowledge drift [36]. To this end, we introduce Quality Assessment Module (QAM) that explicitly evaluates the quality of memory elements (km, Um } in the vanilla buffer for dynamic stereo matching. To activate QAM, we define two complementary scores that quantify each reference frame's contribution to the final accuracy: confidence score S computed over the value embeddings to prioritize reliable evidence, and a redundancy-awee aware relevance score S computed over the key embeddings km to suppress repetitive or low-information entries. The full procedure is summarized in Algorithm/ 1. Sf and S are used together to enable the construction of a compact, high-quality memory M that preserves the most informative cross cross-frame frame cues.
Confidence Score. Memory values um encode pixel-wise horizontal displacements, which are critical for disparity estimation. These features naturally indicate the reliability of its disparity estimation. To this end, we employ lightweight confidence network that transforms um E I into confidence maps u, E xxx*HX quantifying whether memory values Um corresponding to
T x x x
'The confidence network consists of two convolutional layers followed by sigmoid activation, which ensures efficient and effective confidence estimation.

---

Algorithm Pseudo code of Pick-and-Play Memory
Input: Video frames sequence { } of video length T, GRU n-th iterations, K <
Intermediates: Vanilla Memory: .M {lim RI Um ]R L sH sW
The query: Q: E | XxW 88 (1/16, 1/4] is the downsampled scale
Dynamic Memory: MM {Km, E RL'xC, onm R L' K I/l x sW
Output: The residual disparity map at n-th GRU iteration: Adp
1: while
S T do
Memory Pick Process:
2: S, S Sj
# QAM, evaluate the quality of memory elements kmm and Um
3: I {i rank (S] & K) # Select top-K reference frames
4: Mg (K' Cat [( Ii)l v/n Cat [(us E I}}}
Memory Play Process: S. 5 S. E Ie
5:
Balance the contribution of selected memory entries
6: = 4: Ph. km = S. Kn + Pr.
Dynamic memory modulation
7: Ft. = Read-out(q:- M³)
# Aggregate high-quality spatio-temporal cost information
8:
Ad GRU Ft Foostr Fi))
Produce the disparity map at the 7ith iteration
accurate disparity outputs. These confidence maps can provide a frame-level reliability measure by estimating the uncertainty of predicted disparity [42, 49]. During training for N iterations, the confidence maps are supervised using an L1 loss function to enforce consistency with their ground-truth counterparts. The ground-truth confidence score úe is computed as follows:
û; exp (-
de de-
di
(I)
where de and de represent the predicted and ground-truth disparities for the t-th frame, respectively, and is hyper-parameter empirically to 5. Over N iterations, we compute the confidence loss Leonf across all timesteps ME((1T as follows:
Lcon ΣΣ
7
1
0.
where denotes the number of iterations and is decay factor set as 0.9. To obtain frame- level confidence score Si E [R1x we apply average pooling across the spatial dimensions of confidence maps
Redundancy-aware Relevance Score. Relying solely on the confidence score is insufficient, as adjacent frames often exhibit strong spatio-temporal correlations, which can result in higher confidence scores. This, in turn, introduces feature redundancy and suppresses contributions from more diverse frames, ultimately limiting the diversity and effectiveness of the memory buffer. To mitigate this issue, we propose redundancy-avee aware relevance score evaluate memory keys kis balancing semantic consistency and memory diversity. First, we compute an inter- frame similarity score Sim, E RR 1xT between the current query qr and the memory keys ki. measuring semantic alignment while preserving temporal coherence. For computational efficiency, we employ an attention mechanism combined with spatial downsampling. Specifically, average pooling reduces the spatial resolution of the query and memory keys from sHf x sW to sH' xsW', followed by L2 12-normalization
5

---

Go ail Stom
Consataratio on @ @cAsction
C Murtpiaatinn
X Exckaion
Pick
Frums-livall Quality Assosament Moduki
Carisdance
Merory Butter Ad
Confideroo
M
Simdinity
5. barty Assossmenn Moodll
Court roburdancy
Co Contitanial
Bedandwwog
of drrancc
Coot
Project
Project
Project
Boore
Oynamic Memary Are
fooder
Enoooder
Ehoooder
Eroooder
Enooder
Eroodder
Alauttion
M 9.99
Keep top nalerans
M
Dynamio Mamory Extras
Marrory
Fostlio Eioceling
Figure 3: The details of our Pick-and-play Memory Construction Process (PPM).
along the combined feature dimension f 8H' x sW' x The similarity score is computed as:
AvgPool(z)
sim, = ((()))((mm))) )T, where Φ(x) ||AvgPool(<)]]>
where (kkm) E RRTx and AvgPool(.) denotes the average pooling operation. However, focusing solely on the most similar regions may overlook occluded areas. Since occluded regions adjacent frames tend to be highly similar, they can be challenging to reference effectively. To mitigate this, we then introduce redundancy-aware regularizer R.[k] = e + where 44 denotes the the cumulative number of times the k-th frame has been selected for the dynamic memory buffer across previous GRU iterations. This term dynamically downweights overused frames while promoting underutilized yet informative references, ensuring compact yet diverse memory buffer. The final redundancy-awand relevance score S E R IXT combines redundancy and similarity:
S = R sim;
(4)
By jointly considering relevance and diversity, our approach enhances feature aggregation while minimizing redundancy, leading to more robust and efficient memory-based processing.
Memory Updating via QAM. We compute the total quality metric for each memory frame as S, = S S by integrating confidence and redundancy-awarr relevance scores. This integrated scoring enables dynamic memory update by retaining the most informative entries via top-K K selection mechanism, ensuring robust adaptation to varying video scenarios while preventing memory overload. Specifically, for the vanilla memory buffer M (kim, Um) with the corresponding quality scores S, E IxT, we sort the quality scores in descending order and only retain the top-K memory features in the vanilla memory buffer as:
I, = {i rank (S,(i)) < K}
(5)
M = (Cat [(ks E I,)), Cat [{u; i ZJ])
(6)
(where rank(.) denotes the ranking position in descending order, with rank = 1 corresponding to the highest score, I, is the set of selected frames' indices, and Cat denotes the concatenation. The resulting dynamic memory buffer M comprises keys kn, = { ( and values v = {vi (GEZ.)
By enforcing K & T. this strategy efficiently handles arbitrary video sequences while providing high-quality spatio-temporal cues for dynamic memory aggregation.
3.3 Memory Play Process
After the pick process selects the top-K most relevant memory entries for our dynamic memory buffer M we argue that not all selected frames contribute equally to disparity estimation. To further weigh their importance, we introduce a memory play process that dynamically weights the selected memory entries based on learned quality scores. Since dynamic memory construction inherently disrupts temporal ordering, we incorporate temporal position encoding into the framework, ensuring temporal awareness.

---

Dynamic Memory Modulation. Building on this foundation, we propose unified dynamic memory modulation strategy that jointly optimizes feature reliability and temporal consistency. Specifically, given the estimated quality score S₁, we first obtain the relative significance of the frames:
S Σ, S
M
i
(7)
Following [16], we initialize positional encodings (PE) to align with the original memory buffer length T, formalized as PI... This initialization ensures temporal coherence in feature representation. Therefore, the "play' process subsequently operates as follows:
a: q: + Pt: K = S, K'm Pr.
where P. denotes the positional encoding at timestep t, and S, represents the aggregated importance weights over the index set Z. Leveraging the estimated quality scores as reliability indicators, we prioritize more reliable memory entries while maintaining computational efficiency.
Memory Read-out. We aggregate cost features through attention-based memory read-out mecha- nism from the dynamic memory buffer M Specifically, we first compute soft attention weights by measuring the similarity between the query 9, and modulated memory keys K, The aggregated cost are then obtained by weighting the memory values ty through these attention weights:
features Ftt
agy
Ftt cast + Softmax (I/VDR x qi x Kmn)) ) d'm
Fagg
where a is learnable scalar initialized from O. In this way, we employ the attention to gather addi- tional temporal information. With the context, cost, and aggregated cost features, we can now output a residual disparity map through a GRU unit at the иith iteration: Ad, = GRU Foostr agg F.). After N iterations of PPM and GRU, we can get the final disparity map.
Loss Functions.
Our disparity loss functions are inherited from the previous works [24, 22]. Gener- ally, for N iterations, we supervise network with L1 distance between our series of residual flows (di,..., 3 and the ground-truth de with exponentially increasing weights:
T
La
de
(10)
where y and N are set as 0.9 and 10, respectively. Therefore, the total loss function is as follows:
Liotal = La + Leonf-
(11)
4 Experiments
4.1 Datasets
Our work focuses on videos eaptured with moving cameras, rendering standard image benchmarks like Middlebury [39], ETH3D [40] unsuitable. For training and evaluation, we employ three synthetic and one real-world stereo video dataset, all featuring dynamic scenes: SceneFlow (SF) [34] comprising BlyingThings31 Driving, and Monkaa, with Flying Things3D featuring moving 3D objects against varied backgrounds. Dynamic Replica (DR) [24], a synthetic indoor dataset with non-rigid objects such people and animals. Sintel [6]. synthetic movie dataset available in clean and final passes. South Kensington (SV) [23], a real-world stereo dataset without ground truth data, capturing daily scenarios. We use them for generalization evaluation. Following prior work [24, 22], train synthetic datasets (SF and DR + SF) and evaluate the performance on Sintel, DR, and SV.
4.2 Implementation Details
We implement PPMStereo in PyTorch, training on 8x A100 GPUs (batch size = 2) using 320x512 crops from 5-frame sequences, evaluated at full resolution with 20-frame sequences. We use AdamW (Ir = 0.0003) with one-cycle scheduling, training for 180k iterations (Rs 4.5 days). Data augmentation follows DynamicStereo [24], including random crops and saturation shifts. For efficient memory read- out, we employ FlashAttention [13]. Following prior works [22, 24], we set the number of evaluation
7

---

Table 1: Quantitative comparison with SoTA methods. Abbreviations: K KITTI [35], M Mid- dlebury [39], Inf-nnfnngen SV [23], VK Virtual KITT12 [7]. CREStereo utilize datasets for training, including SF [34], Sintel [6]. FallingTThing [48], InStere02K [3], Carla [14), AirSim [41], and CREStereo dataset [27]. The best results are in bold, and the second-best are underlined.
Sintel Stereo
Dynamic Replica
Training data
Mothod
Clean
Final
First 150 frames
A8px
TEPP
i
Spez
8ape
TEPE
4
Sps
81px
TEPP
ipp 33p
CODD [30]
8.68
1.44
10.8
5.65
17:46
2.32
18.56
9.79
6.59
00.105
1.04
0.42
RAFT-Sterco [32]
6.12
0.92
9.33
4.51
10:40
2.10
13.69
7.08
5.51
0,145
2.03
DynamicStereo [24]
6.10
0.77
8.41
3.93
8.97
1.45
11,95
5.98
3.44
0.087
0.75
0.24
SF
BiDAStereo [22]
5.94
0.73
8.29
3.79
8.78
1.26
11.65
5.53
5.17
0.103
1.11 0.40
PPMSteren (Ours)
5.34
0.64
7.38
3,40
7.87
1.14
10.12
4.99
2.95
0.066
0.67
0.23
PPMStereo_ VDA (Ours)
4.62
0.58
6,89
3.08
7.21
1.04
9.84
4.65
2.37
0.059
0.61
0.22
SF
CODD [30]
9.11
1.33
12.16
6.23
11.90
2.01
16.16
8.64
10.03
0.152
2.16
0.77
SF M
RAFT-Sierro [32]
5.86
0.85
8.79
4.13
8.47
1.63
12.40
6.23
3.46
0.114
1.34
0.41
datasets (inel. Sintel)
CREStereo [29]
4,58
0.67
6.36
3,26
8.17
1.90
12.29
6.87
1.75
0.088
0.88
0.29
DR SF
RAPT-Stereo [32]
5.71
0.84
9.15
9.16
2.27
7.17
00.075
0.77
DR SF
DynamicSterco [24]
5.77
0,76
8.46
3.93
8.68
1.42
11.93
5.92
3.32
0.075
0.68
0.23
DR SF
BiDAStereo [22]
5.75
0.75
8.03
3.76
8.52
1.22
11.04
5.30
2.81
0,062
0.62
0,22
DR SF
PPMSteren (Ours)
5.19
0.62
7.21
3.29
7.64
9.98
4.87
2.52
0.057
0.60
0.20
DR SF
PPMStereo, VDA (Ours)
4.47
0.56
6.69
2.97
7.03
1.02
9.65
4.51
1.81
0.052
0.51
0.17
Left Frame
DynamicSterco
PPMStoreo (Ours)
PPMStereo_ VDA (Ours)
Figure 4: Qualitative comparisons on the Sintel final dataset.
iterations N to 20, while setting N 10 during training. Besides, we adopt n-pixel error rate (5mpz) for accuracy analysis. Additionally, we use the temporal end-point-error (TEPE) to quantify error variation over time, and 6m denotes the percentage of pixels with TEPE exceeding pixels. Lower values on metrics indicate greater temporal consistency and disparity estimation accuracy. Besides, we replace our original feature extractor with Video Depth Anything (ViT-Small) [8]. This PPMStereo_ VDA variant leverages pre-trained representations to further boost performance.
4.3 Comparison with State-of-the-Art Methods
Quantitative Results. As shown in Tab. 1, For the SF version, PPMStereo achieves kate-of-the art performance, outperforming BiDAStereo [24] by 12.3% & 9.52% and DynamicStereo by 16.8% & 21.3% in TEPE on Sintel clean/final pass. The method also demonstrates strong generalization on Dynamic Replica, surpassing all previous approaches across all metrics. Remarkably, our PPM- Stereo trained only on synthetic data even largely exceeds the temporal consistency and accuracy of CREStereo [27] on Sintel final pass, despite CREStereo using Sintel data for training. For the SF & DR version, our method achieves superior temporal consistency with a TEPE of 0.057 on Dynamic Replica, significantly outperforming all previous works. Notably, this is achieved with training on only two synthetic datasets, while CREStereo [27] requires seven diverse datasets, demonstrating the efficacy of our long-range temporal modeling. Overall, the results highlight our method's 's robust performance and generalization ability in both seen and unseen domains. Besides, compared to the previous SoTA method BiDAStereo [22], our method achieves better performance with lower computational costs and memory usage (Please see the appendix for details).
Qualitative Results. Our visual comparisons (Fig. 4) using the DR+SF checkpoint show PPM- Stereo produces sharper disparity predictions than DynamicStereo [24] and BiDAStereo [22], especially in textureless regions (e.g., glass surfaces) where competing methods exhibit blurring artifacts. Besides, following prior work [22, 24], we validate temporal consistency on static scenes by rendering depth point clouds at 1e-degree viewpoint increments (Fig. 5). Our method shows significantly smaller high-
Table 2: Ablations of memory buffer module vari- ants trained on DR+SF 'OOM' denotes CUDA out of memory. 'Baseline' refers to our backbone model without any melory-related modules.
Sintel Final
Dynamic Replica
Experiments
Method
8sps
TEPE
81pp
TEPE
Baseline
8.65
1.37
|
3.10
0.074
Full
00M
OOM
MemFlow [15]
8.45
1.28
3.11
0.070
Latest
8.11
1.19
2.89
0.062
Memory Butfer
Random
8.42
1.26
0.064
XMem [9]
8.04
1.18
2.84
0.061
RMem [66]
7.93
1.16
2.77
0.061
Ours
7.64
1.11
2.52
0.057
K=1
7.95
1.18
2.70
0.062
K =3
7,80
1.13
2.58
0.057
Memory Length
==
7.64
1.11
2.52
0.057
==
7.62
1.10
2.50
0.057

---

Mean
DynamicStereo
BidaStereo
PPMStereo (Ours)
Figure 5: Temporal consistency comparison on 50-frame reconstructed stereo video (all trained on DR SF). Our method achieves lower variance, demonstrating superior consistency.
Left Frames
DynamicStereo
BidaStereo
PPMStereo (Ours)
Figure 6: Qualitative generalization comparison on a dynamic outdoor scenario from the SV dataset.
variance regions 40 px². marked red), con- firming superior stability. Furthermore, on the real-world outdoor scenes from the South Kens- ington dataset [23] (Fig. 6), PPMStereo accu- rately recovers thin structures such as the fences while maintaining temporal consistency, demon-
strating robust generalization to unseen domains. More visualizations are provided in the appendix.
4.4 Ablation Studies
Due the huge training cost of PPMStereo VPA, conduct ablation studies exclusively PPMStereo below. Besides, all ablated models below are trained on DR + SF.
Memory buffer construction. We train and evaluate 5 different memory buffer variants, namely, keeping frames from (I) full frames (20 frames), (2) MemFlow (I frame) [15], (3) the latest frames (5 frames), (4) random (5 frames), (5) XMem [9] (distilling all outdated memory features into long-term memory based on attention scores), (6) RMem [66] (5 frames), and (7) ours (5 frames).
Specifically, we replace the memory buffer variants and keep the remaining modules unchanged during training and inference. Table 2 shows three key insights: First, while reference frames improve performance, naive accumulation shows diminishing returns, indicating memory capacity alone is insufficient. Second, frame selection quality critically affects results. The random selection policy underperforms even single-neighbor memory (MemFlow) [15] Sintel final pass, highlighting selection importance. However, on the DR dataset with minimal inter-frame changes, random policy performs comparably to advanced variants. Lastly, direct long-temn memory integration

---

Table 3: Ablation Study of PPM on Sintel and Dynamic Replica. All models are trained on DR+SF. Note that we directly perform the read-out operation for the ablated model without the 'play' process.
Pick-and-PPee Play Memory
Sintel Final
Dynamic Replica
Play
63pe
TEPE
6ips
61pe
TEPE
6ips
Bascline
8.65
1.37
11.72
5.91
3.10
0.074
0.72
0.23
7.81
1.14
10.24
5.07
2.65
0.060
0.64
0.21
7.97
1.17
10.36
5.20
2,80
0.062
0.68
0.21
7.64
1.11
9.98
4.87
2.52
0.057
0.60
0.20
Table 4: Ablation study on the 'pick' process. C, Sim, and R denote confidence score, similarity score, and redundancy factor, respectively.
Table 5: Ablation study on the 'play' process. Weights and PE denote the weighting operation and the temporal position encoding, respectively.
QAM
Sintel Final
Dynamic Replicu
Play Process
Sintel Final
Dynamic Replica
Sim
8ape
TEPE
Aiys
TEPE
619s
Weights
Aapx
TEPE
8pp
81pe
TEPE
61ps
Raseline
7.97
1.17
5.20
2,80
0.062
0.68
Baseline
7.81
1.14
5.07
2.65
0.060
0.64
7.81
1.14
5.06
2.63
0.058
0.65
7.67
1.12
5.00
2.54
0.060
0.62
3
7.74
1.12
4.95
2,57
0.057
0.62
7.77
1.11
4.93
2.63
0.058
0.61
7.64
1.11
4.87
2.52
0.057
0.60
7.64
111
4.87
2.52
0,057
0.60
(XMem) shows limited impact, suggesting that simply using all frames may be less effective than RMem variant. contrast, our PPM mechanism overcomes these limitations by dynamically identifying and modulating valuable reference frames, achieving significant TEPE improvements on these two datasets (+19.0% TEPE on Sintel and +22.9% TEPE on DR) over the baseline.
Memory length. Table shows the impact of memory length on PPMStereo, Performance improves initially (c.g.. +14.8% 8f on Sintel for < when trained and evaluated at this memory length, but performance saturates beyond K = due to feature redundancy. To balance computational efficiency and model accuracy, we select K as the optimal memory length for our final model.
Contribution of each component. Table shows the proposed PPM module outperforms window- based aggregation through key processes: (1) The pick process dynamically selects high-quality memory elements from non-adjacent frames, overcoming fixed-window limitations and improving occlusion handling; (2) The play process adaptively weights features by semantic relevance, reducing noise propagation (ID = shows +0.2 on Sintel and +0.017 TEPE improvements on DR compared to the baseline). By combining them, they provide complementary benefits. The pick ensures feature diversity while play suppresses outliers, yielding superior performance in dynamic stereo matching.
QAM. Our QAM module dynamically assesses frame reliability in the memory buffer using scoring mechanism. We refresh the memory buffer by balancing: (1) cost feature quality (Em) and (2) redundancy-avee semantic relevance (k.,.) (Sec. 3.2). Table 4 shows that quality score improves both depth accuracy and temporal consistency. Fig. further confirms the confidence map's strong correlation with the map, validating it reliable quality indicator for Um-
(b) Predicted Disparity Map Figure 7: Visualization of eπror map and confidence map. Brighter regions denote higher uncertainty.
(a) Left Frame
(c) Error Map
(d) Confidence Map
Memory modulation. Our proposed memory modulation mechanism (Sec. 3.3) further enhances spatio-temporal modeling, achieving a performance gain with +0.17 8zpz and +0.13 dipz improve- ments on the Sintel Final and DR, respectively, as seen in Table The adaptive weighting mechanism dynamically prioritizes the most important spatio-temporal features, highlighting accuracy improve- ments. Meanwhile, learned positional embeddings endow the model with temporal awareness, improving the overall temporal consistency. Experiments show that these components work to- gether to strengthen the model's s ability capture long-range dependencies and distinguish key spatio-temporal patterns.
10

---

Conclusion
In this paper, we introduce PPMStereo, the first framework, our knowledge, to leverage high-quality memory for dynamic stereo matching. By selectively updating and modulating the most valuable memory entries, our proposed pick-and-play memory construction mechanism enables the integration of cost information across long-range spatio-temporal connections, ensuring temporally consistent stereo matching. Extensive experiments demonstrate the effectiveness of our approach across diverse datasets, highlighting its generic applicability.
Acknowledgment
This work was partly supported by the Shenzhen Science and Technology Program under Grant (CB82003322100073666
GuangDong Basic and Applied Basic Research Foundation under Grant 2023A15151 10074. This work was also supported by the InnoHK Initiative of the Government the Hong Kong SAR and the Laboratory for Artificial Intelligence (AT)-Powered Financial Technologies, with additional support from the Hong Kong Research Grants Council (RGC) grant C1042-23GF and the Hong Kong Innovation and Technology Fund (ITF) grant MHP/061/23,
References
[1] Peter Auer. Using confidence bounds for exploitation-exxllraiio trade-offs, Journal of Machine Learning Research (JMLR), 3(Nov):397-422 422, 2002.
[2] Antyanta Bangunharcana, Jac Won Cho, Scokju Lee, In So Kweon, Kyung-Soo Kim, and Soohyun Kim. Correlate-and-exciin Real-time stereo matching via guided cost volume excita- tion. In 202/ IEEE International Conference on Intelligent Robots and Systems (IROS), pages 3542 3548. IEEE, 2021.
[3] Wei Bao, Wei Wang, Yuhua Xu, Yulan Guo, Siyu Hong, and Xiaohu Zhang. Instereo2k: large real dataset for stereo matching in indoor scenes. Science China Information Sciences, 63:1-11, 2020.
[4] Luca Bartolomei, Fabio Tosi, Matteo Poggi, and Stefano Mattoccia. Stereo anywhere: Robust zero-shot deep stereo matching even where either stereo mono fail. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 1013-1027, 2025.
[5] Rajesh Bhargave, Amitav Chakravarti, and Abhijit Guha. Two-stage decisions increase prefer- ence for hedonic options. Organizutional Behavior and Human Decision Processes, 130:123 135, 2015.
[6] Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In Proceedings of the European conference on computer vision (ECCV), pages 1-1.2255. Springer, 2012.
[7] Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual kitti 2. arXiv preprint arXiv:2001.1077 2020.
[8] Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zilong Huang, Jiashi Feng, and Bingyi Kang. Video depth anything: Consistent depth estimation for super-long videos. 2025.
[9] Ho Kei Cheng and Alexander Schwing. Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model. In European Conference Computer Vision (ECCV), pages 640-658. Springer, 2022.
[10] Ho Kei Cheng, Yu-Wing Tai, and Chi- Tang. Rethinking space-time networks with improved memory coverage for efficient video object segmentation. Advances in Neural Information Processing Systems (NeuralIPS), 34:1 34:11781-11794, 2021.
[11] Junda Cheng, Longliang Liu, Gangwei Xu, Xianqi Wang, Zhaoxing Zhang, Yong Deng, Jinliang Zang, Yurui Chen, Zhipeng Cai, and Xin Yang. Monster: Marry monodepth stereo unleashes power. 2025.
11

---

[12] Ziang Cheng, Jiayu Yang. and Hongdong Li. Stereo matching in time: 100+ fps video stereo matching for extended reality. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), pages 8719-8778 8728, 2024.
[13] Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory--fficiint exact attention with io-awareness. Advances in Neural Information Processing Systems (Neural/PS), 35: 63344-6639, 2022.
[14) Jean-Fmmanuel Deschaud. Kitti-carla: kitti-like dataset generated by carla simulator, arXiv preprint arXiv:2109.00880 2021.
[15] Qiaole Dong and Yanwei Fu, Memflow: Optical flow estimation and prediction with memory. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 19068-19078) 2024.
[16] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprins arXfv:2010.11939 2020.
[17] Zhihong Fu, Qingjie Liu, Zehua Fu, and Yunhong Wang. Stmtrack: Template-free visual tracking with space-time memory networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 13774-13783, 2021.
[18] Herbert Gintis. A framework for the unification of the behavioral sciences. Behavioral and brain sciences, 10,,) 2007.
[19] Xianda Guo, Chenming Zhang, Youmin Zhang, Wenzhao Zheng, Dujun Nie, Matteo Poggi, and Long Chen. Lightstereo: Channel boost is all you need for efficient 2d cost aggregation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 4738-87444 IEEE, 2025.
[20] Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xuefei Cao, Ashish Shah, Abhinav Shrivastava, and Ser-Nam Lim. Ma-lmm: Memory-augmentte large multimodal model for long-term video understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 13504-135 13514, 2024.
[21] Heiko Hirschmuller. Stereo processing by semiglobal matching and mutual information. IEEE Transactions Pattern Analysis and Machine Intelligence (TPAMI), 30(2):328 -41,,441111 2007.
[22] Junpeng Jing, Ye Mao, and Krystian Mikolajezyk. siich-stereo-videos: Bidirectional alignment for consistent dynamic stereo matching. In European Conference on Computer Vision (ECCV), pages 415-432. Springer, 2024.
[23] Junpeng Jing, Ye Mao, Anlan Qiu, and Krystian Mikolajezyk. Match stereo videos via bidirec- tional alignment. 2024.
[24] Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Dynamicstereo: Consistent dynamic depth from stereo videos. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2023.
pages
[25] Alex Kendall, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, and Adam Bry. End-to-end learning of geometry and context for deep stereo regression. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 66 06-75,200 7,,75,,00 2017.
[26] Jiahao Li, Xinhong Chen, Zhengmin Jiang, Qian Zhou, Yung-Hui Li, and Jianping Wang. Global regulation and excitation via attention tuning for stereo matching. arXiv preprint arXiv:2509. /5891, 2025.
[27] Jiankun Li, Peisen Wang, Pengfei Xiong, Tao Cai, Ziwei Yan, Lei Yang, Jiangyu Liu, Haoqiang Fan, and Shuaicheng Liu. Practical stereo matching via cascaded recurrent network with adaptive correlation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 16263-16272, 2022.
12

---

[28] Kunhong Li, Longguang Wang, Ye Zhang, Kaiwen Xue. Shumbo Zhou, and Yulan Guo. Los: Local structure guided stereo matching. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2024.
[29] Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In European Conference on Computer Vision (ECCV), pages 280-296 Springer, 2022.
[30] Zhaoshuo Li, Wei Ye, Dilin Wang, Francis X Creighton, Russell H Taylor, Ganesh Venkatesh, and Mathias Unberath. Temporally consistent online depth estimation in dynamic scenes, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pages 2018-3027, 2023.
[31] Zhengfa Liang, Yulan Guo, Yiliu Feng, Wei Chen, Linbo Qiao, Li Zhou, Jianfeng Zhang, and Hengzhu Liu. Stereo matching using multi-level cost volume and multi-scale feature constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019.
[32] Lahav Lipson, Zachary Teed, and Jia Deng. RAFT-Stereo: Multilevel recurrent field transforms for stereo matching. 2021 International Conference on 3D Vision (3DV), pages 218-227, 2021.
[33] Yamin Mao, Zhihua Liu, Weiming Li, Yuchao Dai, Qiang Wang, Yun-Tae Kim, and Hong-Seok Lee. Uasnet: Uncertainty adaptive sampling network for deep stereo matching. Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 631 1-6319, 2021.
[34] Nikolaus Mayer, Eddy 1lg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pages 4040-4048, 2016.
[35] Moritz Menze and Andreas Geiger. Object scene flow for autonomous vehicles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3061-3070, 2015.
[36] Roy Miles, Mehmet Kerim Yucel, Bruno Manganelli, and Albert Saa-Gariiga. Mobilevos: Real-time video object segmentation contrastive learning meets knowledge distillation. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), pages 10480-10490, 2023.
[37] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma. Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yun Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam Segment anything images and videos. arXiv preprint arXiv:2408.00714, 2024.
[38] Laurie R Santos and Alexandra G Rosati. The evolutionary roots of human decision making. Annual review of psychology, 66(1 (311)3334347, 2015.
[39] Daniel Scharstein, Heiko Hirschmüller, York Kitajima, Greg Krathwohl, Nera Nešić, Xi Wang. and Porter Westling. High-resolution stereo datasets with subpixel-accurate ground truth. In German conference on pattern recognition (GCPR), pages 31-42. Springer, 2014.
[40] Thomas Schöps, Johannes L. Schönberger, S. Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. A multi-view stereo benchmark with high-resolution images and multi-camera videos. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 2538-2547, 2017.
[41] Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and Service Robotics: Results of the 1 11th International Conference, pages 621-635. Springer, 2018
[42] Zhelun Shen, Yuchao Dai, and Zhibo Rao. CFNet: Cascade and fused cost volume for robust stereo matching. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition (CVPR), pages 13906-13915, 2021.
13

---

[43] Zhelun Shen, Xibin Song, Yuchao Dai, Dingfu Zhou, Zhibo Rao, and Liangjun Zhang. Digging into uncertainty- based pseudo-label for robust stereo matching. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 30(2) 2023.
[44] Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 18221-18232, 2024.
[45] Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End-to-end memory networks, Advances in Neural Information Processing Systems (NeuralIPS), 28, 2015.
[46] Vladimir Tankovich, Christian Hane, Yinda Zhang, Adarsh Kowdle, Sean Fanello, and Sofien Bouaziz. HITNet: Hierarchical iterative tile refinement network for real-time stereo matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1-362-14372, 2021.
[47] Fabio Tosi, Luca Bartolomei, and Matteo Poggi. A survey on deep stereo matching in the twenties. International Journal of Computer Vision (IJCV), 133(7):4245 4276, 2025.
[48] Jonathan Tremblay, Thang To, and Stan Birchfield. Falling things: A synthetic dataset for 3d object detection and pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2038 2038-2041, 2018.
[49] Chen Wang, Xiang Wang, Jiawei Zhang, Liang Zhang, Xiao Bai, Xin Ning, Jun Zhou, and Edwin Hancock. Uncertainty estimation for stereo matching based on evidential deep learning. Pattern Recognition, 124; 108498, 2022.
[50] Xianqi Wang, Gangwei Xu, Hao Jia and Xin Yang. Selective-stereo: Adaptive frequency information selection for stereo matching, arXiv preprint arXiv:2403.00488, 2024.
[51] Yun Wang, Junjie Hu, Junhui Hou, Chenghao Zhang, Renwei Yang, and Dapeng Oliver* Wu. Rose: Robust self-supervised stereo matching under adverse weather conditions. IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2025.
[52] Yun Wang. Kunhong Li, Longguang Wang, Junjie Hu, Dapeng Oliver Wu, and Yulan Guo. Adstereo: Efficient stereo matching with adaptive downsampling and disparity alignment. JEEE Transuctions on Image Processing (TIP), 2025.
[53] Yun Wang. Longguang Wang. Kunhong Li, Yongjian Zhang, Dapeng Oliver Wu, and Yulan Guo. Cost volume aggregation in stereo matching revisited: A disparity classification perspective. IEEE Transactions on Image Processing (TIP), 2024.
[54] Yun Wang, Longguang Wang, Hanyun Wang, and Yulan Guo. SPNet:
Learning stereo matching with slanted plane aggregation. IEEE Robotics and Automation Letters, 2022.
[55] Yun Wang, Longguang Wang, Chenghao Zhang, Yongjian Zhang, Zhanjie Zhang, Ao Ma, Chenyou Fan, Tin Lun Lam, and Junjie Hu. Learning robust stereo matching in the wild with selective mixture-of-experrs. arXiv preprint arXi::2507.0466 2025.
[56] Yun Wang, Jiahao Zheng, Chenghao Zhang, Zhanjie Zhang, Kunhong Li, Yongjian Zhang, and Junjie Hu. Dualnet: Robust self-supervised stereo matching with pseudo-label supervision. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAT), volume 39, pages 8178-81 8186, 2025.
[57] Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, and Stan Birchfield. Foundationstereo: Zero-shot stereo matching. In Proceedings of the Computer Vision and Puttern Recognition Conference (CVPR), pages 5249-52600 2025.
[58] Gangwei Xu, Xianqi Wang, Xiaohuan Ding, and Xin Yang. Iterative geometry encoding volume for stereo matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 11919-21928, 2023.
14

---

[59] Haofei Xu and Juyong Zhang. AANet: Adaptive aggregation network for efficient stereo matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1959-1968, 2020.
[60] Tianyu Yang and Antoni B Chan. Learning dynamic memory networks for object tracking. In Proceedings of the European Conference on Computer Vision (ECCV), pages 152-167, 2018.
[61] Jiaxi Zeng, Chengtang Yao, Yuwei Wu, and Yunde Jia. Temporally consistent stereo matching. In European Conference Computer Vision (ECCV), pages 341-359, Springer, 2024.
[62] Feihu Zhang, Victor Prisacariu, Ruigang Yang, and Philip HS Torr. GA-Net: Guided aggregation net for end-to-end stereo matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 185-194, 2019.
[63] Jiawei Zhang, Xiang Wang, Xiao Bai, Chen Wang, Lei Huang, Yimin Chen, Lin Gu, Jun Zhou, Tatsuya Harada, and Edwin R Hancock. Revisiting domain generalized stereo matching networks from a feature consistency perspective. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 13001-13011, 2022.
[64] Yongjian Zhang, Longguang Wang, Kunhong Li, Yun Wang. and Yulan Guo. Learning rep- resentations from foundation models for domain generalized stereo matching. In European Conference Computer Vision (ECCV), pages 146-162. Springer, 2024.
[65] Youmin Zhang, Matteo Poggi, and Stefano Mattoccia. Temporalstereo: Efficient spatial- temporal stereo matching network. In 2023 IEEE/RS International Conference on Intelligent Robots and Systems (IROS), pages 9528-95251 9535. IEEE, 2023.
[66] Junbao Zhou, Ziqi Pang, and Yu-Xiong Wang. Rmem: Restricted memory banks improve video object segmentation. In Proceedings of the IEEE Conference Computer Vision and Patlern Recognition (CVPR), pages 18602-18611 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The claims around performance, runtime, and comparison to existing methods are described in the experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: A common limitation among our work and existing works is the inability proactively distinguish between dynamic and static areas, which is the key to ensuring consistency. Moving forward, our focus lies exploring how to integrate our method with high-quality memory cues enhance the performance and in developing lightweight version of the model.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
16

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
No theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the full details necessary reproduce these results, including architecture details and implementation details of the training datasets and procedures, evaluation protocols for dynamic stereo matching. Details can be seen in the Experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
17

---

reproducible, they some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification:
We will release the codes if the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
ceiiionooiicccy for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. wwwppcpppiiccccoccccccc cubbiccguuide, /ooee//dddmmmmmi wentibmisssiin ssissssooooooso ssionnolicy) Policcy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide details on our architecture in the experiment section. The dataset splits were used and shared in previous work.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For dynamic stereo matching metrics, we follow prior work and provide TEPE and 8f mpp to evaluate the temporal consistency.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide comparison of computational resources used in our experiments alongside previous works, with detailed results in the Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't itdn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: / /ww.unnuu.pp..// wttpppp...pppppppeeeeeeeep cc/public/ /Ethhi csGuidelines?
Answer: [Yes]
Justification:
We have reviewed the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work provides generic algorithm to learn high-guality memory buffer for dynamic stereo matching. In the guidelines, it is reported that algorithms to optimize neural networks do not need any societal impact justification.
Guidelines:
19

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite the original paper that produced the code package or datasets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: Our method introduces novel dynamic stereo matching approach without requiring new dataset.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Paper does not involve crowdsourcing nor resarch with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
We only use LLMs for improving writing.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttp///errpp/////ee cc/Conffreecces/2255 /LLM) for what should or should be described.
22


Reviews:
[{"review_id": "55B6RMi7nO", "reviewer": "Reviewer_HYhV", "summary": "This paper proposes a method to improve temporal consistency by capturing long-range dependencies across video frames while maintaining computational efficiency through a novel memory buffer mechanism. The approach first selects the most relevant frames using confidence scores and redundancy-aware relevance metrics, then adaptively weights and aggregates these frames for spatio-temporal feature integration. Experimental results show the proposed method outperforms existing approaches across three benchmark datasets in both accuracy and temporal consistency.", "questions": "1）The paper's organization is somewhat disorganized. \n2）The excessive use of mathematical notation hinders readability. \n3）The paper reads more like a technical report, primarily focusing on implementation details while providing limited explanation of the underlying motivations and theoretical principles.", "limitations": "Does the keyframe extraction and compression process affect inference speed?\nHow many frames can the performance improvement from long-term temporal information be sustained? The paper only tests up to 20 frames?", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1）The proposed method demonstrates significant performance improvements.\n2）The experiments are comprehensive, providing detailed validation of the method's effectiveness.\n\nWeaknesses\n1）The paper's organization is somewhat disorganized. The core idea—selecting and aggregating key frames to balance long-term temporal modeling and computational efficiency could be presented more clearly. A top-down structure (general to specific) would improve readability, rather than introducing excessive details and equations in Section 3.1.\n2）The excessive use of mathematical notation hinders readability. Additionally, some equations (e.g., (1) and (2)) are overly simplistic and could be presented inline rather than as standalone equations to save space.\n3）The paper reads more like a technical report, primarily focusing on implementation details while providing limited explanation of the underlying motivations and theoretical principles.", "was_revised": true, "final_justification": "My main concerns are well addressed. Therefore, I recommend accepting this paper.", "rebuttal": "Thank you for reviewing our paper. We appreciate your valuable feedback and will try to address your concerns below:\n\n**W1:** The paper's organization is somewhat disorganized. The core idea—selecting and aggregating key frames to balance long-term temporal modeling and computational efficiency could be presented more clearly. A top-down structure (general to specific) would improve readability, rather than introducing excessive details and equations in Section 3.1.\n\n**A1**: We thank you for the valuable comment. To improve clarity and follow a top-down structure, we will revise Section 3.1 to provide a clearer and more logically organized presentation of our method.\nIn the revised Section 3.1 (Overview), we will remove non-essential mathematical equations and avoid excessive details,  reorganizing our approach. \n\nWe begin by emphasizing the motivation behind our method:\n\"Dynamic stereo matching aims to generate a sequence of temporally consistent disparity maps from stereo video inputs. While prior methods often struggle to capture long-range temporal dependencies efficiently, the proposed framework, PPMStereo, addresses this challenge by introducing a novel Pick-and-Play Memory (PPM) module. This module selectively aggregates high-quality reference frames into a compact memory buffer, enabling robust spatio-temporal modeling while maintaining computational efficiency.\nAs illustrated in Fig. 2, PPMStereo is built on the DynamicStereo [22] backbone, with our PPM module serving as the core innovation.\"\n\nTo improve the implementation details' readability, we will restructure the pipeline description as follows:\n\n(1) Feature Extraction\n\n(2) Cost Volume Construction\n\n(3) Context Encoding\n\n(4) Memory Buffer Initialization\n\n(5) Pick-and-Play Memory Module\n\n(6) Iterative Refinement\n\nIf you still have concerns about our presentation framework, we would greatly appreciate any further feedback.\n\n**W2:** The excessive use of mathematical notation hinders readability. Additionally, some equations (e.g., (1) and (2)) are overly simplistic and could be presented inline rather than as standalone equations to save space.\n\n**A2:** We thank you for the constructive feedback. In the revised version, we will streamline the presentation by moving simpler expressions, such as Equations (1) and (2), to an inline format where appropriate. \nAdditionally, we will revisit the mathematical content to remove non-essential notations and replace them with clearer explanations. We believe these revisions significantly enhance readability while preserving the depth and clarity of our core contributions.\n\n**W3:** The paper reads more like a technical report, primarily focusing on implementation details while providing limited explanation of the underlying motivations and theoretical principles.\n\n**A3:** We sincerely appreciate your thoughtful suggestions.  To further enhance the clarity of our paper, we will revise the abstract to more clearly state the research problem and the rationale behind our approach, and we will expand the introduction to deepen readers’ understanding of both our motivation and theoretical framework. Furthermore, we will reorganize Section 3.1 to highlight our motivation explicitly, and we will enrich Sections 3.2 and 3.3 with a more detailed discussion of the relevant theoretical principles. If you believe these revisions to the abstract, introduction, and Sections 3.1–3.3 remain insufficient, we would greatly appreciate any further feedback.\n\n**L1:** Does the keyframe extraction and compression process affect inference speed?\n\n**R1:** Thank you for the valuable comment. The keyframe extraction and compression process has a negligible impact on overall inference. The keyframe extraction and compression process corresponds to the \"Pick Process\", which is lightweight when evaluating reference frames. To validate this, we analyzed the \"Pick Process\" within the Pick-and-Play Memory mechanism over 20 iterations using 20-frame video sequences from the Sintel Stereo dataset (resolution: 1024 × 436) on an NVIDIA A100 GPU. The table below presents the average inference time for each component:\n\n|Model Variants|Inference Time (s)|\n|:---------------: | :--------------: |\n|Keyframe Extraction \\& Compression Process (Pick Process) |11.8|\n|Random Keyframe Selection|11.4|\n\nAs shown, the \"Pick Process\" introduces only a negligible increase in inference time, approximately 0.4 seconds, compared to a baseline that uses random keyframe selection. This marginal difference demonstrates that our keyframe extraction and compression pipeline incurs negligible computational overhead, preserving runtime efficiency while enabling reliable reference frame selection. \n\n**L2:** How many frames can the performance improvement from long-term temporal information be sustained? The paper only tests up to 20 frames?\n\n**R2:** We thank you for the insightful question. To ensure a fair comparison, our main experiments are conducted with 20-frame video sequences, following the protocol of prior work [20, 22]. Additionally, to assess performance on longer temporal sequences, we conduct experiments using varying sequence lengths, as shown below:\n|Number of Frames|Sintel Clean(TEPE)|Sintel Clean(Tδ1px)|Sintel Clean(Tδ3px)|Sintel Final (TEPE)|Sintel Final(Tδ1px)|Sintel Final(Tδ3px)|\n| :----------------: | :-----------------------: | :----------------: | :-----------------------: |:-----------------------: |:-----------------------: |:-----------------------: |\n|$T$ = 20|0.62|7.21|3.29|1.11|9.98|4.87|\n|$T$ = 30|0.61|7.17|3.29|1.09|9.92|4.81|\n|$T$ = 40|0.61|7.17|3.28|1.09|9.93|4.82|\n|$T$ = 50|0.61|7.18|3.28|1.09|9.92|4.82|\n\nAs the sequence length increases, performance initially improves, but eventually reaches a saturation point. This saturation arises because overly long sequences introduce redundant or irrelevant information, which eventually contributes little to disparity estimation.\nOverall, the performance improvement of PPMStereo from long-term temporal information can be sustained up to 30 frames without degradation."}, {"review_id": "tgzdRqNPks", "reviewer": "Reviewer_6N4q", "summary": "The paper presents a novel framework called PPMStereo for dynamic stereo matching with temporally consistent disparity estimation in video sequences. The authors address the challenge of capturing long-range temporal relationships efficiently by introducing a Pick-and-Play Memory (PPM) module. The system operates through two primary stages:\n\nPick: Identifying relevant frames based on a Quality Assessment Module (QAM).\nPlay: Adapting the aggregation of features from selected frames to achieve accurate and temporally consistent disparity maps.", "questions": "Please refer to weakness", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.Innovative Memory Mechanism: The introduction of a memory buffer for spatio-temporal consistency in dynamic stereo matching is novel. The \"Pick-and-Play\" strategy improves computational efficiency by dynamically selecting and aggregating relevant frames.\n2.Strong Experimental Results: The method achieves state-of-the-art performance on multiple dynamic stereo benchmarks, including Sintel, Dynamic Replica, and Sintel Final Pass. Notably, it outperforms previous methods such as BiDAStereo and DynamicStereo in terms of temporal consistency and accuracy with reduced computational cost.The ablation studies are comprehensive, demonstrating the impact of different components of the framework.\n3.Generalization across Datasets: The method demonstrates strong generalization, with superior results on both synthetic and real-world datasets.\n\nWeaknesses:\n1.In Table 1, the Memory Length decreases gradually from 1 to 7. The reviewers would like to know the results when k is larger, such as k=9 and k=11.\n2.Computational Efficiency: A more detailed discussion of computational costs, including GPU memory usage and processing time per frame or video sequence, would help clarify the trade-offs in terms of efficiency versus accuracy.\n3.In the \"Deep Stereo Matching\" section of the related work, there is a lack of citations to some latest papers:\n1） \"Openstereo: A comprehensive benchmark for stereo matching and strong baseline.\" arXiv preprint arXiv:2312.00343 (2025).\n2）\"Lightstereo: Channel boost is all you need for efficient 2d cost aggregation.\" (ICRA 2025).", "was_revised": true, "final_justification": "Thanks for the response. I am satisfied with the authors' response, so I maintain my opinion of accepting the paper.", "rebuttal": "Thank you for reviewing our paper. We appreciate your valuable feedback and will try to address your concerns below:\n\n**W1:** In Table 1, the Memory Length decreases gradually from 1 to 7. The reviewers would like to know the results when k is larger, such as k=9 and k=11.\n\n**A1:** We thank you for the helpful suggestion.  In response, we conduct our experiments on larger memory lengths ($K$ = 9, 11). The result is shown in the table below:\n\n| Memory Length (*K*) | Sintel  (δ3px) | Sintel  (TEPE) | Dynamic Replica (δ1px) | Dynamic Replica (TEPE) |\n| :-----------------: | :---------------: | :--------------: | :----------------: | :-----------------------: |\n|Baseline|8.65|1.37|3.10|0.074|\n|5| 7.64 |1.11|2.52|0.057|\n|7| 7.62 |1.10|2.50|0.057|\n|9| **7.61**|**1.10**|**2.49**|**0.057**|\n|11|7.63|1.12|2.52|0.057|\n\nWe observe that increasing the memory length beyond 5 offers only marginal gains or even slight degradations when $K=11$. While intuitively, a longer memory bank might appear beneficial by providing a richer temporal context, it can inadvertently introduce redundant or less relevant information. This surplus of data may obscure the salient cues necessary for effective cost aggregation, ultimately impairing the model’s discriminative capability.  This observation is consistent with findings from RMem [1], which reported that overly long temporal contexts can result in diminishing returns. \nThese results highlight the importance of balancing temporal diversity with information relevance when designing memory systems. Accordingly, we adopt a memory length of $K = 5$ in our implementation, as it offers a good trade-off between accuracy and computational efficiency.\n\n[1] Zhou J, Pang Z, Wang Y X. Rmem: Restricted memory banks improve video object segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2024: 18602-18611.\n\n**W2:** Computational Efficiency: A more detailed discussion of computational costs, including GPU memory usage and processing time per frame or video sequence, would help clarify the trade-offs in terms of efficiency versus accuracy.\n\n**A2:** We appreciate your feedback regarding the computational efficiency. To more clearly illustrate the trade-offs between efficiency and accuracy, we measure GPU memory usage and processing time per video sequence during inference. Specifically, we conduct evaluations on a 20-frame video sequence from the Sintel Final Stereo dataset (resolution: 1024×436) using a single NVIDIA A100 GPU, varying the number of iterations *N*. The results are summarized in the table below:\n\n|Methods|Iterations|GPU Memory (GB)|Inference Time (s)|Sintel Final (TEPE)|Sintel Final (Tδ1px)|Sintel Final (Tδ3px)|\n| :-----------------: | :---------------: | :--------------: | :--------------: | :--------------: | :--------------: |  :--------------: | \n|DynamicStereo |*N*=20|**16.6**|3.4|1.42|11.93 | 5.92 | \n|BidaStereo |*N*=20|19.8|6.7|1.22|11.04| 5.30 |\n|Ours |*N*=5|20.7|**3.0**|1.22|11.84|5.36|\n|Ours |*N*=10|21.9|6.0|1.12|10.10|4.94|\n|Ours |*N*=15|23.4|8.9|**1.11**|9.99| 4.89|\n|Ours |*N*=20|24.4|11.8|**1.11**|**9.98**|**4.87**|\n\nWhile previous approaches require a large number of iterations (*N*=20) to achieve optimal accuracy, our method, as shown above, attains comparable accuracy while requiring fewer inference iterations (e.g., at *N*=5). Under the same inference iterations, our method largely outperforms the SoTA method BidaStereo across all three evaluation metrics. These results indicate that our approach strikes an effective balance between efficiency and accuracy. We will report these results in the revised manuscript to more clearly clarify our contributions.\nThanks to the PPM mechanism, our PPMStereo requires fewer iterations to converge. Notably, even with just 5 iterations (*N*=5), it achieves better temporal consistency than DynamicStereo, clearly demonstrating the effectiveness of the proposed method.\n\n**W3:** In the \"Deep Stereo Matching\" section of the related work, there is a lack of citations to some latest papers: 1） \"Openstereo: A comprehensive benchmark for stereo matching and strong baseline.\" arXiv preprint arXiv:2312.00343 (2025). 2）\"Lightstereo: Channel boost is all you need for efficient 2d cost aggregation.\" (ICRA 2025).\n\n**A3:** Thank you for the valuable suggestion. In the revised manuscript, we will add these citations to the Related Work section and include a corresponding discussion.  If you believe the current references are still insufficient, we would sincerely appreciate any additional suggestions."}, {"review_id": "pw4iBAJiTw", "reviewer": "Reviewer_Y5FR", "summary": "This paper proposes a novel dynamic stereo architecture that uses a memory bank approach with a custom Quality Assessment Module (QAM) to achieve state-of-the-art performance.\nIn particular, the authors propose an iterative-based (i.e., based on RAFT-Stereo architecture) dynamic stereo model that uses features not only from the cost volume and context image, but also from a Dynamic Memory Buffer.\nThe latter is used to store temporal consistency information, built using the proposed Memory Pick'n'Play Processes, that ensure a selection of the most relevant temporal information.\nThe memory bank approach is used not only to store features from different temporal frames but also to reduce memory consumption via a frame selection strategy, based on the outcome of the QAM module.\nExhaustive experiments prove the effectiveness of the architecture design and the performance of the model w.r.t. the previous state of the art.", "questions": "Before asking questions, I would like to introduce my rationale behind my initial rating: the paper proposes an incremental work, extending MemFlow for Dynamic Stereo Matching and introducing a revisited memory bank dubbed as Pick'n'Play Memory. The exhaustive experiments show SOTA performance, thus confirming a valid contribution for NIPS 2025.\n\n- Inside equation 8, there is $k'_m$ on both parts of the right equation. Is it right?\n- How $T$ -- a.k.a. number of reference frames -- hyper-parameter is chosen?\n- Please refer to Strengths and Weaknesses for further suggestions.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- **Exhaustive experiments showing SOTA performance**: The authors validate the potential of their proposal using different standard benchmarks such as Sintel, Dynamic Replica (Tab. 1). Furthermore, they qualitatively investigate in Fig. 6 the performance using a real-world dataset -- i.e., South Kensington. This largely proves the effectiveness of the proposed Pick'n'Play strategy.\n- **Novel Dynamic Stereo Architecture**: The authors revisited the MemFlow architecture for Dynamic Stereo Matching. Additionally, they changed the memory bank module with the improved Pick'n'Play proposal. The benefits of the latter solution w.r.t. MemFlow memory bank are shown in Table 2.\n- **Clear logic structure of the paper**: I personally liked the clear and easy-to-follow structure of the paper. The authors shortly introduce the concept of iterative stereo networks -- assuming a prior knowledge from the reader -- while putting a lot of effort into the Pick'n'Play processes, the core idea of the paper. Even so, chapter 3 was a little hard to digest at first: I suggest that the authors make more references to Fig. 2 and Fig. 3 inside the chapter.\n\nWeaknesses\n\n- **Related works regarding Deep Stereo Matching can be enlarged**: In my opinion, the related works regarding Deep Stereo Matching are quite poor, since the authors mainly report the GRU-based architectures. I suggest at least including a survey paper [1] to include additional literature regarding deep stereo matching.\n- **Missing competitors with VDA features**: The benefits of the VDA encoder are quite impressive. It is true that authors showed SOTA performance even without the VDA encoder; however, I would have liked to see a fairer comparison showing at least one competitor with the VDA encoder.\n- **The paper lacks a runtime memory consumption**: I suggest that the authors enrich Fig. 7 of the appendix by adding a measure of memory consumption during evaluation.\n\n\n[1] Tosi, Fabio, Luca Bartolomei, and Matteo Poggi. \"A survey on deep stereo matching in the twenties.\" International Journal of Computer Vision (2025): 1-32.", "was_revised": true, "final_justification": "After reading the rebuttal and other reviews, I would like to suggest the acceptance of the paper to NIPS 2025.\n\nThe reply made by the authors resolves all my initial concerns.", "rebuttal": "Thank you for reviewing our paper. We appreciate your valuable feedback and will try to address your concerns below:\n\n**S3:** Clear logic structure of the paper: The authors shortly introduce the concept of iterative stereo networks -- assuming a prior knowledge from the reader -- while putting a lot of effort into the Pick'n'Play processes, the core idea of the paper. Even so, chapter 3 was a little hard to digest at first: I suggest that the authors make more references to Fig. 2 and Fig. 3 inside the chapter.\n\n**R3:** Thank you for recognizing the clear logical structure of the paper. For your suggestion that the authors should make more references to Fig. 2 and Fig. 3 inside the chapter, we will organize Section 3.1 into the following coherent workflow:\n\n1. Feature Extraction\n\n2. Cost‑Volume Construction\n\n3. Context Encoding\n\n4. Memory Buffer Initialization\n\n5. Pick‑and‑Play Memory Module\n\n6. Iterative Refinement\n\nIn Section 3.2, we will restructure the description of the \"pick process\" using Figures 3 (b) and 3 (c) to enhance clarity.\n\nIn Section 3.3, we will reintroduce the  \"play process\"  to make it align with the workflow in Figure 3 (d).\nThis revised structure will guide the reader from the overall architecture to progressively more detailed components, helping to clarify our design details and contributions.\n\n**W1:** Related works regarding Deep Stereo Matching can be enlarged: In my opinion, the related works regarding Deep Stereo Matching are quite poor, since the authors mainly report the GRU-based architectures. I suggest at least including a survey paper [1] to include additional literature regarding deep stereo matching.\n[1] Tosi, Fabio, Luca Bartolomei, and Matteo Poggi. \"A survey on deep stereo matching in the twenties.\" International Journal of Computer Vision (2025): 1-32.\n\n**A1:** Thank you for your comments regarding the related works on deep stereo matching. We will revise the related work section accordingly and include the following additional reference as follows:\n\n\"**Deep Stereo Matching.** The field of deep stereo matching, as comprehensively surveyed by Tosi et al. (2025) [1], has undergone rapid evolution, particularly in network design focused on cost volume construction and aggregation. These methods are generally categorized into regression-based approaches (e.g., Guo et al. (2023) [2]; Guo et al. (2025) [3]) and iterative refinement paradigms. Further developments continue along these directions. and so on.\"\n\nIf you believe the current references are still insufficient, we would sincerely appreciate any additional suggestions.\n\n**W2:** Missing competitors with VDA features: The benefits of the VDA encoder are quite impressive. It is true that authors showed SOTA performance even without the VDA encoder; however, I would have liked to see a fairer comparison showing at least one competitor with the VDA encoder.\n\n**A2:** Thank you for highlighting this important issue. To address your concern regarding the impact of the VDA encoder, we conduct additional experiments by integrating the VDA encoder into previous SoTA models to enable a fairer comparison.\nAs shown in the table below, incorporating the VDA encoder into SoTA models consistently improves performance across all benchmarks and metrics:\n|Methods| Sintel Clean (EPE) | Sintel Clean (TEPE)| Sintel Clean (Tδ3px) | Sintel Final (EPE) | Sintel  Final (TEPE) | Sintel  Final (Tδ3px) |\n|:-----------------:| :---------------:|:--------------:|:----------------:| :-----------------------:|:----------------:| :-----------------------:|\n|DynamicStereo|1.34|0.76|3.93 |3.12|1.42|5.92|\n|DynamicStereo_VDA|1.02|0.65 |3.47|2.55|1.20|5.38|\n|BidaStereo|1.38|0.75|3.76 |2.04|1.22|5.30| \n|BidaStereo_VDA|1.04|0.65 | 3.41 |1.67|1.14|4.94|\n|PPMStereo|0.96|0.62|3.29|1.49|1.11|4.87|\n|PPMStereo_VDA|**0.81**|**0.56**|**2.97**|**1.24**|**1.02**|**4.51**|\n\nRemarkably, the proposed method, PPMStereo, still achieves SoTA results even without the VDA encoder, outperforming these VDA-enhanced baselines. When combined with the VDA encoder, PPMStereo further improves and achieves impressive performance. We attribute this performance to the strong priors provided by the Visual Foundation Model. \nThese results highlight the strength and flexibility of our Pick-and-Play Memory mechanism, which effectively captures long-range temporal consistency for consistent dynamic stereo matching.\n\n**W3:** The paper lacks a runtime memory consumption: I suggest that the authors enrich Fig. 7 of the appendix by adding a measure of memory consumption during evaluation.\n\n**A3:** We thank the reviewer for the helpful suggestion. To measure runtime memory during evaluation, we conduct evaluation on a 20‑frame sequence from the Sintel Stereo dataset (resolution 1024×436) on an NVIDIA A100, varying the number of iterations *N* up to 20. The table reports peak GPU memory (in GB) and TEPE on the Clean and Final subsets:\n\n|Methods|Iterations|GPU Memory (GB)|Inference Time (s)|Sintel Clean (TEPE)|Sintel Final (TEPE)|\n| :-----------------: | :---------------: | :--------------: | :--------------: |:--------------: |:--------------: |\n|DynamicStereo |*N=20*|**16.6**|3.4|0.76|1.42|\n|BidaStereo |*N=20*|19.8|6.7|0.75|1.22|\n|Ours |*N=5*|20.7|**3.0**|0.72|1.22|\n|Ours |*N=10*|21.9|6.0|**0.63**|**1.12**|\n|Ours |*N=15*|23.4|8.9|**0.62**|**1.11**|\n|Ours |*N=20*|24.4|11.8|**0.62**|**1.11**|\n\nAs observed, our approach achieves state-of-the-art accuracy (lowest TEPE) on both Sintel Clean and Final benchmarks, with a gradual increase in GPU memory usage and inference time as *N* increases. Notably, for smaller  *N* (e.g., *N*=5), our method matches the accuracy of prior art with only a moderate increase in memory, while larger *N*  provides further accuracy gains at the cost of higher resource usage.\nTo better illustrate this trade-off, we will enrich Fig.7 to include a comparative analysis of runtime memory usage.\n\n**Q1:** Inside equation 8, there is $k'_m$  on both parts of the right equation. Is it right?\n \n**R1:** Thank you for pointing out the typo. We sincerely apologize for the oversight.\nWe will revise the symbols on the left-hand side of Eq.8 ($\\hat{k}_{m} = \\overline{\\mathrm{S}}_t \\cdot k'_m $ +  temporal_postion_encoding) to eliminate any ambiguity and will also update Eq.9 accordingly.\n\n\n\n**Q2:** How  $T$ -- a.k.a. number of reference frames -- hyper-parameter is chosen?\n\n**R2:** Thank you for your concern regarding the selection of reference frame numbers. To ensure a fair comparison, we follow the configuration adopted in prior works (e.g., DynamicStereo, BiDAStereo), using 5 reference frames during training and 20 during inference.\nAdditionally, we conduct further evaluations with varying numbers of reference frames to evaluate their impact. The results are summarized in the table below:\n|Number of Reference Frames|Sintel Clean(TEPE)|Sintel Clean(Tδ1px)|Sintel Clean(Tδ3px)|Sintel Final (TEPE)|Sintel Final(Tδ1px)|Sintel Final(Tδ3px)\n| :----------------: | :-----------------------: | :----------------: | :-----------------------: |:-----------------------: |:-----------------------: |:-----------------------: |\n|$T$ =20|0.62|7.21|3.29|1.11|9.98|4.87|\n|$T$ =30|0.61|7.17|3.29|1.09|9.92|4.81|\n|$T$ =40|0.61|7.17|3.28|1.09|9.93|4.82|\n|$T$ =50|0.61|7.18|3.28|1.09|9.92|4.82|\n\nThe table shows that performance improves as the number of frames increases, but eventually reaches a saturation point."}]

Author rebuttal:

Meta-review: This paper received positive ratings 1) Borderline Accept and (2) Accept from reviewers.
The main contribution of the paper is the introduction of the pick-and-play memory mechanism which can model long-range temporal consistency leading to strong experimental performance for stereo matching in stereo video sequence. 

While reviewers have concerns regarding 1) lack validation of number of frames being able to be handled by the method, 2) lack discussions of computational cost, 3) lack discussion of hyperparameters.

Authors provided detailed response in the rebuttal. All reviewers acknowledge that most of their concerns are addressed and would recommend accept.

AC concurs with reviewers’ comments and would recommend Accept.
Decision: Accept (poster)