Title: PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching
Venue: NeurIPS 2025
Accepted: yes
Keywords: video stereo matching; 3D Vision
Abstract: Temporally consistent depth estimation from stereo video is critical for real-world applications such as augmented reality, where inconsistent depth estimation disrupts the immersion of users.
Despite its importance, this task remains challenging due to the difficulty in modeling long-term temporal consistency in a computationally efficient manner.
Previous methods attempt to address this by aggregating spatio-temporal information but face a fundamental trade-off: limited temporal modeling provides only modest gains, whereas capturing long-range dependencies significantly increases computational cost.
To address this limitation, we introduce a memory buffer for modeling long-range spatio-temporal consistency while achieving efficient dynamic stereo matching.
Inspired by the two-stage decision-making process in humans, we propose a Pick-and-Play Memory (PPM) construction module for dynamic Stereo matching, dubbed as PPMStereo. PPM consists of a pick process that identifies the most relevant frames and a play process that weights the selected frames adaptively for spatio-temporal aggregation.
This two-stage collaborative process maintains a compact yet highly informative memory buffer while achieving temporally consistent information aggregation.
Extensive experiments validate the effectiveness of PPMStereo, demonstrating state-of-the-art performance in both accuracy and temporal consistency.Codes are available at \textcolor{blue}{https://github.com/cocowy1/PPMStereo}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel dynamic stereo architecture that uses a memory bank approach with a custom Quality Assessment Module (QAM) to achieve state-of-the-art performance.\nIn particular, the authors propose an iterative-based (i.e., based on RAFT-Stereo architecture) dynamic stereo model that uses features not only from the cost volume and context image, but also from a Dynamic Memory Buffer.\nThe latter is used to store temporal consistency information, built using the proposed Memory Pick'n'Play Processes, that ensure a selection of the most relevant temporal information.\nThe memory bank approach is used not only to store features from different temporal frames but also to reduce memory consumption via a frame selection strategy, based on the outcome of the QAM module.\nExhaustive experiments prove the effectiveness of the architecture design and the performance of the model w.r.t. the previous state of the art.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- **Exhaustive experiments showing SOTA performance**: The authors validate the potential of their proposal using different standard benchmarks such as Sintel, Dynamic Replica (Tab. 1). Furthermore, they qualitatively investigate in Fig. 6 the performance using a real-world dataset -- i.e., South Kensington. This largely proves the effectiveness of the proposed Pick'n'Play strategy.\n- **Novel Dynamic Stereo Architecture**: The authors revisited the MemFlow architecture for Dynamic Stereo Matching. Additionally, they changed the memory bank module with the improved Pick'n'Play proposal. The benefits of the latter solution w.r.t. MemFlow memory bank are shown in Table 2.\n- **Clear logic structure of the paper**: I personally liked the clear and easy-to-follow structure of the paper. The authors shortly introduce the concept of iterative stereo networks -- assuming a prior knowledge from the reader -- while putting a lot of effort into the Pick'n'Play processes, the core idea of the paper. Even so, chapter 3 was a little hard to digest at first: I suggest that the authors make more references to Fig. 2 and Fig. 3 inside the chapter.\n\nWeaknesses\n\n- **Related works regarding Deep Stereo Matching can be enlarged**: In my opinion, the related works regarding Deep Stereo Matching are quite poor, since the authors mainly report the GRU-based architectures. I suggest at least including a survey paper [1] to include additional literature regarding deep stereo matching.\n- **Missing competitors with VDA features**: The benefits of the VDA encoder are quite impressive. It is true that authors showed SOTA performance even without the VDA encoder; however, I would have liked to see a fairer comparison showing at least one competitor with the VDA encoder.\n- **The paper lacks a runtime memory consumption**: I suggest that the authors enrich Fig. 7 of the appendix by adding a measure of memory consumption during evaluation.\n\n\n[1] Tosi, Fabio, Luca Bartolomei, and Matteo Poggi. \"A survey on deep stereo matching in the twenties.\" International Journal of Computer Vision (2025): 1-32.\n\nQUESTIONS:\nBefore asking questions, I would like to introduce my rationale behind my initial rating: the paper proposes an incremental work, extending MemFlow for Dynamic Stereo Matching and introducing a revisited memory bank dubbed as Pick'n'Play Memory. The exhaustive experiments show SOTA performance, thus confirming a valid contribution for NIPS 2025.\n\n- Inside equation 8, there is $k'_m$ on both parts of the right equation. Is it right?\n- How $T$ -- a.k.a. number of reference frames -- hyper-parameter is chosen?\n- Please refer to Strengths and Weaknesses for further suggestions.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel framework called PPMStereo for dynamic stereo matching with temporally consistent disparity estimation in video sequences. The authors address the challenge of capturing long-range temporal relationships efficiently by introducing a Pick-and-Play Memory (PPM) module. The system operates through two primary stages:\n\nPick: Identifying relevant frames based on a Quality Assessment Module (QAM).\nPlay: Adapting the aggregation of features from selected frames to achieve accurate and temporally consistent disparity maps.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.Innovative Memory Mechanism: The introduction of a memory buffer for spatio-temporal consistency in dynamic stereo matching is novel. The \"Pick-and-Play\" strategy improves computational efficiency by dynamically selecting and aggregating relevant frames.\n2.Strong Experimental Results: The method achieves state-of-the-art performance on multiple dynamic stereo benchmarks, including Sintel, Dynamic Replica, and Sintel Final Pass. Notably, it outperforms previous methods such as BiDAStereo and DynamicStereo in terms of temporal consistency and accuracy with reduced computational cost.The ablation studies are comprehensive, demonstrating the impact of different components of the framework.\n3.Generalization across Datasets: The method demonstrates strong generalization, with superior results on both synthetic and real-world datasets.\n\nWeaknesses:\n1.In Table 1, the Memory Length decreases gradually from 1 to 7. The reviewers would like to know the results when k is larger, such as k=9 and k=11.\n2.Computational Efficiency: A more detailed discussion of computational costs, including GPU memory usage and processing time per frame or video sequence, would help clarify the trade-offs in terms of efficiency versus accuracy.\n3.In the \"Deep Stereo Matching\" section of the related work, there is a lack of citations to some latest papers:\n1） \"Openstereo: A comprehensive benchmark for stereo matching and strong baseline.\" arXiv preprint arXiv:2312.00343 (2025).\n2）\"Lightstereo: Channel boost is all you need for efficient 2d cost aggregation.\" (ICRA 2025).\n\nQUESTIONS:\nPlease refer to weakness\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method to improve temporal consistency by capturing long-range dependencies across video frames while maintaining computational efficiency through a novel memory buffer mechanism. The approach first selects the most relevant frames using confidence scores and redundancy-aware relevance metrics, then adaptively weights and aggregates these frames for spatio-temporal feature integration. Experimental results show the proposed method outperforms existing approaches across three benchmark datasets in both accuracy and temporal consistency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1）The proposed method demonstrates significant performance improvements.\n2）The experiments are comprehensive, providing detailed validation of the method's effectiveness.\n\nWeaknesses\n1）The paper's organization is somewhat disorganized. The core idea—selecting and aggregating key frames to balance long-term temporal modeling and computational efficiency could be presented more clearly. A top-down structure (general to specific) would improve readability, rather than introducing excessive details and equations in Section 3.1.\n2）The excessive use of mathematical notation hinders readability. Additionally, some equations (e.g., (1) and (2)) are overly simplistic and could be presented inline rather than as standalone equations to save space.\n3）The paper reads more like a technical report, primarily focusing on implementation details while providing limited explanation of the underlying motivations and theoretical principles.\n\nQUESTIONS:\n1）The paper's organization is somewhat disorganized. \n2）The excessive use of mathematical notation hinders readability. \n3）The paper reads more like a technical report, primarily focusing on implementation details while providing limited explanation of the underlying motivations and theoretical principles.\n\nLIMITATIONS:\nDoes the keyframe extraction and compression process affect inference speed?\nHow many frames can the performance improvement from long-term temporal information be sustained? The paper only tests up to 20 frames?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}