Title: RobIA: Robust Instance-aware Continual Test-time Adaptation for Deep Stereo
Venue: NeurIPS 2025
Accepted: yes
Keywords: continual test time adaptation

3D computer vision
Abstract: Stereo Depth Estimation in real-world environments poses significant challenges due to dynamic domain shifts, sparse or unreliable supervision, and the high cost of acquiring dense ground-truth labels. While recent Test-Time Adaptation (TTA) methods offer promising solutions, most rely on static target domain assumptions and input-invariant adaptation strategies, limiting their effectiveness under continual shifts. In this paper, we propose RobIA, a novel Robust, Instance-Aware framework for Continual Test-Time Adaptation (CTTA) in stereo depth estimation. RobIA integrates two key components: (1) Attend-and-Excite Mixture-of-Experts (AttEx-MoE), a parameter-efficient module that dynamically routes input to frozen experts via lightweight self-attention mechanism tailored to epipolar geometry, and (2) Robust AdaptBN Teacher, a PEFT-based teacher model that provides dense pseudo-supervision by complementing sparse handcrafted labels. 
This strategy enables input-specific flexibility, broad supervision coverage, improving generalization under domain shift. Extensive experiments demonstrate that RobIA achieves superior adaptation performance across dynamic target domains while maintaining computational efficiency.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article introduces a novel method for Continual Test-Time Adaptation in Deep Stereo. The paper's main contributions are: a new Mixture of Experts-based module for convolutional networks, and a new strategy for generating pseudo-labels during training.\nRegarding the first contribution, the proposed module uses row-wise self-attention to extract an embedding that is fed into a gating network. This approach is both efficient and preserves some spatial structure from the input images. The resulting embedding is used to produce channel-wise modulations for each training instance, enabling greater model flexibility. Moreover, using a sigmoid activation function instead of ReLU helps retain information across all channels of the base model, which is particularly beneficial in scenarios where the backbone is frozen.\nAs for the second contribution, the method proposes generating dense pseudo-labels by combining sparse pseudo-labels obtained through traditional methods (e.g., SGM) with dense pseudo-labels generated by a teacher network for the remaining regions. Specifically, the teacher network is implemented using an AdaptBN version of the base model, rather than a conventional EMA approach. This choice is motivated by the intuition that AdaptBN is less prone to reinforcing the errors of hand-crafted pseudo-labels, thus providing a more reliable supervision signal.\n\nSTRENGTHS AND WEAKNESSES:\n__Strength__\n\n- The paper is well-written, well-structured, and well-curated. \n\n- The various ideas have been thoroughly motivated, both logically and experimentally. \n\n- The experiments are rigorously conducted and also well-described in the paper.\n\n__Weaknesses__\n\nIn order of importance:\n\nA. Some stereo networks have architectural differences compared to CoeX. For instance, could the proposed method be applied to frameworks such as HITNet [a] or TemporalStereo [b]?\n\n[a] Tankovich et al. HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching. CVPR 2021.\n\n[b] Zhang et al. TemporalStereo - Efficient Spatial-Temporal Stereo Matching Network. IROS 2023\n\nB. The AttEX-MoE strategy appears to be particularly tailored to CNN-based networks. However, many recent stereo architectures rely on transformer-based approaches. Is the proposed method applicable to these types of architectures as well?\n\nC. The pseudo-label generation mechanism using AdaptBN implicitly assumes that the network employs Batch Normalization (BN) layers. How could this technique be extended to scenarios where the base network uses a different type of normalization?\n\nD. While row-wise self-attentions improve computational efficiency, the emphasis on epipolar lines is unclear. If leveraging epipolar geometry to estimate embeddings is a key factor for performance, this should be validated experimentally. For example, one possible approach could be to compare it with a strategy that does not exploit epipolar geometry but still partially preserves spatial structure and offers similar efficiency (e.g., using column-wise instead of row-wise embeddings).\n\n\n__Minor Weaknesses__\n\na.\tMissing bolds and underlines to highlight the best results in Tables 2, 3, and 4\n\nb.\tSeveral highly relevant results are only presented in the appendix (e.g., Tables 8, 9, and 10). While it's understandable that these results were moved to the appendix due to space constraints, they should at least be referenced or briefly discussed in the main paper to emphasize their importance.\n\n__Overall Comments__\n\nThe paper is well-organized and structured. It presents novel ideas that are well-motivated and interesting. The main limitation of the work lies in its specificity to a particular architecture, relying on specific normalization layers and a convolution-based design. A better understanding of this limitation could lead me to provide a more grounded result. It would also be interesting to investigate whether the emphasis on epipolar geometry is necessary for the AttEX-MoE component.\n\nQUESTIONS:\nSee Weaknesses section above.\n\nLIMITATIONS:\nSome limitations are presented in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes **RobIA**, a novel Robust, Instance-Aware framework for Continual Test-Time Adaptation (CTTA) in stereo depth estimation. RobIA integrates two key components: (1) Attend-and-Excite Mixture-of-Experts (AttEx-MoE), a parameter-efficient module that dynamically routes input to frozen experts via lightweight self-attention mechanism tailored to epipolar geometry, and (2) Robust AdaptBN Teacher, a PEFT-based teacher model that provides dense pseudo-supervision by complementing sparse handcrafted labels. Extensive experiments demonstrate that RobIA achieves superior adaptation performance across dynamic target domains while maintaining computational efficiency.\n\nSTRENGTHS AND WEAKNESSES:\n### **Strengths**\n\n1. RobIA presents a simple and efficient framework for real-time continual test-time adaptation (CTTA) in stereo depth estimation, addressing an important challenge for reliable real-world applications.\n2. The author proposes a new benchmark from existing TTA sequences that feature frequent domain shifts. This benchmark more accurately reflects real-world conditions, where environmental changes occur regularly and previously encountered scenarios may reappear. It offers a more appropriate criterion for evaluating CTTA frameworks.\n3. The paper is well-organized and easy to follow. Each component of the proposed framework is well-motivated and clearly explained.\n\n### **Weaknesses**\n\n1. **Lack of direct comparison with key baseline.** RobIA builds upon CoEx [1] stereo network, but the Modular ADaptation (MAD++) method from MADNet2 [2] is only evaluated on top of the original MADNet2 network. Comparing RobIA with MAD++ in this setting appears unfair. Furthermore, MAD++ with MADNet2 runs significantly faster than RobIA with the AdaptBN teacher (18ms vs. 204ms as reported in Table 3). A fairer comparison under similar runtime constraints (*e.g.*, by selecting additional modules in MAD++) would better clarify whether RobIA offers substantial improvements over the Modular ADaptation approach.\n2. **Limited overall performance gain.** According to Table 1, 2 and 8 (in the supplementary material), the performance improvements with RobIA seem marginal compared to the AdaptBN approach with minimal trainable parameters (0.03M compared to 1.22M). Notably, as shown in Table 8, CoEx with AdaptBN even outperforms RobIA (w/ AdaptBN teacher) in KITTI RAW CTTA benchmark by a noticeable margin. This raises concerns about the effectiveness of RobIA.\n3. **Minor presentation issues and typos.** I would suggest the author use `w/ Teacher` instead of `w/ AdaptBN` to reduce ambiguity with tunning batch normalization layers; In line 242, the loss should be $\\mathcal{L}_{\\text{teacher}}$ in the third equation; Bold and Underline fonts are missing from Tabel 2 onward.\n\n\n\n[1] Bangunharcana, Antyanta, et al. \"Correlate-and-excite: Real-time stereo matching via guided cost volume excitation.\" *2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE, 2021.\n\n[2] Poggi, Matteo, et al. \"Continual adaptation for deep stereo.\" *IEEE Transactions on Pattern Analysis and Machine Intelligence* 44.9 (2021): 4713-4729.\n\nQUESTIONS:\nPlease refer to the Weaknesses section. I will adjust my rating according to the author's response.\n\nLIMITATIONS:\nThe authors have briefly discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the problem of tuning online a deep stereo network to unseen domains. The authors introduce two main novelties compared with previous works: the introduction of architecture modifications (mixture of experts) to make the model more robust to domain shifts and a new loss function using the prediction from a “teacher” model to speed up and stabilize convergence. The two modifications turned out to be effective across the different scenarios the author tested resulting in faster and more stable performance. The authors also introduce a new testing protocol by looping multiple times over the same video sequences. This setting should stress the ability of methods to adapt without catastrophic forgetting. The authors however do report performance also on the established way of testing test time trained stereo methods.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n+ The test time training area is slightly under-explored in the depth estimation literature although it is one of the fields where it has proven to be most effective and practical. This work continues the existing exploration proposing both a new architecture which can be updated fast and a new loss that seems to be competitive and sometimes more effective than SOTA.\n\n+  The use of a mixture of experts (MoE) in CNN architecture is another under-explored area. I found the proposal from the authors interesting and somehow novel, although I’m not entirely convinced in calling it a MoE since (per my understanding) there is no sparsity enforced and all “experts” are often active at the same time. \n\n+ The work identifies one issue with existing SOTA around performance diverging for invalid regions and proposes an effective fix for it. The authors also designed  experiments to support their claim and solution (Fig. 2). \n\n## Weaknesses\n\na. **Computational cost**: If my understanding is correct, the proposed solution relies on keeping two versions of the same model (one w/AdaptBN training and one with MoE training) and performing 2 fwd pass and 2 bwd pass for each frame being processed. This results in a 10x latency increase compared to pure inference (Tab. 3 row (j) vs row (d)) which is significantly more than the previous SOTA (row (b) and (c) ) in Tab. 3.  \n\nb. **Missing motivation/implementation details**: From the current version of the manuscript it is not clear to me why the MoE architecture proposed should help generalization. The parameters of the mixture will anyway be trained on the source domain so doing inference with the mixture should in theory suffer from domain shifts as much as the full trained model. Where the mixture might help is in regularizing test time training avoiding modifying the architecture too much, though the motivation presented in the work are different. Can the author clarify whether there is anything special in the way the mixture is trained that would help it generalize better.\n\nC. [minor] **New evaluation protocol**: The authors propose a new evaluation protocol by looping over the same video sequence multiple times claiming that this scenario claiming to be the first method to look at test time adaptation for stereo under the \"continuous\" setting, while the previous methods operated under the assumption of a “single stationary target domain” (line 27-29). I would like to challenge this as the previous methods mentioned for SDE don’t have any assumption on the scenario considered and as such operate indeed on a continuous domain. Moreover a setting very similar to the one the authors considered is also reported in [1] Fig. 5 where the authors of [1] also test on looping over the same sequence. Challenging the motivation to develop a new test protocol would also imply that the methods should be compared using the old established protocol which the authors to their credits did in Tab. 9 Supplementary. In this table however the performance gains seem smaller with respect to the proposed new setting. This would benefit from some additional discussion in the document.\n\nD. [minor] **Possible missing comparison**: [2] uses a combination of proxy labels loss + smoothing + image reconstruction to compensate for the sparsity of the proxy labels. This is exactly the problem the authors are trying to solve with using an AdaptBn teacher. While [2] tests this solution offline there is no reason why it cannot be run online. A comparison would be interesting as it would save to the authors the cost of running the expensive AdaptBN teacher. \n\n## References \n\n1. [Tonioni, Alessio, et al. \"Real-time self-adaptive deep stereo.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.](https://openaccess.thecvf.com/content_CVPR_2019/papers/Tonioni_Real-Time_Self-Adaptive_Deep_Stereo_CVPR_2019_paper.pdf)\n2. [Tonioni, Alessio, et al. \"Unsupervised domain adaptation for depth prediction from images.\" IEEE transactions on pattern analysis and machine intelligence 42.10 (2019): 2396-2409.](https://arxiv.org/pdf/1909.03943)\n\nQUESTIONS:\n* Are the timing in Tab. 3 taking into account the time spent generating pseudo labels using SGM or the AdaptBN teacher?\n\n* [weakness b] Can you comment on weakness b?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents RobIA, a novel framework for Continual Test-time Adaptation (CTTA) in stereo depth estimation under dynamic domain shifts. The proposed method incorporates two key components: the Attend-and-Excite Mixture-of-Experts (AttEx-MoE) module for instance-aware adaptation and a Robust AdaptBN Teacher to enhance pseudo-label coverage. The authors show that RobIA outperforms existing methods in adapting to continually shifting domains, demonstrating strong performance in terms of both accuracy and computational efficiency. Overall, the approach is promising, and the experimental results are well-supported, though there are some areas for potential improvement.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nInnovation and Novelty: The introduction of AttEx-MoE and the dual-source pseudo-supervision strategy is a contribution to the field of continual adaptation for stereo depth estimation. The use of instance-aware, epipolar geometry-guided adaptation offers a unique approach to overcoming domain shifts in real-world settings.\n\nExperimental Evaluation: The authors conduct extensive experiments across several challenging benchmarks (KITTI RAW, DrivingStereo, and DSEC), showing that RobIA performs robustly in dynamic, real-world conditions. The results demonstrate superior adaptation performance when compared to existing methods like MADNet2 and CoEx.\n\nEfficient Use of Resources: RobIA achieves high performance while maintaining computational efficiency, which is essential for real-time applications such as autonomous driving. The proposed architecture leverages parameter-efficient modules, which helps in maintaining a balance between accuracy and resource usage.\n\n\nWeakness:\n1.In the KITTI15 leaderboard, among the top 15 models, 5 models with attached paper links include MonSter[1], DEFOM-Stereo[2], IGEV++ (DepthAny.)[3], StereoBase[4], and TC-Stereo[5]. MonSter[1], DEFOM-Stereo[2], and IGEV++ (DepthAny.)[3] utilize monocular Foundation models. IGEV[6] is a recently widely experimented baseline for stereo tasks, so the review recommends verifying the effectiveness of the method proposed in the paper by choosing one of IGEV or StereoBase. To the best knowledge of review, LightStereo[7] is the SOTA 2D-based model. The reviewer believes that the authors need to demonstrate the effectiveness of their method using at least one of the above methods. \n\n[1] Cheng, Junda, et al. \"MonSter: Marry Monodepth to Stereo Unleashes Power.\" (CVPR2025).\n\n[2] Jiang, Hualie, et al. \"DEFOM-Stereo: Depth Foundation Model Based Stereo Matching.\"  (CVPR2025).\n\n[3] G. Xu, X. Wang, Z. Zhang, J. Cheng, C. Liao and X. Yang: IGEV++: Iterative Multi-range Geometry Encoding Volumes for Stereo Matching. IEEE TPAMI 2025.\n\n[4] X. Guo, J. Lu, C. Zhang, Y. Wang, Y. Duan, T. Yang, Z. Zhu and L. Chen: OpenStereo: A Comprehensive Benchmark for Stereo Matching and Strong Baseline. arXiv preprint arXiv:2312.00343 2023.\n\n[5] J. Zeng, C. Yao, Y. Wu and Y. Jia: Temporally Consistent Stereo Matching. （ECCV2024).\n\n[6]  Xu, Gangwei, et al. \"Iterative geometry encoding volume for stereo matching.\" (CVPR2023).\n\n[7] \"Lightstereo: Channel boost is all you need for efficient 2d cost aggregation.\" (ICRA 2025).\n\n2. In Tables 2, 3 and 4, the best results need to be bolded to facilitate readers' reading.\n\nQUESTIONS:\nPlease refer to weakness\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}