Title: Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance
Venue: NeurIPS 2025
Accepted: yes
Keywords: Semantic Scene Completion; autonomous driving
Abstract: 3D Semantic Scene Completion (SSC) provides comprehensive scene geometry and semantics for autonomous driving perception, which is crucial for enabling accurate and reliable decision-making. However, existing SSC methods are limited to capturing sparse information from the current frame or naively stacking multi-frame temporal features, thereby failing to acquire effective scene context. These approaches ignore critical motion dynamics and struggle to achieve temporal consistency. To address the above challenges, we propose a novel temporal SSC method FlowScene: Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance. By leveraging optical flow, FlowScene can integrate motion, different viewpoints, occlusions, and other contextual cues, thereby significantly improving the accuracy of 3D scene completion. Specifically, our framework introduces two key components: (1) a Flow-Guided Temporal Aggregation module that aligns and aggregates temporal features using optical flow, capturing motion-aware context and deformable structures; and (2) an Occlusion-Guided Voxel Refinement module that injects occlusion masks and temporally aggregated features into 3D voxel space, adaptively refining voxel representations for explicit geometric modeling.
Experimental results demonstrate that FlowScene achieves state-of-the-art performance, with mIoU of 17.70 and 20.81 on the SemanticKITTI and SSCBench-KITTI-360 benchmarks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents FlowScene, a novel framework for temporal 3D Semantic Scene Completion that explicitly uses optical flow to guide spatio-temporal dependencies. Specifically, the Flow-Guided Temporal Aggregation module is designed to align features from past frames using motion cues, while the Occlusion-Guided Voxel Refinement module intelligently combines current and historical information using occlusion masks. Experiments on SemanticKITTI and SSCBench-KITTI-360 benchmarks demonstrate the effectiveness of the proposed modules.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n- Utilizing optical flow as explicit guidance for temporal feature aggregation provides more fine-grained motion and contextual cues.\n- The introduction of occlusion masks in 3D voxel refinement enhances representation quality in occluded regions.\n- Extensive experimental validation with detailed ablation studies demonstrates the effectiveness of each component.\n\n**Weaknesses**:\n- The main concern is about the role of occlusion masks in refining voxel features. According to the paper's description, occlusion masks are primarily used to enhance the fusion of features from non-occluded regions in historical frames, while using current frame features for occluded parts. However, for constructing 3D voxels of the current frame, it's more reasonable and helpful to obtain 3D features about the occluded parts of the current frame from historical frames to achieve a more complete 3D representation. Conversely, when performing 2D feature fusion, it's better to consider the interference of occluded features, and for the current frame, it will make sense to fuse non-occluded 2D features.\n- It would be better to provide more explanation on the limited improvement on the KITTI360 test set compared to the SOTA.\n- It would be better to unify the notation of occlusion masks in Figure 3 and Figure 4 for better clarification.\n- Based on the supplementary video materials, the occlusion masks generated from optical flow appear to have significant errors. It would be beneficial to further elaborate on the role and significance of generating occlusion masks through optical flow.\n\nQUESTIONS:\nIt would be better to provide more evidence to address my concerns about the role of occlusion masks in refining voxel features and the limited improvement on the KITTI360 test set. I would raise the evaluation score when these concerns are solved.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces FlowScene, a novel method for temporal 3D Semantic Scene Completion (SSC) that leverages optical flow to improve performance. The authors argue that previous temporal SSC methods, which often just stack features from multiple frames, fail to capture crucial motion dynamics and temporal consistency. To address this, FlowScene incorporates two key modules: a Flow-Guided Temporal Aggregation (FGTA) module that uses optical flow to warp and intelligently aggregate features from past frames, and an Occlusion-Guided Voxel Refinement (OGVR) module that uses occlusion masks (derived from flow consistency checks) to refine the 3D voxel representation. This approach explicitly models motion and handles occlusions, leading to more accurate geometric and semantic predictions. The method achieves state-of-the-art results on the SemanticKITTI and SSCBench-KITTI-360 benchmarks, demonstrating significant improvements, especially in the completion of dynamic objects.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The core concept of using optical flow to guide temporal feature aggregation and explicitly handle occlusions in 3D SSC is a strong and intuitive contribution. It directly addresses a clear limitation of prior work—the naive handling of temporal information and scene dynamics.\n\n2.  Despite the added complexity of optical flow estimation, the proposed method is shown to be efficient. It achieves its state-of-the-art results with fewer parameters than many competing methods and maintains a comparable inference time. The use of only two historical frames also makes it practical.\n\n3. The paper is well-written and easy to follow. The figures (especially Figures 2 and 3) effectively illustrate the core differences between FlowScene and previous methods, and the qualitative results in Figures 6 and 10 clearly showcase the method's superiority in handling occlusions and dynamic objects.\n\nWeakness\n\n1. The method's performance is inherently tied to the quality of the pre-trained optical flow network. The authors acknowledge this as a limitation, noting that performance could be limited in visually degraded conditions. While the ablation study shows robustness to different flow networks, a failure of the flow estimator would likely cause a catastrophic failure in the SSC prediction.\n\n2. The paper does not report error bars or standard deviations for its results, citing computational expense. This makes it difficult to assess the statistical significance of the reported improvements over other methods.\n\nQUESTIONS:\n1. In the Flow-Guided Temporal Aggregation (FGTA) module, you use cosine similarity between the warped historical features (F _warp) and the current features (F_t) to generate weights for aggregation. Could you provide more intuition on why this is effective? Does this implicitly down-weight regions where the flow is inaccurate (leading to poor warping and low similarity) or regions that have changed in appearance for reasons other than motion?\n\n2. In Table 6, you compare your \"Flow-Guided\" alignment against a \"Stack\" baseline. How exactly is this \"Stack\" baseline implemented? Does it simply concatenate the features from the current and two historical frames along the channel dimension before the view transformation?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors propose a 3D Semantic Scene Completion (SSC) method tailored for autonomous driving perception tasks. Unlike existing approaches that fuse temporal information through simple stacking or attention mechanisms, the proposed method leverages estimated optical flow to align features across frames more effectively. Specifically, the authors introduce Flow-Guided Temporal Aggregation (FGTA) to aggregate historical features in the 2D space, and Occlusion-Guided Voxel Refinement (OGVR) to refine features in the 3D voxel space. Experimental results on the SemanticKITTI and SSCBench-KITTI-360 benchmarks demonstrate that the proposed method achieves state-of-the-art performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The motivation for introducing optical flow to guide temporal feature fusion is well-justified and conceptually sound.\n\n2. The writing is clear, and the diagrams are well-designed and easy to follow, which helps in understanding the methodology.\n\n3. Experimental results on benchmark datasets convincingly demonstrate the proposed method's superiority over existing approaches.\n\nWeaknesses:\n\n1. While the integration of optical flow into the framework is reasonable, it inherently depends on the quality of flow estimation from pre-trained networks. As shown in Table 7, aggregating more historical frames will instead degrade the performance, likely due to inaccuracies in flow estimation. This outcome contradicts intuitive expectations. It may be beneficial to explore joint optimization of the flow estimation and the SSC framework to mitigate this limitation.\n\n2. The additional computational cost introduced by flow estimation may result in slower inference compared to other methods. However, this trade-off might be acceptable depending on the application context.\n\nQUESTIONS:\nI have a few questions about the details:\n\n1. In the OGVR, it seems that the 2D features are projected to 3D space through the depth bins. But I do not quite get the definition or acquirement of the depth bins. Are they acquired through other depth estimation models? Or just frustum?\n\n2. During the updating of $F_t$ in Eq.5, only the non-occluded information in historical frames is introduced for fusion. Won't the other regions help the completion? Besides, I'd also appreciate it if the authors can provide more details about the flow consistency check used in this work.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}