Title: NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding
Venue: NeurIPS 2025
Accepted: yes
Keywords: Underwater scene understanding

vision-language model

large language model
Abstract: Underwater exploration offers critical insights into our planet and attracts increasing attention for its broader applications in resource exploration, national security, etc. We study the underwater scene understanding methods, which aim to achieve automated underwater exploration. The underwater scene understanding task demands multi-task perceptions from multiple granularities. However, the absence of large-scale underwater multi-task instruction-tuning datasets hinders the progress of this research. To bridge this gap, we construct NautData, a dataset containing 1.45 M image-text pairs supporting eight underwater scene understanding tasks. It enables the development and thorough evaluation of the underwater scene understanding models. Underwater image degradation is a widely recognized challenge that interferes with underwater tasks. To improve the robustness of underwater scene understanding, we introduce physical priors derived from underwater imaging models and propose a plug-and-play vision feature enhancement (VFE) module, which explicitly restores clear underwater information. We integrate this module into renowned baselines LLaVA-1.5 and Qwen2.5-VL and build our underwater LMM, NAUTILUS. Experiments conducted on the NautData and public underwater datasets demonstrate the effectiveness of the VFE module, consistently improving the performance of both baselines on the majority of supported tasks, thus ensuring the superiority of NAUTILUS in the underwater scene understanding area. Data and models are available at https://github.com/H-EmbodVis/NAUTILUS.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents NAUTILUS, a large vision-language model tailored for underwater scene understanding. To tackle the challenges of underwater image degradation and the lack of diverse datasets, the authors introduce NautData—a new instruction-tuning dataset with 1.45 million image-text pairs spanning eight tasks across image, region, and object levels. They also propose a plug-and-play Vision Feature Enhancement (VFE) module based on physical underwater imaging principles, which improves feature quality by correcting for light scattering and absorption. Integrated into strong LMM backbones like LLaVA-1.5 and Qwen2.5-VL, NAUTILUS achieves state-of-the-art performance across a range of benchmarks, demonstrating robust and comprehensive underwater perception capabilities.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The paper introduces a high-quality dataset, NautData, which is large, diverse, and well-annotated across eight underwater tasks. This is a meaningful contribution to a relatively underexplored domain.\n2. The Vision Feature Enhancement (VFE) module leverages physical priors to address underwater image degradation at the feature level. \n3. The paper is well-written and easy to follow.\n\n---\n\n**Weaknesses**\n\n- While the proposed VFE module is grounded in a physical imaging model, the paper does not clearly define the applicability limits of the model, such as how well it generalizes under extreme lighting or turbidity conditions. The robustness of the dark-pixel prior is also not thoroughly analyzed—its reliability may vary depending on scene composition and noise.\n- In addition, the scope of contributions is relatively narrow, focusing primarily on the construction of a dataset and the design of the VFE module. While both are valuable, the paper would benefit from broader methodological innovations or deeper analysis of the interplay between physical priors and large-scale vision-language models.\n\nQUESTIONS:\n- While the model shows strong performance on NautData and select public benchmarks, it’s unclear how well NAUTILUS generalizes to truly unseen underwater environments or rare object categories not covered in the training data. Have the authors considered evaluating on additional real-world or long-tail settings to test robustness? Results or discussion in this direction would strengthen the claims on general applicability.\n- Since much of NautData is constructed using LMMs like Gemini, Qwen, and GPT-4o, it raises concerns about annotation accuracy—especially given that Table 2 shows GPT-4o itself performs poorly on the benchmark. Could the authors clarify how they ensured label correctness during dataset construction? Additionally, quantitative measures of annotation quality (e.g., human agreement rates or error analyses) would help justify the dataset’s reliability as a benchmark.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work makes two primary contributions to underwater scene understanding:\n\n(1) the introduction of NautData, a novel large-scale multimodal dataset encompassing 1.45M image-text pairs across eight tasks spanning image-, region-, and object-level understanding; and\n\n(2) NAUTILUS, a new underwater-adapted LMM that integrates a physics-inspired vision feature enhancement (VFE) module to\nexplicitly mitigate degradation effects in underwater imagery.\n\nThe proposed VFE module demonstrates strong plug-and-play compatibility when incorporated into established LMMs (LLaVA-1.5, Qwen-VL), with experiments showing consistent performance gains across multiple underwater understanding tasks.\nBy addressing both the data scarcity challenge through NautData and the degradation robustness challenge through the VFE design,\nthis work provides a valuable foundation for future research in underwater multimodal perception systems.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n(1) The construction of NautData represents a significant contribution, being the first large-scale dataset to\ncomprehensively cover eight underwater understanding tasks across image-, region-, and object-level granularities.\n\n(2) The proposed NAUTILUS model demonstrates consistent and measurable improvements across multiple underwater tasks when\nbenchmarked against established baselines.\n\n(3) The paper is well-written with clear organization and minimal grammatical errors, facilitating efficient comprehension of the technical content.\n\n\nWeaknesses:\n\n(1) This paper lacks explanation for performing enhancement in feature space rather than image space\n\n(2) This paper lacks rigorous mathematical proof establishing the connection between the VFE module's design principles and classical underwater imaging physics\n\n(3) Critical components of the VFE design lack empirical validation, including the impact of feature-space vs. pixel-space enhancement and the choise of the dark pixel prior.\n\nQUESTIONS:\n(1) What motivated the choice of feature-space enhancement over traditional image-space approaches?\nAre there specific underwater degradation characteristics that make this more effective?\n\n(2) Could you provide either: A more rigorous theoretical proof connecting VFE's operations to underwater imaging models,\nor visualizations showing how VFE module removes underwater degradation?\n\n(3) Could you add experiments isolating the contribution of the dark pixel prior in VFE and performance comparison between feature-space and image-space enhancement?\n\n(4) In the 4th row of Supplementary Figure II, NAUTILUS incorrectly localizes the leftmost cuttlefish. Could you explain why?\n\nMy evaluation scores could be positively revised should the authors substantively address Q1-Q3 with new analyses and discuss the Figure II anomaly's implications.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study addresses the challenges of data scarcity and poor image quality in underwater scene understanding by proposing a large vision-language model named NAUTILUS. To achieve this, the researchers first constructed a large-scale multi-task instruction dataset for underwater environments, NautData, containing 1.45 million image-text pairs. They then designed an innovative plug-and-play Visual Feature Enhancement (VFE) module, which leverages the physical principles of underwater imaging to effectively mitigate image degradation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The researchers developed NautData, a large-scale underwater instruction learning dataset comprising 1.45 million question-answer pairs. This dataset supports eight different tasks.\n- The study introduces a plug-and-play Visual Feature Enhancement (VFE) module inspired by the physical principles of underwater imaging.\n- NAUTILUS is the first large vision-language model capable of handling eight underwater tasks. \n- Experiments show that NAUTILUS outperforms models including GPT-4o and Gemini 2.0 on most tasks in the NautData test set.\n\nWeaknesses:\n- The paper does not analyze the noise level within the LLM-generated portion of the NautData dataset, nor does it discuss how this potential noise might affect the model's training and final performance.\n- The research lacks a qualitative analysis of failure cases. Discussing specific examples where the model fails would offer valuable insights into its limitations and areas for future improvement.\n- The reported performance improvements over the LLaVA-1.5 baseline are quite marginal on several tasks. This limited gain could suggest that the proposed method's effectiveness and generalization capabilities may not be as significant.\n- The study would be strengthened by an ablation that compares this method to other, more naive weighting strategies to justify its design choice.\n\nQUESTIONS:\nCould the authors analyze the noise levels and perform error analysis?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduced an underwater large vision-language model, namely, NAUTILUS, alongside a new large-scale dataset, i.e., NautData, comprising 1.45 million image-text pairs covering eight underwater scene understanding tasks. It tackled the long-standing challenge of underwater image degradation by incorporating a physically grounded Vision Feature Enhancement (VFE) module based on underwater imaging principles. The authors integrated the VFE into two LMM baselines (i.e., LLaVA-1.5 and Qwen2.5-VL). The experimental results demonstrated consistent improvements over strong baselines on multiple datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The VFE module was well-motivated by the physics-based imaging theory and effectively improved robustness under underwater image degradation.\n2. The paper was clearly written and well-organized. Figures effectively supported the technical content.\n3. This work made a high-impact contribution to a critical domain—underwater vision-language modeling. Both NAUTILUS and NautData could form a strong foundation for future research and applications in marine robotics, ocean monitoring, and scientific exploration, given that the authors would make them publicly available.\n\nWeaknesses:\n1. Although zero-shot evaluations were provided on MarineInst20M, more cross-domain generalization tests would strengthen the claims.\n2. The VFE module involved depth extraction and attention mechanisms. A runtime/memory cost analysis was missing. This is important for practical deployment on underwater devices.\n\nQUESTIONS:\n1. Can the VFE module be plugged into other LMM architectures? This would clarify its universality.\n2. What is the runtime overhead introduced by the VFE module?\n3. Will the trained checkpoints and data annotation tools be released on the acceptance of the paper? This is of great significance for the community’s reproduction and further development.\n4. How does NAUTILUS handle domain shifts between shallow and deep-sea environments? Have the authors tested the model across varying light/color conditions?\nPositive responses or additional evidence addressing the first three questions would increase my confidence in the model’s robustness and practical applicability, and potentially raise my score.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-04"}