NAUTILUS: A Large Multimodal Model for Underwater Scene Understanding
Authors: Wei Xu
Cheng Wang
Dingkang Liang
Zongchuang Zhao
Xingyu Jiang
Peng Zhang
Xiang Bai
Venue: NeurIPS 2025 poster ()
Keywords: Underwater scene understanding
vision-language model
large language model
Primary area: deep_learning

Abstract: Underwater exploration offers critical insights into our planet and attracts increasing attention for its broader applications in resource exploration, national security, etc. We study the underwater scene understanding methods, which aim to achieve automated underwater exploration. The underwater scene understanding task demands multi-task perceptions from multiple granularities. However, the absence of large-scale underwater multi-task instruction-tuning datasets hinders the progress of this research. To bridge this gap, we construct NautData, a dataset containing 1.45 M image-text pairs supporting eight underwater scene understanding tasks. It enables the development and thorough evaluation of the underwater scene understanding models. Underwater image degradation is a widely recognized challenge that interferes with underwater tasks. To improve the robustness of underwater scene understanding, we introduce physical priors derived from underwater imaging models and propose a plug-and-play vision feature enhancement (VFE) module, which explicitly restores clear underwater information. We integrate this module into renowned baselines LLaVA-1.5 and Qwen2.5-VL and build our underwater LMM, NAUTILUS. Experiments conducted on the NautData and public underwater datasets demonstrate the effectiveness of the VFE module, consistently improving the performance of both baselines on the majority of supported tasks, thus ensuring the superiority of NAUTILUS in the underwater scene understanding area. Data and models are available at https://github.com/H-EmbodVis/NAUTILUS.

NAUTILUS A Large Multimodal Model for Underwater Scene Understanding
Wei Xu',, Cheng Wang"", Dingkang Liang', Zongchuang Zhao', Xingyu Jiang', Peng Zhang', Xiang Bai HHazzoong University of Science and Technology 2National University of Defense Technology {wxu2023, cwang666, dkliang, kbaiiHhustt... edu. cn
Coarse Fine-grained Classification
Counting
VQA
Identffi the taxonomic class of the fish shown in the image
Check the number fah in the image.
Q: What is on the sea floor?
Monaly the obiea Inside thesppotfee Feigr Feigia siiing
"Coral and
What color the fishes?
"43"
A: "Yelloo and silver."
"Acinopeen"
Q: Is the water bright or dim?
A: "Dim and blue."
Detection
Grounding Outline deseriiped region.
Region & Image Caption
Detect cuttlefish ofjects.
Can you give description of this image?
Descrbbe Oae
"Two scuba divers swim a coral reef with many small, silver fish surrounding them. Bright coral and sea plants decorate
retaton
Di The water Hos
"Twe divers in scubba gear are
its Tins spread oi
swimming with fish." m-
floor."
Figure 1: The underwater environment presents a visually rich and dynamically evolving landscape. NAUTILUS addresses eight diverse underwater tasks, encompassing baare-ggrained classification, fine-grained classification, counting, visual question answering (VQA), detection, grounding, region caption, and image caption, enabling comprehensive understandings across multiple granularities.
Abstract
Underwater exploration offers critical insights into our planet and attracts increas- ing attention for its broader applications resource exploration, national security, etc. We study the underwater scene understanding methods, which aim to achieve automated underwater exploration. The underwater scene understanding task de- mands multi-task perceptions from multiple granularities. However, the absence of large-scale underwater multi-task instruction-tuning datasets hinders the progress of this research. To bridge this gap, we construct NautData, dataset containing 1.45 M image-text pairs supporting eight underwater scene understanding tasks. It enables the development and thorough evaluation of the underwater scene under- standing models. Underwater image degradation is widely recognized challenge that interferes with underwater tasks. To improve the robustness of underwater scene understanding, we introduce physical priors derived from underwater imag- ing models and propose and-play vision feature enhancement (VFE) module, which explicitly restores clear underwater information. We integrate this module into renowned baselines LLaVA-1.5 ..5 and Qwen2.5-VI. and build our underwater LMM, NSUTILUS. Experiments conducted on the NautDDta and public underwater
"Equal contribation. *Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

datasets demonstrate the effectiveness of the VFE module, consistently improving the performance of both baselines on the majority of supported tasks, thus ensuring the superiority of NAUTILUS in the underwater scene understanding area. Data and models are available at https: /// github. cww/H Comm//E-EnboolissAAAAIII
Introduction
The underwater world plays pivotal role in global well-being, as it covers more than 70% of the
Earth's surface and encompasses the largest ecosystem on our planet [14, 7, 39, 46]. Advancing underwater scene understanding methods facilitates automated underwater robot exploration [48], benefiting sufficient environmental protection [23] and resource development [57]. Comprehensive underwater scene understanding comprises both perception (e.g.. detection and counting) and semantic understanding tasks (c.g., region and image caption). However, most underwater methods [66, 49, 59] typically tailored to specific task, limiting their understanding of underwater scenes task-specific perspective.
Recent achievements in the general domain have driven the advancement of specialized LMMs in understanding tasks [62, 52], leading to prominent applications in fields like autonomous driving [41], embodied intelligence [16], and document understanding [26]. The promising abilities of large multimodal models (LMMs) provide a considerable solution for the underwater scene understanding area. Nonetheless, empirically find that directly adopting general LMMs in underwater scenes cannot serve as an ideal solution, as they face inherent challenges arising from 1) the significant domain shift between in-air and underwater data, and 2) image degradation due to light scattering and absorption in water.
OceanGPT [8] infuses the richness of knowledge into large language model (LLM), uncovering its potential in the field of ocean science while failing to interpret multimodal inputs. MarineGPT [65] empowers the LLMs to sense vision-language information and has been the first-ever publicly available underwater LMM. As shown in Tab. this pioneering work constructs a large-scale underwater vision-language dataset for instruction tuning while only focusing on the image-level understanding, neglecting the hierarchical underwater scene information. We take a valuable step in constructing NautData, an underwater instruction-follle following dataset containing 1.45 million question- answer pairs that cover eight diverse underwater tasks. It establishes a solid foundation for the development of underwater LMMs, bypassing the in-air domain shift prevalent in most current instruction-tume datasets.
The image degradation problem hinders reliable underwater scene understanding. Most methods leave this challenge to the training process, driving the models to learn underwater representations on their own, which could be sub-optimal due to the intricate complexity of underwater conditions. To bridge this gap, we propose a glug-and-plaa vision feature enhancement (VFE) module that explicitly removes noise responses introduced by image degradation and enhances the understanding performance of underwater LMMs. Specifically, the underwater imaging model provides a physical representation of image degradation in underwater scenarios [67, 61, 3, 38], emphasizing backscat- tering from the surroundings as a primary interference. We adopt a dark pixel prior to quantifying the intensity of backscattering, paving the way for removing its adverse effects. The optical medium influences imaging quality, with underwater imaging facing sigmificant light absorption compared to in-air imaging, and is another crucial factor contributing to underwater image degradation. To address this, we extract depth information to restore the scene signals attenuated by medium absorption. The VFE module can be flexibly employed in general LMMs, and integrate it into two renowned baselines, 1.a [32] and Qwen2.5-VV. [6], to build our underwater LMM NAUTILUS.
We evaluate the NAUTILUS and prestigious LMMs [32, 56, 10, 6] on the NautData to analyze their underwater scene understanding performance in a supervised manner. Then, we directly evaluate them on MarineInst [63], recent large-scale vision-language underwater dataset. This zero-shot experiment indicates the generalization capabilities of our method.
This work contributes to three aspects: 1) We construct NautData, a large-scale underwater instruction- following dataset containing 1.45 M image-text pairs, enabling developments and evaluations of underwater LMMs. 2) We build the first eight-task underwater LMM NAUTILUS, achieving un- derwater scene understanding from image, region, and object levels. empowers comprehensive underwater scene understanding through aggregating hierarchical scene information. 3) We design

---

Table 1: The comparisons between NautData and recent underwater vision-language datasets. Our dataset is more comprehensive for eight-task annotations involving understandings at three granulari- ties, making it valuable contribution to the community.
Supported tasks
Granularity
QA pains
Dalusets
Reference
Classification
Caption
VQA Detection
Grounding
Counting Img.
Reg. Obj.
source
Coarse Fine
Image Region
MarineCiPT [65]
arXiv
1.12
Avail.
Murinelns20M [63]
ECCV 24
/
v
V
20 M
Part, Avail."
CoralMask [64]
CVPR
/
/
46.61 K Avail.
AquatieCLIP
arXiv 25
V
2
Not Avail.
SynTIDE [31]
CVPR 25
54.51 K
Avail.
NautData (ours)
/
v v 1.45M
Avail.
CCrrrent public version contains 2.2 QA pairs.
plug-and-play VFE module motivated by physical underwater imaging model. It restores degraded information explicitly in the feature space. Experiments renowned baselines demonstrate its effectiveness on all the annotated tasks.
Related Work
Underwater Vision-Language Analysis. Pioneering studies [34, 68, 32] have aligned linguistic and visual representations using projection layers, enabling L.MMs to process multimodal information and advance into LMMs. Through visual instruction tuning, LMMs have presented impressive performance in both general-purpose and domain-specific areas. For instance, LLaVA-NeXTI [33), IL [11], and MiniGPT-v2 [9] have achieved remarkable success in the general domain by advancing LMMs through improvements in data, vision encoder structures, and training strategies. PaLM-E [16] and Dolphins [36] find LMMs effective in embodied intelligence and autonomous driving, respectively. MarineGPT [65] trains LMMs with marine science knowledge, focusing primarily on image-level understanding while lacking attention to region- and object-level [37, 30] scene information. AquaticCLIP [5] employs contrastive lcarning-based pretraining on aquatic image-text pairs to align the underwater image and text representations. The AquaticCLIP enhances performance on downstream underwater tasks while lacking the capability to follow instructions directly. Marinelnst [63] localizes objects and generates linguistic descriptions for each of them. The object-level vision and text responses contribute to detailed marine image analysis. CoralSCOP [64] can be driven by both vision and text prompts, enabling the mask generation of corals described by users. Despite increasing attention, research on underwater L.Ms remains limited and requires further efforts to achieve underwater vision- vision-language dialogues. This paper advances this field by exploring the capabilities of LLMs to deliver hierarchical underwater scene understandings.
Underwater Image Enhancement. Underwater images often exhibit poor visibility, low contrast, and severe color distortions, primarily due to light absorption and scattering in aquatic environments. To address these issues, underwater image enhancement methods aim to mitigate such degradations and restore visual quality comparable to in-air images. Conventional methods employ typical image augmentation strategies, such as histogram stretching [15, 60] and image fusion [20] enhance the underwater images. These methods are easy to deploy, but they suffer from the generalization limitations of handcrafted features. Another direction of underwater image enhancement research [19, 54, 17] explores deep learning-based methods, particularly those leveraging generative adversarial networks (GANs) to improve the quality of underwater images. However, these methods are hindered by the inadequate availability of high-quality training data, as collecting underwater images is often constrained by the complexity of underwater environments, high equipment costs, and the challenges associated with accurate data annotation. Physical model- based methods [67, 61, 51] offer a feasible solution these problems. They reduce the search space for parameters by incorporating handcrafted priors, thus mitigating dependency on large-scale training data. However, directly applying image enhancement to underwater images may result information loss, thereby limiting the effectiveness in underwater scene understanding, as demonstrated in our experiments. Building upon the previous discussion, we innovatively introduce an enhancement in feature space within an LMM, providing an efficient solution to extract underwater visual information.
3

---

Inpuf
LMMs
Process
NautData
information
Counting, Detection, Coorse/Fine-graine classification
Rule-based
Gemini 2.0 Flash
Integration
Grounding. Image/Region caption
Lmage
Qwen2.5-VL ChatGPT-4o
Free-form
Task-specific prompt
VQA
Illustration of the data construction framework. Eight tasks are involved, and the data generation process is tailored to each task. Rule-based generation utilizes predefined templates to generate drestion-answer pairs, Integration generation integrates question-answer pairs using both templates and outputs from LMMs. Free-form generation enables LMMs to construct questions and answers based on the content they focus on.
Figure 2:
Dataset Construction
Underwater vision-language datasets [65, 63] lack multi-granular and multi-task annotations. To
address this gap, we construct NautData, which provides a pioneering resource for advancing research
on underwater scene understanding. The distinct advantages of NautData lic in three aspects: (1)
extensive underwater information encompassing image- region-, and object-level understanding; (2)
diverse conversational structures, including both rule-based and free-form content; and (3) significant scale, comprising 158 images and 1.45 QA pairs.
Attributes of Individual Entities. Public underwater object detection datasets such as RUOD [18], Deepfish [47], and Brackish [42] provide bounding-bos annotations reflecting the position information of entities. As shown in Fig. 2. we employ a rule-based procedure to reformulate the labeled coor- dinates into linguistic answer "Class: [x1, 1/1, 22, y2]. ...., Class: [21, y1, 22, y2]". Then, concatenate it with pre-defined question "Detect all underwater objects in the image."
create an object-level perception conversation.
Classifying aquatic targets constitutes an expert-level task due to the specialized nature of underwater knowledge. We design both grariee and fine-gr the-grained classification conversations, facilitating knowledge sharing in this domain. Specifically, a standard detection task distinguishes underwater objects in coarse- graiiedd categories, e.g., fish, turtles, reefs. FishNet [25] further fine- tne-grains the fish into 8 taxonomic classes. We pre-define questions with corresponding detection categories as answers deliver coarse-grained classification conversations. Subsequently, for the FishNet dataset, we couple questions with ground-truth taxonomic class names as answers to construct fine-grained classification dialogs.
Apart from the formalized annotations positions and categories, we employ LMMs [50, 6, 40] to generate textual descriptions, which could contain rich individual properties, such color, texture, and shape. For instance, we inject object coordinates textual form into prompts, directing concentrations of LMMs target local regions. The generated descriptions are subsequently integrated with these coordinates to construct grounding conversations.
Annotations on Regional Groups. Underwater species often exhibit collective behaviors essential for understanding their survival and ecosystem dynamics. 1OCfishKK [49] a densely distributed underwater object counting dataset, averaging 117 targets per image. We design conversations on group counts and behaviors to enhance regional understanding. Specifically, we first couple pre- defined questions with ground-truth counts answers, prompting models to regress numerical results directly. We then convert this regression task into a single-choice question by randomly selecting intervals from (5, 50, 100} } to construct four-term arithmetic sequences that include the ground-truth count. Additionally. we employ LMMs [50, 6, 40] to generate descriptions for regional groups, which may describe collective behaviors and relationships. These descriptions are integrated with pre-defined questions to create region caption dialogs.
Descriptions for Holistic Semanties. Image-caption pairs have been demonstrated to significantly benefit LMMS in aligning vision and language modalities [6, 34, 24]. We feed collected images into LMMs [50, 6, 40] to obtain a holistic understanding of the given scenes. The outputs are coupled

---

with pre-defnee questions to construct image caption conversations. Underwater images record rich information rarely discussed in existing datasets, such as brightesss and geomorphology, We employ LMMs 150, 6, 40] to generate free-form visual question-answering (VQA) conversations discussing these elements.
During the generation process utilizing LMMs, we first employ Gemini 2.0 Flash to produce initial outputs. Subsequently, these outputs are evaluated by Qwen2.5-VL-72BB and any responses identified as low-quality are replaced with newly generated answers. Leveraging the constructed NautData, we collect image-text pairs to develop the NautData test set, which is further assessed by GPT-4o.
Answers flagged as low-quality undergo additional manual verification by our research team.
The current Naut test comprises 3, 920 images paired with 7, 7,916 question question-answering answering (QA) examples. These QA pairs encompass a diverse range of tasks, including image and region captioning. pearse-grained and fine-grained classification, grounding, detection, counting, and visual question answering (VQA). This comprehensive benchmark is designed to facilitate rigorous evaluation of methods underwater scene understanding, thereby promoting further advancements in this field.
4 Methodology
Underwater environments, characterized by intricate conditions and entities of diverse colors, shapes, and scales, require multi-grained perception to achieve comprehensive scene understanding. NAU- TILUS is the first model empowering vision- langua- conversations spanning image-, region-, and object-level underwater scene understanding tasks, potentially facilitating seamless human-compp compter interaction and underwater knowledge sharing.
To illustrate the design of NAUTILUS, we first review the physical underwater imaging model to depict the motivation of explicitly dealing with underwater image degradation. Then, we detail the implementation of the model architecture involving the overall framework and the vision feature enhancement (VFE) module.
4.1 Preliminaries
Underwater imaging model [67, 61, 38] typically formulates the captured underwater image I. as the combination the direct reflection D. from the underwater subjects and the backscattering B. from the surroundings:
= D. B D. = Jac -_c
(1)
where Jo is the original color at the c-th channel without light absorption during propagation. The underwater imaging model assumes one attenuation coefficient 0-),(*)** for each color, decreasing exponentially with the imaging distance z. The 3, (z) is an unavailable parameter related to data collection conditions, also primarily regulated by imaging depth. Inspired by this physical imaging model, we attempt restore the representations of J, as enhanced vision features:
J. (1 J.)
(=)2
In practice, we fit the attenuation coefficient by employing a learning module with depth information inputs. Furthermore, we introduce dark pixel prior [67] to quantify the impact of backscattering B.. For instance, due to the surrounding backscattering in underwater environments, dark pixels often lose their original black appearance and instead exhibit a blue-green color. The dark pixel prior emphasizes that these distorted pixel values located at dark pixels reflect influences of backscatterings. Drawing on this analysis, we localize the dark pixels of given image and regard their responses as backscattering intensities. Subsequently, simply substitute the obtained parameters in Eq. 2 to complete the feature enhancement.
The underwater imaging model offers a physical principle to regularize the learning phase, which we consider explainable and sufficient optimization direction for model design.
4.2 Model Architecture
As shown in Fig. 3, the framework of NAUTILUS primarily consists of an image encoder I,, a depth encoder Ig, a iision-to-language projector P.-1. VFE module, and an LLM. Given underwater
5

---

Image encoder
Image-level unders fanding
feature
V-L
"This image
Vision Feature Enhancement (VFE)
Region-level understanding
Dork pixel
LLM
"This region
Depth encoder
Objech-level unders fanding
Depth
modules
LoRA t
Con give description of this image?
Tunable modules
Figure The framework of NAUTILUS. Inspired by underwater physical priors [67, 3, 38], we sample dark pixels to quantify the responses of underwater degradation. The vision feature enhancement (VFE) module improves underwater LMMs with depth information as auxiliary information. Outputs of the image encoder and the VFE module are fed into an LLM to facilitate multimodal processing.
image æ, the image encoder extracts vision features v II, (z). While employing LLaVA-1.5 [32] as the baseline, we employ a CLIP VYT-L/14 [45] with a base resolution of 336 as the image encoder. The vision-to-language projector is a multi-layer perceptron aligning vision and language representations. empowers the LLLM Vicuna-v1.5 [12] reason about visual and textual information.
Motivated by the underwater imaging models [67, 3,38] a physical prior illustrates that the imaging distance closely influences the degree of image degradation. In particular, subjects farther away from the camera suffer more pronounced color degradation, deviating further from their original appearances. Therefore, adopt frozen Depth Anything V2 [55] encoder extract depth features = Ia (a)) from given scenes. In addition, inspired by the dark pixel prior introduced in Sec. 4.1, 1, it reasonable measure the backscattering influence by analyzing the responses of dark pixels. In practice, as pixels are processed in a patch-wise manner, we identify the k-th image patch that exhibits the lowest average RGB value and treat the pixels within this patch as dark pixels. Subsequently, we feed the vision feature, index k, and depth feature into the VFE module M to obtain an enhanced vision feature V. M (v, k d). We would like to emphasize that both the original and enhanced vision features are essential for understanding underwater scenes. Specifically, on the one hand, the degradations in the original vision feature reflect authentic underwater environments, facilitating the study of real ecosystems. On the other hand, the restored information in the enhanced vision feature reduces the adverse effects of imaging conditions, enabling reliable underwater perceptions. Therefore, we feed them forward in parallel. And due to the homologous representations of the two features, we utilize a shared projector to align them with the linguistic modality. This process can be formulated as follows:
i Po-1 (v), 6. = Po
where è and é. are aligned vision features derived from v and ve, respectively. Afterward, we employ the LLM integrate user instructions and vision information, finally achieving multi-granular underwater scene understandings from multi-task aspects.
4.3 Vision Feature Enhancement
According to Eq. 1, the underwater vision feature enhancement comprises two steps: 1) removing backscattering and 2) restoring light absorptions. We reveal the entire process in Fig. 4 and clarify the reasons for taking them.
Remove Backscattering. Assuming v = [fo.)" E RRdd where n and d denote the length and dimension of the vision feature, we regard the k-th slice fv, E 1xddd as dark token, representing the response of the dark pixels. Simply subtracting in feature space would filter out responses from the backscattering. However, the dark token is encoded through multiple attention layers, which infuses global semantics beyond the backseattering. Therefore, we further isolate the global semantic responses from this token to estimate a pure backscattering intensity. Specifically, we employ cross-attention layer with vision feature as the key and value to aggregate global information into a learnable query. This query is also embedded with global average feature, which is obtained by applying average pooling over the vision feature, guiding the learnable query to be more familiar with the global semantics. Assuming the output of this cross-attention layer q I2 Rxxdd the

---

backscattering responses E I 1xd can be refined into fu.k q. As the backscattering is added to the whole underwater scene, disturbing every image pixel, we remove backscattering by pixel-wise subtracting it from the vision feature. This process can be simply formulated as 8, where is broadcast to Raxd, keeping the same shape as e.
Restore Light Absorption. According to the underwater imaging model, underwater light is absorbed over imaging distances. We predict an absorption weight W Rnxd using a light- weight multilayer perceptron MLP with the depth feature an input, i... W MLP (d). Finally, we obtain the enhanced vision feature v. by calculating:
Vision Feature Enhancement
Visian festure
Output
K, v
Cross- athention
indip
Averoge
Learnable query
Dark
Dark tokien
Backscathering response
piwel
MLP
exp(
Restore weight
Ue (V- exp (-W),
(4)
Depth
where @ and exp (-) denote element-wise di- vision and exponentiation, respectively. Eq. 4 conforms to Eq. 2, explicitly injecting human priors into network structure and regularizing restorations of visual responses in feature space.
Figure 4: The structure of the vision feature en- hancement (VFE) module. The inputs consist of the vision feature, the index of the dark pixel, and the depth feature. outputs enhanced vision fea- tures capturing restored underwater information.
5 Experiments
We explain the implementation details and conduct experiments to evaluate the performance of our model. As this is pioneering work towards underwater understanding and perception employing LMMs, we select distinguished general-purpose baselines as counterparts to present comprehensive comparisons,
Implementation Details. We enhance vision features address the native underwater image degradation, as described in Sec. 4.3. To demonstrate how this design benefits current LMMs, we adapt it to LL.VA-1.5 [32] and Qwen2.5-Vi [6]. two prevalent LMM frameworks. For both of them, we adopt warameter-efficien fine-tuning (PEFT) strategy [53, 29, 22]. and the trainable components are the vision-to- lo-nuuuue projector, LoRA [22], and the vision feature enhancement module. In our instruction tuning, we preserve the default hyperparameters of LA-V.5 to pursue optimal performance and ensure fair comparison with its original implementations. since the official repository only supports full fine-tuning, we reproduce LoRA fine-tuning setting the learning rate as x 10-5 with the batch size as 16. The LoRA ranks in LLaVA-1.5 and Qwen2.5-VL are set 128. Unless otherwise specified, we use the 7B variants, except for Intern for which we employ the VL-2..58 model. Our experiments are conducted on four NVIDIA A800-80GE GPUs, training each model for one epoch, taking around days.
for Qwen2.5-VE VL,
Datasets. NautData is the first underwater instruction-followin dataset providing cight-task anno- tations. Experiments are primarily conducted on the NautData. MarineInsI20M [63] is recently impressive underwater lision-language dataset containing high-quality image-caption pairs. Among this dataset, we conduct pero-shot evaluations on its Shutterstock, and Gettyimages subsets. 1OCfish5k [49] the unique underwater object-counting dataset among the collected datasets. We evaluate the counting performance on its test set.
part involving the Flickr,
5.1 Comparison to SOTA Methods
Compared to Renowned LMMs. We conduct zero-shot experiments using GPT-4o [40], Qwen2.5- VL-72B [6], and Gemini 2.0 Flash [44] on the NautData test set. As shown in Tab. 2. despite their advanced image understanding capabilities, these methods struggle to achieve satisfactory performance due to the inherent complexity of underwater environments. Fine-tuned open-source baselines achieve significant improvements over the commercial LMMs on all tasks, demonstrating the effectiveness of domainn-ppciif adaptation in underwater seene understanding. The proposed NAUTILUS explicitly addresses underwater image degradation, empowering reliable underwater perceptions under various adverse conditions. Specifically, while employing L.a. [32] as the baseline, our method improves its performance on seven of eight tasks, presenting strong practical potential. Furthermore, while employing Qwen2.5-VL [6] the base model, the NAUTILUS achieves
7

---

Table 2: The comparison of our NAUTILUS and renowned LMMs the Naut test set. The best results are highlighted in bold, and the second-best results are underscored.
Classification
Caption
Grounding
Delection
VQA
Methods
Reference
Course
Image
Region
acc f
acc 7
METEOR t METEORT t m PR@0.51 1 MAPt mAP@0.5 METEOR
Zhoo experiments
4P [40]
55.2
54.4
0.179
0.148
14.2
4.3
0.3
1.4
0.242
Qwen2.5.VV --- [6]
55.2
$4.2
0.171
0.126
42.3
46.4
8.8
14.7
0.222
Gomini 2.0 Flash
55.5
0.185
20.6
2.4
7.8
0.223
Instraction-turiin tuning experiments
MiniGPTv2 [9]
arXiv 23
80.0
90.0
0.204
0.178
47.0
51.0
6.9
12.9
0.372
mPLUG-OWLB [56]
arXiv
1ntemVL.22.5 [10]
arXiv 24
91.3
90.4
0.208
0.195
50.4
54.6
18.3
30.5
0.382
1.aVA.155 [32]
90.0
Qwen2.5-VL [6]
arXiv 25
85.3
88.2
0.232
0.196
52.5
57.6
24.5
41.7
0,380
NAUTILUSILLNN-13
91.0
0.208
0.191
46.2
52.2
11.1611
20.9
0.365
NAUTILUS
90.3(45.0
93.8
(.223 0.199))0.003) 53.8k-1.))
58.8
25.8r+1.3)
45.3
0,381
the best performance on four tasks, including fine- fine-grained classification, image caption, grounding, and detection, demonstrating remarkable underwater scene understanding capabilities.
Group Perception. The object counting task provides insight into group behaviors. We eval- uate NAUTILUS the underwater object count- ing task to assess its capability for group un- follow the
Table Counting accuracy on the IOCfish5k [49] test set.
derstanding. As shown in Tab. official data splits of IOCfish5k [49] to construct training and subsets. Our methods outper- form other LMMs by at least 0.7 MAE and 0.3 RMSE, delivering 8.0 MAE and 15.9 RMSE improvements on 1.55,- which exhibits impressive group perception performance. Nev- ertheless, there is slight performance drop in the accuracy metric, which we attribute to the challenges of multi-task optimization. Specif- ically, the single-choice question text clas- sification task substantially different from the count regression task in both objectives and out- put formats, introducing further challenges.
Methods
Reference
MAE
RMSE
acc 1
Zero-shot experiments
GPT-4o [40]
51.2
94.0
55.4
Gemini 2.0 Flsh [44]
50.2
123.3
50.7
Qwwen.55VVL-22)
49.8
124.7
58.8
buning experiments
MiniGPTy2 [9]
arXiv 23
55.0
139.6
43.0
mPL.G Owl3 [56]
arXiv 24
36.2
102.9
70.5
IntemVVL-2.5 [10]
arXiv 24
39.1
95.0
69.2
[32]
61.7
Qwen2.5.VL. [6]
arXiv 25
31.6
96.7
70.8
NAUTILUS/LLWW-II
51.2(+800)
136.0(-15.9)
62.6-49)
30.9
93.7:+100
70.7
Generalization. After fine-tuning the well-
established baselines on the NautData, we di- rectly evaluate their grounding performance on the Marinelnst20M [63]. The grounding task necessitates instance-leve understanding and text comprehension capabilities, which reflect the ability to achieve fine-grat -ra-ggained understand- ing of underwater scenes. As shown in Tab. 4, our method improves VA- [32] and Qwen2.5-VI. [6] by 0.6 and 0.4 PR 0.5, re- spectively, demonstrating its generalization abil- ity across domains and models.
Table 4: Zero-shot grounding on the Marine- Inst20M [63].
Methods
Reference
mloli 1
PR
MiniGPTv2 [9]
arXiv 23
36.6
38.3
mPLU'G-OWB [56]
arXiv
32.9
36.7
VL-25 [10]
arXiv 24
37.6
38.4
LL.VA-1.5
CVPR 24
29.6
33.6
Qwen2.5-VT.
arXiv 25
38.0
39.2
NAUTHLUSILLMALS))
30.7
34,2/-00)
NAUTILUSiQenn25
38.3
39.6-044
5.2 Analysis and Ablation
Fig. presents qualitative visualizations of NAUTILUS across eight underwater tasks. Our model responds to user instructions and outputs multi-granularity results for underwater scene understanding. demonstrating its versatility and effectiveness. this section, we perform ablation studies and provide analysis of the key insights. Models in Tab. 5 and Tab, 6 are trained one-third of the

---

Gointing Region Coption
Finegrsihed Gesuficatier
GTHOT
Rlese count how mary fan are the Image
Colegarize the object localed within the prov regresion box (St, ali, 337, 447)
X04
Deer
Identity the tocenamic dlas of the fah wither prosited regrastion hox (CPS.
(6, 333,
205. 510,
school of yellow fish with boee olripes owin near corol reef.
Achiosperrgh
Grounting Tmage Ception
coarcinahe that corraspond the givn region "A fun brew and yullon color rg ISON formet.
Extract
Fiat the lergact organim the pichune.
Sea furtle.
V bock 26l" 179, stzl]
Can you give a description Inage?
imoge
divers
1'abalt's
fowards fish. Sexendl small the bother..
Sall""aaa Virah'' R'stor 183, Vlebel V'tanty
Figure 5: Qualitative results underwater understanding. NAUTILUS perceives image-, region-, and object-level information while addressing eight tasks. Our underwater LMM exhibits remarkable multimodal instruction-following performance, serving meaningful contribution to this field.
Table 5: Ablation on components. We employ Qwen2.5-VL. [6] as the baseline model and sequentially add our modules. Evaluations conducted on eight tasks to provide comprehensive analysis.
Vision Feature Enhancement
Classification
Caption
Depth encoder
Counting Grounding Detection
VQA
Baseline
Restore light absorption
Remove backscattering
Coarse
Fine
Image
Region
acc f
acc
METEOR
METEOR f MAEL
PR@0.5
AP@0.5 METEOR
87.9
89.1
0.223
0.197
36.8
55.4
35.9
0.367
89.5
89.1
0.218
0,194
37.2
55.0
36.4
0.369
91.2
0.372
90.0
91.4
0.222
0.198
36.4
55.9
36.2
0.373
NautData training set and evaluated on the full NautData test set. Tab. 7 presents the performance of models fine-tuned the complete NautData training set when evaluated under various degraded conditions.
Design of the VFE Module. As shown in Tab. 5, evaluate the effectiveness of components. The depth encoder provides rich depth information, which is expected to enhance the LMM's understanding of underwater scenes. However, due to the discrepancy in feature distribution, simply adding depth encoder results in performance degradation on four tasks. contrast, we utilize depth information to enhance light absorption and achieve improvements in five tasks compared to the second line, emphasizing the feasibility of feature fusion guided by the underwater imaging model. Subsequently, remove backscattering to complete the restoration phase, which further surpasses the third line on five tasks, highlighting the benefits of each component.
Enhancement in Feature Space. We train the Qwen2.5- 5-V baseline with all images enhanced by Reti-Diff [21], SMDR-IS [58], and CCL-Net [35], three ofatc-of-the-arr underwater image restoration methods, to assess the benefits of image enhancement. As shown in Tab. 6, there are consistent performance drops in the coarse-gras granied classification, image caption, region caption, and detection tasks. We attribute these inferior results to the information loss introduced by image enhancement during image pre- mrepprceessing. information to the greatest extent, leading to higher reliability and thus deserves wide application.
contrast, feature enhancement preserves the image's 's original
Evaluation under Degraded Conditions. Underwater environments often exhibit distinct char- acteristics in lighting and turbidity. To evaluate the robustness of NAUTILUS under such domain shifts, we assess its performance across varying environmental conditions. Specifically, we employ Gemini 2.5 Flash [13] to categorize the NautData test set based on lighting conditions (low-light,

---

Table 6: Ablation components. We employ Qwen2.5-VL [6] as the baseline model and sequentially add our modules. Evaluations are conducted on eight tasks to provide comprehensive analysis.
Classification
Caption
Counting
Grounding
Delection
VQA
Methods
Coarse
Imuge
Region
acc 1
acc 1
METEOR 1
METEOR f
MAE
PR.00.5 f
AP@0.5
METEOR
Baseline
87.9
89.1
0.222
0.197
36.8
55.4
35.9
0.367
+Reti Diff [21]
0.221
0.194
+SMDR-IS PARRR-SSERR [58]
86.8
86.3
0.220
0.195
36.5
54.4
31.4
0.371
+CCL-Not [35]
82.5
0,220
0.193
37.8
54.2
0.365
*VFR (ours)
90.0
91.4
0.222
0.198
36.4
$5.9
36.2
0.373
Table Ablation on degraded conditions. divide the NautDDta test set into subsets based on the degradations of data and evaluate grounding performance these subsets using the PR @.5 metric.
Methods
Reference
Low-light
Normal-light
Green-tinted
Blue-tinted
Turbid
Clear
MiniGPTv2 [9]
arXiv 23
44.7
52.8
47.0
52.7
46.6
53.5
m [56]
arXiv
IntemVLL-2.5 [10]
arXiv 24
52.7
55.1
53.4
55.0
52.7
55.6
LLaVA-1.5 [32]
CVPR 24
44.1
49.1
43.5
50.1
41.9
51.8
NAUTILUS.LLNA.JS
51.66-7.5)
$2.0:22)
$1.848.3)
51,9:+15)
50.0k-8.1)
53.1.413)
Qwen2.5.
arXiv
56.9
58.5
56.5
58.9
53.4
59.3
58.5
58.7.00)
(7.7
59.2.
55.40200
normal-light), water turbidity (turbid, clear), and color casts (green-tinted, blue-tinted). As shown in Tab. 7, NAUTILUS demonstrates exceptional robustness, particularly under challenging conditions. Compared to the baseline LLaVA-1.5, NAUTILUS achieves substantial improvements 7.5, 8.3, and 8.1 @.5 in low-light, green-tinted, and turbid scenarios, respectively. Even under less challenging conditions, NAUTILUS maintains consistent performance gains. Compared to the bascline Qwen2.5- VL. equipped with strong grounding performance, NAUTILUS still achieves notable improvements for at least 1.2 PR @0.5 facing degradations. These results demonstrate the remarkable robustness and practical applicability of NAUTILUS across diverse underwater environments.
6 Conclusion
We introduce physical principles to regularize the model architecture and propose a vision feature enhancement (VFE) module. Integrating this module into renowned I.L.VA- and Qwen2.5- VL, we develop NAUTILUS, the first underwater L.M addressing underwater image degradation explicitly. Furthermore, we construct NautData to bridge this gap of the absent underwater multi-task instruction--follwinn dataset. Experiments demonstrate the effectiveness of the VFE module, consistently improving baselines across almost all tasks. Comparisons with current state-of-the-art methods further highlight the superiority of NAUTILUS, establishing our work as a valuable contribution to the community.
both NautData and public underwater benchmarks
Limitation. The vast diversity of underwater environments and species poses substantial challenges for exhaustively representing relevant categories and scenarios in current datasets. Therefore, underwater scene understanding algorithms must possess open-vocabee voccaulayy or few-shot learning capabilities to generalize effectively novel and unseen cases, which is under-explored our work.
Acknovledgements
This work was supported by the NSFC (U2341227 and 62225603).
References
[1] Aquarium dataset. httpa////ooo///.oo.oo////////oo.
[2] Underwater
trash
detection
dataset.
07 071bct525prasanga http:eaprraaaaaapaapppuuee wwwpppsaapppppppp.pppppp edu- madu-nn/aae
10

---

[3] Derya Akkaynak and Tali Treibitz. A revised underwater image formation model. In Proc. of IEEE Inil. Conf. on Computer Vision and Pattern Recognition, 2018.
[4] Basit Alawode, Fayaz Ali Dharejo, Mehnaz Ummar. Yuhang Guo, Arif Mahmood. Naoufel Werghi, Fahad Shahbaz Khan, Jiri Matas, and Sajid Javed. Improving underwater visual tracking with a large scale dataset and image enhancement. arXiv preprint arXiv:2308. 15816, 2023.
[5] Basit Alawode. lyyakutti Iyappan Ganapathi, Sajid Javed, Naoufel Werghi, Mohammed Bennamoun, and Arif Mahmood. Aquaticelip: A vision-language foundation model for underwater scene analysis. arXiv preprint arXiv:5502.07885 2025.
[6] Shuai Bai, Kegin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-v1 technical report. arXiv preprint (rXiv:2502.13922 2025.
[7] David Beauchesne, Kevin Cazelles, Rémi M Daigle. Dominique Gravel, and Philippe Archambault. Ecological interactions amplify cumulative effects in marine ecosystems. Science Advances, 11. 2025.
[8] Zhen Bi, Ningyu Zhang. Yida Xue, Yixin Ou, Daxiong Ji, Guozhou Zheng. and Huajun Chen. Oceangpt:
A large language model for ocean science tasks, In Association for Computational Linguistics, 2024.
[9] Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengehuan Zhang, Raghuraman Krish- namoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint or Xiv::2330000000 130.0994, 2023.
[10] Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries open- source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.052777 2024.
[11] Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing. Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic iisual-linguise tasks. In Proc. of IEEE Intl, Conf. on Computer Vision and Pattern Recognition, 2024.
[12] Wei-Lin Chiang, Zhuohan Li, Ziqing Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang. Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90% chatgpt quality. See https://vicuna. Imsys. org (accessed 14 April 2023), 2, 2023.
[13] Gheorghe Comanici, Eric Bieher, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with ad- vanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.000 006661 2025.
[14] Roberto Danovaro, James Aronson, Silvia Bianchelli, Christoffer Boström, W Chen, R Cimino, Cinzia Corinaldesi, J Cortina-Segarra, Ambrosio, C Gambi, et al. Assessing the success of marine ecosystem restoration using meta-analysis. Nature Communications, 16, 2025.
[15] Lili Dong, Weidong Zhang, and Wenhai Xu. Underwater image enhancement via integrated rgb and lab color models. Signal Processing: Image Communication, 104, 2022.
[16] Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery. Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: an embodied multimodal language model. In Proc. of Inl. Conf. on Machine Learning, 2023.
[17] Cameron Fabbri. Md Jahidul Islam, and Junaed Sattar. Enhancing underwater imagery using generative adversarial networks. In Proc. of the IEEE Int. Conf. on Robotics and Automation, 2018
[18] Chenping Fu, Risheng Liu, Xin Fan, Puyang Chen, Hao Fu, Wanqi Yuan, Ming Zhu, and Zhongxuan Luo. Rethinking general underwater object detection: Datasets, challenges, and solutions. Neurocomputing. 517, 2023.
[19] F Fu, P Liu, 2 Shao, J Xu, and M MEvo-GAN Fang. A multi-scak evolutionary generative adversarial network for underwater image enhancement. J. Mar. Sci. Eng, 12, 2024.
[20] Pengfei Guo, Delu Zeng, Yunbo Tian, Shuangyin I. Liu, Hantao Liu, and Daoliang Li. Multi-scale enhance- ment fusion for underwater sea cucumber images based on human visual system modelling, Computers and Electronics in Agriculture, 175, 2020.
[21] Chunming He, Chengyu Fang. Yulun Zhang, Tian Ye, Kai Li, Longxiang Tang, Zhenhua Guo, Xiu Li, and Sina Farsiu. Reti-diff: Illmiiation degradation image restoration with retinex based latent diffusion model. In Proc. of Intl. Conf. on Learning Representations, 2025.
11

---

[22] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In Proc. of Intl. Conf. on Learning Representations, 2022.
[23] Juliette Jacquemont, Charles Loiseau, Luke Tornabene, and Joachim Claudet. 3d ocean assessments reveal that fisheries reach deep but marine protection remains shallow, Nature communications, 15. 2024,
[24] Oguzhan Fatih Kar, Alessio Tonioni, Petra Poklukar, Achin Kulshrestha, Amir Zamir, and Federico Tombari. Brave: Broadening the visual encoding of vision-language models. In Proc. of European Conference on Computer Vision, 2024.
[25] Faizan Farooq Khan, Xiang Li, Andrew Temple, and Mohamed Elhoseiny. Fishnet: A large-scale dataset and benchmark for fish recognition, detection, and functional trait prediction. In Porc. of IEEE Intl. Conf. on Computer Vision, 2023.
[26] Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xiang Bai. Monkey: Image resolution and text label are important things for large multi-modal models. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2024.
[27] Shijie Lian, Hua Li, Runmin Cong, Suqi Li, Wei Zhang, and Sam Kwong, Watermask: Instance segmenta- tion for underwater imagery. In Porc. of IEEE Intl. Conf. on Computer Vision, 2023.
[28] Shijie Lian, Ziyi Zhang, Hua Li, Wenjie Li, Laurence Tianruo Yang. Sam Kwong, and Rummin Cong. Diving into underwater: Segment anything model guided underwater salient instance segmentation and a large-scale dataset. In Proc. of Intl. Conf. on Machine Learning, 2024.
[29] Dingkang Liang. Tianrui Feng. Xin Zhou, Yumeng Zhang, Zhikang Zou, and Xiang Bai. Parameter- efficient fine-tuning in spectral domain for point cloud learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[30] Dingkang Liang, Wei Hua, Chunsheng Shi, Zhikang Zou, Xiaoqing Ye, and Xiang Bai. Sood++: Leveraging unlabeled data to boost oriented object detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[31] Hongkai Lin, Dingkang Liang, Zhenghao Qi, and Xiang Bai. A unified image-dense annotation generation model for underwater scenes. In Proc. IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2025.
[32] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jac Lee. Improved baselines with visual instruction tuning. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2024.
[33] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang. Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024.
[34] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jac Lee. Visual instruction tuning. Proc, of Advances in Neural Information Processing Systems, volume 36, 2023.
[35] Liu, Qiuping Jiang, Xinyi Wang, Ting Luo, and Jingehun Zhou. Underwater image enhancement with cascaded contrastive lcarning. IEEE Transactions on Multimedia, 2024.
[36] Yingzi Ma. Yulong Cao, Jiachen Sun, Marco Pavone, and Chaowei Xiao. Dolphins: Multimodal language model for driving. In Proc. of European Conference on Computer Vision, 2024.
[37] Dilxat Muhtar, Zhenshi Li, Feng Gu, Xueliang Zhang, and Pengfeng Xiao. Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model. In Proc. of European Conference on Computer Vision, 2024.
[38] Opher Bar Nathan, Deborah Levy. Tali Treibitz, and Dan Rosenbaum. Osmosis: Rgbd diffussion prior for underwater image restoration. In Proc. of European Conference on Computer Vision, 2024.
[39] A Justin Nowakowski, Steven WJ Canty, Nathan Bennett, Courtney E Cox, Abel Valdivia, Jessica L Deichmann, Thomas S Akre, Sara E Bonilla-Anaribe Anariba, Sebastien Costedoat, and Melanie McField. Co- benefits of marine protected areas for nature and people. Nature Sustainabiliry, 6, 2023.
[40] OpenAL. Chatml documents, 2024.
[41] Chenbin Pan, Burhaneddin Yaman, Tommaso Nesti, Abhirup Mallik, Alessandro G Allievi, Senem Velipasalar, and Liu Ren. Vlp: Vision language planning for autonomous driving. In Proc. of IEEE Inil. Conf. on Computer Vision and Pattern Recognition, pages 14760- 14700,111991111 2024.
12

---

[42] Malte Pedersen, Joakim Bruslund Haurum. Rikke Gade, and Thomas B Moeslund. Detection of marine animals in new underwater dataset with varying visibility. In Proc. of IEEE Intl. Conf. Computer Vision and Pattern Recognition Workshops. 2019.
[43] Malte Pedersen, Daniel Lehotský, Ivan Nikolov, and Thomas B Moeslund. Brackishmot: The brackish oulti-object tracking dataset. In Scandinavian Conference on Image Analysis, 2023.
[44] Sundar Pichai, Demis Hassabis, and Koray Kavukcuoglu. Introducing gemini 2.0: our new ai model for the agentic era. Google Blog.
[45] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In Proc. of Intl. Conf. on Machine Learning, 2021.
[46] Eva Ramirez-Llodra Angelika Brandt, Roberto Danovaro, Ben De Mol, Elva Escobar, Christopher German, Lisa A Levin, Pedro Martinez Arbizu, Lenaick Menot, Pal Buhl-Mortensen, ct al. Deep, diverse and definitely different: unique attributes of the world's largest ecosystem. Biogeosciences, 7, 2010.
[47] Alzayat Saleh, Issam H Laradji, Dmitry A Konovalov, Michael Bradley, David Vazquez, and Marcus Sheaves. A realistic fish-habitat dataset to evaluate algorithms for underwater visual analysis. Scientific reports, 10, 2020.
[48] Kjetil Skaugset, João Borges de Sousa, and Asgeir Sørensen. Autonomous robotic organizations for marine operations. Science Robotics, 10, 2025.
[49] Guolei Sun, Zhaochong An, Yun Liu, Ce Liu, Christos Sakaridis, Deng-Ping Fan, and Luc Van Gool. Indiscemible object counting in underwater scenes, In Proc. of IEEE Intl. Conf. Computer Vision and Pattern Recognition, 2023.
[50] Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnel,, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024.
[51] Jiayan Tong. Tianjun Zhang, and Lin Zhang. Towards underwater image restoration: A physical ahysical-accurate accurate pipeline and large scale full-reference benchmark. In IEEE International Conference on Multimedia and Expo, 2022.
[52] Weizhen Wang, Chenda Duan, Zhenghao Peng, Yuxin Liu, and Bolei Zhou. Embodied scene understanding for vision language models via metavqa. In Proc. of IEEE Inal. Conf. on Computer Vision and Pattern Recognition, 2025.
[53] Yanbin Wei, Shuai Fu, Weisen Jiang, Zejian Zhang, Zhixiong Zeng, Qi Wu, James Kwok, and Yu Zhang Gita: Graph to visual and textual integration for vision-languee graph reasoning, In Proc. of Advances in Neural Information Processing Systems, volume 37, 2024.
[54] Junjun Wu, Xilin Liu, Qinghua Lu, Zeqin Lin, Ningwei Qin, and Qingwu Shi. Fw-gan: Underwater image enhancement using generative adversarial network with multi-scale fusion. Signal Processing: Image Commanication, 109, 2022.
[55] Lihe Yang. Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. In Proc. of Advances in Neural Information Processing Systems, volume 37, 2024.
[56] Jiabo Ye, Haiyang Xu, Haowei Liu, Anwen Hu, Ming Yan, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou, mplug-ow13: Towards long image-sequence understanding in multi-moda large language models. arXiv preprint arXiv:2408.00 04840, 2024.
[57] Dana R Yoerger, Annette F Govindarajan, Jonathan C Howland, Joel K Llopiz, Peter H Wiebe, Molly Curran, Justin Fujii. Daniel Gomez -banez, Kakani Katija, Bruce H Robison, et al. A hybrid underwater robot for multidisciplinary investigation of the ocean twilight zone. Science Robotics, 6, 2021.
[58] Dehuan Zhang, Jingchun Zhou, Chunle Guo, Weishi Zhang, and Chongyi Li. Synergistic multiscale detail refinement via intrinsic supervision for underwater image enhancement. Proc. of the AAAI Conf. on Artificial Intelligence, volume 38, 2024.
[59] Peng Zhang, Jinsong Tang, Heping Zhong, Mingqiang Ning. Dandan Liu, and Ke Wu. Self-trained target detection of radar and sonar images using automatic deep learning. IEEE Transactions on Geoscience and Remote Sensing, 60, 2021.
13

---

[60] Weidong Zhang, Xipeng Pan. Xiwang Xie, Lingqiao Li. Zimin Wang, and Chu Han. Color correction and adaptive contrast enhancement for underwater image enhancement. Computers & Electrical Engineering. 91, 2021.
[61] Weidong Zhang, Peixian Zhuang, Hai-Han Sun, Guohou Li, Sam Kwong, and Chongyi Li. Underwater image enhancement via minimal color loss locally adaptive contrast enhancement. IEEE Transactions on Image Processing, 31. 2022.
[62] Duo Zheng, Shijia Huang. and Liwei Wang. Video-3d llm: Learning position aware video representation for 3d scene understanding. In Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition, 2025.
[63] Ziqjang Zheng, Yiwei Chen, Huimin Zeng, Tuan-Anh Vu, Binh-Son Hua, and Sai-Kit Yeung, Marineinst: A foundation model for marine image analysis with instance visual description. In Proc. of European Conference Computer Vision, 2024.
[64] Ziqiang Zheng, Haixin Liang, Binh-Son Hua, Yue Him Wong, Put Ang. Apple Pui Yi Chui, and Sai-Kit Yeung. Coralscop: Segment any coral image this planet. In Proc. of IEFE Intl Conf. on Computer Vision and Pottern Recognition, 2024.
[65] Ziqiang Zheng, Jipeng Zhang. Tuan-Anh Vu, Shizhe Diao, Yue Him Wong Tim, and Sai-Kit Young. Marinegpt: Unlocking secrets ocean to the public. arXiv preprint arXiv:2310.13500 2023.
[66] Jingehun Zhou, Zongxin He, Kin-Man Lam, Yudong Wang, Weishi Zhang, Chunle Guo, and Chongyi Li. Amsp-uod: When vortex convolution and stochastic perturbation Proc. the AAAI Conf. on Artificial Intelligence, volume 38, 2024.
underwater object detection. In
[67] Jingehun Zhou, Qian Liu, Qiuping Jiang, Wenqi Ren, Kin- Kin-Man Lam, and Weishi Zhang, Underwater camera: Improving visual perception via adaptive dark pixel prior and color correction. International Journal of Computer Vision, 2023.
[68] Deyao Zhu, Jun Chen, Xiaogian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.1054)
2023.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See limitation part.
15

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't ' acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
This paper does not involve theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
See experiments part.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code will be made available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /CodeSoommmiiicc SonoddeubmissiooPooo
/ //nips... /tipps.ccpppi wttp:ccccpppppccccccceocooocccca
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
17

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: See experiments part.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
See experiments part.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See experiments part.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
18

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //terr///.e.....eeeeee cc wwtpcccccppcccteeeeeeee httprr//ehhhisueeeelnielnenneeee
Answer:
[Yes]
Justification:
We follow the NeurIPS Code of Ethic.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: No such risks.
Guidelines:
The answer NA means that the paper poses such risks.
19

---

Released models that have high risk for misuse or dual use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We will release the code.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We use the public assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [NA]
20

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https ctterrpsssccecooofennnneeeeeeeee /etpeencceee//////////// for what should or should not be described.
21

---

Appendix
In this appendix, we provide additional content to complement the main manuseript:
Appendix A: Detailed information about the dataset, including 1) Distributions the NautData. 2)
Specific prompts for each task.
Appendix B: More results, including 1) Comparisons employing metrics specific tasks. 2) More ablation studies. 3) More visualizations.
Appendix C: Add discussions about our core insights.
A Detailed Information of the NautData
A.1 Dataset Characteristics
Table I: Distribution of QA pairs generated from different datasets. "Images" indicates the number of images collected from each dataset. "Proportion" refers to the percentage of QA pairs derived from each dataset relative to the total number of QA pairs.
Counting 5,137
Course classification 50,872
Detection 130,058
.% 10.1%
Image caption
47,909
9.6%
35,8%
Dataset
Images
QA pairs
Proportion
32. 22.5%
Rule-based 280,169
USIS10K [28]
10,632
179,772
12.5%
11.3%
162,484
UIS [27]
4,628
123,492
8.6%
Grounding 56,672
RUOD [18]
14,000
326,068
22.6%
Froe-farm $5.605
18.8%
Fine-grained classification 94,102
Deepfish [47]
4,505
97,007
6.7%
Brackish [43]
12,444
245,358
17.0%
11.7%
10Cfish5k [49]
5,637
64,037
4.4%
Region eaption 57,903
UVOT-400 [4]
9,064
169,988
11.8%
Aquarium [1]
638
15,076
1.0%
58,608
Underwater Trash [2]
5,130
33,429
2.3%
FishNet [25]
94,806
188,450
13.1%
Figure I: The distribution images involved in different tasks.
We collect source data from 10 public underwater datasets and present the number of QA pairs generated from each dataset in Tab. I. The RUOD [18] and Aquarium [1] contribute to the most and least QA pairs, with the numbers of 326, 068 and 15, 076, respectively. The NautDDta provides rich annotations covering 8 perception and understanding tasks. As shown in Fig. I, we count images involved in different tasks. The counting task merely contains images from the 10Cfish5k [49] dataset, presenting 1.0% proportion of the total images. Most images are used to generate rule-based multi-modal instructions, constituting 55.8 35.8% all images.
A.2 Generation Prompts
We display the task-specific prompts in Tab. VII. These prompts guide LMMs in generating descrip- tions a specified granularity and in a prescribed format. The generated descriptions are subsequently combined with pre-defined questions to construct QA pairs. In particular, provide pre-defined question templates Tab. VIII.
B
More Results
B.1
Detailed Results on Each Task
We perform comparisons on all supported tasks to provide comprehensive evaluation of the proposed NAUTILUS. As shown Tab. and Tab. IV, our model ranks first on five of metrics, highlighting its superior object-level perception capability. Tab. III illustrates the underwater image and region caption performance of current SOTAs. Our method presents superior performance on the image caption task, surpassing other methods on both the CIDEr and METEOR metrics. However,
22

---

Table III: Comparisons the image caption and region
Table II: Comparisons on the detection task.
caption tasks.
Image Caption
Region Caption
Method mAP@0.75 75 @ 100 t
Method
BLEU-4 1CIDEr 1METEOR 4BLEU-4 TCIDEr 1METEOR
Zero-shot experiments
Zero-shoe experiments
GPT-4o [40]
0.03
0.99
0.30
Qwen2.5.VV V.L.7722 [6]
8.58
GPT-4o [40]
0.047
0.322
0.179
0.018
0.137
0.148
Gomini 2.0 Flash [44]
0.64
5,14
2.36
Qwen2.5-VL-722 [6]
0.037
0.205
0.171
0.043
0.353
0.126
Gemini 2.0 Flash [44]
0,050
0.248
0.185
0,022
0,146
0.141
Instruction nwving experiments
Instruction-tuning experiments
MiniGPTv2 [9]
6.78
10.56
mPLUG-Ow[3 [56]
6.13
18.51
10.32
MiniGPTV2 [9]
0.107
0.794
0.204
0.097
0.792
0.178
Ln.em [10]
18.11
24.17
18.28
mPLLG- Owl [56]
0.130
0.979
0.219
0.156
1.245
0.207
LLaVA- [32]
8.59
16.81
9.79
InternV1-2255 [10]
0.126
0.924
0.208
0.147
1.177
0.195
Qwen2.5-VL [6]
23.64
33,39
24.50
LLaVA-1 [32]
0.126
0.902
0.208
0.141
1.104
0.189
Qwen2.5.V1. [6]
0.140
1.015
0.222
0.149
1,230
0.196
NAUTILUSILLWWWI
10:25
19.37
11.06
NAUTILLU
23.50
34.88
25.75
NAUTILUUELLLAAAS)
0.132
0.947
0.208
0.140
0.109
0.191
0.139
1.023
0.223
0.148
1,258
0.199
Table V: Comparisons on the coarse-grained and fine-grained Table IV: Comparisons on the ground- classification tasks.
ing task.
Coarse Coarse-grained
Fine-grained
Method
AP@:0.5 1PR 7055
10.55
Method
PR
Fl
PR
Fl
Zem shot experiments
Zero-shot experiments
GPT-4o [40]
4.31
Swen2.5-VLL-22 [6]
30,90
33
46.36
GPT-44 [40]
84.40
63.46
55.18
67.58
43.43
54.44
Gemini 2.0 Flash [44]
6.42
7.55
20.60
Swea2.5-VLL722- [6]
55.18
$4.65
Gemini 2.0 Flash [44]
64.35
58.29
55.45
31.27
39.16
$4.34
Instriction tuning experiments
Instruction-tutiic experiments
MiniGPTv2 [9]
32.75
41.19
50.99
mPLUG-Owt [56]
28.15
26.62
45.70
MiniGPTv2 [9]
79.95
Intem [10]
37:50
44.69
54.64
mPI.IGGOOI3 [56]
92.07
91.65
91.92
92.
92.02
92.02
LLaVA-1.5 132]
31.04
30.46
48.21
IntemVLL-2.5 [10]
91.01
90.79
91.25
90.34
90.33
90.40
Qwen2.5. [6]
39.34
47.42
57.62
LLaVA-1.5 [32]
90.11
89.46
90.04
89.72
89.72
89.80
Qwen2.5.VL [6]
82.31
82.94
85.33
90.02
88.19
88.18
NAUTILUS/LL1N0115
34.19
36.16
52.19
NAUTILUSQQNSEES
40.89
48.08
58.81
NAUTILUSLLLVA-11
91.06
90.49
90.98
89.95
89.86
89.90
NAUTILUSQQee2..
90.98
90.31
93.80
93.80
struggles to achieve optimal performance on the region caption task, second to mPLUG-OW13 [56], recent LMM renowned for its high-level image understanding performance, which also performs best on the coarse-grained classification task, demonstrated in Tab. V. Nonetheless, it is worth noting that the NAUTILUS presents impressive capabilities on the fine- graigga classification task, surpassing other methods on three metrics. We attribute this to the effectiveness of the VFE module in enhancing the underwater vision feature, which empowers our models to perceive more fine-grained information.
B.2 Ablation Study
We conduct ablation studies on the weighting strategies, with results presented in Tab. VI. In particular, explore three designs of weighting strategies, including "wo/ weighting", "Learned from image feature", and "Learned from depth feature (ours)". Among them, the "wo/ weighting" strategy can be considered as "a norm weight 1". The "Learnd from image feature" strategy, which means loss of depth information, presents comparable performance compared with our design in most tasks. However, depth information is essential for distance perceptions of complex underwater objects, benefiting underwater scene understanding intuitively. Our strategy with the fusion of depth features surpasses other strategies, indicating the effectiveness of our design choice.
B.3 Visualizations
We provide visualizations employing our NAUTILUS more underwater scenes, as shown in Fig. II. Despite the significant diversity in these scenarios, including substantial variations in lighting
23

---

Table VI: Ablation studies the weighting strategies. We employ Qwen2.5-VL. [6] as the baseline model. Evaluations are conducted on eight tasks to provide a comprehensive analysis.
Classification
Caption
Counting
Grounding
Detection
VQA
Methods
Imuge
Region
acc
acc 7
METEOR
METEORT 1
MAE
PR600.
AP@0.51
METEOR
wal weighting
87.9
89.1
0.222
0.197
36.8
55.4
35,9
0.367
Learned image
0.375
Learned from depth feature
90.0
91.4
0.222
0.198
36.4
55.9
36.2
0.373
conditions and viewpoints, the NAUTILUS accurately localizes the targets and provides reasonable textual descriptions. These qualitative results further illustrate the effectiveness of our method.
C
Discussion
The underwater imaging model leads physics- driven research direction tackle image degra- dation. Explicit information restoration fea- ture space presents distinct advantage for NAUTILUS, especially in high-level understand- ing tasks. Specifically, image augmentation changes pixel values, resulting in compromised preservation of the image's fidelity and seman- ties. In contrast, the proposed feature enhance- ment method incorporates physical priors reg- ularize the feature extraction and interaction pro- cess, preserving the original image information and demonstrating greater application potential.
Car you give description of thi imags?
Original
image shows leged
Image Festures
ray
lying
sandy
camouiteged ogainet lighting
among
digon. The surmunndege
natural and mederahely bright, likely
uneven typical shallwr reaf
From sunlight fiitering thrsugh shallow surnonndigg include creating tectured underwater enwireement. ray's coloration allows it to blane ancriisslly with ccnon
lighting
costing greenish-Hlue
typen algen,
As shown in Fig. III, the left and right columns show original and corresponding augmented un- derwater images. Feeding them into the repre- sentative multimodal large language model GPT- 4o [40] produces markedly different feedback. In particular, while employing the data augmen- tation process, the GPT-4o misinterprets the dim and diffuse lighting as nature and moderately bright. illustrating the necessity of exploring feature enhancement in the underwater scene understanding task.
subdued light suggests photo was taken underwater, possibly at moderate deptt.
Figure III: Feeding pair of original and aug- mented underwater images into an LMM yields output descriptions with notable differences, in- dicating that data augmentation can change the semantic interpretation of the image.
24

---

Users:
Could you provide short overview of the inaget
NAUTILUS: Two scuba divers, explore on underwoter scene filled with a dense school of fish. One diver swins among the fish, while the other is positioned on the sandy ocean floor. Several larger fish swin near the surface.
Users: Detect all diver in the image Provide the categary and negression box for each detected object. NAUTILUS, ('bbox 8d%; (180, 6t9, 345, B69), "labal" "diver"?.
('box 2d : [381, 340, 767, Stll "labal: "diver")
Please give brief descriphor swimming c tight formahon.
region
1500,
underwater mage.
NAUTILUS large school
Users: Please give brief description of the diver on the right.
NAUTILUS: toubo diver swims through a school of fish while holding o camero.
Gould you provide short oversiew of the inage? NAUTILUS: Three and white clownfish reef.
Users:
the tentacies of a large onemone coral
Users: Detect all flsh the image. Provide the category
box each detected object.
602, "labal" "Fiih" MM A""bbox__dd" NAUTILUS: ("bbax__ 8d": 1008, bbbox
756, "label"
"fee'"
Users: Please provide the bounding box coordinates the region "A lange with white tentsciess base. sheltering clownfish."
NAUTILUS: ("bbox__ Rd": (86, 1400, 7793
Users: Defect all Rish the image, Provide the cntegary and regression box wath detected sbject.
NAUTILUS: ("bbox
356, 384, 659), "tabe": : 1,f" b'oobb 2d':: [603, 54, 780, 3671, "label" "fisk bbbb 2d"; [826, 334, 1094, 571). "label"
"fish"l.""boo ".""oo__o [699, $29, 787, 7671. "label":
"Rish 3,0 [691, 540, 1056, 8211, "Izbel": "fis' "dTbboo 2d"; (846, 670, 1133, 907), "label". "fish 1.0 _ebox_o__ddd______________""__"_ 0249, 616, 1499, 831), "labal": "fish" 24 bbook B4%; [1400, 637, 1670, 874], "tabel": "Rih")
Users: How many fish are visible in this image? NAUTILUS:
appeorance Fish the picture. NAUTILUS: The are siver and block stripes.
Plesse
Write short description NAUTILUS: Several small, light-colored fish with dark spots swim on sondy bottom on underwater cave.
Image.
figh
Defect fish the image. Provide the category regression
each detected object.
Vlabal Vifuhl "Efu "Elee//boo__ Vidbell": Virah "Annl 2el [309, 129, 470, 185), Viabelt": ("Fehh""AAl "boox_ ady's (aac, at,
NAUTILUS:Y,
340, 1851,
fianl "Anl "bV" [634, 105, 1971, Vlaball". Vifunn")
the hounding coondinates for "amellest fish'' NAUTILUS: (nV boobboo___ee [541, 105, 393, 162/\r)
the visiiliit lighting condition this underwaler scere?
NAUTILUS: The lighting appeors to creating somewhat murky ensiromment. fish level detail
overall
lighting condition.
Users: Detect all fish in the image. Provide the category and regression box For each detected object
NAUTILUS: ("hhox_ Re%s jo, 17, 78, 2813, "loal" " "Ran", ('boor_ [135, 160, 310, 2541, "lobel":
"fish'), ("boox_ 20% [213, 516, 466, 617)."" "hai"", ('abox 2d" [87, 342, 301, 399), "label" "fss"),
("bbox_ 20": 1242. 22, 426, 981, "iabe"" "fis"). (' ("obox_ ad": (223, 210, 436, 276). "label": "fish"..
["bbox__ 8e" [494, 429, 747, 550). "label: "Fd'"), ('abor 28%; [593, 10, 755, 155), "label": "Fat")
("abox_ 90" (698) 399, 901, 580), "lobel: "fah"), ('bhox_2d": 2d" 1698, 610, 969, 6951, "lobel": "Fish"
Users: Plaase provids the bounding box coordinates of the region "The fish in the upper right corner.' NAUTILUS: { "bbox, 25t (193, 16, 767, 183]
Describe appearance the fish
the upper right selly twims
NAUTILUS:
coral
Users:
"Describe the the distribution of the fish in the image in detail"
NAUTILUS: group of eight blue fish are swimming in loose formation in the middle of the frame,
with same pasi fioeed near the coral rasf and others swimming lowords the sandy hotlom.
Users: Plaase provde the bounding box coordinates of the negion "Large sno turtle'
NAUTILUS: ('beox_ 2d" (672, 285, 1935, 575)
Users: Write short descrpption
image.
NAUTILUS: large sso furlle swims in clear blux water, surrounded by achool ailvery
Users: Describe detai the ture, skin, and appearence this turlile
Itss shell has mottled pattern Users: Please provide a detalled description of the lighting in the image.
NAUTILUS: The turtle swimming with herd faring the of and white,
smooth and
NAUTILUS: The image is lit with natural light, likely from the surface of the waler, illurinsling the furlle and fish with a seft glow.
Figure II: Visualizations across various underwater scenes encompassing different illumination conditions, target categories, individual scales, environments, etc. Our NAUTILUS consistently presents remarkable underwater scene understanding performance.
25

---

Table VII: Prompts designed generate text annotations.
Prompt 1: Image caption
You Al visual assistant analyzing underwater image.
Task Overview:
Generate a detailed and accurate description of the image one sentence.
Image caption guidelines:
1. Ensure that **cch description is affirmative and be inferred clearly from the image** Identify the main targets such various fish, sea turtles, jellyfish and other marine organisms, shipwrecks ruins, coral reefs, seagrass, divers, etc. When describing, mention the accurate number of targets and determine their category.
3. Consider the action state of the objects, relationship hetween objects, background and environ- ment.
Prompt 2: Region caption
Given an image, bounding box (bbox), and additional textual context, generate high-quality region description. The description must adhere the following principles: Input:
bounding box: [bbox]
This hounding box represents the normalized xycoordinates of the top-left and bottom-right corners of the target region in the image.
Accuracy: Ensure the description precisely reflects the content within the specified bbox without adding speculative or unrelated details.
Specificity: Provide concrete details about the object's attributes (e.g.. shape, color, texture) and relevant contextual information.
Objectivity: Avoid any subjective interpretations, emotions, assumptions about the object's purpose intent.
Conciseness: Keep the deseription informative yet succinct, avoiding unnecessary elaboration. Context Awareness: Consider the surrounding elements only if they are relevant to understanding the object in the bbox.
Output Format:
description; A darkcolored fish with a broad body and a slightly pointed head, swimming near the coral reef.
Prompt 3: VQA
You AI visual assistant analyzing underwater image. Generate structured dialogue hetween yourself and person asking questions about the image. Your task is to create precise question-answer pairs based purely on the observable visual content of the image. Each answer should be as short as possible, preferably single word or short phrase, while maintaining accuracy. Task Overview:
Generate variety of structured question-answer pairs that reflect the image's 's content. Questions should cover different aspects of the image and fall into one of the following categories: Object Recognition Questions: Identifying or detecting object types and categories (c.g., fish species, coral structures). Attribute Questions: Describing the properties of objects (e.g., color, size, shape, material). Counting Questions: Asking about the number of specific objects (e.g., number fish coral formations). Spatial Relation Questions: Asking about relative position or spatial layout objects (c.g.. where objects are located or their relative positions ). Guidelines:
For "Spatial Relation Questions" ,do answer them by "In the ocean". Ensure that every question has definite and clear answer hased what is visually observahle in the image. Avoid speculative or amhiguous questions. Questions should be answerable with confidence and based on visible content. Include hoth simple (object identification, counting) and moderate (relative positioning, behaviors) questions.
Format:
Follow this exact format for each question-answer pair and no need to include other content:
Q: [Question]
A: [A single word or phrase]
26

---

Table VIII: A list of question templates employed construct task specific conversions.
Task I: Image caption
1. Write description for the image.
Offer a concise description the image.
3. Give a description of the scene.
4. Give short description of the image.
5. Provide brief description of the image.
Please give succinct description of what shown this image.
Could you describe image briefly?
8. Could you provide description the image?
9.
Can you give brief description of this image?
10. Could you provide short overview of the image?
Task 2: Region caption
1. Please provide concise description of this region [bbox] in this underwater image.
2. Please give brief description of the region [bbox] in this underwater image.
3. Could you provide concise description of the region [bbox] this underwater image?
Please offer succinet description of region [bbox].
5. Please provide short yet informative description of the region [bbox].
6. Describe this region [bbox] in the underwater image.
7. Briefly deseribe the content of this region [bbox].
this underwater image, please provide concise description the region [bbox].
9. For this underwater image, concisely describe the content of the region [bbox].
10. What's in this region [bhox] of the underwater image? Describe it concisely.
Task 3: Grounding
Please locate the bounding box coordinates the [region].
2. Find and return the bounding box coordinates of the [region].
3. Give the bounding box coordinates for the [region]. Extract the precise bounding box coordinates that correspond to the given [region]. Detect and outline the bounding box coordinates enclosing the [region].
4.
Task 4: Detection
1. Detect all [class] object in the image.
2.
Detect all underwater object in the image, including [class]], [class2], .... [classn].
Task 5: Counting
1. How many fish can find in this image?
2. Please count the fish in the image.
3. Identify the number of fish present in this image.
4. Count the total number of fish visible this image.
5. What is the count of fish this image?
6. How many fish can you see in this image?
7. How many fish visible this image?
Please count how many fish are this image.
9. Can you determine the number of fish in this image?
10. What is the total number of fish shown in this image?
Task 6: Coarse-grained classification
Identify the object inside the specified regression box [bhox].
2. Categorize the object located within the provided regression hox [bbox] in the underwater image.
3. Classify the items found inside the given regression box [bbox].
4. Determine the category of the object inside the specified regression box [bbox].
Assign a category to object inside the provided regression box [bbox] in image.
Task 7: Fine-grained classification
1. Please identify the biological class of fish depicted in the image.
2. Can you recognize the fish taxonomic class shown in this image? Could you determine
taxonomic class the fish the image?
4. What is the biological class of the fish in the image?
5. You are requested to identify the biological class of fish present in the image.
27


Reviews:
[{"review_id": "BzKjXK56XS", "reviewer": "Reviewer_5R6T", "summary": "This paper introduced an underwater large vision-language model, namely, NAUTILUS, alongside a new large-scale dataset, i.e., NautData, comprising 1.45 million image-text pairs covering eight underwater scene understanding tasks. It tackled the long-standing challenge of underwater image degradation by incorporating a physically grounded Vision Feature Enhancement (VFE) module based on underwater imaging principles. The authors integrated the VFE into two LMM baselines (i.e., LLaVA-1.5 and Qwen2.5-VL). The experimental results demonstrated consistent improvements over strong baselines on multiple datasets.", "questions": "1. Can the VFE module be plugged into other LMM architectures? This would clarify its universality.\n2. What is the runtime overhead introduced by the VFE module?\n3. Will the trained checkpoints and data annotation tools be released on the acceptance of the paper? This is of great significance for the community’s reproduction and further development.\n4. How does NAUTILUS handle domain shifts between shallow and deep-sea environments? Have the authors tested the model across varying light/color conditions?\nPositive responses or additional evidence addressing the first three questions would increase my confidence in the model’s robustness and practical applicability, and potentially raise my score.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The VFE module was well-motivated by the physics-based imaging theory and effectively improved robustness under underwater image degradation.\n2. The paper was clearly written and well-organized. Figures effectively supported the technical content.\n3. This work made a high-impact contribution to a critical domain—underwater vision-language modeling. Both NAUTILUS and NautData could form a strong foundation for future research and applications in marine robotics, ocean monitoring, and scientific exploration, given that the authors would make them publicly available.\n\nWeaknesses:\n1. Although zero-shot evaluations were provided on MarineInst20M, more cross-domain generalization tests would strengthen the claims.\n2. The VFE module involved depth extraction and attention mechanisms. A runtime/memory cost analysis was missing. This is important for practical deployment on underwater devices.", "was_revised": true, "final_justification": "I think that the authors have responded to most of my comments. Thanks. I will maintain my scores.", "rebuttal": "Thanks for providing feedback and taking the time to review our work! We promise that the training/inference codes, checkpoints, and the entire NautData, alongside the data annotation tools, will be released.\n\n**Weaknesses:**\n\n- **To Weakness 1 & Question 4**: “Although zero-shot evaluations were provided on MarineInst20M, more cross-domain generalization tests would strengthen the claims.” / “How does NAUTILUS handle domain shifts between shallow and deep-sea environments? Have the authors tested the model across varying light/color conditions?”\n\n**Reply:** Thanks! Following your advice, we additionally provide zero-shot evaluations on UIIS10K [1] and TrashCan [2]  datasets, which are **never involved in our training data**, to test the cross-domain capabilities of our models. Specifically, both of them only provide bounding box annotations. We randomly sample 200 images from each of these two datasets and employ the Gemini 2.5 Flash model to generate descriptions of each instance. Subsequently, our team manually verifies each description to ensure its accuracy. We train models on our NautData and directly evaluate their zero-shot grounding performance on the sampled data. The results demonstrate that the NAUTILUS surpasses other SOTA baselines by a notable margin, presenting the effectiveness of our methods.\n\n| Method | UIIS10K [1] | TrashCan [2] |\n| --- | --- | --- |\n| MiniGPTv2 | 43.5 | 19.5 |\n| mPLUG-Owl3 | 53.5 | 31.5 |\n| InternVL-2.5 | 56.0 | 41.7 |\n| LLaVA-1.5 | 51.5 | 24.5 |\n| **NAUTILUS**(LLaVA-1.5) | 53.0(+1.5) | 29.0(+4.5) |\n| Qwen2.5-VL | 62.5 | 47.5 |\n| **NAUTILUS**(Qwen2.5-VL) | 65.5(+3.0) | 49.0(+1.5) |\n\nShallow and deep-sea environments can often be distinguished by low-light and normal-light conditions. We also test our models across varying light and color conditions to evaluate the ability of our NAUTILUS to handle domain shifts. Specifically, we employ Gemini 2.5 Flash to filter for low-light,  normal-light, green-color, and blue-color images from the NautData test set. Then, we separately evaluate performance on each of these subsets. It is worth noting that our method presents +5.2 and +4.2 PR\\@0.5 improvements while suffering significant image degradations in low-light and green-color scenarios, respectively, demonstrating the remarkable robustness and practical applicability of our methods. We will add these experiments to our revised version.\n\n| **Method** | **Low-light** | **Normal-light** | **Green-color** | **Blue-color** |\n| --- | --- | --- | --- | --- |\n| MiniGPTv2 | 26.6 | 42.0 | 20.1 | 39.2 |\n| mPLUG-Owl3 | 28.3 | 35.9 | 32.1 | 35.1 |\n| InternVL-2.5 | 29.1 | 42.6 | 36.1 | 40.7 |\n| LLaVA-1.5 | 26.2 | 35.7 | 28.4 | 35.4 |\n| **NAUTILUS**(LLaVA-1.5) | 27.7(+1.5) | 37.6(+1.9) | 33.2(+4.8) | 36.4(+1.0) |\n| Qwen2.5-VL | 30.6 | 43.6 | 36.6 | 42.5 |\n| **NAUTILUS**(Qwen2.5-VL) | 35.8(+5.2) | 44.4(+0.8) | 40.8(+4.2) | 42.9(+0.4) |\n\n[1] UWSAM: Segment Anything Model Guided Underwater Instance Segmentation and A Large-scale Benchmark Dataset. arXiv 25.\n\n[2] TrashCan: A Semantically-Segmented Dataset towards Visual Detection of Marine Debris. arXiv 20.\n\n- **To Weakness 2 & Question 2**: “The VFE module involved depth extraction and attention mechanisms. A runtime/memory cost analysis was missing. This is important for practical deployment on underwater devices.” / “What is the runtime overhead introduced by the VFE module?”\n    \n    \n\n**Reply:**  Thanks. We analyze the runtime and memory cost of our NAUTILUS on a single NVIDIA RTX 4090 GPU. Specifically, we fix the resolution of an input image as 336$\\times$336 and observe that the runtime of the depth encoder and VFE module are 17.48 ms and 22.71 ms, respectively, each consuming 1.07 M and 1.31 M GPU Memory. We will add the runtime and memory cost analysis in the revised version.\n\n| Component | Runtime (ms) | GPU Memory (M) |\n| --- | --- | --- |\n| Depth encoder | 17.48 | 1.07 |\n| VFE module | 22.71 | 1.31 |\n\n\n- **To Question 1**: “Can the VFE module be plugged into other LMM architectures? This would clarify its universality.”\n    \n    \n\n**Reply:** Thanks! We have plugged the VFE module into Qwen2.5-VL and LLaVA-1.5, demonstrating effectiveness based on these two LMM architectures. To evaluate its universality more comprehensively, we further plug the VFE module into InternVL-2.5 [1]. In particular, we conduct training on a third of the NautData dataset and evaluations on eight tasks to enable an extensive comparison. As shown in the following table, the VFE module brings 2.0 acc, 3.9 MAE, and 2.1 AP\\@0.5 improvements on the fine-grained classification, counting, and detection tasks, respectively, presenting a remarkable universality of this method. We will add experiments and discussions in the revised version.\n\n| Method | Coarse-grained Classification | Fine-grained Classification | Image Caption | Region Caption | Counting | Grounding | Detection | VQA |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | acc ↑ | acc ↑ | meteor ↑ | meteor ↑ | MAE ↓ | PR\\@0.5 ↑ | AP\\@0.5 ↑ | meteor ↑ |\n| InternVL-2.5 | 91.3 | 85.7 | 0.206 | 0.107 | 42.1 | 37.4 | 24.6 | 0.350 |\n| **NAUTILUS**(InternVL-2.5) | 92.0(+0.7) | 87.7(+2.0) | 0.206(+0.0) | 0.108(+0.001) | 38.2(+3.9) | 38.4(+1.0) | 26.7(+2.1) | 0.353(+0.003) |\n\n[1] Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling. arXiv 24.\n\n- **To Question 3**: “Will the trained checkpoints and data annotation tools be released on the acceptance of the paper? This is of great significance for the community’s reproduction and further development.”\n\n**Reply:** Thanks! We promise to release the training/inference codes, trained checkpoints, and data annotation tools, facilitating further development of this community."}, {"review_id": "f5TgOWOGaU", "reviewer": "Reviewer_R364", "summary": "This study addresses the challenges of data scarcity and poor image quality in underwater scene understanding by proposing a large vision-language model named NAUTILUS. To achieve this, the researchers first constructed a large-scale multi-task instruction dataset for underwater environments, NautData, containing 1.45 million image-text pairs. They then designed an innovative plug-and-play Visual Feature Enhancement (VFE) module, which leverages the physical principles of underwater imaging to effectively mitigate image degradation.", "questions": "Could the authors analyze the noise levels and perform error analysis?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The researchers developed NautData, a large-scale underwater instruction learning dataset comprising 1.45 million question-answer pairs. This dataset supports eight different tasks.\n- The study introduces a plug-and-play Visual Feature Enhancement (VFE) module inspired by the physical principles of underwater imaging.\n- NAUTILUS is the first large vision-language model capable of handling eight underwater tasks. \n- Experiments show that NAUTILUS outperforms models including GPT-4o and Gemini 2.0 on most tasks in the NautData test set.\n\nWeaknesses:\n- The paper does not analyze the noise level within the LLM-generated portion of the NautData dataset, nor does it discuss how this potential noise might affect the model's training and final performance.\n- The research lacks a qualitative analysis of failure cases. Discussing specific examples where the model fails would offer valuable insights into its limitations and areas for future improvement.\n- The reported performance improvements over the LLaVA-1.5 baseline are quite marginal on several tasks. This limited gain could suggest that the proposed method's effectiveness and generalization capabilities may not be as significant.\n- The study would be strengthened by an ablation that compares this method to other, more naive weighting strategies to justify its design choice.", "was_revised": true, "final_justification": "The paper proposed the large vision-language model for underwater tasks, trained on the a large scale dataset and enhanced by a physics-inspired VFE module, achieving superior performance over GPT-4o and Gemini 2.0 across most tasks. Most of my concerns have been addressed during the rebuttal phase thus I choose to keep my rating.", "rebuttal": "Thank you for your valuable feedback!\n\n- **To Weakness 1 & Question 1**: “The paper does not analyze the noise level within the LLM-generated portion of the NautData dataset, nor does it discuss how this potential noise might affect the model's training and final performance.” / “Could the authors analyze the noise levels and perform error analysis?”\n\n**Reply:** Thanks. Hallucinations in LLMs are a fundamental issue that is difficult to avoid entirely [1][2][3], introducing noises into the construction of large-scale instruction-tuning datasets. The demand for data scaling makes it impractical for us to assess the quality of each image-text pair by humans. Thus, to evaluate the quality of our NautData, we randomly select two hundred image-text samples and recruit four volunteers at the master’s level to give their scores on the quality of each sample. \n\nSpecifically, we adopt a ten-point scoring scale. For example, score 10 denotes “The description is exhaustive, internally consistent, and entirely accurate; no factual, logical, or interpretive errors are observed”, score 8 denotes “Generally accurate, but contains several fine-grained errors that exert limited impact on core understanding”, score 3 denotes “Poor: Predominantly inaccurate; only a few statements are correct, and the overall presentation is misleading”, and score 1 denotes “The content is entirely wrong, irrelevant, or contradicts established knowledge; no redeeming factual accuracy is present”. Fine-grained errors like misinterpreting “clownfish” as “acrossocheilus” demand a score of eight, while misinterpreting “fish” as “crab” will just obtain a score of six.\n\nThe final average score of our dataset is 8.06, illustrating an overall high quality of the NautData with tolerable hallucinations. We attribute this final assessment of scoring 8.06 to two reasons: 1) four of the eight supported tasks, including the coarse-grained, fine-grained, grounding, and detection tasks, are manually annotated and thus free of hallucinatory noise, and 2) we integrate LMM filtering followed by manual correction, which is a practical approach to minimize dataset noise.\n\nBesides, training our models on the 1/3 portion and the entire portion of the NautData, we observe an obvious scaling law of our dataset for 1.5 acc, 1.8 acc, 3.2 MAE, 2.0 PR\\@0.5, and 2.8 AP\\@0.5 improvements on the coarse-grained, fine-grained, counting, grounding, and detection tasks, respectively. This finding demonstrates that the advantages of data scaling still outweigh the detrimental effects of the noise present in our dataset. We will add these analyses of data quality in the revised version.\n\n| Training data | Coarse-grainedClassification | Fine-grainedClassification | Counting | Grounding | Detection |\n| --- | --- | --- | --- | --- | --- |\n|  | acc ↑ | acc ↑ | MAE ↓ | PR\\@0.5 ↑ | AP\\@0.5 ↑ |\n| 1/3 of NautData | 93.1 | 92.3 | 35.3 | 40.3 | 44.8 |\n| Entire NautData | 94.6(+1.5) | 94.1(+1.8) | 32.1(+3.2) | 42.3(+2.0) | 47.6(+2.8) |\n\n[1] Hallucination Augmented Contrastive Learning for Multimodal Large Language Model. CVPR 24.\n\n[2] LLM-Check: Investigating Detection of Hallucinations in Large Language Models. NeurIPS 24.\n\n[3] A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM TIS 25.\n\n- **To Weakness 2**: “The research lacks a qualitative analysis of failure cases. Discussing specific examples where the model fails would offer valuable insights into its limitations and areas for future improvement.”\n\n**Reply:** Good suggestion! We conduct qualitative analysis on the NautData test set and observe failure cases in extremely dense scenes, i.e., the counting error could increase to ten percent while thousands of fish swim in groups. Another cases indicate that while instances of the same fine-grained category occur in adjacent clusters, grounding errors occasionally arise. We will add these analyses with corresponding qualitative results in the revised version.\n\n- **To Weakness 3**: “The reported performance improvements over the LLaVA-1.5 baseline are quite marginal on several tasks. This limited gain could suggest that the proposed method's effectiveness and generalization capabilities may not be as significant.”\n\n**Reply:** Thanks. We provide exhaustive evaluations on this phenomenon, analyzing the factors contributing to LLaVA-1.5’s performance improvements. Specifically, we divide the NautData test set into subsets characterized by low-light, normal-light, green-color, and blue-color conditions. Subsequently, we separately evaluate our models on these subsets and observe a notable improvement of +4.8 PR\\@0.5 under the green-color condition, presenting a remarkable effectiveness for handling challenging underwater degradations. Therefore, we will provide fine-grained subsets of our NautData to better illustrate the advantages of our methods.\n\n| **Method** | **Low-light** | **Normal-light** | **Green-color** | **Blue-color** |\n| --- | --- | --- | --- | --- |\n| MiniGPTv2 | 26.6 | 42.0 | 20.1 | 39.2 |\n| mPLUG-Owl3 | 28.3 | 35.9 | 32.1 | 35.1 |\n| InternVL-2.5 | 29.1 | 42.6 | 36.1 | 40.7 |\n| LLaVA-1.5 | 26.2 | 35.7 | 28.4 | 35.4 |\n| **NAUTILUS**(LLaVA-1.5) | 27.7(+1.5) | 37.6(+1.9) | 33.2(+4.8) | 36.4(+1.0) |\n| Qwen2.5-VL | 30.6 | 43.6 | 36.6 | 42.5 |\n| **NAUTILUS**(Qwen2.5-VL) | 35.8(+5.2) | 44.4(+0.8) | 40.8(+4.2) | 42.9(+0.4) |\n\nBesides, to evaluate the generalization performance of our methods, we also conduct experiments on InternVL-2.5 and Qwen2.5-VL. As shown in the following table, the VFE module brings consistent improvements on almost all tasks, demonstrating remarkable generalization capabilities.\n\n| Method | Coarse-grainedClassification | Fine-grainedClassification | Image Caption | Region Caption | Counting | Grounding | Detection | VQA |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | acc ↑ | acc ↑ | meteor ↑ | meteor ↑ | MAE ↓ | PR\\@0.5 ↑ | AP\\@0.5 ↑ | meteor ↑ |\n| InternVL-2.5 | 91.3 | 85.7 | 0.206 | 0.107 | 42.1 | 37.4 | 24.6 | 0.350 |\n| **NAUTILUS**(InternVL-2.5) | 92.0(+0.7) | 87.7(+2.0) | 0.206(+0.0) | 0.108(+0.001) | 38.2(+3.9) | 38.4(+1.0) | 26.7(+2.1) | 0.353(+0.003) |\n| Qwen2.5-VL | 89.1 | 92.1 | 0.220 | 0.189 | 40.7 | 37.6 | 41.9 | 0.357 |\n| **NAUTILUS**(Qwen2.5-VL) | 93.1(+4.0) | 92.3(+0.2) | 0.221(+0.001) | 0.189(+0.0) | 35.3(+5.4) | 40.3(+2.7) | 44.8(+2.9) | 0.354(-0.003) |\n\n- **To Weakness 4**: “The study would be strengthened by an ablation that compares this method to other, more naive weighting strategies to justify its design choice.”\n\n**Reply:** Good suggestion! Following your advice, we conduct ablation studies on the weighting strategies, with results presented in the following table. In particular, we explore four designs of weighting strategies, including “wo/ weighting”, “a norm weight e”, “Learned from image feature”, and “Learned from depth feature (ours)”. Among them, the “wo/ weighting” strategy can be considered as “a norm weight 1”. It achieves similar performance with the “a norm weight e” strategy, slightly inferior to other learning-based strategies. The “Learned from image feature” strategy, which means a loss of depth information, presents comparable performance compared with our design in the image caption, region caption, and VQA tasks. However, depth information is essential for distance perceptions of complex underwater objects, benefiting underwater scene understanding intuitively. Our strategy with the fusion of depth features surpasses other strategies, indicating the effectiveness of our design choice. We will add this ablation study in the revised version.\n\n| Weighting Strategy | Coarse-grained Classification | Fine-grained Classification | Image Caption | Region Caption | Counting | Grounding | Detection | VQA |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | acc ↑ | acc ↑ | meteor ↑ | meteor ↑ | MAE ↓ | PR\\@0.5 ↑ | AP\\@0.5 ↑ | meteor ↑ |\n| wo/ weighting | 88.3 | 88.5 | 0.213 | 0.181 | 49.8 | 36.8 | 37.1 | 0.341 |\n| a norm weight $e$  | 88.4 | 87.6 | 0.211 | 0.185 | 50.0 | 37.0 | 36.0 | 0.346 |\n| Learned from image feature | 90.7 | 91.8 | **0.221** | 0.188 | 35.9 | 37.5 | 40.6 | **0.364** |\n| Learned from depth feature (ours) | **93.1** | **92.3** | **0.221** | **0.189** | **35.3** | **40.3** | **44.8** | 0.354 |"}, {"review_id": "VDuQOmFG1p", "reviewer": "Reviewer_qKHH", "summary": "This work makes two primary contributions to underwater scene understanding:\n\n(1) the introduction of NautData, a novel large-scale multimodal dataset encompassing 1.45M image-text pairs across eight tasks spanning image-, region-, and object-level understanding; and\n\n(2) NAUTILUS, a new underwater-adapted LMM that integrates a physics-inspired vision feature enhancement (VFE) module to\nexplicitly mitigate degradation effects in underwater imagery.\n\nThe proposed VFE module demonstrates strong plug-and-play compatibility when incorporated into established LMMs (LLaVA-1.5, Qwen-VL), with experiments showing consistent performance gains across multiple underwater understanding tasks.\nBy addressing both the data scarcity challenge through NautData and the degradation robustness challenge through the VFE design,\nthis work provides a valuable foundation for future research in underwater multimodal perception systems.", "questions": "(1) What motivated the choice of feature-space enhancement over traditional image-space approaches?\nAre there specific underwater degradation characteristics that make this more effective?\n\n(2) Could you provide either: A more rigorous theoretical proof connecting VFE's operations to underwater imaging models,\nor visualizations showing how VFE module removes underwater degradation?\n\n(3) Could you add experiments isolating the contribution of the dark pixel prior in VFE and performance comparison between feature-space and image-space enhancement?\n\n(4) In the 4th row of Supplementary Figure II, NAUTILUS incorrectly localizes the leftmost cuttlefish. Could you explain why?\n\nMy evaluation scores could be positively revised should the authors substantively address Q1-Q3 with new analyses and discuss the Figure II anomaly's implications.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n(1) The construction of NautData represents a significant contribution, being the first large-scale dataset to\ncomprehensively cover eight underwater understanding tasks across image-, region-, and object-level granularities.\n\n(2) The proposed NAUTILUS model demonstrates consistent and measurable improvements across multiple underwater tasks when\nbenchmarked against established baselines.\n\n(3) The paper is well-written with clear organization and minimal grammatical errors, facilitating efficient comprehension of the technical content.\n\n\nWeaknesses:\n\n(1) This paper lacks explanation for performing enhancement in feature space rather than image space\n\n(2) This paper lacks rigorous mathematical proof establishing the connection between the VFE module's design principles and classical underwater imaging physics\n\n(3) Critical components of the VFE design lack empirical validation, including the impact of feature-space vs. pixel-space enhancement and the choise of the dark pixel prior.", "was_revised": true, "final_justification": "Most of my concerns have been addressed in the rebuttal and discussions. The additional experiments provided by the authors during the rebuttal phase effectively address the limitations in the paper. Incorporating these experimental results and other visualizations in the final version would significantly enhance the completeness and effectiveness of the article. I am upgrading my score to 4 (borderline accept).", "rebuttal": "Thanks for providing feedback and taking the time to review our work!\n\n- **To Weakness 1 & Question 1**: “… lacks explanation for performing enhancement in feature space rather than image space” / “What motivated the choice of feature-space enhancement over traditional image-space approaches? Are there specific underwater degradation characteristics that make this more effective?”\n\n**Reply:** Thanks. There are three observations motivating us to explore feature-space enhancement over image-space approaches: \n\n1) Traditional image restoration methods align with human visual quality preferences. However, “the factors contributing to **visual quality often differ from those determining recognition quality**” **[1][2]**, while the high recognition quality is our purpose.\n\n2) Image-space restoration methods directly **alter the raw signal**, hurting the original image information. Instead, the feature-space enhancement completely preserves the input of original pixel values, enhancing underwater information more softly.\n\n3) Decoupling image enhancement from model training can accumulate errors. This **risk is magnified in our large-scale underwater dataset** captured under highly variable conditions. Enhancement in feature space enables end-to-end optimization, allowing task-oriented supervision from downstream tasks, which could be more suitable for our eight-task understanding models.\n\nWe pursue performance improvement on underwater understanding rather than prioritizing human-oriented visual quality, and therefore, we investigate feature-space enhancement approaches. Besides, we provide empirical validations to evaluate the effectiveness of feature-space enhancement. As shown in the following table, our method improves the counting, grounding, and detection performance by at least 2.7 points, demonstrating a remarkable potential to explore a feature-space enhancement approach. We will add these discussions to the revised version.\n\n| Restoration Methods | Coarse-grained Classification | Fine-grained Classification | Image Caption | Region Caption | Counting | Grounding | Detection | VQA |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | acc ↑ | acc ↑ | meteor ↑ | meteor ↑ | MAE ↓ | PR\\@0.5 ↑ | AP\\@0.5 ↑ | meteor ↑ |\n| Qwen2.5-VL(Baseline) | 89.1 | 92.1 | 0.220 | **0.189** | 40.7 | 37.6 | 41.9 | **0.357** |\n| +Reti-Diff [3] | 92.3 | 91.7 | 0.217 | **0.189** | **33.7** | 38.1 | 36.3 | 0.355 |\n| +SMDR-IS [4] | 88.3 | 85.8 | 0.214 | 0.182 | 38.9 | 39.9 | 32.4 | 0.352 |\n| +CCL-Net [5] | 89.7 | 88.6 | 0.217 | 0.183 | 45.7 | 36.9 | 38.5 | 0.353 |\n| +VFE (ours) | **93.1** | **92.3** | **0.221** | **0.189** | 35.3 | **40.3** | **44.8** | 0.354 |\n\n[1] UniRestore: Unified Perceptual and Task-Oriented Image Restoration Model Using Diffusion Prior. CVPR 25.\n\n[2] Visual Recognition-Driven Image Restoration for Multiple Degradation with Intrinsic Semantics Recovery. CVPR 23.\n\n[3] Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model. ICLR 25.\n\n[4] Synergistic multiscale detail refinement via intrinsic supervision for underwater image enhancement. AAAI 24.\n\n[5] Underwater Image Enhancement With Cascaded Contrastive Learning. TMM 24.\n\n- **To Weakness 2 & Question 2**: “...establishing the connection between the VFE module's design principles…”/ “… visualizations showing how VFE module removes underwater degradation?”\n\n**Reply:** Thanks for this insightful comment! The VFE module consists of two components motivated by the physical underwater imaging model: 1) employing dark pixels to localize the backscattering response, and 2) introducing depth features to estimate the restoration weight, alleviating information degradation during underwater propagation. Method in [1] provides strategies to estimate backscattering response and restoration weight within the physical underwater imaging model, and achieves underwater image restoration. Taking inspiration from this method, we revive its idea in feature space, first estimating representations of unavailable items in the physical underwater imaging model and subsequently substituting them into the equation of the underwater imaging physics to obtain an enhanced vision feature.\n\nBesides, following your advice, to intuitively illustrate the effectiveness of our VFE module, we provide visualizations of the original and enhanced vision features. Due to the policy restrictions during the NeurIPS rebuttal phase, we are unable to directly present figures. Thus, we upload these visualizations as Fig.1 **via the anonymous link embedded in the supplementary file** (this does not violate the rebuttal requirements).\n\nWe can see that the original feature struggles to focus on the targets of interest while the enhanced features clearly find them with high responses, demonstrating the remarkable performance of the VFE model in understanding underwater scenes. We will meticulously refine the relevant discussions and analysis in the revised version.\n\n[1] What is the Space of Attenuation Coefficients in Underwater Computer Vision? CVPR 17.\n\n[2] UWSAM: Segment Anything Model Guided Underwater Instance Segmentation and A Large-scale Benchmark Dataset. arXiv 25.\n\n- **To Weakness 3 & Question 3**: “Critical components of the VFE design lack empirical validation, including the impact of feature-space vs. pixel-space enhancement and the choise of the dark pixel prior.” / “Could you add experiments isolating the contribution of the dark pixel prior in VFE and performance comparison between feature-space and image-space enhancement?”\n\n**Reply:** Thanks. Following your advice, we conduct empirical validation to evaluate the critical components of the VFE design. First, we compare the effectiveness of feature-space and pixel-space enhancements. As shown in the table below, SMDR-IS and CCL-Net introduce a performance decline on most tasks. Reti-Diff suffers a 5.6 AP\\@0.5 performance drop on the detection task. We attribute this to challenges from complex underwater scenes and information loss during pixel-space enhancement. In contrast, our VFE module preserves the original information, achieving 4.0 acc, 5.4 MAE, 2.7 PR\\@0.5, and 2.9 AP\\@0.5 improvements on the coarse-grained classification, counting, grounding, and detection tasks.\n\n| Restoration Methods | Coarse-grained Classification | Fine-grained Classification | Image Caption | Region Caption | Counting | Grounding | Detection | VQA |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | acc ↑ | acc ↑ | meteor ↑ | meteor ↑ | MAE ↓ | PR\\@0.5 ↑ | AP\\@0.5 ↑ | meteor ↑ |\n| Qwen2.5-VL(Baseline) | 89.1 | 92.1 | 0.220 | 0.189 | 40.7 | 37.6 | 41.9 | 0.357 |\n| +Reti-Diff [1] | 92.3 | 91.7 | 0.217 | 0.189 | 33.7 | 38.1 | 36.3 | 0.355 |\n| +SMDR-IS [2] | 88.3 | 85.8 | 0.214 | 0.182 | 38.9 | 39.9 | 32.4 | 0.352 |\n| +CCL-Net [3] | 89.7 | 88.6 | 0.217 | 0.183 | 45.7 | 36.9 | 38.5 | 0.353 |\n| +VFE (ours) | 93.1 | 92.3 | 0.221 | 0.189 | 35.3 | 40.3 | 44.8 | 0.354 |\n\nSubsequently, we evaluate the choice and contribution of the dark pixel prior. In particular, treating the design of “wo/ dark prior” as the baseline, randomly sampled dark pixels may misinterpret clear responses as backscattering, destroying original semantic information, and suffering a significant performance decline of 12.1 AP\\@0.5 on the detection task. Afterwards, we design a strategy to sample the image patch containing the most pixels with the minimum RGB values. However, we also observe a decline in performance on six of eight takes, which we attribute to the noises involved in underwater environments containing pixels with meaningless artifacts. Therefore, we compute mean RGB values to mitigate the interference of noise and achieve remarkable improvements, illustrating the contribution of the dark pixel prior in our method. We will add these validations and discussions in the revised version.\n\n| Choice of dark prior | Coarse-grained Classification | Fine-grained Classification | Image Caption | Region Caption | Counting | Grounding | Detection | VQA |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | acc ↑ | acc ↑ | meteor ↑ | meteor ↑ | MAE ↓ | PR\\@0.5 ↑ | AP\\@0.5 ↑ | meteor ↑ |\n| wo/ dark prior | 92.7 | 91.3 | 0.217 | 0.191 | 36.5 | 38.5 | 43.9 | 0.356 |\n| random pixel | 87.7 | 86.2 | 0.201 | 0.185 | 51.4 | 36.4 | 31.8 | 0.330 |\n| containing most pixels with minimum RGB value | 89.6 | 88.5 | 0.209 | 0.192 | 45.6 | 37.6 | 38.2 | 0.341 |\n| minimum of mean RGB value (ours) | 93.1 | 92.3 | 0.221 | 0.189 | 35.3 | 40.3 | 44.8 | 0.354 |\n\n[1] Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model. ICLR 25.\n\n[2] Synergistic multiscale detail refinement via intrinsic supervision for underwater image enhancement. AAAI 24.\n\n[3] Underwater Image Enhancement With Cascaded Contrastive Learning. TMM 24.\n\n- **To Question 4**: “ In the 4th row of Supplementary Figure II, NAUTILUS incorrectly localizes the leftmost cuttlefish. Could you explain why?”\n\n**Reply:** Thank you for carefully reading this paper. Although our NAUTILUS exhibits remarkable underwater scene understanding performance, there are still challenging cases that occasionally raise errors. The 4th row of Supplementary Figure II is a failure case for the grounding task. It is challenging for a group of fish with the same fine-grained categories represented in clusters, requiring the method to distinguish the detailed instance appearance from the image modality and the delicate position descriptions from the text modality. We study to address these failure cases in future work.\n\n- My evaluation scores could be positively revised should the authors substantively address Q1-Q3 with new analyses and discuss the Figure II anomaly's implications.\n\n**Reply:** Thanks for your insightful comments. We hope we have already addressed your concerns. If you have any further questions, please feel free to let us know."}, {"review_id": "KeyPaJ2UbG", "reviewer": "Reviewer_Aru7", "summary": "This paper presents NAUTILUS, a large vision-language model tailored for underwater scene understanding. To tackle the challenges of underwater image degradation and the lack of diverse datasets, the authors introduce NautData—a new instruction-tuning dataset with 1.45 million image-text pairs spanning eight tasks across image, region, and object levels. They also propose a plug-and-play Vision Feature Enhancement (VFE) module based on physical underwater imaging principles, which improves feature quality by correcting for light scattering and absorption. Integrated into strong LMM backbones like LLaVA-1.5 and Qwen2.5-VL, NAUTILUS achieves state-of-the-art performance across a range of benchmarks, demonstrating robust and comprehensive underwater perception capabilities.", "questions": "- While the model shows strong performance on NautData and select public benchmarks, it’s unclear how well NAUTILUS generalizes to truly unseen underwater environments or rare object categories not covered in the training data. Have the authors considered evaluating on additional real-world or long-tail settings to test robustness? Results or discussion in this direction would strengthen the claims on general applicability.\n- Since much of NautData is constructed using LMMs like Gemini, Qwen, and GPT-4o, it raises concerns about annotation accuracy—especially given that Table 2 shows GPT-4o itself performs poorly on the benchmark. Could the authors clarify how they ensured label correctness during dataset construction? Additionally, quantitative measures of annotation quality (e.g., human agreement rates or error analyses) would help justify the dataset’s reliability as a benchmark.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. The paper introduces a high-quality dataset, NautData, which is large, diverse, and well-annotated across eight underwater tasks. This is a meaningful contribution to a relatively underexplored domain.\n2. The Vision Feature Enhancement (VFE) module leverages physical priors to address underwater image degradation at the feature level. \n3. The paper is well-written and easy to follow.\n\n---\n\n**Weaknesses**\n\n- While the proposed VFE module is grounded in a physical imaging model, the paper does not clearly define the applicability limits of the model, such as how well it generalizes under extreme lighting or turbidity conditions. The robustness of the dark-pixel prior is also not thoroughly analyzed—its reliability may vary depending on scene composition and noise.\n- In addition, the scope of contributions is relatively narrow, focusing primarily on the construction of a dataset and the design of the VFE module. While both are valuable, the paper would benefit from broader methodological innovations or deeper analysis of the interplay between physical priors and large-scale vision-language models.", "was_revised": true, "final_justification": "Most of my concerns have been addressed in the rebuttal and discussions. I am satisfied with the technical quality and clarity of the work, and will maintain my current score. My only remaining concern is the limited scope of contributions, which may affect the broader impact of the paper. I remain open to either outcome depending on how this aspect is weighed by the AC and other reviewers.", "rebuttal": "Thanks for providing feedback and taking the time to review our work!\n\n- **To Weakness 1**: “While the proposed VFE module is grounded in a physical imaging model, the paper does not clearly define the applicability limits of the model, such as how well it generalizes under extreme lighting or turbidity conditions. The robustness of the dark-pixel prior is also not thoroughly analyzed—its reliability may vary depending on scene composition and noise.”\n\n**Reply:** Thanks. Following your advice, we evaluate the performance of our models under various lighting and turbidity conditions. Specifically, we employ Gemini 2.5 Flash to filter for images of low-light, normal-light, turbid, and clear from the NautData test set. The low-light and turbid images suffer from underwater degradations with extreme lighting and turbidity conditions, respectively. We conduct comparisons separately on the low-light, normal-light, turbid, and clear subsets, with results presented in the table below. It is worth noting that the VFE module brings remarkable performance gains, especially under the low-light and turbid conditions for up to +5.2 and +2.8 PR\\@0.5 improvements, respectively, demonstrating the great generalization capabilities of the VFE module.\n\n| Method | Low-light | Normal-light | Turbid | Clear |\n| --- | --- | --- | --- | --- |\n| MiniGPTv2 | 26.6 | 42.0 | 34.7 | 41.1 |\n| mPLUG-Owl3 | 28.3 | 35.9 | 27.3 | 38.8 |\n| InternVL-2.5 | 29.1 | 42.6 | 36.7 | 40.9 |\n| LLaVA-1.5 | 26.2 | 35.7 | 28.8 | 36.8 |\n| **NAUTILUS**(LLaVA-1.5) | 27.7(+1.5) | 37.6(+1.9) | 31.6(+2.8) | 37.9(+1.1) |\n| Qwen2.5-VL | 30.6 | 43.6 | 38.0 | 42.3 |\n| **NAUTILUS**(Qwen2.5-VL) | 35.8(+5.2) | 44.4(+0.8) | 39.6(+1.6) | 44.0(+1.7) |\n\nSubsequently, we evaluate the robustness of the dark-pixel prior. As shown in the following table, randomly selected dark pixels lead to a significant performance decline of up to 16.1 MAE and 13.0 AP\\@0.5 compared with adopting the dark-pixel prior, illustrating the effectiveness of the dark-pixel prior in most scenarios. We will add analysis and discussions in the revised version.\n\n| Strategy of sampling dark pixel | Coarse-grained Classification | Fine-grained Classification | Image Caption | Region Caption | Counting | Grounding | Detection | VQA |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  | acc ↑ | acc ↑ | meteor ↑ | meteor ↑ | MAE ↓ | PR\\@0.5 ↑ | AP\\@0.5 ↑ | meteor ↑ |\n| random pixel | 87.7 | 86.2 | 0.201 | 0.185 | 51.4 | 36.4 | 31.8 | 0.330 |\n| dark-pixel prior | 93.1 | 92.3 | 0.221 | 0.189 | 35.3 | 40.3 | 44.8 | 0.354 |\n\n- **To Weakness 2**: “In addition, the scope of contributions is relatively narrow, focusing primarily on the construction of a dataset and the design of the VFE module. While both are valuable, the paper would benefit from broader methodological innovations or deeper analysis of the interplay between physical priors and large-scale vision-language models.”\n\n**Reply:** Thanks for admiring the value of our work! Underwater understanding enables automated explorations into the most extensive scope of our planet, facilitating applications in resource exploration, robot navigation, etc. However, the absence of multi-granular underwater LMMs hinders the deep development of this area. We take a pioneering step towards underwater LMMs from dataset construction and model design aspects, contributions we believe are valuable foundations for subsequent research.\n\nPhysical priors present an interpretable way to construct LMMs. In particular, the underwater imaging model inspires us to take advantage of depth modality while overcoming the underwater degradation. Specifically, the underwater imaging model highlights light absorption during underwater propagation, challenging the quality of underwater images, especially the targets far away from the cameras. Thus, we integrate depth encoder in the design of our VFE module, regularizing feature fusion of depth and vision modalities in a reasonable framework.\n\nAdditionally, we provide visualizations to give an intuitive impression of the VFE module’s effectiveness. Due to the policy restrictions during the NeurIPS rebuttal phase, we cannot directly present figures. Thus, we upload these visualizations as Fig.1 via the anonymous link embedded in the supplementary file (this does not violate the rebuttal requirements). We sincerely request that the reviewer click this anonymous link for more details. The visualizations demonstrate that the original vision features struggle with focusing on objects of interest, while the enhanced features present high responses to them, illustrating remarkable potential to achieve reliable underwater scene understanding. We will add these discussions to the revised version.\n\n\n- **To Question 1**: “While the model shows strong performance on NautData and select public benchmarks, it’s unclear how well NAUTILUS generalizes to truly unseen underwater environments or rare object categories not covered in the training data. Have the authors considered evaluating on additional real-world or long-tail settings to test robustness? Results or discussion in this direction would strengthen the claims on general applicability.”\n\n**Reply:** Thanks. We have evaluated zero-shot performance on the Marine-Inst20M [3], an unseen real-world dataset. Our NAUTILUS surpasses other methods on this dataset, presenting a strong generalization performance. Following your advice, we add zero-shot evaluations on UIIS10K [1] and TrashCan [2], two additional datasets not involved in the training data. In particular, the UIIS10K covers rare categories such as “Artiodactyla” and “Mollusk,” and the TrashCan is captured in entirely unseen underwater environments. Both of them are real-world datasets and provide bounding box annotations.\nWe sample 200 images from each of the two datasets in our experiments for evaluation. Specifically, we employ Gemini 2.5 Flash to generate descriptions for annotated instances and manually verify each sentence by our team to ensure the accuracy of the data quality. Subsequently, we conduct zero-shot evaluations on these data, with performances recorded in the following table. Our NAUTILUS surpasses the counterparts by at least 3.0 PR\\@0.5 and 1.5 PR\\@0.5 on the UIIS10K and TrashCan datasets, respectively, presenting a remarkable performance of general applicability. We will add these evaluations in the revised version.\n\n| **Method** | **UIIS10K** [1] | **TrashCan** [2] | Marine-Inst20M [3] |\n| --- | --- | --- | --- |\n| MiniGPTv2 | 43.5 | 19.5 | 37.3 |\n| mPLUG-Owl3 | 53.5 | 31.5 | 36.0 |\n| InternVL-2.5 | 56.0 | 41.7 | 38.3 |\n| LLaVA-1.5 | 51.5 | 24.5 | 36.0 |\n| **NAUTILUS**(LLaVA-1.5) | 53.0(+1.5) | 29.0(+4.5) | 37.1(+1.1) |\n| Qwen2.5-VL | 62.5 | 47.5 | 40.0 |\n| **NAUTILUS**(Qwen2.5-VL) | 65.5(+3.0) | 49.0(+1.5) | 40.4(+0.4) |\n\n[1] UWSAM: Segment Anything Model Guided Underwater Instance Segmentation and A Large-scale Benchmark Dataset. arXiv 25.\n\n[2] TrashCan: A Semantically-Segmented Dataset towards Visual Detection of Marine Debris. arXiv 20.\n\n[3] MarineInst: A Foundation Model for Marine Image Analysis with Instance Visual Description. ECCV 24.\n\n- **To Question 2**: “Since much of NautData is constructed using LMMs like Gemini, Qwen, and GPT-4o, it raises concerns about annotation accuracy—especially given that Table 2 shows GPT-4o itself performs poorly on the benchmark. Could the authors clarify how they ensured label correctness during dataset construction? Additionally, quantitative measures of annotation quality (e.g., human agreement rates or error analyses) would help justify the dataset’s reliability as a benchmark.”\n\n**Reply:** The NautData can be divided into three parts according to their generation method. Specifically, data for counting, detection, coarse-grained, and fine-grained classification tasks are generated rule-based, without involving LMMs. Annotations of this part of the data are obviously accurate. Indeed, the performance of GPT-4o is significantly inferior to that of other models. However, the GPT-4o is not employed to generate bounding box annotations and is only used to filter out low-quality text descriptions, unleashing the superior multi-modality performance of GPT-4o. Besides, employing LMMs in data scaling of large-scale instruction-following datasets is a prevailing practice [1][2], which introduces noise while maintaining the overall accuracy.\n\nTo evaluate the label correctness of our NautData, following your advice, we conduct human studies, providing a quantitative measure of annotation quality. Specifically, we adopt a ten-point scoring scale. For example, score 10 denotes “The description is exhaustive, internally consistent, and entirely accurate; no factual, logical, or interpretive errors are observed”, score 8 denotes “Generally accurate, but contains several fine-grained errors that exert limited impact on core understanding”, score 3 denotes “Poor: Predominantly inaccurate; only a few statements are correct, and the overall presentation is misleading”, and score 1 denotes “The content is entirely wrong, irrelevant, or contradicts established knowledge; no redeeming factual accuracy is present”. Fine-grained errors like misinterpreting “clownfish” as “acrossocheilus” demand a score of eight, while misinterpreting “fish” as “crab” will just obtain a score of six. Our dataset finally obtains an average score of 8.06, illustrating an overall high quality of the NautData with tolerable noise. We will add discussions of our data quality in the revised version.\n\n[1] ShareGPT4V: Improving Large Multi-modal Models with Better Captions. ECCV 24.\n\n[2] GROUNDHOG: Grounding Large Language Models to Holistic Segmentation. CVPR 24."}]

Author rebuttal:

Meta-review: This paper presents NAUTILUS, a vision-language model specifically designed for underwater scene understanding. Also, they propose a large-scale dataset called NautData, comprising 1.45M image-text pairs across eight tasks. The core contribution is the Vision Feature Enhancement module, which improves robustness under low-visibility conditions and can be seamlessly integrated into mainstream multimodal models. While reviewers initially raised concerns about generalization, the motivation for feature-space enhancement, and the design of the dark-pixel prior, the authors addressed these points with extensive rebuttal experiments, including cross-dataset zero-shot evaluations, robustness tests under varying lighting and turbidity, and human-rated dataset quality assessments. Following the rebuttal, most reviewers shifted to a positive stance. Thus, the AC recommends accepting this paper.
Decision: Accept (poster)