Title: Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image Generation

Object Detection

Computer Vision
Abstract: Maritime object detection is essential for navigation safety, surveillance, and autonomous operations, yet constrained by two key challenges: the scarcity of annotated maritime data and poor generalization across various maritime attributes (e.g., object category, viewpoint, location, and imaging environment). To address these challenges, we propose Neptune-X, a data-centric generative-selection framework that enhances training effectiveness by leveraging synthetic data generation with task-aware sample selection. From the generation perspective, we develop X-to-Maritime, a multi-modality-conditioned generative model that synthesizes diverse and realistic maritime scenes. A key component is the Bidirectional Object-Water Attention module, which captures boundary interactions between objects and their aquatic surroundings to improve visual fidelity. To further improve downstream tasking performance, we propose Attribute-correlated Active Sampling, which dynamically selects synthetic samples based on their task relevance. To support robust benchmarking, we construct the Maritime Generation Dataset, the first dataset tailored for generative maritime learning, encompassing a wide range of semantic conditions. Extensive experiments demonstrate that our approach sets a new benchmark in maritime scene synthesis, significantly improving detection accuracy, particularly in challenging and previously underrepresented settings. The code is available at https://github.com/gy65896/Neptune-X.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Neptune-X, a data-centric framework that improves maritime object detection by combining synthetic data generation with task-aware sample selection. It introduces X-to-Maritime, a multi-modality generative model with a Bidirectional Object-Water Attention module for realistic scene synthesis, and Attribute-correlated Active Sampling to enhance training efficiency. The authors also present the Maritime Generation Dataset to support benchmarking. Experiments show significant improvements in both image synthesis and detection performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1.\tLeveraging image generation for data augmentation is a valuable and promising approach to expand the training set for maritime object detection.\n\n2.\tThe paper is well-written and easy to follow.\n\n3.\tThe experiments demonstrate high-quality image synthesis and substantial improvements in detection performance.\n\nWeaknesses\n\n1.\tThe idea of using image generation for data augmentation is not novel for maritime detection [C] and other domain-specific detection tasks. Prior works such as DriveDreamer [A] and Panacea [B] in autonomous driving have explored similar approaches and evaluated their effectiveness for improving training utility.\n\n2.\tThe technical contributions are highly domain-specific, and some components—such as the AAS—appear hand-crafted. While these designs are effective within the maritime context, they lack generality and may offer limited methodological insights for the broader machine learning community. As a result, the work may be better suited to a domain-focused venue rather than a general-purpose conference like NeurIPS.\n\n[A] Wang, Xiaofeng, et al. \"DriveDreamer: Towards Real-World-Drive World Models for Autonomous Driving.\" ECCV 2024\n\n[B] Wen, Yuqing, et al. \"Panacea: Panoramic and controllable video generation for autonomous driving.\" CVPR. 2024.\n\n[C] Tang, Datao, et al. \"AeroGen: Enhancing remote sensing object detection with diffusion-driven data generation.\" CVPR. 2025.\n\nQUESTIONS:\nNA\n\nLIMITATIONS:\nOne potential limitation that is not discussed in the paper is the dual-use nature of the proposed techniques. Although the work is positioned for maritime safety and monitoring, the enhanced object detection and scene synthesis capabilities could also be adapted for military surveillance or defense-related applications. It would be helpful if the authors could briefly acknowledge this potential and clarify the intended scope of deployment.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- The authors propose Neptune-X1, a unified framework for maritime object detection that combines diverse scene generation with task-aware data selection.\n\n- Authors introduce a Bidirectional Object-Water Attention (BiOW-Attn) module that enhances realism by modeling object-water interactions under multi-condition inputs. \n\n- To select data samples that are most beneficial for detection performance, authors propose an Attribute-dependent Active Sampling (AAS) strategy that estimates training difficulty across semantic dimensions and selects high-value samples through difficulty-aware weighting.\n\n- Authors also construct a Maritime Generation Dataset (MGD) for maritime detection, by pooling together 11,900 images from various publicly available and newly captured datasets – with corresponding captions, water surface masks, and bounding box annotations. The dataset covers five object categories, three viewpoints, four locations, and six imaging environments.\n\n- Table 2 demonstrate that Neptune-X is better generator compared to baselines, and Table 3 and Fig 5 demonstrate that training on generated samples leads to improved performance across the board.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- This work proposes a novel technique specifically tailored to improving methods for maritime detection, which represents a valuable contribution to the field.\n\n- The thorough experiments and results clearly demonstrate the effectiveness of both the generator (Neptune-X) and the post-processing components (AAS strategy and ATDF factors).\n\n- The authors have collected and annotated a dedicated dataset, the Maritime Generation Dataset (MGD), which could be beneficial for the broader research community.\n\n- The paper is well-written, includes clear illustrative figures, and provides sufficient details about the experiments and the collected dataset.\n\nWeaknesses:\n- The modeling contribution, particularly the Bidirectional Object-Water Attention (BiOW-Attn) module, while effective, lacks strong novelty.\n\n- The AAS strategy shows clear benefits in the low-data regime (~5K samples), but its marginal gains at 10K suggest diminishing returns. A more detailed explanation for this saturation would be helpful.\n\nQUESTIONS:\n- I wonder if the authors have ablations on ATDF factors to evaluate their individual contribution to the AAS strategy?\n\n- Can the authors provide examples of generation diversity — for instance, using the same layout with different seeds or text prompts, and vice versa? This would help assess how well the model supports diverse outputs.\n\n- Is the model capable of editing existing real images (e.g., inserting new objects into real maritime scenes), and if so, could this be used to further improve detection performance?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of using synthetically generated images to improve object detection in a new domain. It proposes a framework named Neptune-X to actively select and balance synthetically generated images for domain specific fine-tuning. To facilitate this, this paper introduces a conditional image generator called X-to-Maritime, building on top of existing Stable Diffusion [Ref 25] framework. Specifically, X-to-Maritime leverage several application-specific design including multi-conditional guidance, layout condition embedder, and bidirectional object-water attribution for improved control and quality. For end-to-end evaluation, the paper applies the Neptune-X to curate a large-scale synthetic dataset, which in turn boosts the YOLO object detection in the Maritime domain.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- [S1] This paper addresses a very practical domain adaptation problem. Specifically, it tackles both data sacristy and poor generation in maritime object detection, building on top of a foundational text-to-image diffusion model.\n\n- [S2] While most of the work in the literature focused on generative model design, this paper presents an end-to-end solution to the problem, featured by the attribute-dependent active sampling technique.\n\n**Weaknesses**\n\n- [W1] The current evaluation does not clearly indicate  the specific impact of the proposed X-to-Maritime image synthesis engine on the final object detection performance. While Table 3 demonstrates performance gains when augmenting training data with generated samples, it remains ambiguous how much of this improvement is attributable to the enhanced synthesis capabilities of X-to-Maritime itself, as opposed to the overall data augmentation strategy. \n  - [W1.1] To provide a more comprehensive understanding of the X-to-Maritime engine's contribution, it would be beneficial to include a comparative analysis of YOLO detector performance when retrained on synthetic data generated by existing state-of-the-art text-to-image synthesis methods, such as SD1.5 [Ref 25], RC-L2I [Ref 5], InstDiff [Ref 33], GLIGEN [Ref 16] , and LayoutDiff [Ref 42]. This would clearly highlight the direct impact of X-to-Maritime's generated data quality on object detection improvement. \n  - [W1.2] Furthermore, extending the ablation studies (currently focused on generation quality in Table 4 ) to demonstrate the contribution of different components of the X-to-Maritime generation model (e.g., BiOW-Attn, multi-condition guidance) directly to the object detection improvement metrics (rather than YOLO score from a pre-trained detector) would offer deeper insights into the effectiveness of each generative module for downstream task performance.\n  - [W1.3] To more comprehensively illustrate the impact of the proposed methods, it would be beneficial to present the results as a 2D curve, with mean Average Precision (mAP) on the Y-axis and the number of synthetic samples on the X-axis. This visualization would allow for a clearer understanding of the performance trends as the quantity of generated data increases. Specifically, it would be insightful to determine if adding, for example, 10x the current amount of synthetic data continues to yield improvements in object detection performance. Furthermore, such a curve would help elucidate the sustained importance of both the AAS strategy and the X-to-Maritime model in scenarios with significantly larger synthetic datasets.\n\n- [W2] The paper states that the YOLO score was computed using an object detector pre-trained on synthetic data (L533-L537). This approach appears to be an unfair measurement when evaluating the quality of the synthetic data, as it would make more sense to use other object detectors trained on a larger, independent dataset to provide auxiliary signals.\n\n- [W3] Figure 2 depicts the proposed Embedder taking separate Layout and Water Surface images, suggesting distinct processing streams. Clarification is needed regarding whether the embedder employs a separate auto-encoder for encoding Layout and Water Surface information in the actual implementation. If this is the case, it raises concerns about the generalizability of the approach to other related domains. Such a design might imply a significant reliance on domain-specific expertise or extensive pre-training tailored to maritime scenes, potentially limiting its direct applicability or requiring substantial adaptation for different environments or object types.\n\nQUESTIONS:\nPlease address the comments especially [W1.1], [W1.2], [W1.3] and [W2] mentioned in the previous question.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Neptune-X, a framework for improving maritime object detection by generating synthetic training data with a generative model and selecting the most useful synthetic samples via an active sampling strategy. The authors highlight that maritime datasets suffer from scarce, costly annotation and poor diversity across key attributes (object category, viewpoint, environment, etc.).\n\nNeptune-X addresses this in two main ways:\n\n1. X-to-Maritime Generation: A diffusion-based generator (built on Stable Diffusion) extended with a Bidirectional Object-Water Attention (BiOW-Attn) module to model interactions between objects and the water surface. This improves the realism and semantic coherence of synthetic maritime scenes under multi-modal conditions.\n\n2. Attribute-dependent Active Sampling (AAS): An approach that scores generated samples by how difficult they are for an object detector, using Attribute-correlated Training Difficulty Factors (ATDFs) across dimensions like category, viewpoint, location, and environment, to prioritize hard or underrepresented cases.\n\nThey also create the Maritime Generation Dataset (MGD), the first large-scale dataset designed for training generative maritime models, with over 11,000 images and rich annotations.\n\nExtensive experiments show improvements in synthetic image quality (measured via FID, CAS, and YOLO Score) and object detection accuracy (via mAP) when augmenting detectors with selected synthetic data. An ablation study demonstrates the benefit of BiOW-Attn and AAS over baselines.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. Clear Motivation & Importance: The maritime domain has real and specific data scarcity issues that make synthetic generation important and meaningful. I like the way how the authors adjust existing techniques in ML community to support the domain-specific applications.\n\n2. Technical Contributions: The BiOW-Attn module is a well-motivated addition, specifically modeling object–water boundary interactions for more realistic generation. The AAS sampling framework integrates active learning principles into generative augmentation, addressing the problem that not all synthetic data is equally useful.\n\n3. Strong Empirical Evaluation: Multiple metrics (FID, CAS, YOLO Score) on generation. Detection performance improvements (7–10% mAP gains) on multiple YOLO baselines. Ablation studies isolating the effect of attention components and sampling strategies.\n\n4. Dataset Contribution: The new Maritime Generation Dataset (MGD) is well-documented, diverse in viewpoints, locations, and conditions, with an explicit plan for release.\n\n## Weaknesses\n\n\n1. Evaluation Scope: The object detection experiments only use YOLO variants; it’s unclear if improvements generalize to other detection architectures. For example, can grounding dino be used as a object detector to evaluate the dataset?\n\n2. Limited Baseline Comparisons in Sampling:\nWhile random sampling is compared to AAS, other active learning sampling strategies (e.g., uncertainty-based, diversity-based) are not evaluated. Also, ATDF is introduced with some mathematical sophistication, but the benefits over simpler scoring are only superficially explored.\n\n3. Assumptions and Generalization:\nThe ATDF method relies on predefined discrete attributes (category, viewpoint, etc.) which the authors admit limits granularity. But there’s no experiment showing how much that matters.\nThe approach is tightly tailored to maritime scenes; it's unclear if BiOW-Attn or AAS generalize to other domains with different context interactions---However, I don't think this point affect the value of this paper as we already claimed we are focusing on the maritime domain.\n\nQUESTIONS:\n1. BiOW-Attn: Could the authors provides qualitative result of the data generated without water condition? I am not fully motivated the purpose of water condition. Can I understand the water condition is just like the semantic mask condition telling the model here is the water? Can we just use traditional semantic mask to do this instead of designing a new attention layer?\n\n2. Sampling Strategy Comparison: How does the Attribute-dependent Active Sampling compare to standard active learning heuristics (e.g., entropy sampling, core-set)? Could the authors include such baselines?\n\n3. Detector Dependence: The pre-trained detector used for ATDF scoring is critical. How sensitive is the overall framework to the quality of this detector? Would a worse initial detector degrade AAS selection?\n\nBroader Impact Discussion: The paper briefly notes no negative societal impacts, but generated imagery could plausibly be misused for deception (e.g., fake maritime surveillance images). Would the authors consider adding even a short discussion acknowledging this risk?\n\nLIMITATIONS:\nThe authors do include a limitations discussion acknowledging reliance on discrete attribute categories and proposing more continuous or hierarchical modeling in future work. They don't discuss possible misuse (e.g., generating fake surveillance imagery). I recommend adding that. Overall they are upfront about technical limitations but could improve societal impact reflection.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-04"}