SeaDiff: Delve into Underwater Image Generation with Symmetrical Parameter Control

Authors: Haonan Wu, Shuang Hao, Shi Wang, Hua Li, He Tang
Primary area: generative models
Keywords: diffusion model, controllable generation, underwater

Abstract:
With the advancement of diffusion models, the controllability of image generation has significantly improved. However, due to the refraction and absorption of light in water, underwater images often exhibit notable variations in luminance and color cast. This leads to challenges for generative models pre-trained on terrestrial images, as they struggle to produce underwater images with a diverse range of these variations, severely limiting the appearance diversity of generated underwater images. To address this issue, we focus on the precise control of appearance in underwater images. We model the appearance of underwater images using three attributes: luminance, dynamic range, and color cast. We propose a new method, SeaDiff, which introduces a Symmetrical Parameter Control structure to achieve precise control over the appearance of underwater images. The proposed method comprises two modules: Appearance Writer, which encodes and injects appearance attributes into the U-Net encoder, and Appearance Reader, which ensures that the generated images align with the desired appearance by analyzing the feature maps. Experimental results demonstrate that the proposed SeaDiff method significantly improves control over underwater image appearance while maintaining image quality, validating its effectiveness in underwater image generation.

Decision: reject
Reviewer scores: 5, 3, 5, 3

Reviews and discussion:
<thread>
creation date:
1730006415368

thread_type:
official_review

<thread_object>
creator: reviewer_pbau

<content>
summary:
This paper proposed a new underwater data generation method, which can control the luminance, dynamic range, and color cast of the generated images. Experiments prove that this method achieves superior control over appearance while maintaining image quality and layout-consistency. However, its application is limited and its techniques have been widely studyed. Specific reasons are given in weakness.

strengths:
1) This article achieves precise control over underwater image generation, and the performance is good.
2) To some extent, I believe that this method can serve as an alternative to underwater image enhancement, but the paper itself did not discuss this point.
3) This paper introduces the simultaneous use of luminance, dynamic range, and layout control for underwater image generation.

weaknesses:
1) its application is limited. From my opinion, the underwater image enhancement and some downstream applications, e.g., object detection and semantic segmentation, are significant. Only generating underwater images is meaningless but this paper only discusses how the proposed method can effectively and controllably generate underwater images, but does not mention the significance of generating underwater images. Only generating images of a specific scene is meaningless. 

2) Besides, the techniques it used have been widely adopted. If this paper proposed a novel technique for underwater image generation, it will still be a valuable work. However, the A-Writer and A-Reader are not novel and interesting enough. Their key contributions are cross-attention and feature supervision. The layout control is also achieved by cross-attention. These techniques have been widely used and are not designed for underwater image generation.

This paper lacks application impact and technological innovation.

questions:
1) The impact on mainstream applications should be fully discussed, including image enhancement and downstream applications. Generating underwater images only is not important.

2) Underwater-oriented techniques should be studied. Considering the content of this task, I believe it would be more meaningful to design specialized generative components for underwater scenes.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730276652084

thread_type:
official_review

<thread_object>
creator: reviewer_hkdy

<content>
summary:
This work models the appearance of underwater images from three aspects: luminance, dynamic range, and color cast,  achieving precise control over the appearance of underwater images. Additionally, this paper proposes a new method, SEADIFF, which comprises two primary contributions: the Appearance Writer, responsible for encoding and injecting appearance attributes into the U-Net encoder; and the Appearance Reader, which ensures that the generated images align with the desired appearance by analyzing the feature maps.

strengths:
1.This work models the appearance of underwater images across three aspects: luminance, dynamic range, and color cast. The idea is interesting.

2.This paper is written with clarity and is highly accessible to readers.

weaknesses:
1. The proposed method is relatively simple, lacking novelty and insight. This work appears to be more of an engineering effort rather than an academic innovation.

2. Although the proposed method achieves superior results based on quantitative metrics, the visualizations make it difficult to discern its advantages. For instance, in Figure 1, it is challenging to observe that the images generated by our method exhibit higher quality compared to those produced by GeoDiffusion.

3.The experiments are not robust, and additional studies are needed to demonstrate that the images generated by the proposed method are beneficial.

questions:
1. Why do the input resolutions differ across the various methods presented in Table 1?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730301251548

thread_type:
official_review

<thread_object>
creator: reviewer_cfjp

<content>
summary:
This paper tackles the challenge of managing underwater image appearance by focusing on three key attributes: illumination, dynamic range, and color cast. The method comprises two primary components: Appearance Writer (A-Writer) and Appearance Reader (A-Reader). The former encodes and injects appearance attributes into the U-Net encoder, while the latter ensures that the generated image aligns with the desired appearance by analyzing the feature maps.

strengths:
This paper is structured in a clear and accessible manner, making it easy to follow for readers. The topic of generating diverse and realistic underwater images is particularly intriguing, as it addresses the challenges faced in underwater imaging and highlights the potential for advancements in this field.

weaknesses:
1. The results presented are not satisfactory. As shown in Fig. 1, the images fail to meet the expected standards of realism. 
2. Why is it important to generate underwater images, particularly those of low quality (severely degraded)? Will this be beneficial for downstream tasks, or does it serve other purposes? 
3. The author creates new underwater images based on three attributes: illumination, dynamic range, and color cast. However, the degree of turbid, a key attribute of underwater images is neglected.

questions:
1. Can the proposed method generate high-resolution underwater images? 
2. What is the inference efficiency of the proposed method?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730626980658

thread_type:
official_review

<thread_object>
creator: reviewer_5nfm

<content>
summary:
This paper presents SeaDiff, an effective approach for controlling appearance attributes in underwater image generation using diffusion models. The authors model three key attributes (luminance, dynamic range, and color cast) and propose a Symmetrical Parameter Control framework consisting of an Appearance Writer (A-Writer) and an Appearance Reader (A-Reader). Evaluated on the RUOD dataset, the method demonstrates significant improvements over existing approaches in both appearance control and image quality.

strengths:
The paper addresses controllable image attribute editing, which is a fundamental challenge. The focus on precise control of specific attributes (luminance, dynamic range, and color cast) demonstrates a clear understanding of key image manipulation needs. I think the research direction would leverage a wide range of applications.

weaknesses:
In method section, I think the paper's unique contributions appear limited, primarily focusing on transferring existing pre-trained priors to a specific field. The proposed model is essentially a fine-tuned version of the pre-trained Stable Diffusion model. While leveraging pre-trained models is common practice, the core contributions and motivations of this work should be more explicitly highlighted to distinguish it from existing approaches.

In Table 1 of the experimental section, there is no analysis of the model's parameters or inference latency, which is essential given the reliance on a computationally heavy pre-trained model. Such an analysis would provide valuable insights into the method's efficiency and practical applicability.

In Section 5.1, the paper acknowledges a bias in existing datasets, which consist of similar frames extracted from the same video. However, the simple re-division of the dataset does not adequately address this distribution bias. Additionally, all experiments were conducted exclusively on the RUOD dataset, raising concerns about the method's generalization capabilities to other underwater datasets.

Meanwhile, the experimental section highlights successful outcomes but lacks a systematic analysis of failure cases or the method's limitations. A thorough examination of these aspects is crucial for understanding the robustness and applicability of the proposed method in diverse scenarios.

questions:
Underwater image attributes encompass more than just luminance, dynamic range, and color cast. Can the proposed method be adapted to handle additional attributes or support more extensive attribute editing? If so, how would this be achieved?

The results presented in the experimental section demonstrate the method's performance under relatively ideal underwater conditions. However, there is no testing under extreme lighting conditions or severe turbidity. How does the method handle extreme underwater conditions such as severe turbidity, low visibility, and uneven lighting? 

How does the method perform on underwater datasets other than RUOD? Has any cross-dataset evaluation been conducted to assess its generalization capabilities?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732790777686

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear Reviewers,

First, we would like to express our sincere gratitude for your thorough review and valuable feedback. We have given careful consideration to each of your suggestions, and in response, we have updated our paper and conducted additional experiments to further validate and improve our work. Below are the key updates we made to the experimental section:

1. **Analysis of Model Parameters and Inference Latency**: We have updated the analysis of model parameters and inference latency, exploring how different parameter settings impact the model's performance and efficiency. This update addresses the suggestions from Reviewer 5nFM and Cfjp.

2. **Zero-Shot Quantitative Experiments on the UTDAC Dataset**: In response to Reviewer 5nFM’s feedback, we have extended our validation of the zero-shot task and included new quantitative experiments on the UTDAC dataset to ensure the stability and effectiveness of the model in real-world applications.

3. **Validation of Generated Image Effectiveness and Enhancement of Downstream Object Detection Methods**: We have further validated the effectiveness of the generated images and, based on the feedback from Reviewers Cfjp, HkDY, and PBAu, enhanced the downstream object detection methods to improve the model's performance across various tasks.

The updates and additional experiments have been incorporated into the latest version of the paper, and the updated PDF file has been submitted for your review. We deeply appreciate the constructive feedback you provided, which has greatly helped us improve the quality and depth of our research. We hope these updates provide stronger support for our work. If you have any further suggestions or questions, we would be happy to continue the discussion and make any additional improvements.

Thank you once again for your time and review!

Best regards,  
The Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper addresses appearance discrepancies in underwater images by modeling three attributes: luminance, dynamic range, and color cast. It introduces the Appearance Writer (A-Writer) module that encodes these attributes and employs a cross-attention mechanism within a U-Net encoder to enhance realism and consistency in the images. Additionally, the Appearance Reader (A-Reader) module is introduced to analyze feature maps from the U-Net decoder, predicting appearance attributes and providing deep supervision to ensure alignment with expected appearance, ultimately improving image quality and fidelity.

***Strengths:***
- The paper targets the specific topic generation of controllable and realistic underwater images, that may have potential usage for underwater applications.  
- The manuscript is well-structured and written clearly,
- The use of a pre-trained SD model for fine-tuning is a practical approach

***Weaknesses:***
- The novelty of the proposed method is limited which just  fine-tuning an existing model with minimal new contributions to the method itself.
- The paper lacks critical analysis regarding the model's efficiency. This part is added, but the efficiency is not an advantage for this method. 

We thank the authors for their rebuttal and additional explanations and experiments provided during the review process. After careful consideration and discussion, we conclude the paper does not currently meet the threshold for acceptance in its present form. The manuscript does not sufficiently differentiate its approach from existing methods. The primary contribution appears to be an application of widely used techniques without significant modifications or novel insights. The research presented does not meet the high standards expected for significant impact or advancement.