Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Model

Authors: ting  chen
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-26 05:06:25

Abstract: Existing unpaired image deraining approaches face challenges in accurately capture the distinguishing characteristics between the rainy and clean domains, resulting in residual degradation and color distortion within the reconstructed images.
To this end, we propose an energy-informed diffusion model for unpaired photo-realistic image deraining (UPID-EDM).
Initially, we delve into the intricate visual-language priors embedded within the contrastive language-image pre-training model (CLIP), and demonstrate that the CLIP priors aid in the discrimination of rainy and clean images.
Furthermore, we introduce a dual-consistent energy function (DEF) that retains the rain-irrelevant characteristics while eliminating the rain-relevant features.
This energy function is trained by the non-corresponding rainy and clean images.
In addition, we employ the rain-relevance discarding energy function (RDEF) and the rain-irrelevance preserving energy function (RPEF) to direct the reverse sampling procedure of a pre-trained diffusion model, effectively removing the rain streaks while preserving the image contents.
Extensive experiments demonstrate that our energy-informed model surpasses the existing unpaired learning approaches in terms of both supervised and no-reference metrics.

Main paper: 
 introduction: 
 Images taken during rainy weather condition frequently exhibit reduced visibility caused by rain streaks [3, 35, 36, 43] . This degradation severely impedes multiple computer vision tasks [10, 34, 42] , including detection, segmentation, and video surveillance. There are significant interests in developing methods to mitigate the rain degradation and reconstruct the photo-realistic details. Despite recent progress in data-driven learning techniques [4, 5, 33, 44, 49, 50] , fully-supervised learning that depend on the paired synthetic images frequently fall short in accurately capturing the underlying rain features. Therefore, these approaches encounter difficulties when confronted with real-world rainy images due to the disparity between the datasets utilized for training and testing.Due to the scarcity of precisely labeled data for image deraining, several strategies have surfaced to tackle this issue. These encompass semi-supervised methodologies, which leverage both labeled and unlabeled data, alongside unpaired learning techniques that function without explicit correlations between rainy and clean images. The former researches [8, 17, 23, 39, 56] prioritize extracting features that remain consistent across different domains and integrating supplementary objectives to enhance the performance, while unpaired techniques [6, 14, 41, 47, 57] frequently utilize domain adaptation strategies to achieve superior generalization. However, existing methods encounter challenges in attaining superior restoration quality because of the absence of clearly defined constraints for both rainy and clean images, leading to problems like residual degradation and color distortion. Therefore, there is a necessity to create thorough mapping representations that effectively capture the inherent relationship between rainy and clean domains. Fortunately, score-based diffusion model (SBDM) [15, 30] presents a promising departure from traditional generative adversarial network (GAN) [11] . SBDM perturbs data through a diffusion process and master the reversal of this process, yielding image generation performance that is either competitive or superior. However, to the best of our knowledge, there has been no the exploration of utilizing the score-based diffusion model within the unpaired image deraining.In this work, we propose an energy-informed diffusion model for unpaired photo-realistic image deraining, identified as UPID-EDM. This approach utilizes a dual-consistent energy function (DEF) that has been pre-trained across both rainy and clean domains. It directs the reverse sampling procedure of a pre-trained stochastic differential equation (SDE), facilitating the removal of rain streaks and the restoration of image details. Image deraining can be conceptualized as comprising two distinct processes, namely preserving the rain-irrelevant features (image content) and discarding the rainrelevant features (rain streaks). Based on this, we decompose the energy function into the summation of two potential functions. Although the contrastive language-image pre-training model (CLIP) [26] possesses the capability to discern between images depicting rain and those depicting cleanliness, its practical implementation still presents considerable challenges. For instance, although clean images and rain-free images are related concepts, we notice varying CLIP scores when comparing with the same given images [21] . To this end, we introduce the trainable domain-representation prompts (LDP), initialized randomly and trained using non-correspondence images. Specifically, we employ the rainy and clean images as inputs for a fixed image encoder, while the learnable negative and positive prompts are utilized as inputs for a fixed text encoder. Subsequently, we adopt the binary cross-entropy loss to classify the rainy and clean images, aiding in the acquisition of learned domainrepresentation prompts. Furthermore, we establish a rain-relevance discarding energy function (RDEF) to ascertain whether a given image belongs to the clean domain. Meanwhile, we also formulate a rain-irrelevance preserving energy function (RPEF) that ensures the generated images maintain consistency with the given rainy images in terms of image content. With the guidance of our suggested dual-consistent energy function, the pre-trained stochastic differential equation applied to the clean domain can produce the photo-realistic derained images when only given the rainy images.Figure 2 illustrates the overall pipeline of our proposed approach. The main contributions can be summarized as follows.• Our work pioneers the utilization of diffusion models in unpaired image deraining, showcasing the potential for the first time.• We decompose the energy function into two components, aiming to retain rain-irrelevant features while eliminating rain-relevant features during the reverse sampling procedure for generating reconstructed images. • By leveraging the perceptual abilities of contrastive languageimage pre-training model, we propose the learnable domainrepresentation prompts that guarantee the generated images adhere to clean domain. • Experiments on publicly available datasets demonstrate our proposed approach achieves the best photo-realistic unpaired deraining performance. 
related work 2.1 unpaired image deraining: 
 In the realm of unpaired image deraining, generative adversarial network [58] has emerged as a widely favored model. Several recent works [14, 57] improve CycleGAN [58] Score-based diffusion model [22, 30, 31] have recently made significant progress in a series of conditional image generation tasks. SBDM provides a diffusion model to guide how image shaped data from a Gauss distribution is iterated step by step into an image of the target domain. In each step, SBDM gives score guidance which, from an engineering perspective, can be mixed with energy and statistical guidance to control the generation process. Saharia et. al. [27, 28] develop a conditional SBDM to achieve paired superresolution and colorization. Choi et. al. [7] employ the reference image to refine the generated images after each denoising step with a low-pass filter. Recently, Zhao et. al. [54] and Sun et. al. [32] decompose the score into several components to guide the reverse sampling process and achieve the competitive image translation performance. The characteristic that SBDM can be influenced by energy or statistics serves as a strong motivation for us to employ the domain representation approaches in order to effectively transition from rainy domain to clean domain. However, there is currently no approach that employ SBDM to achieve unpaired image deraining. 3 Preliminary SBDM [31, 53] initially perturb the training data through a forward diffusion process and subsequently acquire the capability to invert this process, thereby constructing a generative model that approximates the underlying data distribution. Let 𝑞(𝒚 0 ) be the unknown data distribution on R 𝐷 . The forward diffusion process 𝒚 𝑡 , indexed by time 𝑡, can be represented by the following forward SDEEQUATIONwhere𝑡 ∈ [0,𝑇 ], 𝒘 ∈ R 𝐷 is a standard Wiener process, 𝒇 (•, 𝑡) : R 𝐷 → R 𝐷 is the drift coefficient and 𝑔(𝑡) ∈ R is the diffusion coefficient. Denote 𝑞 𝑡 |0 (𝒚 𝑡 |𝒚 0 )the transition kernel from time 0 to 𝑡, which is decided by 𝒇 (𝒚, 𝑡) and 𝑔(𝑡). In practice, 𝒇 (𝒚, 𝑡) typically exhibits an affine behavior, resulting in the perturbation kernel being a linear Gaussian distribution, allowing for straightforward sampling in a single step. Let 𝑞 𝑡 (𝒚) be the marginal distribution of the SDE at time t in Eq. ( 1), its time reversal can be described byEQUATIONwhere w is a reverse-time standard Wiener process, and 𝑑𝑡 is an infinitesimal negative timestep. Song et. al. [31] adopts a scorebased model 𝒔 (𝒚, 𝑡) to approximate the unknown ∇ 𝒚 log𝑞 𝑡 (𝒚) by score matching, thereby inducing a score-based diffusion model, which is defined asEQUATION)4 Proposed Method 
overview: 
 Based on Eq. ( 3), incorporating a supplementary guidance function 𝜖 (𝒚, 𝒙 0 , 𝑡) proves beneficial, leading to the derivation of a revised time-reversed SDE followingEQUATIONwhere𝒔 (•, •) : R 𝐷 × R → R 𝐷 is the score-based diffusion model in the pre-trained SDE and 𝜖 (•, •, •) : R 𝐷 ×R 𝐷 ×R → R is the proposed dual-consistent energy function. The start point 𝑦 𝑇 𝑠 is sampled from the perturbation distribution 𝑞 𝑇 𝑠 |0 (𝑦 𝑇 𝑠 |𝑥 0 ), where 𝑇 𝑠 = 0.4𝑇 empirically. We acquire the derained images by sampling at the time point 𝑡 = 0 according to the stochastic differential equation outlined in Eq. ( 4). Meanwhile, our methodology utilizes a scorebased diffusion model exclusively trained on the pristine domain. This model delineates a marginal distribution of clean images and predominantly enhances the authenticity of the derained samples. The suggested dual-consistent energy function incorporates data from both rainy and clean domains in an unpaired setting, aiming to maintain rain-irrelevant features while discarding rain-relevant ones. This approach enhances both the fidelity and naturalness of the reconstructed images. After individually completing training, we employ Eq. ( 13) to sample the reconstructed images based on the given rainy images.Image deraining can be described as a dual process involving the preservation of rain-irrelevant features (image content) and the removal of rain-relevant features (rain streaks). Therefore, heavily based on [54] , we express the energy function 𝜖 (𝒚, 𝒙, 𝑡) as the combination of two logarithmic potential functions,EQUATIONwhere To specify 𝑠 1 (•, •, •), we investigate the diverse priors within the contrastive language-image pre-training model, which exhibit a robust capability to capture domain discrimination [21, 45] . However, we discover that texts conveying similar meanings exhibit considerable separation within the latent space. As shown in Figure 5 , we present two instances featuring a single image alongside varied but analogous texts, followed by an assessment of their similarities within the CLIP space. Therefore, we employ the learnable domain-representation prompts to depict the rainy and clean domains without the need for paired images. Based on the inherent latent resemblance between images and prompts, we employ binary cross-entropy loss to classify images as either rainy or clean. This loss function is derived from (6) where ẑ (𝒗, 𝒑) = 𝑝 (𝑧|E 𝑖 (𝒗)) ∈ {0, 1} denotes the target label and prediction probability, 𝒗 ∈ {𝒙 𝑡 , 𝒚}, 𝑝 (•) indicates probability, 𝒑 𝑝 is the learnable positive prompt. 𝑦 = 0 is the label of rainy images and 𝑦 = 1 is the label of clean images. In our setting, the original prediction probability can be formulated as𝜖 1 (•, •, •) : R 𝐷 ×R 𝐷 ×R → RL 𝑝𝑟𝑜𝑚𝑝𝑡 = -[𝑧 log( ẑ (𝒗, 𝒑 𝑝 )) + (1 -𝑧) log(1 -ẑ (𝒗, 𝒑 𝑝 ))],EQUATIONwhere 𝑠𝑖𝑚(•) denotes the cosine similarity, E 𝑖 (•) and E 𝑡 (•) indicate the image encoder and text encoder in CLIP, respectively. 𝒑 𝑛 is the learnable negative prompts. As illustrated in Figure 5 , our learnable domain-representation prompts facilitate the accuracy of CLIP in distinguishing between rainy and clean images. Following the training of the learnable prompts, we also select the negative prompt to represent the underlying correlation between the provided images and prompts [21] , a formulation of which can be described as 𝒙 ∼ 𝑞 𝑛 |0 (𝒙 |𝒙 0 )EQUATION)7:𝜖 (𝒚 𝑛 , 𝒙 0 , 𝑛) = 𝜆 1 𝜖 1 (𝒚 𝑛 , 𝒙 0 , 𝑛) -𝜆 2 𝜖 2 (𝒚 𝑛 , 𝒙 0 , 𝑛) 8: 𝒚 𝑡 = 𝒚 𝑛 -[𝒇 (𝒚 𝑛 , 𝑛) -𝑔(𝑛) 2 (𝒔 (𝒚 𝑛 , 𝑛) -∇ 𝒚 𝜖 (𝒚 𝑛 , 𝒙 0 , 𝑛))]ℎ + 𝑔(𝑛) √ ℎ𝜂, 𝜂 ∼ N (0, I ) if 𝑖 > 1,else 𝜂 = 0 9: end for 10: return 𝒚 0 Therefore, we have the rain-relevance discarding energy function 𝒔 1 (𝒚, 𝒙 𝑡 , 𝑡) followingEQUATIONLowering this energy value indicates the generated sample to eliminate rain-relevant characteristics, ensuring its alignment with the clean domain [2, 12] . Furthermore, to maintain rain-irrelevant attributes and ensure contextual coherence between generated samples and clean images, we utilize two CLIP image encoders to assess latent similarity. This approach aims to enhance the alignment of derained images with their rainy counterparts in terms of contextual features [21] , thereby fostering the fidelity of content within the derained images. Therefore, we formulate the rain-irrelevance preserving energy function 𝒔 2 (𝒚, 𝒙 𝑡 , 𝑡) asEQUATIONwhere 𝑚 denotes the number of image encoder layers utilized to represent the distances, E 𝑖 (•) is the weight of 𝑘-th layer in image encoder, 𝜆 𝑘 indicates the coefficient corresponding to the 𝑘-th layer. By decreasing this energy value, it prompts the improved outcome to closely resemble the clean images in content, thereby safeguarding the rain-irrelevant features. Therefore, our decomposed dual-consistent energy function aids our model in filtering out rain-relevant features while retaining those that are rainirrelevant. Compared to the energy function in [54] , our decomposed energy function achieves better domain representation and structural preservation in the complex generative process of image deraining.In this work, we utilize the Euler-Maruyama solver [24] to tackle the energy-guided reverse-time stochastic differential equation. Based on the pre-trained score-based model 𝒔 (𝒚, 𝑡) and energy function 𝜖 (𝒚, 𝒙, 𝑡), we can solve the proposed energy-informed score-based diffusion model to generate samples from conditional distribution 𝑝 (𝒚 0 |𝒙 0 ). We select a step size ℎ, and utilize the iteration rule starting from 𝑡 = 𝑛ℎ followingEQUATIONwhere 𝜂 ∼ N (0, I ). The anticipated value within 𝜖 (𝒚 𝑠 , 𝒙 0 , 𝑠) is evaluated using the Monte Carlo method with a single sample to enhance efficiency. We outline the overarching sampling process of our approach in Algorithm 1. In our experiments, we employ the variance-preserving energy-informed diffusion model, as explicitly defined byEQUATION)where 𝛽 (•) is a positive function. The perturbation kernel𝑞 𝑡 |0 (𝒚 𝑡 |𝒚 0 ) ∼ N (𝒚 0 exp -1 2 ∫ 𝑡 0 𝛽 (𝑛)𝑑𝑛, (1-exp -∫ 𝑡 0 (𝛽 (𝑛)𝑑𝑛)I ) and 𝛽 (𝑡) = 𝛽 𝑚𝑖𝑛 + 𝑡 (𝛽 𝑚𝑎𝑥 -𝛽 𝑚𝑖𝑛 ). We set 𝛽 𝑚𝑖𝑛 = 0.1 and 𝛽 𝑚𝑎𝑥 = 20 [15, 24, 54] . Therefore, the iteration rule from 𝑛 to 𝑡 = 𝑛ℎ isEQUATIONThe iteration rule in Eq. ( 13) is equivalent to that using Euler-Maruyama solver when ℎ is small [31] , thereby the score network is modified to s (𝒚, 𝒙 0 , 𝑡) = 𝒔 (𝒚, 𝑡) -∇ 𝒚 𝜖 (𝒚, 𝒙 0 , 𝑡) in our proposed UPID-EDM. Therefore, we can readily adjust the noise prediction network and integrate it into the reverse sampling process [15] . 
implementation details: 
 We implement our proposed method with PyTorch on a single NVIDIA GeForce RTX 4090 GPU. The rain-relevance discarding energy function is trained on the combined dataset, where we ensure the rainy images are never paired with their corresponding clean images. We utilize the Adam optimizer [20] with 𝛽 1 = 0.9 and 𝛽 2 = 0.99. Each learnable domain-representation prompt contains 16 embedded tokens, with a total of 50 000 iterations. The learning rate for the energy function learning is set to 5 × 10 -6 , and the batch size is 8. In the rain-irrelevance preserving energy function, 𝜆 𝑘 ∈1,2,3,4,5 = 0.5, 1, 1, 1, 1. The initial time 𝑇 𝑠 is configured as 0.4𝑇 .We employ five commonly recognized benchmark rainy datasets to assess the effectiveness of our proposed approach, namely Rain800 [52] , Rain1400 [9] , Rain1200 [51] , RainCityscapes [16] , SPA-Data [38] datasets. In this work, our training regimen integrates a combination of these datasets, while testing experiments are conducted independently. A detailed description of these rainy datasets is provided in Table 1 . To optimize our training, we utilize a feature clustering approach to eliminate data that deviates significantly from the center of the testing distribution.We employ the learned perpetual image patch similarity (LPIPS) [53] to evaluate the supervised metrical scores between the reconstructed images and corresponding clean images. Meanwhile, we Test1188 Test1000 also utilize the contrastive language-image pre-training image quality assessment (CLIPIQA) [37] , multi-dimension attention network image quality assessment (MANIQA) [46] and multi-scale image quality transformer (MUSIQ) [18] to evaluate the no-reference metrical scores of the derained images. The lower LPIPS score indicates better image quality, and vice versa for the others.To evaluate the effectiveness of our proposed approach on synthetic image deraining, we employ the combined training dataset to train the involved methods and conduct evaluation on the corresponding five testing datasets, respectively. As Table 2 depicted, our proposed method demonstrates substantial enhancements in performance metrics, surpassing existing methods in both reference-based and no-reference evaluations. To further illustrate the effectiveness of our model, we additionally compute the average quantitative metrics and offer intuitive comparisons among various methods across LPIPS and the three image naturalness assessment metrics shown in Figure 1 . As illustrated, our model excels in average performance under supervised protocols and exhibits significantly enhanced naturalness compared to the other methods. Furthermore, we also provide several visual samples of the comparative methods in Figure 3 . Specifically, although EGSDE [54] with a similar setup can ensure that the reconstructed images fall within the clean domain, which leads to severe artifacts and hallucinations due to the lack of strict constraints on details. Meanwhile, the visual results of the other approaches exhibit either notable residual degradation or substantial color distortions. In contrast, our model effectively removes rain streaks while maintaining consistent color consistency.We conduct ablation experiments to evaluate the effectiveness of our model for unpaired photo-realistic image deraining, and all the experimental results are calculated by averaging the metrical scores across the five testing datasets.We validate the effectiveness of our proposed dual-consistent energy function by changing the weighting parameters 𝜆 1 and 𝜆 2 to control the contribution level of the rain-relevant discarding and rain-irrelevance preserving energy functions in the overall performance. As shown in Figure 4 , although the results generated without using our energy functions clearly fall within the clean domain, the textures of reconstructed images are inaccurate and there are many artifacts. This result intuitively proves the effectiveness of our dual-consistent energy function in constraining the domain and textures of reconstructed images. Meanwhile, Table 3 reports the quantitative comparisons We also compare our proposed rain-relevance discarding energy function with the texts and classifier. We substitute the learnable domain-representation prompts with the negative text ′ 𝑟𝑎𝑖𝑛𝑦 𝑖𝑚𝑎𝑔𝑒 ′ and the positive text ′ 𝑐𝑙𝑒𝑎𝑛 𝑖𝑚𝑎𝑔𝑒 ′ in the texts. Additionally, we utilize a basic VGG16 network [29] to train a classifier for clarifying the rainy and clean images. As Table 4 reported, our LDP demonstrates the significant image fidelity and naturalness improvements over the simple texts and classifier. Therefore, our proposed energy function for discarding rain-relevance aids the pre-trained score-based diffusion model in eliminating the rain-relevant features.To further demonstrate the effectiveness of our proposed rain-irrelevance preserving energy function, we compare the performance of it with the commonly used cycle consistency constraint (CCC) [40] and the Gaussian-guided mean squared error (GMSE) [54] . As the rain-irrelevant features, and further improves the quality of the reconstructed images.Prompts. Table 3 verifies the effectiveness of our proposed learnable domain-representation prompts. To clarify the motivation of our LDP, we additionally demonstrate the comparative latent similarities between various prompts and the provided rainy and clean images in Figure 5 , where the texts similarities are compared with the given images, and the and clean images are ′ 𝑟𝑎𝑖𝑛-𝑓 𝑟𝑒𝑒 𝑖𝑚𝑎𝑔𝑒 ′ and ′ 𝑟𝑎𝑖𝑛-𝑑𝑒𝑔𝑟𝑎𝑑𝑒𝑑 𝑖𝑚𝑎𝑔𝑒 ′ , respectively. As illustrated, although the texts present similar meanings in depicting the provided images, they exhibit notable distinctions in the underlying context. This issue have also been demonstrated in [21] . Therefore, employing the texts to compute energy values in our rain-relevance discarding energy function would lead to instability in the reverse sampling procedure. In contrast, our LDP can classify rainy images and clean images more accurately. 6 .As depicted, our energy-informed diffusion model demonstrates comparable performance at both 𝑇 𝑠 = 0.4𝑇 and 𝑇 𝑠 = 0.6𝑇 , and the under-performance at the other starting times. Therefore, we set 𝑇 𝑠 = 0.4𝑇 to strike a balance between the reconstruction performance and inference time in our experiments.  
limitations: 
 Although our approach successfully eliminates the rain streaks without the requirements of paired rainy and cleans images, its major limitations arise from the significant computational resources required for generating the reconstructed images and the processing time for individual images surpasses that of the existing methods. Additionally, although our proposed approach is effective in removing the real-world rain degradation, which may inadvertently exhibit hallucinations (seen in Figure 6 ), leading to the degradation of image details and distortion of texture in several cases.  
conclusion: 
 In this work, we introduce an innovative method for generating photo-realistic images without paired images. We delve into the rich visual-language priors within the contrastive language-image pre-training model, along with the capabilities of the score-based diffusion model in this context. Additionally, we showcase how our proposed dual-consistent energy function aids in filtering out the rain-relevant features while preserving the rain-irrelevant features during the reverse sampling process of a pre-trained diffusion model. 
: 
 Figure 1 : Intuitive comparisons of our proposed method and the other existing approaches between the average learned perpetual image patch similarity and three image naturalness assessment metrics. Our model achieves the currently best performance in the supervised protocols, while preserving the significant improved naturalness. 
 
 Appendix:  


Decision: This paper was reviewed by three experts in the field. The recommendations were mixed initially, but after authors’ feedback and discussion, the reviewers reached a consensus of acceptance, including Accept, Weak Accept, Borderline Accept. However, the reviewers still raise some valid concerns that authors did not fully resolve.

First, in the previous work EGSDE, the idea of domain specific and domain consistent features are also proposed. In the rebuttal, the authors only show quantitative comparison that shows the proposed method is better, but did not explain the reason and no additional experiment to verify why it is better. Second, experimental results for generalization are not unclear. It uses all five dataset, including real ones, and claims it can generalize to real images.

Still, AC agreed that this is an interesting idea and quantitative and qualitative results are good. Given that, the decision is to recommend the paper for acceptance to ACM Multimedia 2024.  We congratulate the authors on the acceptance of their paper!

Still, we recommended the authors to carefully read all reviewers’ final feedback, particularly the ones we mentioned above and revise the manuscript.

Reviews: review: paper_summary: paper_summary: This paper proposes an Energy-informed Diffusion Model to solve the image-deraining problem. The authors delve into the intricate visual-language priors embedded within the contrastive language-image pre-training model (CLIP). The authors further employ the domain-relevance discarding energy function and the domain-consistency preserving energy function to direct the reverse sampling procedure of a pre-trained diffusion model, effectively removing the rain streaks while preserving the image contents. Extensive experiments demonstrate that our energy-informed model surpasses the existing unpaired learning approaches in terms of both supervised and no-reference metrics.
strength_weakness: strength_weakness: 1, This paper is well-written.
2, This paper may be interesting.
3, The proposed UPID-EDM may be effective.
limitations: limitations: 1, It would be better if the authors could state the difference between this paper and [A].

[A] EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations. NIPS2022.

2, Missing some related references:
1) Dcsfn: Deep cross-scale fusion network for single image rain removal. ACM MM2020.
2) Joint self-attention and scale-aggregation for self-calibrated deraining network. ACM MM2020. 
3) Online-updated high-order collaborative networks for single image deraining. AAAI2022.
4) PromptRestorer: A Prompting Image Restoration Method with Degradation Perception. NeurIPS2023.
paper_summary: paper_summary: This paper introduces an energy-informed diffusion model for unpaired image deraining called UPID-EDM. The model utilizes the CLIP to distinguish between rainy and clean images and employs a dual-consistent energy function (DEF) to guide the reverse sampling process in a pre-trained diffusion model. This approach decomposes the energy function into two parts: one that discards domain-related features and another that preserves domain-consistent features. Sufficient experiments validate the effectiveness of the proposed model, demonstrating superior restoration quality and naturalness in the derained images.
strength_weakness: strength_weakness: This paper is well-organized. It effectively utilizes a dual-consistent energy function to separate and preserve domain-consistent features while discarding domain-related features, leading to more accurate and natural image reconstruction.

The authors present a novel view to handle this problem,  which is to train the domain-representation prompts based on the energy-based diffusion model to guide the deraining tasks during the denoising process.
limitations: limitations: [1]. In your Dual-consistent energy function, the energy function is divided into two parts: one is domain-consistent, and the other one is domain-related, which is similar to the EGSDE paper [51]. Meanwhile, Algorithm 1 is also similar to the algorithm presented in the EGSDE paper. Are there any improvements or differences in your algorithm and energy function?

[2]. Is there another way to choose the better value of $\lambda_1$ and $\lambda_2$? Could you show some qualitative results with different $\lambda_1$ and $\lambda_2$​​​?

[3]. The symbol in Line 316. is $q_{T_s|0} (y_{T_s}|x_0)$. The symbol in Line 345 is $q_{t|0}(.|.)$
paper_summary: paper_summary: This work leverages contrastive language-image pre-training models and score-based diffusion models to generate photo-realistic images without paired images. The proposed dual-consistent energy function filters domain-related features while preserving domain-consistent features during reverse sampling.
strength_weakness: strength_weakness: 1. The reported quantitative and qualitative results are good.
2. The proposed method uses rich priors from contrastive language-image pre-training to enhance image quality and diversity.
3. The proposed method combines score-based diffusion models, improving the stability and effectiveness of the generation process.
limitations: limitations: 1. The authors mention in the paper that they are preserving the domain-consistent features (image content) while discarding the domain-related features (rain streaks). What is the basis for this assertion? Why are rain streaks categorized as domain-related features, and what exactly does "domain" refer to in this context?
2. The paper is hard to read and contains multiple instances of non-standard expressions. For example, in fig. 1, which dataset is used? Line 115, there are two dots. Line 178, "Specifically, We-->we ". 
3. The experimental data presented in this paper need scrutiny. For example, the "DerainCycleGAN [17]" paper reports test results on the Rain800 dataset with PSNR 24.32 and SSIM 0.843, whereas this paper reports results of PSNR 24.257 and SSIM 0.7802. Given that this work utilizes more datasets for training compared to DerainCycleGAN, yet shows inferior results on the same test set, an explanation is required. Furthermore, Section 5.4 claims to test generalization ability, but according to my understanding, SPA-Data is a real dataset and was included in the training process. The authors should provide a clear explanation regarding this.