Real-world Image Dehazing with Coherence-based Pseudo Labeling and Cooperative Unfolding Network
Authors: Chengyu Fang
Chunming He
Fengyang Xiao
Yulun Zhang
Longxiang Tang
Yuelin Zhang
Kai Li
Xiu Li
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Semi-supervised Learning
Real-world image dehazing
Primary area: machine_vision
TL;DR: The cooperative unfolding network (CORUN) and the first plug-in-play iterative mean-teacher framework (Colabator) for real-world image dehazing.

Abstract: Real-world Image Dehazing (RID) aims to alleviate haze-induced degradation in real-world settings. This task remains challenging due to the complexities in accurately modeling real haze distributions and the scarcity of paired real-world data. To address these challenges, we first introduce a cooperative unfolding network that jointly models atmospheric scattering and image scenes, effectively integrating physical knowledge into deep networks to restore haze-contaminated details. Additionally, we propose the first RID-oriented iterative mean-teacher framework, termed the Coherence-based Label Generator, to generate high-quality pseudo labels for network training. Specifically, we provide an optimal label pool to store the best pseudo-labels during network training, leveraging both global and local coherence to select high-quality candidates and assign weights to prioritize haze-free regions. We verify the effectiveness of our method, with experiments demonstrating that it achieves state-of-the-art performance on RID tasks. Code will be available at https://github.com/cnyvfang/CORUN-Colabator.

Real-world Image Dehazing with Coherence-based Pseudo Labeling and Cooperative Unfolding Network
Chengyu Fang!." Chunming H. Fengyang Xiao Yulun Zhang
Longxiang Tang' Yuelin Zhang", Kai Li Xiu Li
Shenzhen International Graduate School, Tsinghua University, Sun Yat-sen University, Duke University, *Shanghai Jiao Tong University, Thee Chinese University of Hong Kong, Meta Reality Labs t chengyufang, thu @ gmail.com
Abstract
Real-world Image Dehazing (RID) aims to alleviate haze-induced degradation in real-world settings. This task remains challenging due the complexities in accurately modeling real haze distributions and the scarcity of paired real-world data. To address these challenges, we first introduce cooperative unfolding network that jointly models atmospheric scattering and image scenes, effectively integrating physical knowledge into deep networks restore haze-contaminated details. Additionally, we propose the first RID oriented iterative mean- mean-teacher framework, termed the Coherence-basee pseudo labels for network training. Specifically, we provide an optimal label pool to store the best pseudo-labels during network training, leveraging both global and local coherence to select high-quality candidates and assign weights to prioritize haze-free regions. We verify the effectiveness of our method, with experiments demonstrating that achieves ofte-of-thee-r performance on RID tasks. Code will be available at ht tps: / / thub. com/ htt////waaag///uu-oolaaaaaaaa
Label Generator, to generate high-quality
Introduction
Real-world image dehazing (RID) is a challenging task that aims to restore images affected by complex haze in real-world scenarios. The goal is to generate visual- appeaaing results while enhancing the performance of down- stream tasks [1, 2]. The atmospheric scattering model (ASM), providing a
Input
DGUN
CORUN
physical framework for real-world de- hazing, is formulated as follows:
(1) where (ax) and J(x) are the hazy image and the haze-free counterpart.
P(x) = +(z)(())+AA11(()))))
PDN
DGUN +
CORUN
signifies the global atmospheric light. f(x) characterizes the transmis- sion map reflecting varying degrees haze visibility across different regions.
Figure Results cutting-edge methods. Our CORUN better restores hazy hazy-contaminated details. Furthermore, techniques op- timized by our Colabator framework, indicated by "+" suffix, exhibit strong generalization in haze removal and color correction.
Equal Contribution, f Corresponding Author, æ Email Address
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Conventional methods [3, 4] are limited by fixed feature extractors, which struggle to handle the complexities of real haze. Although existing deep saaring--bsedd methods [5-9] demonstrate im- proved performance, they face two significant challenges: (1) These methods do not accurately model the complex distribution of haze, leading to color distortion, as illustrated in fig. DGUN [10]. (2) Real-world settings lack sufficient paired data for network training while optimizing the network with synthesized data brings a domain gap, limiting the generalizability of the models.
To overcome the first challenge, PDN [1I] first introduces unfolding network [12, 13] to the RID field. In specific, PDN unfolds the iterative optimization steps of an ASM-based solution into a deep network for end-to-end training, incorporating physical information into the deep network. However, PDN does not effectively leverage the complementary information between the dehazed image and the transmission map, bringing overfitting problems and resulting in detail blurring (see fig. 1).
In this paper, we introduce the COopeRative Unfolding Network (CORUN), also derived from the ASM-based function, to address PDN's limitations and better model real hazy distribution. CORUN cooperatively models the atmospheric scattering and image scene by incorporating Transmission and Scene Gradient Descent Modules at each stage, corresponding to each iteration of the traditional optimization algorithm. To prevent overfitting, we introduce global coherence loss, which constrains the entire pipeline to adhere to physical laws while alleviating constraints on the intermediate layers. These design choices collectively ensure that CORUN effectively integrates physical information into deep networks, thereby excelling in restoring caze-contaminated details, as depicted in fig. 1.
To enhance generalizability in real-world scenarios, we introduce the first RID-oriented iterative mean-teacher framework, named Coherence- label generator (Colabator), designed to generate high-quality dehazed images as pseudo labels for training dehazing methods. Specifically, Colabator employs a teacher network, a dehazing network pretrained on synthesized datasets, to generate dehazed images on label-free real-world datasets. These restored images are stored in a dynamically updated label pool as pseudo labels for training the student network, which shares the same structure as the teacher network but with distinct weights. During network training, the teacher network generates multiple pseudo labels for a single real-world hazy image. We propose selecting the best labels to store in the label pool based on visual fidelity and dehazing performance.
To achieve this, we design a compound image quality assessment strategy tailored to the dehazing task, evaluating the global coherence of the dehazed images and selecting the most visually appealing ones without distortions for inclusion in the label pool. Additionally, we propose a patch-level certainty map to encourage the network focus well-restored regions of the dehazed pseudo labels, effectively constraining the local coherence between the outputs of the student model and the teacher model. As shown in fig. 1, Colabator, generating high-quality pseudo labels for network training, enhances the student dehazing network's 's capacity for haze removal and color correction.
Our contributions are summarized follows:
(1) We propose a novel dehazing method, CORUN, to cooperatively model the atmospheric scattering and image scene, effectively integrating physical information into deep networks.
(2) We propose the first iterative mean-teacher framework, Colabator, to generate high-quality pseudo labels for network training, enhancing the network's 's generalization in haze removal.
(3) We evaluate our CORUN with the Colabator framework on real-world dehazing tasks. Abundant experiments demonstrate that our method achieves state-of-the-art performance.
2
Related Works
2.1 Real-world Image Dehazing
The dissonance between synthetic and real haze distributions often hinders existing Learning-based dehazing methods [14-18] from effectively dehazing real-world images. Consequently, there's 's a growing emphasis tackling challenges specific to real-world dehazing [19-23].
Given the characteristics of real haze, RIDCP [7] and Wang et al. [24] proposed novel haze synthesis pipelines. However, relying solely on synthetic data limits models' robustness in real-world dehazing scenarios. Recognizing the distributional disparities between synthetic and real haze, methods like
CDD-GAN [25], D4 [26], Shao et al. [27], and Li et al. [28] have utilized CycleGAN [29] for

---

37
Learabl paramster
Stage 1
Stage
Stage
Stage
adtie summmattio
SODM
TGDM
SODM
TODM
Elermeni-sciee
(Initaaf)
S-OPNM
T-CPMM
S-OPMM
S-CPM
T-CPMIn
S-CPMM
Ehemeni-eiix muliipication
TODM
SGDM
TGDM
X
Doi product
Cherne -win summation
Motrox tronsposttion
Iniia
Muliipiccttion
0) The anchiberbrii the piopoeed DORLUN
D
P
TOOM
BCDM
Figure 2: The architecture of the proposed CORUN with the details at 1oth stage.
dehazing. Despite this, the challenges inherent in GAN [30] training often result in artifacts. Some approaches combine synthetic and real-world data, applying unsupervised loss to supervise real-world dehazing learning [19]. However, these losses lack sufficient precision, leading to suboptimal results. Other methods leverage pseudo-labels [31, 32], but the erroneous pseudo-labele cause degrade quality.
To address these challenges, we introduce a coherence-based pseudo labeling method termed Cola- bator. Our approach selectively identifies and prioritizes high-quality regions within pseudo labels, leading to enhanced robustness and superior generation quality for real-world image dehazing.
2.2 Deep Unfolding Image Restoration
Deep Unfolding Networks (DUNs) integrate model-based and dearning-based approaches [33, 34] and thus offer enhanced interpretability and flexibility compared to traditional learning- based methods. Increasingly, DUNs are being utilized for various image tasks, including image super-resolution [35], compressive sensing [36], and hyperspectral image reconstruction [37]. DGUN [10] proposes general form of proximal gradient descent learn degradation. However, it fails decouple prior knowledge, relying solely on single-path DUN to model degradation and construct mappings, posing challenges comprehending complex degradation. Yang and Sun first introduced DUNs to the image dehazing field and proposed PDN [1]]. However, PDN does not exploit the complementary information between the dehazed image and the transmission map, resulting in detail blurring. Our CORUN optimizes the atmospheric scattering model and the image scene feature through dual proximal gradient descent, thus preventing overfitting and facilitating detail restoration.
Methodology
3.1
Cooperative Unfolding Network
We propose the Cooperative Unfolding Network (CORUN), the first Deep Unfolding Network (DUN) method utilizing Proximal Gradient Descent (PGD) optimize image dehazing performance by levaraging the Atmospheric Scattering Model (ASM) and neural image reconstruction in cooperative manner. Each stage of CORUN includes Transmission and Scene Gradient Descent Modules (T&SGDM) paired with Cooperative Proximal Mapping Modules (T&S-CPMM). These modules work together to model atmospheric scattering and image scene features, enabling the adaptive capture and restoration of global composite features within the scene.
#x we initialize transmission map T In gradient descent, we simplify the atmospheric light A E R and implicitly estimate it in the CORUN pipeline focus on the detailed characterization of the scene and the relationship between volumetric haze and scene. Hence, (1) can be rewrite as
According eq. (I), given hazy image W x x
P=J P=J.T+I-17
(2)
3

---

Where J means the clear image without hazy. is the all-one matrix. Based on eq. (2), we can define our cooperative dehazing energy function like
LJJ, T) - T I1l (J) (TT),
(3)
where (J) and (T) are regularization terms on and J. We introduce two auxiliary variables T and J to approximate T and J, respectively. This leads to the following minimization problem:
(J. T arg min (J, J,T
Transmission optimization. Give the estimated coarse transmission map T and dehazed image JA-1 at iteration k 1, the variable T can be updated as:
||p |-- Ja-Ja-1 T 1/ 2
Tk arg min T
(TT).
(5)
We construct the proximal mapping between T and T by encoder-decoder like neural network which we named T-CPMM and denoted as proxØ-
T& = prox o (J+-1, Tx),
(6)
the auxiliary variables T, which we calculate by our proposed TGDM can be formulated as:
Au )
TTTx-1
Tx
Σ jI-
(-) (I -I-P" P
(I ji-
(I jI-
(E(R,G,B)
The variable Å is learnable parameter, we enable CORUN to learn this parameter at each stage during the end-to-end learning process, allowing the network to adaptively control the updates in iteration.
Scene optimization. Give T4 and J, the variable J can be updated as:
1 |P J Te + Tx I (J).
J arg min J
8
Same as the proximal mapping process the transmission optimization, S-CPMM has the similar structure as T-CPMM but different inputs, we denote S S-CPMM as prox.:
J prox (Jx, Ts),
where the J& we process by our SGDM can be presented as:
j (T Tx + m=1) (T P T. - TI + paJa-1);
(10)
the A in transmission optimization, μ4 is also learnable parameter bring more generalization capabilities to the network.
Details about CPMM. T-CPMM and S-CPMM share the same structure for improved mapping quality. Each CPMM block uses a 4-channel convolution to embed T and J into 30-dimensional feature map. The distinction between T-CPMM and S-CPMM lies in their outputs: T-CPMM produces 1-channel result to aid TGDM in predicting scene-compliant transmission map, whereas S-CPMM generates 3-channel RGB image. This enables S-CPMM to learn additional scene feature information, such as atmospheric light and blur, assisting SGDM generating higher-quality dehazed results with more details. For more efficient computation, each CPMM comprises only 3 layers with [1, blocks, doubling the dimensions with increasing depth.
3.2 Coherence-based Pseudo Labeling by Colabator
We generate and select pseudo labels using our proposed plug-and-play coherence-baseee label generator, Colabator. Colabator consists of teacher network with weights Orea shared with the student network Osta. via exponential moving average (EMA). It employs tailored mean-teacher strategy with a trust weighting process and an optimal label pool to generate high-quality pseudo labels, addressing the scarcity of real-world data. Figure illustrates the pipeline of our Colabator.

---

|mc1 Dehaca Network
Welk
Teyn
16
A Sadtnt
H0
X W
Laad
dv
Luyon
IA
DM
| Teeehe Neewwrk
betterr?
Updats
Partboring
CLiPkoeee Agen
Trasted weightr
Eaita
Optinal
Pre-training phose
(i) Fine-tuning phass with our Colsbator
Figure 3: The plug-and-play Coherence-based Pseudo-label Generator paradigm.
LQ E HRx we initially apply augmentations to generate corresponding strongly degraded data using strong augmentor A.(-), which randomly applies adjustments such as contrast, brightness, posterize, sharpness, JPEG com- pression, and Gaussian blur. Unlike the common wreanttecchrr strategy, omit functions like solarize, equalize, shear, and translate to prevent unnecessary degradation that might mislead model learning. We use the augmented image the input for the teacher network and the strongly augmented image for the student network, generating the following results:
Iterative mean-teacher dehazing. Given real hazy image P
PR R HQ HQ fo... (P P.R
T HQ- HQ fo... (A, (P
(11)
where PR E HRRx the result from the teacher network using the non-augmented input, and HQ PHg E ER represents the result from the student network by strong augment input and T HQ' HQ
T HQ are the corresponding transmission map. The different degrees of data augmentation lead to having better quality than P
varying dehazing results, typically resulting in P HQ
This approach ensures the model descends in the correct direction and helps mitigate the overfitting issues often associated with direct pseudo-label learning methods. By iterating, our teacher network generates increasingly high-quality pscudo-labels, providing more reliable supervision.
Label frust weighting. To better leverage the pseudo-dehazed images PR #Q generated by the teacher
network for model supervision, we designed composite image quality assessment strategy for further processing these pseudo-dehazed images and get the trusted weight w which means the reliability of each location an image. Our composite strategy primarily consists a haze density evaluator D(-) based on pre-trained CL.IP [38] model and fixed text feature, and a non-reference image quality evaluator (--). We partition P into sequence S E (FX)) and use D(-) HQ and ((-) to predict the density score and quality score. The final trusted weight w we can get from:
HQ
w (norm (D(Om (())))))
)) norm ( HQ
(12)
where V is compose sequence to map and resize as P. norm(-) means normalize scores from 0 to HQ'
that higher score means lower haze density and better image quality.
Optimal label pool. To ensure the use of optimal pseudo-labels and avoid domain adaptation collapse due to instability during training, we proposed optimal label pool P maintain the pseudo-labels in their optimal state. The overall procedure of our optimal label pool process is summarized in algorithm 1, compare pseudo-dehazed image PR with previous pseudo-label P4 and update HQ; pseudo-dehazed image as pseudo-label if it better than previous. To summarize the algorithm 1 and eq. (I1), the overall process of Colabator can be formalize as:
Pses
P HQ:
HQ: Pf.e: T
Peer Wpsee C(P Oten,
D(.), Q(.),
(13)
where C is our Colabator framework, PB... corresponding pesudo transmission map. Wpse means the trusted weight of the pseudo label.
Pse. is the paired pseudo label of A. T Psc is the
Weights update. The teacher network's weights Ozea are updated by exponential moving average (EMA) of the student network's weights Ostus which is denoted as follows:
0 8tce 1/0ea + (1 7))
(14)
where y) is momentum and r) E (0, 1). Using this update strategy, the teacher model can aggregate previously learned weights immediately after each training step, ensuring updating stability.
5

---

Algorithm 1 Optimal label pool process
Require: Haze density evaluator (-)) and image quality evaluator Q()):
Optimal label pool P;
Sample batch of real hazy images (P Liq for each Pf.q T.Q, do
Get teacher network prediction: Pn T HQ;
for. (P
HQQ
Partition Pft HQ; into N x N and get S $10 "º;
Compute score map of S HQ; d; = norm (D no and di = norm n ( @1
Load P if d; > desei and > QPse; then Compute trusted weight: wi (d, + q:)
T Upeeis decc ap P(i)
Psess
Update P(i) (P T HQ, Wi di 9:)
Return Pf Tt HQ,' wi as pesudo label.
"Q."" "Q²,"
else
Return P T
Panti Psess paee as pesudo label.
end if
end for
3.3 Semi-supervised Real-world Image Dehazing
To achieve success in real-world dehazing, we designed several loss functions for our CORUN and Colabator to constrain their learning process. We introduce a reconstruction loss using the L1 norm ||- I]1. To enhance visual perception, we employ contrastive and common perceptual regularization to ensure the consistency of the reconstruction results with the ground truth in terms of features at different levels. The perceptual loss is defined as follows:
common
(PHQ. PcT) |P PHoll 11 + 3e E Rec
(15)
(PP Gr), & (Pr)lll Ts (P. p. (H
L
contra Ren
PLQ.PHO Per) = |ar. PHolli + Be
(16)
where P PиQ is the dehazed result, (px()) means the in hidden layer of pre-trained VGG-19 [39], r, is the weight coefficient. Besides, to constrain the entire pipeline to obey physical laws while alleviating constraints on the intermediate layers, and prevent overfitting, we introduce a global coherence loss:
Lc (P P PHQ. THQ)= = [[(HQQ O THQ + -I-THQQ) PLQl11.
(17)
where @ is the Hadamard product, I means the all-ones matrix as the same size of P.. The global coherence loss ensures that CORUN can more efficiently integrate physical information into the deep network to facilitate the recovery of more physically consistent details. In addition, we introduce density loss Ldens based on D(-) to score and constraint the model to dehaze in the semantic domain:
Dens (P) D(P).
(18)
Pre-training phase. To ensure the capacity in dehazing and transmission map estimation, we pre- trained CORUN on synthetic paired datasets which contained clear image S E H x3 and synthetic hazy image P E HRWxx x3, Setting P as input, we can get the result by
PS T fe... (An (Pīq))
(19)
where Au means weakly geometric data augment, P means the dehazed result of synthetic hazy image, and S is the corresponding transmission map. In the pre-training phase, our CORUN is

---

Metrics
Hazy
PDN [11] MBDN [14]
DH [15]
DAD [27]
PSD [19]
D4 [26]
RIDCP [7]
DGUN [10]
Ours
FADEL
2.484
0.876
1.363
1.895
1.130
0.920
1.358
0,944
1.111
0.824
BRISQUEL
36.642
30.811
27.672
33.862
32.241
27.713
33.210
17.293
27.968
11.956
NIMAT
4.483
4.464
4.529
4.522
4.312
4.598
4.484
4,965
4.653
5.342
Table 1: Quantitative results on RTTS dataset. Red and blue indicate the best and the second best.
Hazy image
PDN
DAD
PSD)
D)
DGUN
RIDCP
Ours
Figure 4: Visual comparison on RTTS[40]. Please zoom in for a better view.
optimized end-to-end end using two supervised loss functions. The overall loss of the pre- pre-training phase:
Lpre Lie (Aw (P LQ) P P.q::P. P&T)
(20)
+ PeTicCon (PLq): Piro: The) PSS
+ T.Dens (Pi)) (H)):
where Pr is the trade-off weight of Leontro, Pc is the trade-off weight of LCoh- Leee
Fine-tuning phase. In fine-tuning phase, we adapt our CORUN pre-trained on synthetic data to real-world domain by our Colabator framework. For more steady learning, in this phase, we train with both synthetic and real-world data. As eq. (13), we generate P& TB PR TR HQ: HQ: Pse: Ti Wee
from P R LQ and we get Pirq: The use the eq. (19). The overall loss of the fine-tuning phase:
Line Lee contra (A, (P]) (Q), (P P HQ. Pfsc) + p, Commmon
Rec Pe PP
(21)
+ +wpeCcon
(A. (A, ( LQ), #Q: T -Dens (P w WL (
PR
RR))
4 Experiments
4.1
Experimental Setup
Data Preparation. We use RIDCP500 [7] dataset, comprising 500 clear images with depth maps estimated by [41], and follow the same way RIDCP [7] for generating paired data. During the fine-tuning phase, we incorporate the URHI subset of RESIDE dataset [40], which only consists of 4,807 real hazy images, for generating pseudo-labels and fine-tuning the network. We evaluate our framework qualitatively and quantitatively on the RTTS subset, which comprises over 4,000 real hazy images featuring diverse scenes, resolutions, and degradation. Fattal's dataset [42], comprising 31 classic real hazy cases, serves supplementary source for cross-dataset visual comparison.
Implementation Details. Our framework implemented using Py Torch [43] and trained on four NVIDIA RTX 4090 GPUs. During the pre-training phase, we train the network for 30K iterations,

---

Hazy image
PDN
DAD
DGUN
RIDCP
Qurs
Figure 5: Visual comparison Fattal's 's data [2]]
wlo Colabator DGUN
w/ Colabator DGUN
w/o Colabator CORUN
Colabator CORUN (Ours)
Datasets
Metrics
FADEL
1
0.857
1.091
0.824 11.956
RTTS
BRISQUEL
25.085
20.731
16.541
NIMA
4.813
5.190
4.856
5.342
Table 2: Generalization and Effect of our Colabator.
Datasets Metrics
w/o Mcan-
teacher
weight
Trusted wio Optimal
label pool
Datasets
Metrics
Stages (Ours)
6
FADEL
0.912
0.827
0.846
FADEL
0.785
0.808
0.824
0.839
RTTS
BRISQUEJ
15.728
16.606
15.707
RTTS
BRISQUEL
15.520
15.151
11,956
16.227
NIMA
4.921
4.867
5,285
NIMAT
5.228 5.281
5.342
5.187
Table 3: Module's Effect of our Colabator.
Table
Effect of stage number.
optimizing it with Adam [44] using momentum parameters (31 = 0.9, B2 = 0.999) and an initial learning rate of 2 x 10 gradually reduced to 1 10 with cosine annealing. In Colabator, the initial learning rate set to x 10 with only 5K iterations. Following [7]. we employ random crop and flip for synthetic data augmentation. We use DA-CLIP [45] as haze density evaluator and MUSIQ [46] as the image quality evaluator. Our CORUN consists of 4 stages and the trade-off parameters in the loss are set Ber Pr. Pe set to 0.2, 5, 10-2 respectively.
Metrics. We utilize the Fog Aware Density Evaluator (FADE) [47] to assess the haze density in various methods. However, FADE focuses on haze density exclusively, overlooking other crucial image characteristics such as color, brightness, and detail. To address this limitation, we also employ Blind/Referenceleess Image Spatial Quality Evaluator (BRISQUE) [48], and Neural Image AsSesSmen((NIMAA) [49] for a more comprehensive evaluation image quality and aesthetic. Higher NIMA scores, along with lower FADE and BRISQUE scores, indicate better performance. We use PyIQA [50] for BRISQUE and NIMA calculations, and the official MATLAB code for FADE calculations. All of these metrics non-reference because there is no ground-truth in RTTS [40].
4.2 Comparative Evaluation
We compare our method with 8 state-of-the-ard methods: PDN [1J, MBDN [14), DH [15], DAD [27], PSD [19], D4 [26], RIDCP [7], DGUN [10]. The quantitative results, presented table show that our method achieved the highest performance, outperforming the second-best method (RIDCP) by 17.0%. Specifically, our method improved FADE, BRISQUE, and NIMA scores by 12.7%. 30.8%, and 7.6%, respectively. This demonstrates that our method surpasses current state-of-the-a art techniques in both dehazing capability and the quality, and aesthetics the generated images.
The visual comparisons of our proposed method and state-of-thed the-tht algorithms are shown in figs. 4 and We can observe that these methods have demonstrated some effectiveness in real-world dehazing tasks, but when images containing white objects, sky, or extreme haze, the results from PDN, DAD, PSD, and RIDCP exhibited varying degrees of dark patches and contrast inconsistencies. Conversely, D4 caused an overall reduction in brightness, leading to detail loss in darker areas. Under these conditions, DGUN produced relatively aesthetically pleasing results but lost significant local detail, impairing overall visual quality. Notably, PSD achieved higher brightness but suffered from

---

Dataset
PDN [1T]
MBDN [14]
DH[[S]
DAD [27]
PSD [19]
D4 [26]
RIDCP [7]
DGUN [10] Ours
RTTS|40]
4.52
3.47
3.23
4.35
3.90
4.66
7.14
6.04
7.76
Fattal [attal's[42]
4.85
3.33
3.19
4.80
4.28
4.38
7.28
6.33
8.04
Table 5:
User study scores on [TTS[40] and Fattal's Fattal's[42].
Class(AP)
Hazy
PDN [11]
MBDN [14]
DH [15]
DAD [27] PSD [19] D4 [26] RIDCP [7]
DGUN [10] Ours
Bicycle
0.51
0.55
0.54
0.47
0.52
0.52
0.54
0.57
0.55
0.59
Bus
0.25
0.29
0.27
0.23
0.29
0.25
0.28
0.32
0.31
0.31
Car
0.61
0.65
0.63
0.51
0.65
0.63
0.64
0.67
0.66
0.68
Motor
0.38
0.45
0.43
0.37
0.38
0.42
0.42
0.47
0.46
0.49
Person
0.73
0.76
0.75
0.69
0.74
0.74
0.75
0.76
0.76
0.77
Mean
0.50
0.54
0.52
0.45
0.52
0.51
0.53
0.56
0.55
0.57
Table 6: Object detection results on RTTS[40].
Hazy image
Oar
Figure 6: Visual comparison of object detection on RTTS [40].
severe oversaturation. CORUN+ consistently outperforms others by producing clearer images with natural colors and better contrast, effectively removing haze while preserving image details.
4.3 Ablation Study
Generalization and Effect of Colabator. We evaluates the performance and the impact of our proposed Colabator framework across different metrics. As shown in table 2, removing the fine- tuning phase of Colabator led to significant performance drops. highlighting its critical role in the dehazing process. To evaluate the generalizability Colabator, we conducted additional experiments by replacing CORUN with the DGUN [10], while maintaining consistent training settings. Results in table and fig. indicate that Colabator substantially enhances DGUN's performance, demonstrating its effectiveness as a plug-and-pla paradigm with strong generalization capabilities.
Effeet of Colabator. validate the effect of our Colabator. In table 3, we systematically removed critical components, such as mean-teacher, trusted weighting, and the optimal label pool, from the model architecture. The outcomes indicate the performance deteriorates when these components are removed, highlighting their essential role in the system.
Ablations on stage number. The number stages in a deep unfolding network significantly impacts its efficiency and performance. To investigate this, we experimented with different stage numbers for CORUN+, specifically choosing & values from the set [1, 4, 6). The results detailed in table indicate that CORUN+ achieves high-quality dehazing with 4 stages. Notably, increasing the number of stages does not necessarily improve outcomes. Excessive stages can increase the network's complexity, hinder convergence, and potentially introduce errors in the results.
4.4
User Study and Downstream Task
User Study. We conducted a user study to evaluate the human subjective visual perception of our proposed method against other methods. We invited five experts with an image processing background and 16 naive observers as testers. These testers were instructed to focus on three primary aspects: (i) Haze density compared to the original hazy image, (ii) Clarity of details in the dehazed image, and (iii) Color and aesthetic quality of the dehazed image. The results for each method, along with the corresponding hazy images, were presented to the testers anonymously. They scored each method
on scale from 1 (worst) to 10 (best). The hazy images were selected randomly, with a total of 225 images from RTTS[51] I] and images from Fattal's s[42] dataset. The user study scores are reported in table 5, showing that method achieved the highest average score.

---

Hazy image
PDN
DAD
PSD
D4
DGUN
RIDCP
Ours
Figure 7: Failure cases. Our results show low quality texture details.
The performance of high-level vision tasks, e-g. object detection and semantic segmentation, is greatly affected by image quality, with severely degraded images often leading erroneous results [52, 53J. To address this performance degradation, some methods have incorporated image restoration as a preprocessing step for high-level vision tasks. To validate the effectiveness of our approach for high-level vision, utilized pretrained YOLOv3 [54], and tested it on the RTTS [40] dataset, and evaluated the results using the mean Average Precision (mAP) metric. As shown in table and fig. our method demonstrates a substantial advantage over existing methods, verifying our efficacy in facilitating high-level vision understanding.
Downstream Task Evaluation.
5 Limitations and Future Work
In fig. 7, our CORUN+ model struggles maintain result quality and preserve texture details when dealing with severely degraded inputs, such strong compression and extreme high-density haze. This challenge persists across existing methods and remains unresolved. We attribute this difficulty to the model's s struggle in reconstructing scenes from dense haze, where information is often severely lacking or entirely lost, affecting the reconstruction of both haze-free and low haze density areas. Moreover, the model solely focuses dehazing and lacks the capability address other image degradations, such as image deblurring(55) and low-light image enhancement [56, 57], limiting its ability to achieve high-quality reconstruction results from complex degraded images. To address this limitation in future research, we propose not only focusing on environmental degradation but also considering additional information about image degradation when solving real-world dehazing problems. In addition to this, we can integrate robust generative methods to improve the network's ability restore dense haze regions [58-62]. synthesize haze that matches real-world distributions [63- 66], and introduce more modalities as supplements to RGB images, enhancing the model's ability to effectively recover details [67].
Conclusions
In this paper, introduce CORUN to cooperatively model atmospheric scattering and image scenes and thus incorporate physical information into deep networks. Furthermore, we propose Colabator, an iterative meen-teacher framework, to generate high-quality pseudo-labels by storing the best-ever results with global and local coherence in a dynamic label pool. Experiments demonstrate that our method achieves state-of-the-art performance in real-world image dehazing tasks, with Colabator also improving the generalization of other dehazing methods. The code will be released.
Acknowledgments and Disclosure of Funding
This work was supported by the STI 2030-Major Projects under Grant 2021ZD020144 404. The authors thank the NeurIPS committee for granting us the NeurlPS 2024 Scholar Award, which has helped us participate in the conference.
10

---

References
[1] Chunming He, Kai Li, Guoxia Xu, Jiangpeng Yan, and Longxiang Tang. Hqg-net: Unpaired medical image enhancement with high-quality guidance. IEEE Trans. Neural Netw. Learn. Syst., 2023.
[2] Christos Sakaridis, Dengxin Dai, Simon Hecker, and Luc Van Gool. Model adaptation with synthetic and real data for semantic dense foggy scene understanding. In Proceedings of the European Conference on Computer Vision (ECCV), pages 687-704, 2018. 1
[3] Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 33(12):2341-25 2010. 2
[4] Mingye Ju, Chumming He, Can Ding, Wenqi Ren, Lin Zhang, and Kai-Kuang Ma. All-inclusive image enhancement for degraded images exhibiting low low-freuuency corruption. Trans. Circuits Syst. Video Technol., 2024. 2
[5] Yeying Jin, Wending Yan, Wenhan Yang, and Robby T. Tan. Structure representation network and uncertainty feedback learning for dense non-uniform fog removal. In Proceedings of the Asian Conference Computer Vision (ACCV), pages 2041-2058, December 2022. 2
[6] Chunming He, Yuqi Shen, Chengyu Fang, Fengyang Xiao, Longxiang Tang. Yulun Zhang, Wangmeng Zuo, Zhenhua Guo, and Xiu Li. Diffusion models in low-level vision: A survey. arXiv preprint arXiv:2406. 11138, 2024.
[7] Ruiqi Wu, Zhengpeng Duan, Chunle Guo, Zhi Chai, and Chongyi Li. Ridep: Revitalizing real image dehazing via high-quality codebook priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2, 7, 8, 9, 18
[8] Shenghai Yuan, Jijia Chen, Jiaqi Li, Wenchao Jiang, and Song Guo. Lhnet: A low-cost hybrid network for single image dehazing. In Proceedings of the 31st ACM International Conference on Multimedia, pages 7706-7717, 2023.
[9] Shenghai Yuan, Jijia Chen, Wenchao Jiang, Zhiming Zhao, and Song Guo. Lhnetv2: A balanced low-cost hybrid network for single image dehazing. IEEE Transactions on Multimedia, 2024.
[10] Chong Mou, Qian Wang, and Jian Zhang. Deep generalized unfolding networks for image restoration. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17399-17410, 2022. 2, 3, 7, 8,8,9
[11] Dong Yang and Jian Sun. Proximal dehaze-net: A prior learning-based deep network for single image dehazing. In Proceedings of the european conference on computer vision (ECCV), pages 702-717, 2018. 2, 3, 7, 8,
[12] Chunming He, Kai Li, Guoxia Xu, Yulun Zhang, Runze Hu, Zhenhua Guo, and Xiu Li. Degradattion-resistan unfolding network for heterogeneous image fusion. In ICCV, pages 12611-12621, 2023.
[13] Guoxia Xu, Chunming He, Hao Wang, Hu Zhu, and Weiping Ding. Dm-fusion: Deep model- driven network for heterogeneous image fusion. IEEE Trans. Neural Netw. Learn. Syst., 2023.
2
[14] Hang Dong, Jinshan Pan, Lei Xiang, Zhe Hu, Xinyi Zhang, Fei Wang, and Ming- Hsuan Yang. Multi-scale boosted dehazing network with dense feature fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2157-2167. 2020. 7,
[15] Chun-Le Guo, Qixin Yan, Saeed Anwar, Runmin Cong, Wenqi Ren, and Chongyi Li. Image dehazing transformer with transmission-aware 3d position embedding. In Proceedings of the IFFF/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 5812-5820. 2022. 7. 8,8,
11

---

[16] Xiaohong Liu, Yongrui Ma, Zhihao Shi, and Jun Chen. Griddehazenet: Attention based multi- scale network for image dehazing. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 7314-7323, 2019. 18, 19
[17] Xu Qin, Zhilin Wang, Yuanchao Bai, Xiaodong Xie, and Huizhu Jia. Ffa-net: Feature fusion attention network for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 2020. 18, 19
[18] Tian Ye, Yunchen Zhang, Mingchao Jiang, Liang Chen, Yun Liu, Sixiang Chen, and Erkang Chen. Perceiving and modeling density for image dehazing. In European conference on computer vision, pages 130-145. Springer, 2022.
[19] Zeyuan Chen, Yangchao Wang. Yang Yang, and Dong Liu. Psd: Principled synthetic-to -raael dehazing guided by physical priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7180-7189, 2021. 7,
[20] Yuwei Qiu, Kaihao Zhang, Chenxi Wang, Wenhan Luo, Hongdong Li, and Zhi Jin. Mb- taylorformer: Multi-branch efficient transformer expanded by taylor formula for image dehazing. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 12802-
12813, 2023.
[21] Haiyan Wu, Yanyun Qu, Shaohui Lin, Jian Zhou, Ruizhi Qiao, Zhizhong Zhang, Yuan Xie, and Lizhuang Ma. Contrastive learning for compact single image dehazing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10551-10566 2021.
[22] Yu Zheng, Jiahui Zhan, Shengfeng He, Junyu Dong, and Yong Du Curricular contrastive regularization for physics- aware single image dehazing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5785-5794, 2023. 18, 19
[23] Yafei Zhang, Shen Zhou, and Huafeng Li. Depth information assisted collaborative mutual promotion network for single image dehazing. arXiv preprint arXiv: 2024. 2
[24] Jing Wang, Songtao Wu, Zhiqiang Yuan, Qiang Tong, and Kuanhong Xu. Frequency compen- sated diffusion model for real-scene dehazing. Neural Networks, 175:106281, 2024. 2
[25] Xiang Chen, Zhentao Fan, Pengpeng Li, Longgang Dai, Caihua Kong, Zhuoran Zheng, Yufeng Huang, and Yufeng Li. Unpaired deep image dehazing using contrastive disentanglement learning. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022, pages 632-648, Cham, 2022. Springer Nature Switzerland. 2
[26] Yang Yang, Chaoyue Wang, Risheng Liu, Lin Zhang, Xiaojie Guo, and Dacheng Tao. Self- augmented unpaired image dehazing via density and depth decomposition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2037-2046, 2022. 2. 8,
[27] Yuanjie Shao, Lerenhan Li, Wenqi Ren, Changxin Gao, and Nong Sang. Domain adaptation for image dehazing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages
2020. 2, 7, 8,
[28] Yi Li, Yi Chang, Yan Gao, Changfeng Yu, and Luxin Yan. Physically disentangled intra- and inter-domain adaptation for varicolored haze removal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5841-5850, 2022.
[29] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-iman translation using cycle-consistent adversarial networks. In Computer Vision (ICCV), 2017 IEÉE International Conference on, 2017.7
[30] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sher- jil Ozair, Aaron C Courville, and Yoshua Bengio. Generative adversarial nets. In Neural Information Processing Systems (NeruIPS), 2014.
[31] Xiaofeng Cong, Jie Gui, Jing Zhang, Junming Hou, and Hao Shen. A semi-supervised nighttime dehazing bascline with peatial-frequeng aware and realistic brightness constraint, 2024. 3
12

---

[32] Ming Tong, Yongzhen Wang, Peng Cui, Xuefeng Yan, and Mingqiang Wei. Semi-uformer: Semi-supervised uncertainty-awee aware transformer for image dehazing, 2022. 3
[33] Ran He, Wei-Shi Zheng, Tieniu Tan, and Zhenan Sun. Half-quadratic-baat iterative mini- mization for robust sparse representation. IEEE transactions on pattern analysis and machine intelligence, 36(2):261-277 2013. 3
[34] Mingye Ju, Chumming He, and Juping Liu. Ivf-net: An infrared and visible data fusion deep network for traffic object enhancement in intelligent transportation systems. IEEE Trans. Intell. Transp. Syst. 2022.
[35] Kai Zhang, Luc Van Gool, and Radu Timofte. Deep unfolding network for image super- resolution. Proceedings of the IEEE/CVE conference on computer vision and puttern recognition, pages 3217-3226, 2020. 3
[36] Di You, Jingfen Xie, and Jian Zhang. Ista-net++: Flexible deep unfolding network for compres- sive sensing. In 202/ IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEE, 2021. 3
[37] Miaoyu Li, Ying Fu, Ji Liu, and Yulun Zhang. Pixel adaptive deep unfolding transformer for hyperspectral image reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12959-12968, 2023.
[38] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning.
pages 1744--7733. PMLR, 2021.
[39] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scall scage- image recognition. In Yoshua Bengio and Yann LeCun, editors, International Conference on Learning Representations (ICLR). 2015. 6
[40] Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, and Zhangyang Wang. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 28(1):492-505, 2019. 8, 9,
[41] Mu He, Le Hui, Yikai Bian, Jian Ren, Jin Xie, and Jian Yang, Ra-depth: Resolution adaptive self-supervised monocular depth estimation. In European Conference on Computer Vision, pages 565-581 Springer, 2022. 7
[42] Raanan Fattal. Dehazing using color-lines. ACM transactions on graphics 2014. 7, 8,
[43] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pegforerformanc deep learning library. Advances in neural information processing systems, 32, 2019. 7
[44] Kingma Diederik. Adam: method for stochastic optimization. (No Title), 2014.
[45] Ziwei Luo, Fredrik K Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas Schön. Controlling vision-language models for universal image restoration. arXiv preprint arXiv:2310.01018 2023,
18
[46] Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5148-5157, 2021. 8
[47] Lark Kwon Choi, Jachee You, and Alan Conrad Bovik. Referenceless prediction of perceptual fog density and perceptual image defogging. IEEE Transactions on Image Processing (TIP), 24(1 (A(11)3888-399 2015. 8
[48] Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. No-reference image quality assessment in the spatial domain. IEEE Transactions on image processing, 21(12)449554708
2012. 8
13

---

[49] Hossein Talebi and Peyman Milanfar. Nima: Neural image assessment. IEEE Transactions on
Image Processing (TIP), 27(8)33998 10118,01111011 2018. 8
[50] Chaofeng Chen and Jiadi Mo. IQA-PyTorch: Pytorch toolbox for image quality assessment.
[Online]. Available: https://gi thub. wtt.cco//cccc/cc
2022. 8
[51] Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432. 2013.
[52] Xiaobo Wang, Lizhen Deng, and Guoxia Xu. Image threshold segmentation based on glle histogram. In CPSCom, pages 410-415. IEEE, 2019. 10
[53] Wenhan Yang, Ye Yuan, Wenqi Ren, Jiaying Liu, Walter J Scheirer, Zhangyang Wang, Taiheng Zhang, Qiaoyong Zhong, Di Xie, Shiliang Pu, et al. Advancing image understanding poor visibility enviromments: A collective benchmark study. IEEE Transactions on Image Processing,
29:5737-5752, 2020. 10
[54] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint
1rX1v:1804.02767, 2018. 10
[55] Yuelin Zhang, Pengyu Zheng, Wanquan Yan, Chengyu Fang, and Shing Shin Cheng. unified framework for microscopy defocus deblur with multi murlti-pyaamid transformer and contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11 11125-11136, June 2024. 10
[56] Chumming He, Chengyu Fang, Yulun Zhang, Kai Li, Longxiang Tang, Chenyu You, Fengyang Xiao, Zhenhua Guo, and Xiu Li. Reti-diff: Illumination degradation image restoration with vetinex-based latent diffusion model. arXiv preprint arXiv:2311.11633
2023. 10
[57] Xunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang, and Jiayi Ma. Diff-retinex: Rethinking low- light image enhancement with a generative diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12302-12311, 2023. 10
[58] Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 41 17-4125, 2024. 10
[59] Jiangshan Wang, Yue Ma. Jiayi Guo, Yicheng Xiao, Gao Huang, and Xiu Li. Cove: Un- leashing the diffusion feature correspondence for consistent video editing. arXiv preprint arXiv:2406.088 08850, 2024.
[60] Shenghai Yuan, Jinfa Huang, Yujun Shi, Yongqi Xu, Ruijie Zhu, Bin Lin, Xinhua Cheng, Li Yuan, and Jiebo Luo. Magictime: Time-lapse video generation models as metamorphic simulators. arXiv preprint 4rX1v:2404.05014, 2024.
[61] Shenghai Yuan, Jinfa Huang, Yongqi Xu, Yaoyang Liu, Shaofeng Zhang, Yujun Shi, Ruijie Zhu, Xinhua Cheng, Jiebo Luo, and Li Yuan. Chronomagic-bench: A benchmark for metamorphic evaluation of tix-to-timme-appe video generation. arXiv preprint arXiv:2406. /8522, 2024.
[62] Zunnan Xu, Yukang Lin, Haonan Han, Sicheng Yang. Ronghui Li, Yachao Zhang, and Xiu Li. Mambatalk: Efficient holistic gesture synthesis with selective state space models. arXiv preprint arXiv:2403.094/ 09471, 2024. 10
[63] Zanlin Ni, Yulin Wang, Renping Zhou, Jiayi Guo, Jinyi Hu, Zhiyuan Liu, Shiji Song, Yuan Yao, and Gao Huang. Revisiting non-autoregressive transformers for efficient image synthesis. In CVPR, 2024. 10
[64] Zanlin Ni, Yulin Wang, Renping Zhou, Rui Lu, Jiayi Guo, Jinyi Hu, Zhiyuan Liu, Yuan Yao, and Gao Huang. Adanat: Exploring adaptive policy for boken-based image generation. In ECCV, 2024.
[65] Yulin Wang, Gao Huang, Shiji Song, Xuran Pan, Yitong Xia, and Cheng Wu. Regularizing deep networks with semantic data augmentation. IEEE Transactions on Pottern Analysis and Machine Intelligence, 1021.
14

---

[66] Chenyang Zhu, Kai Li, Yue Ma, Chunming He, and Li Xiu. Multibooth: Towards generating all your concepts in an image from text. arXiv preprint arXiv:2404. 142399 2024. 10
[67] Ziqing Wang, Yuetong Fang, Jiahang Cao, and Renjing Xu. Bursting spikes: Efficient and nigh-performance snns for event-based vision. arXiv preprint arXiv:2311.14265 2023. 10
[68] Yifan Pu, Yizeng Han, Yulin Wang, Junlan Feng, Chao Deng, and Gao Huang. Fine- Fine-grained recognition with learnable semantic data augmentation. IEEE Transactions on Image Processing. 2024. 19
[69] Yifan Pu, Weicong Liang, Yiduo Hao, Yuhui Yuan, Yukang Yang, Chao Zhang, Han Hu, and Gao Huang. Rank-detr for high quality object detection. Advances in Neural Information Processing Systems, 36, 2024.
[70] Jiangshan Wang, Yifan Pu, Yizeng Han, Jiayi Guo, Yiru Wang, Xiu Li, and Gao Huang. Gra: Detecting oriented objects through group-wise rotating and attention. arXiv preprint arXiv:2403.1113 11127, 2024.
[71] Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, and Xiu Li. Bridging the gap: A unified video comprehension framework for moment retrieval and highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18709-18719, 2024.
[72] Zhengxi Zhang, Liang Zhao, Yunan Liu, Shanshan Zhang, and Jian Yang. Unified density-aware image dehazing and object detection real-world hazy scenes. In Proceedings of the Asian Conference Computer Vision, 2020. 19
[73] Chumming He, Kai Li, Yachao Zhang, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Camouflaged object detection with feature decomposition and edge reconstruction. In CVPR, pages 22046-22055, 2023. 19
[74] Chunming He, Kai Li, Yachao Zhang, Guoxia Xu, and Longxiang Tang. Weakly- concealed object segmentation with sam-based pseudo labeling and multi-scale feature grouping. NeurlPS, 2024.
[75] Chumming He, Kai Li, Yachao Zhang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Strategic preys make acute predators: Enhancing camouflaged object detectors by generating camouflaged objects. In ICLR, 2024.
[76] Jian Hu, Jiayi Lin, Shaogang Gong, and Weitong Cai. Relax image-specific prompt requirement in sam: A single generic prompt for segmenting camouflaged objects. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12511-12518 2024.
[77] Jian Hu, Jiayi Lin, Junchi Yan, and Shaogang Gong. Leveraging hallucinations to reduce manual prompt dependency in promptable segmentation. arXiv preprint arXivv2408.15205 2024.
[78] Fengyang Xiao, Pan Zhang, Chunming He, Runze Hu, and Yutao Liu. Concealed object segmentation with hicrarchical coherence modeling. In CAAI, pages 16-27. Springer, 2023.
[79] Fengyang Xiao, Sujie Hu, Yuqi Shen, Chengyu Fang, Jinfa Huang, Chunming He, Longxiang Tang, Ziyun Yang, and Xiu Li. A survey of camouflaged object detection and beyond. arXiv preprint arXiv:2408. 14562, 2024.
[80] Zunnan Xu, Zhihong Chen, Yong Zhang, Yibing Song, Xiang Wan, and Guanbin Li. Bridging vision and language encoders: Parameter-efficient tuning for referring image segmentation, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17503- 17512, 2023.
[81] Zunnan Xu, Jiaqi Huang, Ting Liu, Yong Liu, Haonan Han, Kehong Yuan, and Xiu Li. Enhanc- ing fine-grained multi-modal alignment via adapters: A parameter-efficient training framework for referring image segmentation. In 2nd Workshop Advancing Neural Network Training: Computational Efficiency, Scalability, and Resource Optimization (WANT@ ICMI. 2024), 2024.
19
15

---

[82] Yuelin Zhang, Kim Yan, Chun Ping Lam, Chengyu Fang, Wenxuan Xie. Yufu Qiu, Raymond Shing- Yan Tang, and Shing Shin Cheng. Mlotion-guided dual cume-camera tracker for endoscope tracking and motion analysis in mechanical gastric simulator, 2024. 19
[83] Longxiang Tang, Zhuotao Tian, Kai Li, Chumming He, Hantao Zhou, Hengshuang Zhao, Xiu Li. and Jiaya Jia. Mind the interference: Retaining pre-trained knowledge in parameter efficient continual learning of vision- lasounlanguage models. arXiv preprint arXhv:2407.05342 2024. 19
[84] Sixiang Chen, Tian Ye, Jun Shi, Yun Liu, JingXia Jiang, Erkang Chen, and Peng Chen. Dehrformer:
Real-time transformer for depth estimation and haze removal from varicolored haze scenes. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023. 19
[85] Lei Xu, Hui Wu, Chunming He, Jun Wang, Changqing Zhang, Feiping Nie, and Lei Chen. Multi-moda sequence learning for alzheimer's disease progression prediction with incomplete variable-length longitudinal data. MIA, 82: 82:102643, 2022. 19
[86] Yicheng Xiao, Lin Song, Shaoli Huang, Jiangshan Wang, Siyu Song, Yixiao Ge, Xiu Li, and Ying Shan. Grootvl: Tree topology is all you need in state space model. arXiv preprint a4Xiv:2406.02335, 2024. 19
[87] Yang Yue, Rui Lu, Bingyi Kang, Shiji Song, and Gao Huang. Understanding, predicting and better resolving q-value divergence in offline-rl. Advances in Neural Information Processing Systems, 36, 2024.
[88] Yang Yue, Bingyi Kang, Zhongwen Xu, Gao Huang, and Shuicheng Yan. Value-consistent representation learning for data-cfficient reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11069-11077, 2023.
[89] Le Yang, Haojun Jiang, Ruojin Cai, Yulin Wang, Shiji Song, Gao Huang, and Qi Tian. Con- densenet v2: Sparse feature reactivation for deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3569-3578, 2021.
[90] Le Yang, Yizeng Han, Xi Chen, Shiji Song, Jifeng Dai, and Gao Huang. Resolution adaptive networks for efficient inference. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 2369-2376 2020.
[91] Yizeng Han, Zeyu Liu, Zhihang Yuan, Yifan Pu, Chaofei Wang, Shiji Song, and Gao Huang. Latency-aware unified dynamic networks for efficient image recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1-17, 2024.
[92] Yizeng Han, Yifan Pu, Zihang Lai, Chaofei Wang, Shiji Song, Junfeng Cao, Wenhui Huang, Chao Deng, and Gao Huang. Learning to weight samples for dynamic early-exiting networks. In European conference computer vision, pages 362-378. Springer, 2022.
[93] Dongchen Han, Tianzhu Ye, Yizeng Han, Zhuofan Xia, Shiji Song, and Gao Huang. Agent attention: On the integration of softmax and linear attention. ECCV, 2024.
[94] Dongchen Han, Yifan Pu, Zhuofan Xia, Yizeng Han, Xuran Pan, Xiu Li, Jiwen Lu, Shiji Song, and Gao Huang. Bridging the divide: Reconsidering softmax and linear attention. In NewrlPS, 2024. 19
16

---

Hazy image
PDN
DAD
PSD
DGUN
RIDCP
Our
Figure 8: Detailed Comparison of fig. Red region reflects all past methods have had haze residues, but our method have the least in the same case. Purple region shows our method restores richer detail and truer colors. Please zoom for a better view.
A Appendix
A.1 Declaration of eq. (7) and eq. (10)
Having gotten eq. (5) and eq. (6), the solution of T can be formulated according to the proximal gradient algorithm:
A I
T(J 1 2 |P Ju-1 T+ T 1
1
(22)
Then we obtain the partial derivative:
0 TJ Tx-1) (I -I- - -
(23)
Let the partial derivative be equal zero, archieve the closed-form solution for T in eq. (7).
Similarly, the solution of J can be formulated
J (Tr, 2 - IP j
T
+ Tx
1 +
Mk
|J J 11
(24)
2
The corresponding partial derivative is JT k. J+-1) =
+
I) + 4 - JR-1).
(25)
The closed-form solution for is presented in eq. (10) when let the partial derivative be equal to zero.
A.2 Declaration of CLIP module.
The haze density evaluator D(-) can be formulated as:
D(-) = EnCimage (-) Encrest (Text)
|[nnaaaae (-)ll EnCrert (Text)]l T
(26)
The text used is "hazy" from the DA-CLIP provided in the text list.
17

---

Table 7: Ablation of CPMM module of CORUN.
Table 8: Ablation of trusted weights present
Modules
NNIA 1
BRISQUE
FADE
map or value.
Hazy(Input)
4,483
36.642
2,484
Methods
NIMA 1 BRISQUE
FADE $
w/o CPMM
4.836
38.197
1.362
Only Full
5.229
13.099
0.803
w/ CPMM (CORUN+)
5.342
11.956
0.824
Partttin=Full(CORII
5.342
11.956
0.824
Table 9: Effects of more Colabator components.
Table 10: Ablation of mainstream datasets setting.
Modules
NIMA
BRISQUE & FADE
RIDCP[7] Pipeline
Metrics
w/o Colabator
4.856
16.541
1.091
Data.+
Aug. OTS NIMA
BRISQUE 1 FADEL
w/o Strong aug.
5,084
12.671
0.813
4.845
20.779
0.765
w/o DA-CLIP[4S]
5.358
11.200
0.856
4.991
16.478
0.840
CORUN+
5.342
11.956
0.824
5.342
11.956
0.824
A.3 Ablation Study
Effect of the CPMM Module in CORUN. We evaluate the impact of the Cooperative Proximal Mapping Modules (CPMM) in our CORUN architecture. As shown table 7, incorporating CPMM leads to a significant improvement in performance across all metrics. The model with CPMM (CORUN+) outperforms the variant without CPMM, highlighting the importance of CPMM in enhancing dehazing efficiency and image quality.
Impact of Trusted Weight Representations. We investigate the effect of using trusted weights as either map or a value. As seen table 8, the combination of partitioned and full trusted weights (CORUN+) achieves better results compared using only full trusted weights. This emphasizes the value our trusted weight representation in improving the accuracy of dehazing.
Effects of Additional Colabator Components. To analyze the contribution of individual components within the Colabator framework, we performed ablation studies, with the results presented table 9. Removing essential elements, such strong augmentation DA-CLIP, results in performance deterioration, confirming the importance of each Colabator component ensuring optimal dehazing outcomes.
Ablation on Dataset Configurations. We evaluate our methods with different dataset settings. As shown in table 10, the results verify our method still achieves leading place under the three settings compared with existing methods.
Effectiveness of the Simplified ASM Formula. Scattering Model (ASM) formula. The results table II indicate that using the simplified ASM formula will lead to slight decrease in the dehazing ability, but it can evidently improve the image quality of the results.
We We assess the impact of simplifying the Atmospheric
Influence of Loss Functions. We compare the effect of using different loss functions (eq. (15) and eq. (16)). Table 12 shows that combining both loss functions yields better performance than using either one alone, demonstrating the advantage of this combined loss strategy refining the dehazing process.
Table II: Ablation of our simplified ASM formula.
Table 13:
Effects of integrating our Colabator with more cutting edge dehazing methods. The gains brought by Colabator are significant.
ASM formula
NIMA 1 BRISQUE FADEL
wo simplify w/ )OmpliY
5,203
14.469
0.817
5.342
11.956
0.824
Methods
NIMA t BRISQUE 1 FADE|
Table 12: Ablations of eq. 15.15 and eq. 16 loss functions.
1q.16 Our strategy achieves a better result.
C2PNet[22]
4.715
34,314
2.064
C2PNet+Colabator
4.823
23.662
1.329
Loss
NIMA
BRISQUE
FADE 1
FFA-Net[17]
4.822
33.235
2,080
Coo-aat+Colabbtor
4.839
29.219
0.958
15 Only
5.249
13.997
1.035
16.16 Only
5.220
12.484
0.795
GDN[16]
5.074
33,051
2.611
Both Loss (Ours)
5.342
11.956
0.824
GDN+Colabator
5.258
23.691
0.947
18

---

Integration with Other Dehazing Methods. To test the generalizability of Colabator, we inte- grated with various state-of-ther the-art dehazing models, such as C2PNet [22]. FFA-Net [17], and GDN [16]. As shown in table 13, incorporating Colabator leads to performance gains across all metrics, demonstrating its effectiveness as a plug-and-play module for improving dehazing in various architectures.
A.4 Broader Impacts
Real-world image dehazing is a crucial task in image restoration, aimed at removing haze degra- dation from images captured in real-world scenarios. In computer vision, dehazing can benefit downstream tasks such as object detection [68-72], image segmentation [73-81], tracking [82, 83], depth estimation [84, 85]. and more vision related tasks [86-94], with applications ranging from autonomous driving security monitoring. Our paper introduces a cooperative unfolding network and plug-and-play pseudo-labeling framework, achieving state-of-the-arr performance in wollw dehazing tasks. Notably, image dehazing techniques have yet to exhibit negative social impacts. Our proposed CORUN and Colabator methods also do not present any foreseeable negative societal consequences.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Relevant information included in section 1.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Relevant information included in section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
20

---

For each theoretical result we declared in section and introduced in section has full set of assumptions and complete and correct proof in section 3 and section 4. Guidelines:
Justification:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Relevant information is included in section 3 and section 4.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [Yes]
Justification: We provide a clear algorithm description section 3, which is conducive to reproducing, and the code and data will be open.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nipp) cc/ Moolic/guides/Coole Submi ssionPolicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (ht /nips.cc/pbbll.. wwabblc/guuu tttgii/iiiii httgguiddes//oooeo Sose/SodeSSuii ssionPooicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Relevant information included section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The results of both our ablation experiments and the comparison experi- ments section 4.3 effectively demonstrate the validity of our method and claims.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
22

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide the GPU model, video memory and quantity used by the computers we use for training and testing. section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the ines?
NeurlPS Code of Ethics http::///eeriips.ccpppltt///eeee
Answer: [Yes]
Justification: The research conducted in the thesis complies with the NeurIPS Code Ethics in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: There is no societal impact of the work performed as we explained in ap- pendix A.4
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite all the work covered in this article and follow their license.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our code, data and model will be open.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: Our experiment included a User study, and we reported our requirements and work content of the volunteers invited. table 5
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our user study does not involve any potential risks.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "G6cCz23CKl", "reviewer": "Reviewer_Udmv", "summary": "This paper focuses on Real Image dehazing. They propose a cooperative unfolding network (CORUN) to integrate physical knowledge for image dehazing. The proposed CORUN exploits the complementary information between components in Atmospheric Scattering Model. Besides, due to the lack of real paired data, this paper proposes a Coherence-based Label Generator to generate pseudo labels for real hazy inputs. The experimental results demonstrate that the proposed method achieves good dehazing performance.", "questions": "Please see the weakness for more details.\nBesides, I was wondering:\n1.\tWhat are the essential advantages of CORUN compared to state-of-the-art end-to-end dehazing models? Are there any experiments?\n2.\tWhether the proposed method can remove medium or dense haze correctly. Since the dehazing results shown in Fig. 4 still have haze residues even if the hazy input is not very dense. \n3.\tWhy use both perceptual loss and contrastive loss simultaneously? There are overlapping parts between them.\n4.\tHow robust is the estimation of haze density by DA-CLIP?", "limitations": "The authors have discussed the limitations.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.\tThe research direction of this paper is meaningful.\n2.\tThe quantitative results show that the proposed CORUN+ achieves state-of-the-art performance in terms of NRIQA metrics.\n3.\tThe visual results shown in Fig.5 outperform other competing methods.", "weaknesses": "1.\tThere is still room for improvement in the writing of this paper.\na)\tTypo in line 167\nb)\tHow to simplify A in line 111 is unclear. According to my understanding, after simplification, P in Eq. 2 has different physical meaning as defined in line 110.\nc)\tIn lin3 172, the same symbols are used to indicate different images.\nd)\tLine 187 presents two kinds of reconstruction loss, which is inconsistent with the perceptual loss in line 186.\n2.\tThe experiment is a bit unfair. This paper utilizes the data generation pipeline proposed in RIDCP, which has been proved that can improve the real image dehazing performance of dehazing networks. However, the quantitative results of PDN and MBDN reported in Table 1 seem to be calculated by using their original models, rather than re-training on the same training data used in this paper.\n3.\tThe experiment is a bit insufficient. \na)\tThe author only validates the effectiveness of Colabator on DGUN, neglecting other widely-used dehazing networks.\nb)\tThe full-reference evaluation on real dehazing benchmarks (e.g., O-HAZE, I-HAZE, etc.) can also be an important evidence to support the effectiveness of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments. If not stated, all experiments are conducted on the RTTS dataset.\n```  \n*Experiment results are placed in the attached PDF for space limitations*\n```\n**W1: Writing and formula**\n\n- The redundant comma will be removed.\n- The simplified $P$ in eq.(2) differs from the $P$ in line 110. In line 111, the original atmospheric scattering model $P=J\\cdot T+A\\cdot (I-T)$ is divided by the atmospheric light A, resulting in $P/A=J/A\\cdot T+I-T$. We then simplified $P/A$ and $J/A$ to $P$ and $J$, leading to $P=J\\cdot T+I-T$. We will reorganize it. \n\nThis simplification aims to help integrate atmospheric light $A$, which is closely related to image features, with the scene for better results. Experiments in **Table A1** show that this improves performance by **10.7%**.\n    \n- We will correct the notion: pseudo-dehazed image ${\\mathbf{P}^{R}\\_{\\widetilde{HQ}}}\\_{i}$ and previous pseudo-label ${\\mathbf{P}^{R}\\_{Pse}}\\_{i}$.\n- In eqs. (15) and (16), we introduced two loss functions for different phases. Eq. (15) includes reconstruction loss and perceptual loss, while eq. (16) substitutes the perceptual loss with a stronger supervised contrastive perceptual loss. We will rename eq. (15) and eq. (16) to Reconstruction-Perceptual Loss and Reconstruction-Contrastive Perceptual Loss.\n\n**W2: Unfair in data augmentation and generation pipeline**\n\nTo ensure a fair comparison with RIDCP, we follow its setup by using the same data pipeline (generation + augmentation) and dataset.\n\nTo fully show our results, we provide results under two extra settings: 1) We remove the augmentation pipeline, 2) We further replace RIDCP's dataset and generation pipeline with the OTS dataset (commonly used in previous methods). \n\nAs shown in **Table A2**, the results verify our method still achieves a leading place under the two settings compared with existing methods. \n\n**W3: Effect of Colabator and results on full-reference**\n\n- We integrate our Colabator with cutting-edge dehazing methods and test them in RTTS. In **Table A3**, we find evident performance gains brought by Colabator: **23.2%** in C2PNet, **22.0%** in FFA-Net, **31.7%** in GDN.\n\n- We further evaluate our methods on the full-reference setting with O-HAZE and I-HAZE. Because we have paired data for training, we drop Colabator and only train our CORUN following the practice of MB-TaylorFormer. **Table A4** shows that our method achieves leading performance with a big margin on both datasets.\n\n**Q1: Essential advantages**\n\nPrevious state-of-the-art dehazing methods, such as C2PNet[60], fail to generalize to broader real-world scenarios like the RTTS dataset when trained on synthetic data. RIDCP, with its well-designed data synthesis pipeline, generalizes better to *RTTS*, while is limited by depth estimation errors, causing unrealistic haze residuals.\n\nOur CORUN is specifically designed for real-world scenarios. It integrates deep unfolding networks and proximal gradient descent algorithms to combine physical priors with network learning. Therefore,  as shown in **Table A5**, our method achieves state-of-the-art real-world dehazing results.\n\n**Q2: Haze residues and medium/dense haze removal**\n\nApart from the problem of the synthetic training data, *i.e.*, the inaccurate depth estimation and the hazy residue from the ground truth, the reasons for failing to completely remove the haze lie in our semi-supervised setting with incomplete supervision. In Figs. 4, it is a common problem that also exists in other comparative methods.\n\nHowever, as shown in **Fig. A1**, our method has a better dehazing effect compared to existing methods, which is attributed to the joint estimation capacity of CORUN and the generalizability of Colabator.\n\nBesides, we randomly select 300 data in *RTTS* and invite 10 naive observers to score them based on how hazy they are. Then we select the first 100 images as dense hazy data and the 101st-200th images as medium-density hazy data according to the scores. **Table A6** shows our superiority in removing medium-density and dense haze.\n\n**Q3: Loss functions overlap**\n\nThe two losses, *i.e.*, eq.15 and eq.16, are employed in different phases with different inputs and their specific parameters in eq.20 and eq.21.\n\nIn the pre-training phase (eq.20), we use eq.16 with contrastive learning for constraints, using synthetic data. During the fine-tuning phase (eq.21), we employ both eq.15 and eq.16. Here, eq.15 uses synthetic data to ensure stability in the fine-tuning process, while eq.16, which incorporates contrastive perceptual loss, uses real-world data to provide stronger constraints and align the model training with real-world conditions. As shown in **Table A7**, our ablation verifies that combining these two losses enhances the model's performance. We will reorganize the content.\n\n**Q4: Robustness of DA-CLIP**\n\nWe evaluated the robustness of DA-CLIP by randomly selecting 300 images and assessing their density score. We invited 10 observers to rate haze density in these images. In **Table A8**, our analysis revealed that while DA-CLIP's results generally align with human ratings (with discrepancies under 0.2), significant differences occur in certain situations. Notable discrepancies (over 0.5) were observed in cases involving 1) images with predominant sky coverage on cloudy days, 2) color distortion or severe faults, and 3) insufficient lighting at night (examples shown in **Fig. A2**).\n\nTo address DA-CLIP's limitations in handling complex scenes, we designed a pseudo-label rating strategy. Along with DA-CLIP, we used NR-IQA to evaluate potential pseudo-labels. A pseudo-label is updated only if its scores from both DA-CLIP and NR-IQA surpass the latest label in our optimal label pool. Locally, we partitioned images and used DA-CLIP and MUSIQ to assign weights, encouraging the network to focus on higher-quality regions with better dehazing effects. **Table A9** verifies the effect of our pseudo-label rating strategy.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9IKyVLmuTU", "reviewer": "Reviewer_ivCJ", "summary": "The paper aims for real-world image dehazing, where the paper tries to handle the difficulties of modeling real haze distributions and the scarcity of real data. For modeling haze, the paper proposes to jointly model atmospheric scattering and image scenes by a cooperative unfolding network. To handle the scarcity of data, the paper proposes to generate pseudo labels using an iterative mean-teacher network, which the paper terms Coherence-based label Generator (Colabator).", "questions": "My major concern and questions have been written in the weakness section.", "limitations": "The authors have mentioned the limitation of their work in the supplementary document.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is well-written and easy to follow.\n- The proposed method is effective in that object detection results are improved on images dehazed by the proposed method in comparison to previous works.\n- The iterative mean-teacher framework for generating pseudo labels seems to be effective in image dehazing.", "weaknesses": "- The major concern I have for this paper is that the proposed method seems to be mostly combination of existing works. \n   - The joint estimation of atmospheric scattering/transmission (or atmospheric light) and clean image is not new:\n      - Mondal et al., Image Dehazing by Joint Estimation of Transmittance and Airlight using Bi-Directional Consistency Loss Minimized FCN  \n      - Im et al., Deep Variational Bayesian Modeling of Haze Degradation Process\n      - Zhang et al., Joint Transmission Map Estimation and Dehazing using Deep Networks\n      - Ren et al.,  Single image dehazing via multi-scale convolutional neural network\n    - Deep Unfolding Networks (DUNs) has been utilized for dehazing by Yang and Sun [50], mentioned in the paper\n    - The use of PGD for DUNs in the context of image restoration has been done by DGUN [33], as mentioned in the paper.\n    - Also, using mean teacher for creating pseudo labels is commonly performed in deep learning.\n   - Considering all of the above, the proposed method seems to be the combination of the existing methods.\n   - What is a major difference and technical novelty in comparison to the combination of existing methods above?\n- Since the proposed method is a combination of many components, a detailed ablation study would be needed to justify the effectiveness of each component.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments. If not specifically stated, all experiments are conducted on the real-world image dehazing task with RTTS for space limitation. \n\n**W1: Contribution**\n\nOur work goes beyond simply combining existing methods. We introduce the Cooperative Unfolding Network (CORUN), which iteratively optimizes transmission maps and scene information with theoretical guarantees. We also present Colabator, a semi-supervised fine-tuning framework that requires no extra computational cost and adapts well to real-world scenarios with just a few iterations.\n\nSpecifically, compared to previous methods, our innovations are as follows:\n\n- Methods by Mondal et al.[64], Im et al.[65], and Ren et al.[66] use common networks to estimate atmospheric light and transmission maps through the atmospheric scattering model but overlook other complex scene information. Zhang et al.[27] estimate the transmission map and haze features from hazy images but ignore the atmospheric model during reconstruction, limiting generalization.\n    \n    In contrast, our method mathematically models the relationship between the image scene and transmission map, utilizing a deep unfolding network based on proximal gradient descent for progressive dehazing. This enhances interpretability and detail restoration, resulting in more realistic images. To verify this, we incorporate our joint estimation component into the three open-sourced methods and observe evident performance gain.\n    \n    |Datasets|Metrics|Ren[66]|Ren+CORUN|Mondal[64]|Mondal+CORUN|Im[65]|Im+CORUN|CORUN(Ours)|\n    |-|-|-|-|-|-|-|-|-|\n    |O-HAZE|PSNR↑|15.72|19.91|16.37|18.85|18.85|21.36|25.66|\n    ||SSIM↑|0.703|0.753|0.717|0.750|0.776|0.793|0.847|\n    |I-HAZE|PSNR↑|14.49|18.83|15.52|18.27|17.93|20.15|23.90|\n    ||SSIM↑|0.710|0.760|0.721|0.763|0.752|0.785|0.868|\n\n- PDN[50] introduced a deep unfolding network for image dehazing but couldn't effectively use complementary information between the scene and transmission map, leading to sub-optimal results with real-world data. In contrast, our new dual proximal gradient descent cooperative unfolding network considers both atmospheric light and image information, enhancing robustness and flexibility. We integrated our optimization mode into PDN[50], resulting in improved performance.\n    \n    |Datasets|Metrics|PDN[50]|PDN+CORUN|DGUN[33]|DGUN+CORUN’s model|DGUN+CORUN|CORUN(Ours)|\n    |-|-|-|-|-|-|-|-|\n    |O-HAZE|PSNR↑|16.37|19.02|18.86|20.16|22.74|25.66|\n    ||SSIM↑|0.717|0.763|0.756|0.788|0.805|0.847|\n    |I-HAZE|PSNR↑|15.86|18.80|18.27|19.82|22.63|23.90|\n    ||SSIM↑|0.712|0.757|0.732|0.763|0.791|0.868|\n\n- DGUN models the relationship between degraded and clean images but ignores the interaction between the scene and transmission map in dehazing. Additionally, DGUN[33]'s unfolding strategy overlooks complex hazy conditions in real-world scenarios. Our CORUN addresses this with CPMM modules and a proposed global coherence loss. Replacing DGUN[33]'s dehazing model with ours and integrating our unfolding framework into DGUN[33] both improve performance.\n- Mean-teacher, commonly used in high-level vision tasks, commonly utilizes uncertainty to constrain pseudo-labels which isn't suitable for low-level vision tasks. Some attempts have been made to adapt it for low-level tasks using data augmentation, but these still face issues with erroneous pseudo-labels and overfitting. Colabator improves this by using CLIP and NR-IQA to weight pseudo-label regions, guiding the student network to focus on high-quality areas. It also evaluates pseudo-label quality globally to refine the optimal label pool iteratively, enhancing performance over Mean-teacher. Comparison results can be found in **Table A10**.\n\n- Totally, the major difference and technical novelty in comparison to the combination of existing methods above is:\n    - From a mathematical optimization perspective, we jointly model the transmission map and image scene based on the atmospheric scattering model, creating a novel optimization function that effectively handles complex real-world haze scenarios.\n    - Our unfolding network, CORUN, is tailored to this optimization formula, integrating scene and haze feature commonalities. Using the proximal gradient descent algorithm, CORUN reconstructs high-quality dehazed images progressively from coarse to fine.\n    - Our Colabator architecture is plug-and-play, designed for low-level vision tasks like real-world image dehazing. It evaluates pseudo-labels globally and locally, helping the student network learn from high-quality regions while minimizing errors. It also dynamically maintains an optimal label pool to ensure pseudo-labels stay globally optimal.\n- Therefore, our work is not a mere assembly of existing methods but an effective framework proposed specifically based on the characteristics and challenges of real-world image dehazing.\n\n**W2: Detailed ablation study**\n\nIn the paper, we presented Colabator, along with ablation experiments on the Optimal Label Pool, Trusted Weights, Iterative Mean-Teacher, and the number of layers in the deep unfolding network. **We have supplemented additional ablation experiments in** **Global Rebuttal‘s pdf supplement**, which will be included in the supplementary materials of the final version.\n\nThe results of our CORUN and Colabator on various datasets, data processing methods, and types of Image Quality Assessment (IQA) are presented in **Tables A2, A4, A6, A17**. **Tables A3, A14** demonstrate the generalizability of Colabator across different tasks and networks. **Tables A1, A12** detail the ablation studies of CORUN’s components, while **Table A7** shows the ablation of loss functions. **Table A8** assesses the robustness of DA-CLIP. **Tables A9, A11, A13, A15, A16** demonstrate the ablations of Colabator’s components and functions.\n\n[64]Mondal et al., CVPRW, 2018\n\n[65]Im et al., CIKM, 2023\n\n[66]Ren et al., ECCV, 2016\n\n[67]Zhang et al., TCSVT, 2019", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RFOz2EHNvj", "reviewer": "Reviewer_h5fd", "summary": "This paper focuses on challenging real-world image dehazing problem. The authors develop their network based on unfolding network, while leveraging cooperative proximal mapping modules to facilitate the estimation of transmission map and image content. In addition, the authors propose an interesting teacher network, named as Colabator, with newly developed quality assessment metrics to better provide the high-quality pseudo-labels for dehazing. The subjective metrics on different benchmark, and the user study with downstream tasks evaluation demonstrate the effectiveness of this work.", "questions": "First of all, please address my concerns listed in the weakness part. I have some additional questions:\n\n1.\tAbout density loss: similar as the problems in the second point of weakness, how does this loss function formulate?\n2.\tIn the fine-tuning phase, as the student network is optimized using strong augmented data, is the performance gain coming from this augmentation strategy?\n3.\tWhy partition images before quality assessment? How about using full-resolution images as input?\n\nI hope to see the answers and will adjust my rating accordingly.", "limitations": "The authors have stated the limitations of their article in the supplementary materials.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe Colabator devised by authors sound reasonable, it also provides some insights on training scheme design for restoration problems where paired-images data are hard to collect.\n2.\tThe CPMM modules further improve the representative abilities of unfolding networks to model the gradient descent process of image dehazing problems.\n3.\tThe experimental results give better subjective qualities of the proposed network compared to other methods. Moreover, the results of user study and downstream tasks confirm this point. Judging by myself, I also agree that the authors’ method generates more natural restored images with minimal color shift and the best quality.", "weaknesses": "1.\tThe derivation of equation (7) and (10) are unclear. The authors may need to provide some basic theories before stating these equations.\n2.\tThe details of using DA-CLIP as haze density evaluator is unclear, the statement in line 164 indicates that the authors use a fixed text feature, which is also missing in the article. Moreover, the reason behind the patch partition in line 165 is not clearly stated, the information about patch size N cannot be found in the article.\n3.\tThe article lacks ablation studies about CPMM modules. I wonder how network will performance if removing all CPMM modules.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments. If not specifically stated, all experiments are conducted on the real-world image dehazing task with *RTTS* for space limitation.\n\n**W1: Unclear derivation of equation**\n\nFor space limitation, we only show the detailed derivation of Eq. (7) and the Eq. (10) can be derived similarly. \n\nHaving gotten Eqs. (5) and (6), the solution of $\\hat{\\mathbf{T}}$ can be formulated according to the proximal gradient algorithm: \n\n$$\n\\mathcal{T}(\\hat{\\mathbf{J}}\\_{k-1}, \\mathbf{T}\\_{k-1})=\\frac{1}{2}\\| \\mathbf{P}-\\hat{\\mathbf{J}}\\_{k-1}\\cdot \\hat{\\mathbf{T}}+\\hat{\\mathbf{T}}-\\mathbf{I}\\|^2_2+\\frac{\\lambda_k}{2}\\|\\hat{\\mathbf{T}}-\\mathbf{T}\\_{k-1}\\|^2_2.\n$$\n\nThen we obtain the partial derivative:\n\n$$\n\\partial\\_{\\hat{\\mathbf{T}}}\\mathcal{T}(\\hat{\\mathbf{J}}\\_{k-1}, \\mathbf{T}\\_{k-1})=(\\mathbf{I}-\\hat{\\mathbf{J}}\\_{k-1})^T(\\mathbf{P}-\\hat{\\mathbf{J}}\\_{k-1}\\cdot \\hat{\\mathbf{T}}+\\hat{\\mathbf{T}}-\\mathbf{I})+ \\lambda\\_k(\\hat{\\mathbf{T}}-\\mathbf{T}\\_{k-1}).\n$$\n\nLet the partial derivative be equal to zero, we archieve the closed-form solution for $\\hat{\\mathbf{T}}$ in Eq. (7).\n\nSimilarly, the solution of $\\hat{\\mathbf{J}}$ can be formulated as\n\n$$\n\\mathcal{J}(\\hat{\\mathbf{T}}\\_k, \\mathbf{J}\\_{k-1})=\\frac{1}{2}\\| \\mathbf{P}-\\hat{\\mathbf{J}}\\cdot \\hat{\\mathbf{T}}\\_k+\\hat{\\mathbf{T}}\\_k-\\mathbf{I}\\|^2\\_2+\\frac{\\mu\\_k}{2}\\|\\hat{\\mathbf{J}}-\\mathbf{J}\\_{k-1}\\|^2\\_2.\n$$\n\nThe corresponding partial derivative is\n\n$$\n\\partial\\_{\\hat{\\mathbf{J}}}\\mathcal{J}(\\hat{\\mathbf{T}}\\_k, \\mathbf{J}\\_{k-1})=-\\hat{\\mathbf{T}}\\_k^T(\\mathbf{P}-\\hat{\\mathbf{J}}\\cdot \\hat{\\mathbf{T}}\\_k+\\hat{\\mathbf{T}}\\_k-\\mathbf{I})+\\mu\\_k(\\hat{\\mathbf{J}}-\\mathbf{J}\\_{k-1}).\n$$\n\nThe closed-form solution for $\\hat{\\mathbf{J}}$ is presented in Eq. (7) when let the partial derivative be equal to zero.\n\n\n**W2 & Q1 & Q3:  Details of using DA-CLIP and reasons for partition**\n\nOur method for using DA-CLIP as a haze density estimator involves calculating the cosine similarity between the encoded input image and the encoded input text. The specific formula is\n\n$$\n\\mathcal{D}(\\mathbf{S}^{R}\\_{\\widetilde{HQ}})=\\frac{Enc\\_{image}(\\mathbf{S}^{R}\\_{\\widetilde{HQ}})}{\\|Enc\\_{image}(\\mathbf{S}^{R}\\_{\\widetilde{HQ}})\\|}\\cdot (\\frac{Enc\\_{text}(\\text{Text})}{\\|Enc\\_{text}(\\text{Text})\\|})^\\top\n$$\n\nThe text we used is \"hazy\" from the DA-CLIP provided in the text list. \n\n**(Q1)** As Density Loss, we normalize the result of this formula from 0 to 1, the higher result means higher density.\n\n**(W2)** As Haze Density Evaluator for Colabator, the $norm$ in eq. (12) means $1 - normalized(\\mathcal{D}(\\mathbf{S}^{R}\\_{\\widetilde{HQ}}))$ which has the same value as (1 - density loss), the higher result means lower density.\n\n**(Q3)** We assess the image both globally and locally. We use global assessment to determine whether the overall quality of the image has improved relative to the images in the existing optimal label pool, updating the pseudo-labels. We also use block-based processing to obtain independent haze density and image quality scores for each region of the image. This allows us to assign appropriate confidence weights to the pseudo-labels of each block. This approach enables our network to maximize the use of high-quality dehazed parts of the image while avoiding being misled by poorly dehazed or low-quality parts. By focusing on both global and local aspects of the image, our method achieves better performance.\n\n||NIMA↑|BRISQUE↓|FADE↓|\n|---|---|---|---|\n|Only Full|5.229|13.099|0.803|\n|Partition+Full(Ours)|5.315|11.956|0.751|\n\n**(W2)** The patch size N is set to 32 to balance performance and efficiency. We will add this.\n\n\n**W3: Ablation study on CPMM**\n\nAfter removing all CPMM modules, our method retained only 8 learnable parameters. As shown in the table, removing CPMM  brings a significant performance decline both in the FADE metric, which measures haze removal capability, and the NIMA and BRISQUE metrics, which measure the quality of generated results. However, compared to the original hazy images, the CPMM-removed CORUN still possesses a certain haze removal capability.\n\n||NIMA↑|BRISQUE↓|FADE↓|\n|---|---|---|---|\n|Hazy(Input)|4.483|36.642|2.484|\n|w/o CPMM|4.836|38.197|1.362|\n|w/ CPMM(Ours CORUN+)|5.315|11.956|0.751|\n\nIn the **Figure.A3**, we provide a case of generated result examples. The images include, respectively, the input image, the result with the CPMM module, the result without the CPMM module, and the result without the CPMM module but with manually increased exposure. From our analysis, we can conclude that removing the CPMM module severely affects the flexibility of CORUN, causing the generated results to lose a significant amount of detail, especially in dark areas, and leading to a substantial decrease in haze removal capability.\n\n\n**Q2: Ablation study on strong data augmentation before student network**\n\nWe conducted ablation experiments on the data augmentation strategies used during the fine-tuning phase. In the table below, we provide the quantitative results obtained after disabling strong data augmentation for the student network input. From the results, it can be seen that the strong data augmentation strategy for the student network indeed brings some performance gains, but even without strong data augmentation, Colabator still provides a considerable performance improvement to the network. Therefore, we can conclude that the performance gain during the fine-tuning phase does not entirely come from the strong data augmentation strategy.\n\n||NIMA↑|BRISQUE↓|FADE↓|\n|---|---|---|---|\n|w/o Colabator|4.856|16.541|1.091|\n|w/o Strong aug.|5.084|12.671|0.813|\n|w/ Strong aug. (Ours CORUN+)|5.315|11.956|0.751|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1ptZjnyagO", "reviewer": "Reviewer_wF18", "summary": "This paper posits that real-world image dehazing is particularly challenging due to the intricacies of accurately modeling haze distributions and the limited availability of paired real-world data. Traditional and deep learning-based methods struggle to address the complexities of real haze, often resulting in color distortion and suboptimal outcomes. To tackle these issues, this paper introduces the CORUN model, which integrates atmospheric scattering and image scenes to incorporate physical information into deep networks. Furthermore, it proposes the Coherence-based Label Generator to produce high-quality pseudo labels for network training, thereby improving the network's generalization in haze removal.", "questions": "1. This paper introduce Colabator, a plug-and-play coherence label generation method designed to fine-tune models pre-trained on synthetic datasets using real degraded images, thereby achieving better real-world image processing results. Can the authors provide additional results of this framework's fine-tuning on various tasks to support this claim?\n2. I noticed that the authors assign weights to the local quality of pseudo-labels using a weighted combination of CLIP scores and NR-IQA scores, instead of the more common approach of using their dot product. Can the authors provide experimental results to demonstrate the effectiveness of this choice?\n3. Regarding the choice of NR-IQA, why did this paper select MUSIQ over other methods? What are the reasons behind this decision?\n4. There is a small portion of similar or identical image content between the RTTS and URHI subsets of the RESIDE dataset. Did the authors notice this issue during their experiments? Have they addressed this issue, or did they directly use the full URHI subset for fine-tuning? If the authors used the entire URHI subset for fine-tuning, have they considered the model's performance on the RTTS subset after removing these similar images?", "limitations": "Regarding limitations, the proposed method partially addresses previous issues such as the lack of paired real-world data and the difficulty in modeling complex real-world haze distributions. Section A.1 \"Limitations and Future Work\" in the appendix discusses the current work's limitations and potential future research directions to address these unresolved issues. As for broader impacts, Section A.2 in the appendix provides a thorough discussion. As stated in the paper, this work has no foreseeable negative impacts on the field or society and offers substantial positive contributions.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method, CORUN (COopeRative Unfolding Network), integrates physical information into deep networks to overcome the limitations of existing deep learning-based dehazing methods. It cooperatively models atmospheric scattering and image scenes by incorporating Transmission and Scene Gradient Descent Modules at each stage, effectively restoring haze-contaminated details. \n2. CORUN is constructed on the basis of the atmospheric scattering model using a proximal gradient descent method within a deep unfolding network, which provides strong interpretability to the method.\n3. The proposed Colabator framework significantly improves the performance of models pretrained on synthetic datasets in real-world scenarios within a limited number of iterations by fine-tuning with real degraded images. This plug-and-play framework incurs no additional computational cost during deployment.\n4. Experiments demonstrate the excellent performance of the proposed CORUN model and Colabator framework in real-world dehazing tasks. Both quantitative and visual results validate the network's capability to model real-world haze and restore real-world scenes effectively.", "weaknesses": "1. The paper contains some inconsistencies in the use of symbols. While images are generally denoted by P, in the loss functions (15, 16, 17), the authors use I to represent images. Although the text explains the use of these symbols, it can still impact readability. Additionally, in line 172, the pseudo-label symbol is misspelled and should be corrected to P^{R}\\_{Pse\\_{i}}. \n2. In line 175, the symbols contain unnecessary tildes, which should be removed. The symbols in Figure 2 should also be consistent with those used in the formulas.\n\n3. Equations (5) and (7) do not address the consistency of the number of channels between the Transmission Map and the image. Although this is illustrated in Figure 2, it still needs to be clearly stated in the equations to facilitate readability.\n\n4. This paper adequately demonstrates its method's performance in real-world dehazing tasks through thorough experiments in both visualization and quantitative results. However, the provided ablation studies are insufficient. The authors should conduct more comprehensive ablation studies and experiments on different datasets to further validate the effectiveness of their method and its components.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments. If not specifically stated, all experiments are conducted on the real-world image dehazing task with *RTTS* for space limitation.\n\n**W1 & W2: Typo and Misuse of symbols.**\n\nWe apologize for the errors in our writing and appreciate you pointing out these mistakes in our article. We will correct the issues you mentioned in the final version to improve the readability of the paper.\n\n**W3: Lack of clarity in the formulas**\n\nWe have revised Eq. (5) and Eq. (7) to clearly express the handling of information across different channels. The updated Eq. (5) is as follows:\n\n$$\n\\mathbf{T}\\_k=\\underset{\\mathbf{T}}{\\arg \\min }\\frac{1}{2}\\sum\\_{c\\in\\{R,G,B\\}} \\| \\mathbf{P}^{c} - \\hat{\\mathbf{J}}\\_{k-1}^{c}\\cdot \\mathbf{T} + \\mathbf{T} - \\mathbf{I} \\|^{2}\\_{2}+\\phi(\\mathbf{T}).\n$$\n\nThe updated Eq. (7) is as follows:\n\n$$\n\\hat{\\mathbf{T}}\\_k=\\sum\\_{c\\in\\{R,G,B\\}}(\\mathbf{I}-\\hat{\\mathbf{J}}\\_{k-1}^{c}+\\frac{\\lambda\\_{k}}{(\\mathbf{I}-\\hat{\\mathbf{J}}\\_{k-1}^{c})^{\\top}})^{-1} \\cdot (\\mathbf{I}-\\mathbf{P}^{c}\\frac{\\lambda\\_{k} \\mathbf{T}\\_{k-1}}{(\\mathbf{I} -\\hat{\\mathbf{J}}\\_{k-1}^{c})^{\\top}}).\n$$\n\nWe will update this content in the final version.\n\n**W4: More Ablation studies.**\n\nWe tested our CORUN method on the O-HAZE and I-HAZE datasets, using PSNR and SSIM for evaluation. The results, presented in **Table A4**, illustrate the robustness of CORUN across dehazing datasets.\n\n**Q1: Plug-in-play experiment**\n\nThank you for your suggestion. According to our experiments, Colabator exhibits plug-and-play characteristics and brings enhancements across different networks and real-world tasks. In **Figure. 1** and **Table. 2** of the paper, we validated the effectiveness of Colabator on deep unfolding networks and real-world image dehazing tasks. Here, we integrate our Colabator with more cutting-edge image restoration methods and fine-tune them 5000 iterations in *underwater image enhancement* and *Real robotic laparoscopic hysterectomy desmoke tasks after pretraining*. \n\nIn the UIE task, all methods pretrained on the *UIEB* dataset, finetuned and tested on the *EUVP* trainset’s low-quality part and testset, we find the performance average gain by **11.3%** in SwinIR [68], **20.6%** by Restormer[69], **15.4%** by GRL[70], and **13.0%** by AST[71].\n\n||NIMA↑|BRISQUE↓|FADE↓|\n|---|---|---|---|\n|SwinIR[68]|3.392|31.775|0.536|\n|SwinIR+Colabator|3.650|27.247|0.472|\n|Restormer[69]|3.981|25.688|0.482|\n|Restormer+Colabator|4.185|16.489|0.381|\n|GRL[70]|3.816|25.795|0.506|\n|GRL+Colabator|3.925|18.854|0.423|\n|AST[71]|4.073|22.772|0.473|\n|AST+Colabator|4.186|16.353|0.435|\n\nIn the Desmoke task, all methods pretrained on *Desmoke-LAP* with synthetic paired data, finetuned and tested on *Desmoke-LAP*’s real unpaired data, we find the performance average gain by **7.9%** in SwinIR [68], **4.9%** by Restormer[69], **7.1%** by GRL[70], and **6.4%** by AST[71].\n\n||NIMA↑|BRISQUE↓|FADE↓|\n|---|---|---|---|\n|SwinIR[68]|3.056|40.668|0.655|\n|SwinIR+Colabator|3.317|35.519|0.639|\n|Restormer[69]|3.554|33.356|0.628|\n|Restormer+Colabator|3.738|30.933|0.614|\n|GRL[70]|3.503|31.682|0.601|\n|GRL+Colabator|3.716|27.663|0.585|\n|AST[71]|3.808|29.735|0.563|\n|AST+Colabator|3.952|26.483|0.538|\n\nThe gains brought by our Colabator are significant, demonstrating that Colabator possesses plug-and-play capabilities across different networks and tasks.\n\n**Q2: Ablation study on trusted weights calculation**\n\nWe apologize for not explicitly explaining the reason for using weighted summation in the text. We opted for summation rather than multiplication to calculate the Trusted weight, primarily because both the CLIP score and NR-IQAs score need to contribute their respective effects to the weight distribution. This approach also allows for the adjustment of different score ratios for different tasks to achieve more reasonable and better results. If weighted multiplication were used, it would be impossible to control the importance between different components. As shown in the table, we experimentally demonstrated that using the score summation method to calculate the Trusted weight, compared to using score multiplication, results in better final outcomes.\n\n||NIMA↑|BRISQUE↓|FADE↓|\n|---|---|---|---|\n|Product|5.487|14.949|0.807|\n|Summation (Ours)|5.315|11.956|0.751|\n\n**Q3: Ablation study on NR-IQA**\n\nThank you for your question. In fact, we have previously tested many NRIQA methods and found that MUSIQ has the best alignment with the human visual system. MUSIQ evaluates images at multiple scales, comprehensively considering both the technical and aesthetic quality of images, making it broadly applicable to various image quality assessment tasks. We conducted ablation experiments using different NR-IQA methods, and the results show that using MUSIQ in the scenarios presented in this paper can achieve better overall performance.\n\n||NIMA↑|BRISQUE↓|FADE↓|\n|---|---|---|---|\n|NIMA|5.337|12.242|0.825|\n|BRISQUE|5.245|11.652|0.813|\n|NIQE|5.122|12.900|0.835|\n|MUSIQ(Ours CORUN+)|5.315|11.956|0.751|\n\n**Q4: Similar portion in RTTS and URHI**\n\nThank you for your question. We used the URHI dataset following the common setting as referenced in works like PSD, so the issue does not cause unfairness in the comparative experiments. However, this concern is worth considering. Therefore, we attempted to remove images from the URHI dataset that overlap more than 60% with the RTTS dataset, ultimately retaining 3728 images for the second phase of fine-tuning CORUN. The final results are shown below. From the results, it can be seen that our method still achieves excellent performance even after using the cleaned dataset.\n\nWe found that the NIMA score actually increased, which we believe is due to the removal of some low-quality data during the data cleaning process.\n\n||NIMA↑|BRISQUE↓|FADE↓|\n|---|---|---|---|\n|URHI-|5.348|12.784|0.832|\n|URHI|5.315|11.956|0.751|\n\n[68]SwinIR, ICCVW, 2021\n\n[69]Restormer, CVPR, 2022\n\n[70]GRL, CVPR, 2023\n\n[71]AST, CVPR, 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We extend our sincere gratitude to all the reviewers (**R1**-**Udmv**, **R2**-**ivCJ**, **R3**-**h5fd**, and **R4**-**wF18**) for their insightful and considerate reviews, which help us to emphasize the contributions of our approach. We are pleased to hear that the reviewers approved the novelty of our work, as well as the commendable performance (**R1, R2, R3, R4**).

We are delighted to see reviewers confirm our contributions to the field of real-world image dehazing. These encompass our novel deep unfolding dehazing method, CORUN, and the ingenious plug-and-play pseudo-label generation and fine-tuning framework, Colabator.

In direct response to your thoughtful comments, we have methodically addressed each point in our individual responses, and we provide a summary here:

- We corrected writing errors, revised certain formulations, and added information about the theory, formulas, and derivation process in this paper to enhance clarity.
- We compared our method with more SOTA methods on additional datasets to underscore our superiority in performance.
- We added experiments to verify the generalizability of our plug-and-play Colabator framework and the advancement of CORUN.
- We provided more ablation study results to demonstrate the robustness and effectiveness of the various modules in CORUN and Colabator.

Thanks again for all of your valuable suggestions. We will update the paper accordingly and release our code for community study. We appreciate the reviewers' time to check our response and **hope to further discuss with the reviewers whether the concerns have been addressed or not**. If the reviewers still have any unclear parts about our work, please let us know.

```
Due to space limitations, images and most of our supplementary experiments are stored in the attached PDF. Please download this PDF for more information. Thank you!
```

Meta-review: This paper was reviewed by four experts. The reviewers generally appreciated the central concept and were persuaded by the effectiveness of the proposed technique. Initially, the review scores were varied and included several concerns. However, during the rebuttal phase, the authors effectively addressed most of these issues. As a result, reviewers who had previously given lower scores revised their assessments to favor acceptance. Given the unanimous recommendation for acceptance, AC believes the paper is a clear  acceptance case.
Decision: Accept (spotlight)