Conditional Controllable Image Fusion
Authors: Bing Cao
Xingxin Xu
Pengfei Zhu
Qilong Wang
Qinghua Hu
Venue: NeurIPS 2024 poster (2024)
Keywords: Image Fusion; DDPM; Conditional Bank; Controllable Fusion
Primary area: machine_vision

Abstract: Image fusion aims to integrate complementary information from multiple input images acquired through various sources to synthesize a new fused image. Existing methods usually employ distinct constraint designs tailored to specific scenes, forming fixed fusion paradigms. However, this data-driven fusion approach is challenging to deploy in varying scenarios, especially in rapidly changing environments. To address this issue, we propose a conditional controllable fusion (CCF) framework for general image fusion tasks without specific training. Due to the dynamic differences of different samples, our CCF employs specific fusion constraints for each individual in practice. Given the powerful generative capabilities of the denoising diffusion model, we first inject the specific constraints into the pre-trained DDPM as adaptive fusion conditions. The appropriate conditions are dynamically selected to ensure the fusion process remains responsive to the specific requirements in each reverse diffusion stage. Thus, CCF enables conditionally calibrating the fused images step by step. Extensive experiments validate our effectiveness in general fusion tasks across diverse scenarios against the competing methods without additional training. The code is publicly available.

Conditional Controllable Image Fusion
Bing Cao 1.2
Xingxin Xu² Pengfei Zhu
Qilong Wang'
Qinghua Hu
'College of Intelligence and Computing, Tianjin University, Tianjin, China
2 State Key Laboratory of Integrated Services Networks, Xidian University, Xi'an, China
"School of New Media and Communication, Tianjin University, Tianjin, China {caobing, xuxingxin, zhupengfei, qlwang, huqinghua)et,ju waw.ghuuaattu.eeuu.... edu. cn
Abstract
Image fusion aims to integrate complementary information from multiple input images acquired through various sources to synthesize a new fused image. Existing methods usually employ distinct constraint designs tailored specific scenes, forming fixed fusion paradigms. However, this data-driven fusion approach challenging to deploy in varying scenarios, especially in rapidly changing environ- ments. To address this issue, we propose conditional controllable fusion (CCF) framework for general image fusion tasks without specific training. Due the dynamic differences of different samples, our CCF employs specific fusion con- straints for each individual in practice. Given the powerful generative capabilities of the denoising diffusion model, we first inject the specific constraints into the pre-trained DDPM as adaptive fusion conditions. The appropriate conditions are dynamically selected to ensure the fusion process remains responsive to the specific requirements in each reverse diffusion stage. Thus, CCF enables conditionally calibrating the fused images step by step. Extensive experiments validate our effectiveness in general fusion tasks across diverse scenarios against the competing methods without additional training. The code is publicly available
1 Introduction
Image fusion aims at integrating complementary information from multi-source images, fusing composite image containing richer details [I] It has been applied in various scenarios that single image contains incomplete information, such as multi-modal fusion (MMF) 2 3 multi- exposure fusion (MEF) 4 (), multi-focus fusion (MFF) [6], and remote sensing fusion (2) The fused image inherits the strengths of both modalities, resulting in composite with enhanced visual effects [7]. These fusion tasks have diverse downstream applications in computer vision, including object detection [8 semantic segmentation [|] and medical diagnosis [3] because the comprehensive representation of images with multi-scene information contributes to the improved performance of applications.
Recently, numerous image fusion methods 14 have been proposed, such traditional fusion
methods 18), CNN-based fusion methods 19 20 and GAN-based methods [21]. While these
methods produce acceptable fused images in certain scenarios, they also accompanied by sig- nificant drawbacks and limitations: (i) They are often tailored for specific scenarios or individual tasks, limiting their adaptability across diverse applications; (ii) These methods necessitate training and consume substantial computational resources, posing limitations in terms of time and resource requirements. Lately, denoising diffusion probabilistic models (DDPM) have emerged as an iterative
"Corresoonding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Enhanced conditions dynamic injection
Low-frequency
Random
Detail
High- frequency
Selection frequency
Spatial Frequency
0.6
Edge
C Content
Edge Intensity
7
Standard Deviation
SSIM
MSE
-0.0
The conditions selection statistics during the sampling process of the LLVIP dataset. The distinct process of sampling has different favor of the conditions. The crucial role that diverse conditions play in controlling various image generation processes. Throughout the diffusion sampling, different conditions are dynamically selected best suit the generation requirement each stage.
Figure 1:
generation framework, showcasing impressive capabilities in unconditional generation. Inspiringly, numerous researchers explored its controllable aspects. ILRR [22] proposed iterative latent variable refinement with reference image control image translation. Some recent works (23 24 employed the diffusion model for image fusion, which fuses images fixed fusion paradigms (fixed fusion conditions) by using its inherent reconstruction capacity. However, these approaches are not qualified for
fusion with dynamic conditions. At present, general image fusion with controllable diffusion models is still challenging problem, warranting further exploration.
In this paper, we propose a diffusion- based controllable conditional image fusion (CCF) framework, which controls the fusion process by adaptively selecting optimization conditions. We construct condition bank of generally used conditions, categorizing them into basic, enhanced, and task-specific conditions. CCF dynamically assigns fusion conditions from the condition bank and continuously injects them into the sampling process of diffusion. To enable flexible integrated conditions, we further propose sampling adaptive condition selection (SCS) mechanism that tailors condition selection at different denoising steps. The iterative refinements pre-trained diffusion model without additional training. It is worth noting that the estimated fused images are conditionally controllable during the iterative denoising process. The diffusion process seamlessly integrates these conditions during the sampling process, decreasing potential impacts. As illustrated in Fig. [ the generation process emphasizes different aspects at various sampling steps. In the initial stages, the condition selection influenced by random noise, resulting in random selection. During the intermediate stages, there is a shift towards content components. the final stage, the emphasis moves to generating and selecting texture details. These various conditional factors contribute to different aspects of fusion results and demonstrate the necessity and effectiveness of introducing specific conditions in different stages. To the best of our knowledge, we for the first time propose a conditional controllable framework for image fusion. The main contributions are summarized follows:
sampling-adaptive
the sampling are based on the
We propose a pioneering conditional controllable image fusion (CCF) framework with condition bank, achieving controllability various image fusion scenarios and facilitating the capability of dynamic controllable image fusion.
We propose sampling-adaptive condition selection mechanism to subtly integrate the condition bank into denoising steps, allowing adaptive condition selection on the fly without additional training and ensuring the dynamic adaptability of the fusion process.
Extensive experiments on various fusion tasks have confirmed our superior fusion perfor- mance against the competing methods. Furthermore, our approach qualifies for interactive manipulation of the fusion results, demonstrating applicability and efficacy.

---

2
Related Work
Image fusion focuses on producing a unified image that amalgamates complementary information sourced from multiple source images 25
Specialized. Focus specialized tasks such as VIF, several early approaches 26 28
relied on
CNNs to address challenges across various scenarios. GTF (29] defined the objective of image fusion as preserving both intensity information in infrared images and gradient information in visible images. Besides that, researchers started artificially incorporating prior knowledge to aid in the fusion process.
CDDFuse [30] introduced the concept of high and low-frequency decomposition with dual-branch as prior information. Diverging from approaches tailored to single scenarios, numerous methods are now exploring the development of unified fusion framework. DDFM [24] represents the pioneering training-fie ffree method that employs a diffusion model for multi-modal image fusion.
Generalized. Not limited to specialized applications, researchers aim extend its use to generalized tasks. U2Fusion [1] introduced a unified framework capable of adaptively preserving information and facilitating joint training across various task scenarios. Additionally, SwinFusion [32] proposed a cross-domain distance learning method that has been extended to form a unified framework encompassing diverse task scenarios. Defusion [3] employs self-supervised learning techniques to decompose images and subsequently adaptively fuse them. TC-MoA [64] proposed a novel lask-customized mixture adapters for generating image fusion with a unified model, enabling adaptive prompting for various fusion tasks.
Nevertheless, these methods cannot control image fusion for adaptation to different scenarios. There- fore, we propose a method that enables image control, manipulating the fused image through existing conditions our condition bank.
3 Preliminary
Denosing diffusion probabilistic models (DDPM) is a class of likelihood- based models that shows remarkable performance (35] with stable training objective unconditional image generation. The diffusion process entails incrementally introducing Gaussian noise to the data until it reaches a state of random noise. For a clean sample 2g q(za) each step within the diffusion process constitutes Markov Chain, encompassing total of T steps, relying on the data derived from the preceding step. Gaussian noise is added as follows:
(( = (xx,; 111,,11,,,,11))
where {8,) is the variance schedule of each diffusion step which is fixed and predefined. The generative process learns the inverse of the DDPM forward (diffusion) process, sampling from distribution by reversing gradual denoising process. We can directly sample x; at any step based on the original data q(z)/20) and via the reparameterization, it can be redefined:
Ft V + vI C.CC
where defined a; := 1- 3r and ā II i= O; The diffusion process introduces noise to the data, whereas the inverse process represents a denoising procedure called sampling. In particular, stats with noise ør ~ (0, I), the diffusion model learns to produce slightly less-noisy sample 2T-1, the process can be formulate by:
po (xx))= = 11 (), Eo(x,, ())-
Utilizing the properties of Markov chains, decomposing μs and Le, the process of generation is expressed as:
ae co t)) oz(())
F- (xr
Va
VT- 0z
where, ())) = Et) (1 11--0)01----11 Eg signifies the output of a neural network, commonly a
1-Gs.
U-Net. This neural network predicts the noise Eg at each step, which is utilized for the denoising procedure. can be observed that variance is fixed quantity, because of diffusion process parameters being constant, whereas the mean is a function dependent on vo and zt. However, the stochastic process poses challenges in controlling the generative process.
3

---

Conditions Embedding
Pt
O uuput
Sampling
Top-K K Routing
Detoction
MSE
Eittmyy
Spucil Iroquanyy
SSIM
Classification
High-trequnncy
Gate
Standar Destation
Segmentation
Low Frogurccy
Crses Entryy
Mitral Isformtion
Asonge
Depth
Edge
Conditions
Enhanced Conditions
Task specific
Input
Condition Bank
Figure 2: Illustrates the pipeline of the proposed CCF, The framework comprises two components: sampling process utilizing pre-trained DDPM and condition bank with SCS.
4 Method
Leveraging the reconstruction capability of unconditional DDPM, we introduced new controllable conditional image fusion (CCF) framework. Our approach accomplishes dynamically controllable image fusion progressive condition embedding. particular, we introduced a condition bank that regulates the incorporation of fusion information using conditions. It allows for combining the dynamic selection of multiple conditions to achieve aamplin--daptivv fusion effects. As shown in CCF framework in detail with visible-infrared image fusion (VIF). The goal is R xW x N and infrared E XW N images, where H, W and N denote height, width, and channel numbers, respectively.
Fig. 2 we illustrate to generate a fused image f E R # xN from visible v
4.1 Controllable Conditions
Firstly, we provide the notation for the model formulation. For each sampling instance, a pre-trained DDPM represents unconditional transition po (z))) Our method facilitates the inclusion of conditional during the sampling step unconditional transformation, without no additional training. For this purpose, we sample images from the conditional distribution p(zolc) given condition
po(zolc)=
= /do (o
(5)
(a (2. (0:T))) = p(xT)) II po 1
Each transition (o Te, c) the generative process depends on the condition From the property of the forward process that latent variable Ie can be sampled from To in closed-fone denoised data Tp can be approximated with model prediction cn (xit, t):
(Ir v1 - (eee((2,))
Tope So(
(6)
Von
To compute p(zrlc). we can derive it from the Stochastic Differential Equation (SDE) 36 For brevity, Zoje is abbreviated as Ho, and the expression is given by:
logp(x;) = Ft /
(7)
&e
Classifier Guidance 377 can be intuitively elucidated via the score function, which logarithmically decomposes the conditional generation probability using Bayes' theorem:
V logp(zz e) Vlogp(æ.) + V Vlogp(c/2.).
(8)

---

Infrared
Visible
SwinFusion
DIVFusion
ML Fusion
CDD
DDFM
Text-IF
TC-MoA
Ours(CCF)
Figure 3: Qualitative comparisons of our CCF and the competing methods on VIF fusion task of LLVIP dataset.
Diffusion posterior sampling can yield a more favorable generative trajectory, especially in noisy settings, 38 estimation log p(c]æ)) follows:
v log V log p(c/fo))
Further, demonstrated in
39
express the score function as:
V Vlog(((2..) = V logp(c(ão) fo( AVIIC A(ro)]l2
(10)
Here, A(.) can be linear nonlinear operation. We represent (C--(()))ll öc. Now, the objective to obtain in and incorporate the condition into it. We can minimize dg to regulate the sampling within the diffusion process. In the following section, we will provide detailed explanation of how to build a condition bank and how to select conditions.
4.2 Condition Bank
We empirically construct a condition bank and divide the image constraints into three categories: basic fusion conditions, enhanced fusion conditions, and task-specific fusion conditions. Basic fusion conditions are utilized throughout the entire sampling process, while enhanced fusion conditions dynamically selected. Task-specific fusion conditions manually optional, tailored specific tasks, and may possess unique attributes that can be customized for various task scenarios. All conditions can be part of the enhanced condition set, enabling dynamic selection. The condition bank presented in this paper includes some common conditions, but additional conditions can be explored and utilized in other scenarios.
In the above formulation, each conditional Markov transition with the given condition is shown in Eq. In particular, constructed condition bank that allows to select required conditions [1,,00 .2..., ). subsequently integrating them into the unconditional DDPM for executing conditional image fusion. Let C' represent condition bank comprising series of conditions. The function &c represents the difference between the source images with the given condition. In every sampling step the difference function &z, can be minimized using gradient descent. These conditions help regulate the image information within each modality involved the fusion process.
Basic Conditions. As shown in Fig. basic conditions are essential to select for a basic generation. The basic conditions aim to synthesize foundational fused image, offering an initial, coarse representation. The fused image serves primary fusion output, capturing essential features from the source images, though it may suffer from detail loss or texture blur. Notably, different scenarios may require adjustments to the basic condition, the specific requirements of each fusion task, such as clarity, contrast, and other priorities, can influence its selection. Tailoring the basic condition to align with the unique demands of each task thus ensures an effective fusion process.
Enhanced Conditions. Besides basic conditions, we added enhancement conditions for refining the image generation process. The condition bank contains variety of enhanced conditions, inspired by various Image Quality Assessments (IQA) such as SSIM, and standard dSviationSDD). These conditions can be integrated into the CCF generation process to improve the quality of the generated images. The enhanced conditions can selected with SCS algorithm, allowing different steps the diffusion sampling process to be optimized with different conditions. This targeted approach ensures
5

---

that each phase of the image generation is adapted to the specific requirements of that stage, resulting in higher quality fused images.
The purpose of image fusion is to facilitate various downstream tasks. To meet the specific requirements of these tasks, offer task-specific conditions. These conditions can be manually added to ensure that the fusion process is tailored to suit the downstream applications better. For example, a detection condition can be introduced by utilizing the feature extracted by a detection network F = Dz). The detection condition is formulated as [x) F F(M)l22. where X { and M the set of m modalities. Other task sesk-specfiic conditions can be similarly tailored to optimize the fusion process for different tasks. By integrating these task-specific conditions, the
Tpek-specific Conditions.
image fusion process can be precisely aligned with the demands of various applications. enhancing
the effectiveness and utility of the generated images.
4.3 Campling-adaptiin Condition Selection
During the diffusion sampling process, it is crucial to focus on generating distinct aspects of the image. To address this, we designed an algorithm that dynamically selects the appropriate condition from the condition bank to fit each sampling stage. This selection process can be denoted as Copt = Top,. { GCt) The Copt is the selected condition, and Gate is the gate of selection. We hypothesize that rapidly changing conditions during the sampling process should be prioritized as they indicate greater significance at that generation stage. Inspired by multi-task learning [40], the gate of conditions can be calculated using the following formula:
Gate = [W1, .Wer] (t) wr(t - 1) Vu,
(11)
The wi(t - 1) represents the condition gradient from the previous step, and the V7,, is calculated as:
V43-
G (1) [[] x az(())1.
(12)
where, Gw (1)(( (I) represents the condition gradient step and the value can be calculated using a gradient descent algorithm in minimize dz.- The or:()) is defined as:
L;
a; (t) =
8
(13)
E[L.]
where, L, = L.1
(, is hyper-parameter. By incorporating this SCS algorithm, we can efficiently choose the most relevant condition for each step in the diffusion process, thereby enhancing the quality of the conditional image fusion.
5 Experiments
Datasets. We conduct experiments in three image fusion scenarios: multi-modal, multi-focus, and multi-exposure image fusion. For multi-modal image fusion task, conducted experiments on the LLVIP [41] dataset and referred to the test set outlined in Zhu et al. [34]. For MEF and MFF, our testing procedure followed the test setting in MFFW dataset 42 and MEFB dataset 43], respectively. Additionally, we test our method on the TNO dataset and Harvard medical dataset to assess our method's performance within the multi-modal fusion domain, detailed in App. B and H
Implementation Details. Our method utilizes a pre-trained diffusion model as our foundational model [44). This model was directly applied without any subsequent fine-tuning for specific task requirements during our experiments. The experiments are conducted on Huawei Atlas 800 Training Server with CANN and NVIDIA RTX 3090 GPU. Experimental settings are shown in App- A
Metries. We evaluated the fusion results in both quantitative and qualitative. Qualitative evaluation primarily hinges on subjective visual assessments conducted by individuals. We expect that the fused image will exhibit rich texture details and abundant color saturability. Objective evaluation primarily focuses on measuring the quality assessments of individual fused images and their deviations from the source images. For different task scenarios, the different evaluation metrics used, specifically, we employ six metrics including Structural Similarity (SSIM). Mean squared error (MSE), correlation coefficient (CC), peak signal-to-noise ratio (PSNR), modified fusion artifacts measure (Nabf). In the MFF and MEF tasks, considering the different task scenarios, we employ standard deviation (SD), average gradient (AG), spatial frequency (SF), and sum of the correlations of differences (SCD) for evaluation metrics.
Evaluation

---

Sourte I
Source
U2Fusion
DeFusion
Overspoorre
Und Undausposre
UiFisson
Defusion
DDFM
TC-MoA
Ours(OCF)
DDFM
TC-Mo
Ours(CCF)
Figure 4: Qualitative comparisons of various methods in MFF task from MFFW dataset.
5.1
Evaluation on Multi-Modal Image Fusion
For multi-modal image fusion, compare method with the state-of-the-arr methods: Swin-
Fusion 32). DIVFusion (45), MUFusion [46], CDDFuse (30, DDFM [24). Text-IF [47), and
TC-MoA +4 and the LLVIP dataset. More datasets and comparison methods are shown in
App. Note that our method, like DDFM, does not require additional tuning.
Quantitative Comparisons. We em- play five quantitative metrics to eval- uate model, shown in Ta- ble Our method demonstrated ex- ceptional performance across various evaluation metrics. On the LLVIP dataset, our method achieved the best results in SSIM, MSE, and CC indica- tors. Specifically, our method outper- formed others in SSIM and CC, with improvements of 0.02 and 0.035 over the second-best results, respectively. Additionally, lower MSE values indi- cate better performance, with our method showing reduction of 362 compared to the second-best methods these metries. This indicates that our method retains more information from the source images. The results show suboptimal performance in Nabf but are close to the best values, indicating the fused image with less noise. Notably, our method does not necessitate turning and holds its ground against methods requiring training. Compared to existing LLM tuning methods, our model performs slightly worse in terms of PSNR. This demonstrates the excellent performance of our model, achieving high performance across almost all indicators in a tuning-free model.
Table I: Comparison with SOTAs in the LLVIP dataset. The
red/blue/green indicates the best, runner-up and third best.
LLVIP Dataset
SSIM
MSEI
CC1
PSNR1
Nabf
SwinFusion
32
0.81
28455
0.668
32.33
0.023
DIVFusion
45
0.82
6450
0.655
21.60
0.044
MUFusion
46
1.10
2069
0,648
31.64
0.030
CDDFuse
1.18
2545
0.670
32.13
0.016
DDFM 244
1.18
2056
0.668
36.10
0.004
Text-IF 47
1.20
2135
0.669
31.97
0.023
TC-MoA 34
1.20
2790
0.666
33.00
0.017
CCF (ours)
1.22
1694
0.705
32.71
0.005
Qualitative Comparisons. Furthermore, the incorporation of the basic condition and enhanced conditions enables effective preservation of the background and texture. This comparison underscores our model's 's efficacy in image fusion, resulting outstanding visual outcomes. As shown in Fig. B our method showcases superior visual quality compared to other approaches. Specifically, our method excels in preserving intricate texture details well lid in low light (Fig. 3 red box). Although TC-MoA and MUFusion approach our method in retaining details, they exhibit visible artifacts, blur, and low contrast contrast-ccarrcteriie characteristies absent in CCF (Fig. 3 green box). CCF exhibits the highest contrast, the clearest details, and the most information content, further highlighting its superiority in preserving texture details. Its excellent detail retention and clear background generation further demonstrate the effectiveness of our proposed method.
Evaluation on Multi-Focus Fusion
For multi-focus image fusion, compare CCF with five general image fusion methods: U2Fusion (3I), DeFusion [33], DDFM (4), Text-IF [47), and TC-MoA [34].
Quantitative Comparisons. We employ four quantitative metrics to evaluate our model, as shown in Table (left). Our method significantly outperforms the comparison methods, achieving SOTA across all metrics. Specifically, the SD is 11.19 higher than the suboptimal value, indicating higher
7

---

Table 2: Comparison with SOTAs. The red/blue/green indicates the best, runner-up and third best.
MFFW
Dataset SF
MEEB Dataset
SD
AG 1
SCD 1
SD 1
AG 1
SF 1
SCD f
U2Fusion
31
67.83
8.08
22.19
2.67
64.88
5.56
18.74
2.05
DeFusion
3
54.75
4.76
12.72
0.50
52.75
4.32
14.12
-0.97
DDFM 24
56.34
4.47
12.21
0.90
67.30
3.82
13.40
2.12
Text-IF
66.27
7.72
21.58
2.27
62.51
4.78
17.26
1.46
TC-MoA
34
57.55
6.95
20.67
1.03
50.27
4.82
15.64
0.42
CCF (ours)
79.02
8.18
24.30
2.78
71.88
5.70
20.28
3.00
contrast and clearer images. The SCD is 0.11 higher than the suboptimal value, suggesting lower error between source images and fused images. The AG and SF also rank first demonstrating the retention of more texture details. These results showcase that our method effectively preserves details from the source images and produces high-quality fused images.
Qualitative Comparisons. As illustrated in Fig. our proposed method demonstrates outstanding visual performance, particularly in preserving intricate details. We have carefully selected specific conditions that allow our approach to effectively handle the blurring caused by multi-focus scenes while retaining the original image's lighting and color information. In comparison, other DDPM- based methods such as DDFM are unable to achieve the same level of effectiveness as approach. In Fig. 4 (red). our method excels preserving the details of the watch hand. The closest result to ours is U2Fusion, but it loses texture and color fidelity, appearing blurry, in Fig. 4 (green). In short, our method performs well in maintaining both color and authentic details.
5.3 Evaluation on Multi-Exposure Fusion
For multi-exposure image fusion. we compare our model with five general image fusion methods, i... U2Fusion [1, DeFusion [33), DDFM 24], Text-1F (7), and TC-MoA 34
Quantitative Comparisons. As demonstrated in Table (right), the MEF index, analogous to the results observed for the MFF task. Notably, the metrics SD, AG, and SF signify the highest image quality, while SCD exhibits the highest correlation. Each of these metrics attained state-of-the-art performance levels, underscoring the efficacy of our approach. This consistent performance across multiple metrics illustrates the robustness and versatility of our method in enhancing image quality and fidelity.
method achieved SOTA
Qualitative Comparisons. Fig. 4 demonstrates the excellent visual performance of our method. Our approach effectively addresses the issue overexposure while preserving crucial details. In contrast, DDFM, which relies on finding the middle value of two images, struggles to maintain texture details. Similarly, Text-IF tends to result in higher average brightness, which can lead to content loss in overexposed scenes. Defusion and TC-MoA exhibit similar visual performance, with more blurred edges compared to our method. In comparison, our method strikes a balance between these challenges, resulting in superior visual fidelity and saturation compared other existing methods.
5.4 Task-specific Conditions
The task-specific conditions can be manually se- lected. In this section, we use the detection con- dition an example. The detection model em- ployed is YOLOv5 (48). trained on the LLVIP dataset. We randomly select 287 images from the LLVIP test dataset to validate our method with the detection condition. Before adding the detection condition, the fused image achieved mAP.5 0.737, mAP.5 .95 0.509, and recall of 0.737. After incorporating the detec- tion condition, the mAP.5 increased by 0.049 to 0.907, mAP.5 .95 increased by 0.054 to 0.563,
tank-ppecific
with task-specifie
and recall significantly improved 0.832. Fig. visualizes several cases detected using YOLOv5.
Figure The visualization of the w/o and with task-specific conditions.

---

Figure 6: Ablation study of condition bank. (a) is infrared image, (b) is visible image, (c) basic condition, (d) is CCF w/o SCS, (e) is CCF.
The fused images with the detection condition exhibit higher confidence (columns and 2) and recall (columns I, and 3). This demonstrates that task-specific conditions can enhance the performance of fused images in downstream tasks. By integrating task-specific conditions, the fused image can be precisely aligned with the demands of various applications, enhancing the overall effectiveness and utility of the generated images. More detail shows in App. F
5.5 Ablation Study
Numerous ablation experiments were conducted to assess the effectiveness of different components of our model. The aforementioned four metrics were utilized to evaluate fusion performance across different experimental configurations. The quantitative results are presented in App.[G]
To validate the effectiveness of the condition bank, we systematically add basic and enhanced conditions individually and then verify the effectiveness of SCS. Fig. 6 Gillustrates gradual variation in performance metrics with the progressive addition of conditions.
In Fig. 6 ()), using only the basic condition results in image fusion. When all enhanced conditions injected together without SCS, the metrics all reduced (Fig. G()),, likely due to the conflict between different conditions, as evidenced by the messy lines and random noise in Fig. d). This demonstrates that injecting conditions unevenly without DCS leads to suboptimal results. After introducing DCS, both the metries and visual results (Fig. e))) are well-balanced, with conditions being injected as needed during the generation process. This indicates the effectiveness of the condition bank and DCS in dynamically and appropriately selecting the conditions.
5.6 Discussion Controllable Fusion
Our method enables the dynamic selection of conditions, allowing for the adaptive customization of conditions for each sample. As illustrated in Fig. [ the statisties show the condition selection at various diffusion sampling stages. Initially, the process is stochastic due to random noise (around to T steps). As denoising progresses, the content of the image starts to be dominant fusion, and conditions like MSE. SSIM, and SD are most frequently chosen (around İT to T steps). As the process continues, and the fusion model tends to generate more texture details, conditions like Hl and edge become more dominant (around T to T steps). SSIM also remains selected to constrain the structure, whereas the frequency of SI selection decreases. This demonstrates the crucial role of dynamically selecting conditions during the diffusion sampling process. Our CCF adaptively decomposes diverse conflict fusion conditions into different denoising steps, which is significantly compatible with the reconstruction preferences of different steps in the denoising model. This dynamic fusion paradigm ensures appropriate condition selection at each stage of the sampling.
6 Conclusion
In this paper, we introduce a learning-free approach for conditional controllable image fusion (CCF), utilizing a condition bank to regulate joint information with a pre-trained DDPM. We capitalize on the remarkable reconstruction abilities of DDPM and integrate them into the sampling steps. Sample-adaptive condition selection facilitates fusion in dynamic scenarios. Varied fusion images can personalize their conditions to emphasize different aspects. Empirical findings demonstrate that CCF surpasses the competing methods in achieving superior performance for general image fusion tasks. In the future, we will further explore automatic manners to distinguish basic and enhanced conditions to reduce empirical intervention, thereby enabling more robust and reliable image fusion.

---

Acknowledgements.
This work was sponsored by National Science and Technology Major Project (No. 2022ZD01 16500), National Natural Science Foundation of China (No.s 62476198, 62436002, 62222608, U23B2049, 62106171 and 61925602). Tianjin Natural Science Funds for Distinguished Young Scholar (No. 23JCIQ/C00270) the Zhejiang Provincial Natural Science Foundation of China (No. LD244020004), and CCF-Baide Open Fund. This work was also sponsored by CAAI-CANN Open Fund, developed on Openl Community.
References
[1] Shutao Li, Xudong Kang, Leyuan Fang, Jianwen Hu, and Haitao Yin. Pixel-level image fusion: A survey of the state of the art. information Fusion, 33:100-1122 2017.
[2] Jiaxin Li, Danfeng Hong, Lianru Gao, Jing Yao, Ke Zheng, Bing Zhang, and Jocelyn Chanussot. Deep learning in multimodal remote sensing data fusion: A comprehensive review. International Journal of Applied Earth Observation and Geoinformation, 112:102926, 2022.
[3] Xingchen Zhang, Ping Ye, and Gang Xiao. Vifb: A visible and infrared image fusion benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 105, 2020.
[4] Fang Xu, Jinghong Liu, Yueming Song, Hui Sun, and Xuan Wang. Multi-exposure image fusion techniques: A comprehensive review. Remote Sensing, 14(3):771, 2022.
[5] Jinyuan Liu, Jingjie Shang, Risheng Liu, and Xin Fan. Attention-guider global-local adversarial learning for detail-preserving multi-exposure image fusion. IEEE Transactions on Circuits and Systems for Video Technology, 32(8):5026-504 2022.
[6] Yu Liu, Lei Wang, Juan Cheng, Chang Li, and Xun Chen. Multi-focus image fusion: A survey of the state of the art. Information Fusion, 64:71-91, 2020.
[7] Bing Cao, Yiming Sun, Pengfei Zhu, and Qinghua Hu. Multi-modal gated mixture of local- to-global experts for dynamic image fusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 23555-23564, October 2023.
[8] Yiming Sun, Bing Cao, Pengfei Zhu, and Qinghua Hu. Detfusion: A detection-driven infrared and visible image fusion network. In Proceedings of the 30th ACM International Conference on Multimedia, pages 4003-4011, 2022.
[9] Chenxiao Zhang, Peng Yue, Deodato Tapete. Liangeun Jiang. Boyi Shangguan, Li Huang, and Guangchao Liu. A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 166: 166:183- 183-200, 2020.
[10] Jinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu, Risheng Liu, Wei Zhong, and Zhongxuan Luo. Target-aware dual adversarial learning and multi-scenario multi multi-modality benchmark to fuse infrared and visible for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5802-5811, 2022.
[11] Chaoben Du and Shesheng Gao. Image segmentation- based multi-focus image fusion through multi-scale convolutional neural network. IEEE access, 5:15750- 15761, 2017.
[12] Jinyuan Liu, Zhu Liu, Guanyao Wu, Long Ma, Risheng Liu, Wei Zhong, Zhongxuan Luo, and Xin Fan. Multi-interactive feature learning and full-time multi-modality benchmark for image fusion and segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8518-41 2023.
[13] Abeer D Algarni. Automated medical diagnosis system based on multi multi-modality image fusion and deep learning. Wireless Personal Communications, 111:1033-1055 2020.
[14] Yiming Sun, Bing Cao, Pengfei Zhu, and Qinghua Hu. Detfusion: A detection-driven infrared and visible image fusion network. Proceedings of the 30th ACM International Conference on Multinedia, 2022.
10

---

[15] Han Xu and Jiayi Ma. Emfusion: An unsupervised enhanced medical image fusion network. Information Fusion, 76:177-186, 2021.
[16] Kanika Bhalla, Deepika Koundal, Bhisham Sharma, Yu-Chen Hu, and Atef Zaguia. A fuzzy convolutional neural network for enhancing multi-focus image fusion. Journal of Visual Communication and Image Reprexentation, 84:103485, 2022.
[17] Han Xu, Jiayi Ma, and Xiao-Ping Zhang. Mef-gan: Multi-exposure image fusion via generative adversarial networks. IEEE Trunsactions on Image Processing, 29:7203-7216. 2020.
[18] Krista Amolins, Yun Zhang, and Peter Dare. Wavelet based image fusion techniques- an introduction, review and comparison. ISPRS Journal of photogranmetry and Remote Sensing. 62(4):249 263, 2007.
[19] Yu Liu, Xun Chen, Hu Peng, and Zengfu Wang. Multi-focus image fusion with deep convolutional neural network. Information Fusion, 36:191-201, 2017.
[20] Jinyuan Liu, Runjia Lin, Guanyao Wu, Risheng Liu, Zhongxuan Luo, and Xin Fan. Coconet: Coupled contrastive learning network with multi-level feature ensemble for multi-modality image fusion. International Journal of Computer Vision, 13745)1178 2024.
[21] Jiayi Ma, Wei Yu, Chen Chen, Pengwei Liang, Xiaojie Guo, and Junjun Jiang. Pan-gan: An unsupervised pan-sharpening method for remote sensing image fusion. Information Fusion, 62: 110-120, 2020.
[22] Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Conditioning method for denoising diffusion probabilistic models. In 2021 IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), 2021.
[23] Mining Li, Ronghao Pei, Tianyou Zheng. Yang Zhang, and Weiwei Fu. Fusiondiff: Multi-focus image fusion using denoising diffusion probabilistic models. Expert Systems with Applications, 238: 136612, 2024.
[24] Zixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang, Shuang Xu, Yulun Zhang, Kai Zhang, Deyu Meng, Radu Timofte, and Luc Van Gool. Ddfm: denoising diffusion model for multi-modality image fusion. arXiv preprint arXiv:2303.0m 0660000 2023.
[25] Jinyuan Liu, Xin Fan, Ji Jiang, Risheng Liu, and Zhongxuan Luo. Learning a deep multi-scale feature ensemble and edge-attention guidance for image fusion. IEEE Transactions Circuits and Systems for Video Technology, 32(1):105-119, 2021.
[26] Hui Li, Xiao-Jun Wu, and Josef Kittler. Infrared and visible image fusion using a deep learning framework. In 2018 24th international conference on pattern recognition (ICPR), pages 2705-2710. IEEE, 2018
[27] Mostafa Amin-Naji, Ali Aghagolzadeh, and Mehdi Ezoji. Ensemble of cnn for multi-focus image fusion. Information fusion 51:201-214, 2019.
[28] Yu Liu, Xun Chen, Juan Cheng, and Hu Peng. A medical image fusion method based on convolutional neural networks. In 2017 20th international conference on information fusion (Fusion), pages 1-7. JEEE, 2017.
[29] Jiayi Ma, Chen Chen, Chang Li, and Jun Huang. Infrared and visible image fusion via gradient transfer and total variation minimization. Information Fusion, page 100-109, Sep 2016.
[30] Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Shuang Xu, Zudi Lin, Radu Timofte, and Luc Van Gool. Cddfuse: cirrelation-drivee dral-branch feature decomposition for multi- modality image fusion. In Proceedings of the IEFF/CVE conference on computer vision and pattern recognition, pages 5906-5916. 2023.
[31] Han Xu, Jiayi Ma, Junjun Jiang. Xiaojie Guo, and Haibin Ling, U2fusion: A unified unsuper- vised image fusion network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1):502-518, 2020.
11

---

[32] Jiayi Ma, Linfeng Tang, Fan Fan, Jun Huang, Xiaoguang Mei, and Yong Ma. Swinfusion: Cross-domain long-range learning for general image fusion via swin transformer. IEEE/CAA Journal of Automatica Sinica, 9()):100-1221,, 2022.
[33] Pengwei Liang, Junjun Jiang, Xianming Liu, and Jiayi Ma. Fusion from decomposition: A self- supervised decomposition approach for image fusion. In European Conference on Computer Vision, pages 719-735. Springer, 2022.
[34] Pengfei Zhu, Yang Sun, Bing Cao, and Qinghua Hu. customusto mixture of adapters for general image fusion. arXiv preprint arXhv:2403.12494 2024.
[35] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[36] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.134 2020.
[37] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
[38] Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. Diffu- sion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209. 14687, 2022.
[39] Hyungjin Chung, Jeongsol Kim, MichaelT. Mccann, MareL. Klasky, and JongChul Ye. Dif- fusion posterior sampling for general noisy inverse problems. The Eleventh International Conference Learning Representations, Sep 2023.
[40] Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. Gradnorm: Gra- dient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning, pages 794-803. PMLR, 2018.
[41] Xinyu Jia, Chuang Zhu, Minzhen Li, Wenqi Tang, and Wenli Zhou. Llvip: A visible-infrared paired dataset for low-light vision. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3496-38 3504,350,220 2021.
[42] Xingchen Zhang. Benchmarking and comparing multi-exposure image fusion algorithms. Information Fusion, 14:111-131, 2021.
[43] Xingchen Zhang. Deep learning-based multi-focus image fusion: A survey and a comparative study. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):4819 4838, 2021.
[44] Prafulla Dhariwal and AlexanderQuinn Nichol. Diffusion models beat gans image synthesis. Neural Information Processing Neutels Information Processing Systems, Dec 2021.
[45] Linfeng Tang, Xinyu Xiang, Hao Zhang, Meiqi Gong, and Jiayi Ma. Divfusion: Darkness-fiee infrared and visible image fusion. Information Fusion, 91:477-493, 2023.
[46] Chunyang Cheng, Tianyang Xu, and Xiao-Jun Wu. Mufusion: A general unsupervised image fusion network based on memory unit. Information Fusion, 92:80-99 92, 2023.
[47] Xunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang, and Jiayi Ma. Text-if: Leveraging semantic text guidance for degradation-awe aware and interactive image fusion. arXiv preprint 11X3v:2403.16387. 2024.
[48] Glenn Jocher. Yolov5 by ultralytics, 2020. URL httpps//ggttub..con/ulaaaaaaatat yolov5 doi 52.....//eeoooo.... 3908559
[49] Wenlong Zhang, Xiaolin Liu, Wuchao Wang, and Yujun Zeng. Multi-exposure image fu- sion based on wavelet transform. International Journal of Advanced Robotic Systems, 15(2): 11298814187688 2018.
12

---

[50] Jianming Zhang, Mingshuang Wu, Wei Cao, and Zi Xing. Partition-basee image exposure correction via wavelet-based high frequency restoration. In International Conference on Intelligent Computing, pages 452-463. Springer, 2024.
[51] Linfeng Tang, Hao Zhang, Han Xu, and Jiayi Ma. Rethinking the necessity of image fusion in high-level vision tasks: practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity. Information Fusion, page 101870, 2023.
[52] Hui Li and Xiao-Jun Wu. Densefuse: A fusion approach infrared and visible images. IEEE Transactions on Image Processing, 28(5):2114-2633 2018.
[53] Hui Li, Xiao-Jun Wu, and Josef Kitter. Rfn-nest: An end-to-end residual fusion network for infrared and visible images. Information Fusion, 73:72-86, 2021.
[54] Di Wang, Jinyuan Liu, Xin Fan, and Risheng Liu. Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration. arXiv preprint arXiv:2205. 1876, 2022.
[55] Wei Tang, Fazhi He, and Yu Liu, Ydtr: Infrared and visible image fusion via y-shape dynamic transformer. IEEE Transactions on Multimedia, 2022.
[56] Linfeng Tang, Jiteng Yuan, Hao Zhang, Xingyu Jiang, and Jiayi Ma. Piafusion: A progressive infrared and visible image fusion network based on illumination aware. Information Fusion, 83: 79-92, 2022.
[57] Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelemn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.055177 2017.
[58] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[59] Alex Pappachen James and Belur V Dasarathy. Medical image fusion: A survey of the state of the art. Information fusion, 19:4-19, 2014.
[60] Keith A. Johnson and Alex Becker. Harvard medical website, 2023. URL http://wt::: http:://wwwwmm.m med harvard, acu/AANLIB// www.///........e........eee.ee.e html
13

---

Appendix
A
Experimental Settings
Our method not only excels in individual settings generate customized images but also demonstrates robust performance in general settings. In this section, we elaborate on numerous experiments for image fusion tasks to demonstrate the superiority of our method. For different tasks, the basic conditions vary. Specifically, in the VIF task, MSE 124, 46 31 34] serves as the basic condition, where CMSE can be expressed as IMSE(i, zo) + MSE(U, u, zo In contrast, for the MEF and MFF tasks, to achieve clearer and higher fidelity images and inspired by [29 6 50), the basic conditions include MSE, frequency and edge conditions, detailed in the App. C To ensure convenience and fairness, we select eight enhanced conditions from the condition bank: SSIM, MSE, Edge, Low- frequency, High High-frreqency, Spatial Frequency, Edge Intensity, and Standard Deviation while setting = across all tasks.
B Experiments on More Comparisons
In our evaluation using the TNO dataset, we referred to the test set outlined in the work by Tang et al. [5T] We evaluated the fusion results in both quantitative and qualitative. Qualitative evaluation primarily hinges on subjective visual assessments conducted by individuals. We expect that the fused image will exhibit rich texture details and abundant color saturability. For different task scenarios, the different evaluation metrics used, specifically, we employ six metrics including standard deviation (SD), entropy (EN), spatial frequency (SF), sum the correlations of differences (SCD), Structural Similarity (SSIM), and edge intensity (EI). In the MFF and MEF tasks, considering the different task scenarios, we employ SD, AG, SF, SCD, and MSE for evaluation metrics. For multi-modal image fusion, we compare our method with four task-specific methods: DenseFuse [52), RFN-Nest (53)
UMF-CMGR (54), YDTR (55), and three general approaches U2Fusion 31,, DeFusion 33 and on the LLVIP dataset and TNO dataset.
DDFM 24
Quantitative Comparisons. We employ 6 quantitative metries to evaluate our model, shown in Table 3 Our method demonstrated exceptional performance across various evaluation metrics. On the TNO dataset, our method achieves the best result on the SD, EN, and SCD indicators. The result shows suboptimal performance in SF and is close to the best values. Notably, our method does not necessitate turning and holds its ground against methods requiring training. Compared with the learning-fre free DDFM and DeFusion, our method shows better results. This demonstrates the excellent performance of our model, achieving high performance across almost all indicators in tuning-free model.
e
Qualitative Comparisons. The incorporation of both basic and enhanced conditions enables effective preservation of background and texture. This comparison underscores our model's 's efficacy in image fusion, resulting outstanding visual outcomes. As shown in Fig 7 our method demonstrates superior visual quality compared to other approaches. Specifically. it excels in preserving intricate texture details and color fidelity, such as license plate numbers (highlighted in the red box in Fig 7). The DDFM and DeFusion approaches retain fewer texture details. As depicted in Fig. 7 CCF exhibits the highest contrast, the clearest details, and the most comprehensive information content, further highlighting its superiority in preserving texture details. Its excellent detail retention ability and clear background generation further prove the effectiveness of our proposed method.
C Basic Conditions of the MEF and MFF
High-frreqenccy It's commonly understood that the high-frequency information in both modalities is distinctive to each modality. For instance, texture and detailed information are specific to visible images, while thermal radiation information pertains to infrared images. Specifically, we utilizes wavelet transform to extract high-frequency information from the image. For example, 2D discrete wavelet transformation with Haar wavelets to transform the input image (IM) into four sub-bands can be expressed
{LLk, HL, LHN, HHp} = Wa (LLK-1
(14)
14

---

Visible
Denseluus
REN-Nest
UME-CMGR
YDTR
U2Fusion
DeFusion
DDFM
Ours(CCF)
Figure 7: Qualitative comparisons of our CCF and the competing methods on VIF fusion task of TNO dataset
where L.,,, HT, L,H, H E R RExx [1, K], signify the averages of the input image and the high-frequency details in the vertical, horizontal, and diagonal directions, respectively, at the first-level wavelet decomposition. Denote that the high-frequency details (IM)) from IM. So we can see the high-frequency condition below:
HF (r(z)) F(M, (1 Hr()))
(15)
where |.| stands for the absolute operation and F(.,) can represent a variety of customized functions. In this paper. specifically, we employ the max function. The restored average coefficient and the high-free frequencynny coefficient at scale k are correspondingly converted into the output at scale k - by employing W-1 the 2D inverse discrete wavelet transform:
80, = W- (LL, HF+)
(16)
Low-F now-Frequency Condition. Low-frequency components generally encapsulate information common to both modes. For simplicity, we directly utilize the low-frequency information obtained via wavelet transform. However, each mode may possess slightly varying amounts of low-frequency information. Our conditions allow for the adjustment of the low-frequency LF fusion ratio with hyperparameters, i.e.,
ËL = LF(zo) (A)(z)) 8c, =W HF&}
L))F(e)
(17)
(18)
Edge Condition. We aim the fused image retain the more intricate texture details from both infrared and visible images. To achieve this, we integrated edge conditions into the set of conditions
Table 3: Comparison with SOTAs. Red indicates the best, blue indicates the second best, and green indicates the third best.
TNO Dataset
SD
EN f
SF f
SSIM f
SCD
EI
DenseFuse
52
31.83
6.82
8.99
1.38
1.78
8.75
RFN-Nest
53
36.90
6.96
5.87
1.31
1.78
7.13
UMF-CMGR
54
29.97
6.53
8.17
1.43
1.64
7.32
YDTR [55]
28.03
6.43
7.62
1.41
1.56
6.81
U2Fusion 56
37.70
7.00
11.86
1.28
1.78
12.78
DeFusion
31
30.38
6.58
6.20
1.38
1.53
6.54
DDFM 24
34.45
6.85
7.27
1.08
1.54
7.89
CCF (ours)
40.11
7.01
10.20
1.38
1.84
8.29
15

---

32x32
32x32
16x16
16x16
Sequence layers
Down stream
Down and right stream
8x8
8x8
Skip connection
Figure 8: Framework of our neural network architecture.
to enhance the edge texture the fused image. The condition defined as
de
V (ao) F(A. (1- (v))(())))
(19)
where indicates the Sobel gradient operator, which measures the texture detail information of an image.
We incorporate the information from the original image into the fused image as supplementary content condition. Essentially, we utilize the image reconstruction capability of DDPM to supplement lost details and enhance the informational content pertaining to the target in the fused image, i.c.
MSE Condition.
BeME xo V 1 (x.) (1 AMSE) (yy))
(20)
where V and V represent the downsample and upsample operator, respectively, while can refer either the infrared or visible image.
Totally, these conditions can be combined to govern the generation the final fused image:
8 Mrnen mnde node, Moonteesss
(21)
D More Details of CCF
In the sampling process, the neural network ar- chitecture is similar PixeICN++ [57]. con-
Algorithm 1 CCF
sisting of a U-Net backbone with group nor- malization. The diffusion step is defined by
Input: 1,v
Output: f
incorporating the Transformer (58 sinusoidal
&c <- C(i, v)
position embedding into each residual block. As shown in Fig. [8] six feature map resolutions are utilized, with convolutional residual blocks
2: Sample 2T (0,
3: for I=T, do
4: ~~ 1(0
resolution level. Additionally, self self-attention blocks are incorporated the 16x 16 resolution between the convolutional blocks.
5: Fo 2e ==
yau
6: for k=N.... 1 do
7: in + 20 - V 8es
Representing process of reverse diffusion (sam- pling), starting from a standard normal distribu- tion (0, Z). T' sampling steps performed to generate the final fused image. However, the generation with unconditional DDPM 35 is uncon- trollable. Therefore, conditions selected from the condition bank we constructed are introduced to control the sampling process. More precisely, the conditions can rectify each step's 's estimation of 2g- Finally, after the step, the final fused image can be generated. The algorithm of CCF is presented in Algorithm[]
8: end for
9: end for
E More About Enhanced Conditions
We follow the recent works 46,, and adopt the eight most commonly used constraints as conditions in this paper. However, shown in Table approach is not restricted to these
16

---

Table 4: Comparison across different numbers of enhancement conditions. For the conditions, conditions include SSIM, MSE, Edge, and SD. The 8 conditions expand to include SSIM, MSE, Edge, SD, Low- now-frequency. High-frequency, SF, and EI. The 12 conditions incorporate the previous 8 conditions with four additional conditions: CC, MMS-SSIM, SCD, and VIF. Bold indicates the best
results
Number of conditions
SSIM
MSE
CC
PSNR
Nabf 4
Avg. Runtime(s)
4
1.12
1820
0.699
32.20
0.034
188
8 (ours)
1.22
1694
0.705
32.58
0,005
222
1.20
1545
0.719
33.21
0.002
454
MAEE
MSE
SAM
Sputa Pisqarro
SS0M
Figure Example of enhanced conditions selection in denoising iteration for rapidly changing scenarios.
constraints; fewer conditions be randomly selected (here we select SSIM, MSE, Edge, and SD), and additional enhanced conditions [31] 34 (Coreelatin Coefficient (CC), Multi-Scale Structural Similarity (MS-SSIM), the Sum of the Correlations of Differences (SCD), and the Visual Information Fidelity (VIF)) can also be incorporated. Additionally, we evaluate the runtime across different numbers of conditions to assess their impact on efficiency. While adding more conditions slightly improves performance, it also increases inference runtime.
Fig. 9 illustrates the selection of enhanced conditions at each denoising step in the same location under both daylight and nighttime scenarios, representing a typical rapidly changing environment. demonstrates that CCF can adapt to these changing environments by selecting different conditions at various denoising steps to respond effectively to dynamic environmental characteristics.
F
More About Task-specific Conditions
The task-specific conditions are incorporated to guide the fusion process throughout the denoising procedure. For instance, we take the Euclidean distance of features extracted by the object detection model as the detection condition. on the visible modality, to extract features from both the estimated at each step and the visible image. The Euclidean distance is minimized in the inverse diffusion process iteratively. Consequently, the final fused image progressively integrates the object-specific information, enhancing the fusion performance. We provide additional visualizations of the detection conditions, as shown in Fig. TO directly fused images appear to have lot of missed detection in raw 1-4 and the detection scores are generally higher. The last row of Fig I0 demonstrates a reduction in false- sisss-vosiiive boxes when the detection condition is applicd.
our experiments, we employ YOLOv5, which is pretrained
Table supports this, indicating that metrics such mAP.5, MAP.5:95, and recall have increased remarkably, showing that our method can customize conditions to fit the downstream task effectively. By tailoring conditions to specific tasks, our approach effectively improves the applicability of fused images for particular applications, further explored the use of different levels of features obtained with YoloV5 as detection conditions. Specifically, we evaluate in three scales and each of the scales is evaluated individually, and all features combined involve using all three scale features simultaneously, averaged, to add task-specific perception to enhance task performance. Furthermore, Table indicates that there is significant
evidenced by the improved detection metrics. We
17

---

Table 5: Comparison of task task-specific conditions across different scales of features in YOLOv5s.The
Bold indicates the best results.
Scale of feature
Position
mAP.5
mAP.5:95
Recall
w/o Feature
0.858
0.509
0.737
Feature 1 (160x 128)
Before
0.884
0.537
0.832
Feature 2 x80x64)
Before
0.890
0.540
0.826
Feature 3 (40x32) 32) (ours)
Before
0.907
0.563
0.832
All Features ( 2 Feature i)
Before
0.882
0.541
0.837
Final performance
After
0.894
0.537
0.824
w/o detection condition
with detection condition
w/o detection condition
with detection condition
Figure 10: Qualitative Comparisons of w/o and with the detection condition.
difference between using different features. The constraint primarily enhances the implicit expressive ability of the image, enabling it to better adapt to downstream tasks.
G
Ablation
The quantitative results of the ablation study compare different scenarios: without DDPM, with only basic conditions, with enhanced conditions but without SCS, and the complete CCF results. With the reconstruction capability of DDPM, the fusion performance significantly improved after the conditions were integrated. Some metrics for the enhanced conditions without SCS show decline due to conflicts among the various focusing aspect conditions. However, after incorporating SCS, most metrics improve, demonstrating that SCS can enhance the image fusion process, producing high-quality images.
18

---

Table 6: Ablation studies on the LLVIP dataset. Bold indicates the best.
DDPM
Basic Conditions
Enhanced conditions
SCS
SSIM
MSEJ
CC1
PSNR1
Nabf
0.28
2947
0,452
27.64
0.131
1.16
1785
0.683
33.06
0.032
1.16
1779
0.677
32.17
0.072
1.22
1694
0.705
32.58
0.005
MRI
SPECT
CCF
MRI
CT
CCF
MRI
PET
CCF
Figure 11: Visualization medical image fusion.
H Experiments on Medical Image Fusion
Medical image fusion 559 experiments were conducted validate the efficacy of our proposed method using the Harvard Medical Image Dataset(60). We visualized fused medical images, focusing on MRI-SPECT, MRI-CT, and MRI-PET fusion, with the results presented in Fig. 1T Observing our fused images, it is evident that the preservation of both the skull's shape and intensity is notably mell-maintained. Within the brain region, the CCF method effectively combines intricate details and structures from the two modalities.
I Limitations and Broader Impacts
Even though the proposed CCF model achieves superior performance over existing methods and demonstrates advanced generalization with dynamic condition selection, there are still some potential limitations. We provide a condition bank, but it needs to be constructed empirically. Most of the conditions are inspired by Image Quality Assessment (IQA), but classifying these conditions challenging due to the complexity of IQA. Therefore, it is necessary propose a method to automatically classify the conditions, reducing empirical intervention. Additionally, our method relies on a pre-trained diffusion model, which limits its efficiency and makes the generation process time- consuming. Exploring more effective sampling processes would be valuable to improve efficiency and further enhance the model's performance. For potential social impact, difficult to ensure effectiveness of selected conditions for all scenarios, which may be risky in high-risk scenarios such as medical imaging and autonomous driving.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitation discussed in the App.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
20

---

Justification:
This paper does not provide theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide implementation details in Sec. and experimental setting in App.
A
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [Yes]
Justification:
The source code is provided in httcps::/githuub.com/j/ttee
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pipps.cc/pullc///uuii wwwticccguie//eeieeeeeiii CoodeSubeiiieeee ssi wwwinnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide all the implementation details in Sec. and App.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: All the experiments and comparisons are performed with the same settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
22

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide that in Sec. | and App.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://////rripps.ccpppoopp///pa http://wwwpp.ppppaooccccccaaaaaa
Question:
Answer: [Yes]
Justification: Our research conducted in the paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide that in App.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
All the generated images are carefully checked.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have checked the related requirements.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets provided.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not study human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not conduct experiments with human participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "ofjxXovppX", "reviewer": "Reviewer_spMU", "summary": "This paper proposes a Conditional Controllable Fusion framework called CCF, effectively addressing the issue that existing data-driven fusion methods struggle to adapt to all scenarios. The authors conducted extensive experiments to demonstrate the effectiveness of the CCF. This manuscript is standardized and the writing is fluent.", "questions": "1. Could you kindly elucidate the methodology employed to generate the Selection frequency map depicted in Figure 1? Real or simulated?\n2. Why Use DDPM to achieve conditional image fusion?\n3. Can you give an example to explain the specific process of condition selection in the process of sampling iteration?", "limitations": "Please refer to \"Weakness\".", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The idea of conditional controllable image fusion (CCF) is novel.\n2. Extensive experiments are conducted to demonstrate the effectiveness of the CCF.\n3. This manuscript adheres to a high writing standard, ensuring fluent and easily understandable content.", "weaknesses": "1. How was the Selection frequency map in Figure 1 obtained? Is it based on real data or simulated?\n2. It is not clear exactly what iterative refinement of sampling refers to. Why does sampling need to be iteratively refined? The sampling iteration is derived from the diffusion model DDPM and how is it related to the proposed CCF?\n3. What are the advantages of using the DDPM for controlled image fusion? In other words, how do diffusion models contribute to the adaptability and controllability of image fusion?\n4. Multiple sign ambiguities. Including but not limited to: 1) $c$ denotes both the channels and the given condition in Eq. (5). 2) What does the $f_{\\theta}$ refer to? 3) What is the $s_{\\theta}$?\n5. How are task-specific conditions incorporated into DDPM? Lack of specific implementation details.\n6. The theoretical discussion of adaptive customization of conditions for each sample is limited to sections 4.1 and 4.2, lacking specific customization procedures or visual case studies. Consequently, the credibility of this contribution in terms of condition customization is compromised.\n7. The authors chose 8 enhanced conditions, i.e., SSIM, Content, Edge, Low-frequency, High-frequency, Spatial Frequency, Edge Intensity, and Standard Deviation enhancements. However, their selection lacks a strong foundation or rationale.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We'd like to thank the reviewer for the valuable comments and appreciate your recognition of the **novel idea**, **effective method** and **fluent writing**. We provide detailed responses to the constructive comments.\n\n- W1&Q1: How was the Selection frequency map in Figure 1 obtained? Is it based on real data or simulated?\n\nThanks for your constructive comment. The **statistical result** in Fig. 1 is based on the **real data**. For each sample, the conditions are selected by the Sampling-adaptive Condition Selection (SCS) (L159, Sec. 4.2 ) from the enhanced conditions at each denoising step of diffusion. We counted the frequency of all sample selection conditions in LLVIP. The deeper the color, the higher the frequency.\n\n- W2&Q2:  More explanations for sampling iteration.\n\n\nThanks for your detailed comment. The denoising process is also named as the inverse diffusion process or sampling process [1]. The refinement of sampling in DDPM is the denoising process that removes the noise progressively and produces more realistic images. Please kindly note that we for the first time found the diffusion-based image fusion model requires significant different conditions at different steps. This inspired us to **subtly integrate** **DDPM** into our dynamic fusion framework and proposed SCS to **adaptively select the suitable conditions** at each sampling step. Specifically, we inject the **selected conditions iteratively** during the sampling process to guide the generation and refine the fused images. As illustrated in Fig. 1, it proved our condition selection is consistent with the generation of DDPM.\n\n- W3: How do diffusion models contribute to the adaptability and controllability of image fusion?\n\n\nThanks for your detailed comment. \n\ni) As previously mentioned in **W2&Q2**, DDPM generates images progressively, with conditions being injected at each diffusion denoising step. We decompose the image fusion process into several controllable parts, allowing each part to be controlled individually, thereby **ensuring overall controllability**.\n\nii) In the process of progressively fusing images with diffusion, different steps have varying perceptions of the images, as shown in Fig 1. As discussed in Sec. 5.6, in the initial stage, the conditions are randomly selected, the middle stage prefers conditions related to content, and the final stage focuses on conditions related to the details. Therefore, our CCF is qualified to **adaptively select suitable conditions at different stages** of the denoising process to effectively fuse the image with DDPM.\n\n\n- W4:  Multiple sign ambiguities. Including but not limited to: 1) c denotes both the channels and the given condition in Eq. (5). 2) What does the $f_\\theta$ refer to? 3) What is the $s_\\theta$? \n\n\nThanks for the valuable suggestions.\n\ni) Eq. (5) $c$ represents the given condition. To eliminate ambiguity, we have revised the $n$ signifies the channels in L123-124.\n\nii) $f_{\\theta}$ and $s_\\theta$ both denotes the estimation of $x_0$. To eliminate ambiguity, we have unified them as $f_{\\theta}$.\n\nWe have conducted a thorough review to eliminate all ambiguous notations and to clarify the expressions.\n\n\n- W5:  How are task-specific conditions incorporated into DDPM? Lack of specific implementation details.\n\n\nThanks for the constructive comment. As described in **L189-196**, the task-specific conditions are **incorporated to constrain the fusion across the whole denoising process**. For instance, we take the Euclidean distance of features extracted by object detection model as the detection condition. We deploy YOLOv5 in the experiments, which extracts the features of estimated $x_0$ in each step and visible image, as the YOLOv5 is pretrained on visible modality. We minimize the Euclidean distance in the inverse diffusion process iteratively. Consequently, the final fused image progressively integrates the object-specific information, enhancing the fusion performance.\n\n\n- W6&Q3:  More explanations for the specific process of condition selection in the process of sampling iteration?\n\n\nAs shown in Fig. 1, we analyze the condition selection at each step for all samples in the LLVIP dataset. Furthermore we explain the process of condition selection in detail.\n\nFor example, in the MMF task in our experimental setting, using the visible image $v$ and infrared image $i$.\n\n1. **Build the conditional bank**: Detailed in Appendix A, \"Experimental Settings.\"\n2. **Estimate $x_0$** with Eq.(6).\n3. **Calculate each condition** using the $v$ and $i$.\n4. **Obtain** $\\omega$ with Eq.(11) .\n5. **Select the index of conditions** same as $\\text{topk}(\\omega)$ for image fusion.\n6. **Loop to the end** of the denoising process and finally **get the fused image**.\n\nWe have extended the visual case studies of the sampling process iterations in Fig. A1 of global pdf.\n\n- W7:  The authors chose 8 enhanced conditions, i.e., SSIM, Content, Edge, Low-frequency, High-frequency, Spatial Frequency, Edge Intensity, and Standard Deviation enhancements. However, their selection lacks a strong foundation or rationale\n\nAs shown in **L181-188**, we follow the recent works [2-4] and **choose 8 most used constraints as the conditions**. However, we are not limited to these, as shown Tab. A1 of global pdf, **more enhanced conditions (CC, MS-SSIM, SCD, VIFF)[5-7] can be incorporated** . While adding more conditions slightly improves performance, it also results in a linear increase in inference runtime.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lYXTuvzGHT", "reviewer": "Reviewer_PorP", "summary": "This paper proposes a diffusion-based image fusion method with adaptive fusion conditions. It aims to solve the drawback of existing method, i.e., the application of distinct constraint designs tailored to specific scenes. This method builds a condition bank with basic, enhanced, and task-specific conditions. Then, it employs specific fusion constraints based on these conditions for each individual in practice. The proposed method is tested on various image fusion tasks, including multi-modal, multi-exposure, and multi-focus image fusion.", "questions": "1. What does $s_\\theta(x_t,t)$ in Eq. (10) represent? And what is the form of $C$ in Eq. (10)? As mentioned in the multiple types of conditions, some conditions are in pixel level, such as MSE, SSIM, feature similarity, while some is denoted with the characteristic, such as SD. The form is not unified, and the characteristic-based condition may have some effect on the content.\n2. In the specific conditions, how to build the constraint between F(x_0) and F(V,I), as the first term takes a single image as input while the last term take the two source images as the input.\n3. For the task-specific condition, the extracted features with the task network is used for constraint. Why not use the final performance of the task for constraint?", "limitations": "1. The first and second contributions are essentially the same.\n2. The conditions discussed in the Appendix are almost different basic conditions. And the results of MSE are a bit strange.\n3. It lacks the experiment on challenging and dynamic scenarios.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The proposed method can combine the advantages of multiple types of loss functions and take consideration of downstream tasks, thus dynamically adjust the optimization direction during the sampling process. \n2. This method is applicable to multiple image fusion tasks, including visible-infrared, medical, multi-exposure, and multi-focus image fusion.\n3. The experiments are rich and the results are competitive.", "weaknesses": "1. The basic and enhanced conditions are some widely used image fusion constraints, and the task-specific condition is based on the feature extracted by a downstream network. Thus, the main contribution lies in the design of the gate of conditions. However, the reason why is the gate defined in this way and the principle behind this definition not fully explained in detail.\n2. It focuses on the problem that such data-driven fusion methods are hardly applicable to all scenarios, especially in rapidly changing environments and source images with dynamic differences. However, the experiment fails to show the advantages in these scenarios.\n3. This method states that multiple conditions can be considered, but the actual considerations are limited, such as the SSIM-based enhanced condition and detection-based task-specific condition.\n4. The article basically describes the process of the method, but the motivation, principle, and details of the method are not clear and explicit enough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for recognizing our method as **applicable to multiple fusion tasks, rich experiments** and **competitive results**. We will also make an effort to increase clarity throughout.\n\n- W1&L1: More explanations for contributions and the \"gate\" of conditions.\n\nThanks for the constructive comments. \n\ni) The first contribution focuses on the **dynamically controllable fusion framework**, which integrates various fusion constraints as a condition bank. The condition bank contains basic condition (BC), enhanced condition (EC), and task-specific condition (TC). Thus, we can **select different conditions to perform fusion for different samples**. Please kindly note that this is a sample-level selection for the whole denoising process. \n\nii) Our second contribution lies in the proposed SCS, which adaptively injects suitable conditions at each denoising step of diffusion model. To our knowledge, **we first found that the diffusion-based image fusion model requires significantly different constraints at different steps.** This inspired us to perform **different selections for each step of denoising step**, endowing more adaptivity.\n\nWe have revised the contributions as:\n\n- We propose a pioneering conditional controllable image fusion (CCF) framework with a condition bank, achieving controllability in various image fusion scenarios and facilitating the capability of dynamic controllable image fusion.\n\n- We present the Sampling-adaptive Condition Selection (SCS) to subtly integrate the condition bank into denoising steps of diffusion, allowing adaptively selected conditions on the fly without additional training and ensuring the dynamic adaptability of the fusion process.\n\niii) The \"gate\" refers to the routing process:\n(a) **Calculate all enhanced conditions' gradients** using the error of target and $x_0$. (b) **Calculate the gradient error** ∇ω with Eq.(12). (c) **Update the ω**  acts as '**gate**' in the routing process, with ∇ω. (d) **Sort ω** and **identify the corresponding top-k conditions**. The routing process facilitates the adaptive selection of conditions based on fusion requirements. \n\n- W2&L3: Experiments on challenging scenarios. \n\nThanks for the valuable comment. We have added experiments on rapidly changing scenarios. Shows in Fig. A1 of global pdf, our CCF adapts to changing environments by selecting different conditions at different sampling steps.\n\n- W3: Limitations about actual consideration of condition.\n\ni) The EC and TC are **not limited to SSIM and detection**. The ECs include 8 conditions (L490-491), and TCs are not limited to object detection, classification, segmentation, and depth, as shown in Fig. 2.\nii) The ECs are **adaptively selected at each denoising step by SCS** module, and the **condition bank is expandable** for more potential tasks. Tab. A1 shows that more ECs improve the performance.\n\n- W4: Motivation, principle, and details.\n\n**Motivation:** The dynamically changing environments lead to varying effective information of different modalities, which is sensitive to different constraints. This inspired us to assign different constraints as the conditions to improve flexibility of fusion. Our CCF subtly introduces a condition bank to diffusion model, injecting suitable conditions at each denoising step according to diffierent samples.\n\n**Principle:** We first found the diffusion-based image fusion model prefers significantly different constraints at different denoising steps (see Fig. 1). Based on this, we proposed SCS to adaptively choose suitable conditions at each step, progressively optimizing the fusion results.\n\n**Details:** Our method is based on a pre-trained DDPM. During the denoising process, we use conditions to constrain $x_0$ for image fusion. The BCs ensure fundamental fusion across various fusion tasks, while the ECs are adaptively selected by SCS (L145-151) to improve the fused image at each step. TCs is manually selected to introduce more task-specific constraints.\n\n- Q1: Presentation issues.\n\n$s_\\theta(x_t, t)$ and $f_{\\theta}$ all refers to the estimation of $x_0$ at t-th step. We would unify $s_\\theta(x_t, t)$ into $f_{\\theta}$.\n\nC is the target of the conditions. Take the MSE as an example, the  $||C-\\mathcal{A}(x_0)||_2$ can be express as $||y- x_0||_2$\n\nThe conditions are not classified by pixel level or characteristic. The classification criteria of different conditions are based on the selection frequency. The content condition refers to the basic condition in our paper. To avoid ambiguity, we will replace 'content' with 'basic' in the revision.\n\n- Q2: The constraint of TCs.\n\nThanks for the valuable question. To mitigate the ambiguity, we have revised it to $||F(X), F(M)||_2$ where $X\\in \\{x_0\\}_i^m$ and $M$ is the set of $m$ modalities.\n\n- Q3: The final performance of the task for constraint?  \n\nIn general, the task-specific ground truth is not available. Therefore, the final performance of the task is not used for constraint. Instead, we introduce the feature and final results of task network on source images as the pseudo label to constrain the denoising process, integrating task-specific information for more robust fusion.\n\nIn addition, we have added experiments to compare the effect of feature constraint before and after the detection neck. Tab. A2 in global pdf indicates deep features before detection neck showing superior performance as they balance the detection-specific information and sufficient representation.\n\n\n- L2: More explanations for basic conditions and MSE. \n\nConsidering the significant gaps between different tasks, we propose a set of BCs tailored various task scenarios. E.g., in the MMF task, MSE [3-6] is a commonly used condition. In both MMF and MEF tasks, to achieve clearer and higher fidelity images, Wavelets are widely employed [8-11].\n\nThe MSE metric [12] may be different due to the image size, value range (e.g., 0-1 or 0-255), and channel configuration (e.g., RGB or grayscale).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YKbDFvtIa2", "reviewer": "Reviewer_y2va", "summary": "This paper proposes a novel Controllable Condition Fusion (CCF) framework that utilizes a pre-trained diffusion model to achieve dynamic and adaptive condition selection without requiring specific training for general image fusion tasks. The authors presented a conditional bank conducted by various conditions to fit diverse scenarios. The conditions can be dynamically and adaptively selected according to the diffusion step, allowing CCF to conditionally calibrate the fused images step by step for each individual sample. Experimental results demonstrate the effectiveness of the proposed method.", "questions": "Please refer to the weaknesses above.", "limitations": "The authors have discussed the limitations of this work. And I don’t think there are any direct negative societal implications. Other limitations and opportunities for improvement are addressed in my responses to previous questions.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "(1)The paper is well-structured and the proposed idea is novel. It introduces a conditional controllable fusion (CCF) framework for general image fusion tasks without specific training. The generation and denoising capabilities of DDPM are effectively leveraged to produce high-quality fused images, making its integration with image fusion both ingenious and well-suited.\n(2)A new dynamic conditional paradigm was introduced with a conditional bank that regulates image fusion. This allows for the adaptive selection of multiple conditions, enabling control over various image fusion processes.\n(3)The paper validates the superior performance of the CCF framework over SOTA methods through extensive experiments on various fusion tasks. This enhances the credibility and applicability of the proposed approach.", "weaknesses": "(1)This paper introduces a controllable condition image fusion model. However, this ‘condition’ is not the common guidance condition, such as sketch, location map, pose image, etc, but evaluation metrics, such as SSIM, Edge Intensity et al. Therefore, suggesting that authors give a clear explanation or definition of the condition in Section Introduction.\n(2)An important comparison is missing. From the first impression, the proposed method is to leverage the reconstruction capability of DDPM. However, the article did not demonstrate that the fusion process utilized the reconstruction capability of DDPM, it is important to prove it systematically.\n(3)One of the core innovations in this paper is the Conditional Bank, which contains three types of conditions: basic fusion conditions, enhanced fusion conditions, and task-specific fusion conditions. However, there lacks a clear definition of three conditions to identify the difference between these conditions. \n(4)The paper contains some small mistakes in symbol, such as Line 170 theta, the lack of a subscript for eq 6 \\epsilon and missing the parentheses. Additionally, there is indistinct use of \"x0|t\" and \"x0\" in Line 134. Overall, the work is very readable, but the problems with the writing should be corrected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We'd like to thank the reviewer for the valuable comments, and acknowledgment of our **novel**, **ingenious integration** and over current **state-of-the-art** framework. We appreciate your support and constructive suggestions and address your concerns as follows.\n\n- W1: This paper introduces a controllable condition image fusion model. However, this ‘condition’ is not the common guidance condition, such as sketch, location map, pose image, etc, but evaluation metrics, such as SSIM, Edge Intensity et al. Therefore, suggesting that authors give a clear explanation or definition of the condition in Section Introduction.\n\nIt would kindly note that the condition is not similar to stable diffusion, it is the fusion condition in this paper. **The fusion condition aims to synthesize a unified image that amalgamates complementary information from multiple images**. For typical fixed conditions scenarios, we often design conditions to determine which information needs to be fused such as $||x_0,V||_2$ and $||x_0,I||_2$ . Therefore, we propose the conditional bank with Sampling-adaptive Condition Selection (SCS) for selecting conditions to adapt different task scenarios.\n\n- W2: An important comparison is missing. From the first impression, the proposed method is to leverage the reconstruction capability of DDPM. However, the article did not demonstrate that the fusion process utilized the reconstruction capability of DDPM, it is important to prove it systematically. \n\nThank you for your detailed comment. The refinement of the denoising process in DDPM is the denoising process that removes the noise progressively, step by step, and produces clearer and more realistic images. Unlike other methods with fixed fusion conditions, the CCF framework **dynamically selects the fusion conditions from a condition bank**. DDPM decomposes image fusion across multiple steps, enabling combined CCF so that conditions can be injected iteratively during the denoising process to guide the generation toward the final fused images. We have found that image fusion focuses on different conditions at different steps within the diffusion denoising process. This nuanced understanding inspired the proposal of SCS, which **adaptively selects the suitable condition at each sampling step**, aligning with the generation preferences to optimize image fusion. As illustrated in the Tab. A3 of global pdf shows significant improvement with the reconstruction capability of DDPM.  \n\n\n- W3: One of the core innovations in this paper is the Conditional Bank, which contains three types of conditions: basic fusion conditions, enhanced fusion conditions, and task-specific fusion conditions. However, it lacks a clear definition of three conditions to identify the difference between these conditions.\n\nThanks for the valuable comment. We establish a condition bank that includes basic, enhanced, and task-specific fusion conditions. The basic conditions are **essential for obtaining the fundamental** fused image, while the enhanced conditions are adaptively selected using SCS to **enhance the quality** of the synthesized fused image. Additionally, task-specific conditions can be manually selected to **obtain more task perceptions** in the fused images.\n\n\n- W4: The paper contains some small mistakes in symbols, such as Line 170 theta, the lack of a subscript for eq 6 \\epsilon and missing the parentheses. Additionally, there is indistinct use of \"x0|t\" and \"x0\" in Line 134. Overall, the work is very readable, but the problems with the writing should be corrected.\n\nL170 should be $\\theta$\n\n$x_{0|t} \\approx f_\\theta(x_t, t) = \\frac{(x_t -\\sqrt{1-\\bar{\\alpha_t}}\\epsilon_\\theta(x_t, t))}{\\sqrt{\\bar{\\alpha}_t}}$\n\nWe have carefully checked and revised all the presentation issues throughout the paper. We appreciate the reviewer's constructive comments, which greatly improved our presentation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GTHKSyU7kW", "reviewer": "Reviewer_ghie", "summary": "The paper proposes a controllable conditional image fusion method. This method enables dynamically controllable fusion for each image pairs. The core idea is to empirically construct a conditional bank and dynamically select different control conditions during the diffusion fusion process. The method is suitable for image fusion tasks of different modalities and can also perform controlled fusion given various downstream tasks, such as object detection.", "questions": "- In Fig1, the terms “Random”, “Content”, and “Detail” in the center of the diagram need to be clearly explained.\n\n- In Fig2, Top-K Routing is not contained in the main text.\n\n- In Fig3, the enlarged section does not seem to be as good as methods like Swinfusion and Text-IF.\n\n- Line 193, “the feature extracted by a detection network F = D(x)”—what scale/level of feature serves as a good guide? Are there any selection criteria?\n\n- For basic conditions, are the high-frequency information of both modality images (for example,  infrared and visible images) extracted?\n\n- With so many conditions, low-frequency doesn’t seem as important as shown in Fig.1.  Additionally, are there any other empirical conditions with potential?\n\n- In Fig8, the authors mention the network structure, but in Line 493, they state that they used “a pre-trained diffusion model.” So, this network structure should be derived from the paper on the pre-trained model? If so, a citation needs to be provided in the diagram.", "limitations": "- The conditional bank is manually designed, and different experiments likely require different fine-tuning. This limits the potential of the proposed method.\n- It is necessary to specify the runtime, as time is usually a key metric for diffusion model-based methods.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "- The paper introduces the controllable conditional image fusion algorithm for the first time. This method makes the image fusion process based on the diffusion model controllable through manual feature constraints (empirical).\n\n- This method is suitable for different fusion tasks.", "weaknesses": "- The section Sampling-adaptive Condition Selection (SCS) is mainly derived from \"GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks\", which is cited in the paper, but some of the formulas are not clearly expressed. For example, in Eq 11, Gate(C) has C on the left, but the condition C is not contained on the right. Another example is Line 170, Li(t) = E[wi(t)Li(t)]? Additionally, there are some typos, like Line 170, theta.  \n     Therefore, understanding the routing process without reading the GradNorm paper might be difficult.\n\n- There are some issues with the writing and formatting, such as the order of appearance of Tab1 and Tab2, and the captions for the tables do not specify the task for each tab (e.g., Multi-Focus, Multi-Modal, etc.) in the quantitative comparison results. The captions for the images also have this issue and need improvement.\n\n- There are too many empirical conditions, which may require different settings under various circumstances.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for your valuable comments and appreciate your recognition for our **novelty** on controllable conditional image fusion and **generalization** on different fusion tasks. We believe the constructive feedback will improve the paper and increase its potential impact on the community.\n\n- W1: Formulas and the routing process are not clear.\n\nEq 11 should be : $Gate=[\\omega_1,..\\omega_c], \\omega_i(t)=\\omega_i(t-1)-\\bigtriangledown\\omega_i$\n\nL170 should be: $L(t)=\\mathbb{E}[\\omega_i(t)L_i(t)]$\n\nL170 \\theta should be $\\theta$\n\n**Routing process:**\n(a) **Calculate all enhanced conditions' gradients** using the error of target and $x_0$. \n(b) **Calculate the gradient error** ∇ω with Eq.(12). \n(c) **Update the ω**  acts as '**gate**' in the routing process, with ∇ω. \n(d) **Sort ω** and **identify the corresponding top-k conditions**.  \n\nThe rooting process facilitates adaptively selecting conditions based on fusion requirements. \n\n- W2: Issues with the writing and formatting.\n\nThanks for the constructive comment. The MFFW, MEFB and LLVIP datasets refer to Multi-focus, Multi-exposure and Multi-Modal fusion, respectively. We have specified the tasks for each Table. \n\nIn addition, we have carefully checked and revised all the presentation issues to mitigate the ambiguity. We appreciate the constructive comments, which improved our presentation.\n\n- W3&L1: Limitations about requiring different settings under various circumstances. \n\nThanks for the valuable comment. Please kindly note that, for different fusion scenarios, **the condition bank is the same** in our experiments. We propose a Sampling-adaptive Condition Selection (SCS) to **adaptively select different conditions at each denoising step** for different samples, which is sample-adaptive and denoising-adaptive. We have verified the ability of SCS in Sec. 5.5 and App. F, and most metrics show improvement after incorporating SCS.\n\nIn addition, we have also conducted experiments on an enlarged condition bank based on related works [5-7], which contains 12 conditions without manually screening. As shown in Tab. A1 of global pdf, while adding more conditions slightly improves performance, it also results in a linear increase in inference time. **To balance the time consumption and performance**, we manually selected 8 conditions as our condition bank.\n\n- Q1:  Need to explain “Random”, “Content”, and  “Detail” in Fig1.\n\nThanks for the constructive comment. The \"Random\" refers to the conditions are randomly selected, the \"Content\" refers to the selection of conditions that content information, and the \"Detail\" refers to prefer selection of conditions about texture detail.\n\nAs discussed in Section 5.6, we examine the preferences for condition selection. We found that image fusion focuses on different conditions at different steps within the diffusion denoising process. The SCS is designed to align the condition selection with the denoising process. Fig.1 demonstrates the effectiveness of our method. Initially, when the image is merely noise without any discernible information, conditions are \"**random**\" selected. As denoising progresses and content begin to emerge, conditions that aid in synthesizing the '**content**' are chosen. In the later stages, when the focus shifts to refining details, conditions related to '**detail**' are selected.\n\n- Q2: Problem about Top-K Routing.\n\nThe Gete is the selection gate, which dynamically chooses the top-k conditions to adaptively generate distinct aspects of the images. As we mentioned the **W1**, the **top-k routing in the SCS for adaptive selection conditions.** \n\n- Q3: Problem about qualitative evaluation of Swinfusion and Text-IF in Fig3.\n\nThanks for the comment. Compared with SwinFusion, our method outperforms it in the contrast and outline, especially in the manhole cover region producing clearer outlines within the red box.\n\nCompared with Text-IF, our method exhibits higher image definition, particularly evident in the clearer texture details of the broken texture of a zebra crossing within the blue box.\n\n- Q4: The feature selection criteria.\n\n**The deeper feature contains more task-specific information** and can provide a stronger constraint for the task. As shown in Tab. A2 of global pdf, the deepest feature captures the best performance. Additionally, the deepest feature consumes fewer computing resources, making it a better condition.\n\n- Q5: How high-frequency information extracted?\n\nThanks for the details question. The high-frequency are extracted from both modality images. Then take the max to them for containing more high-frequency information.\n\n- Q6:  Problem about Low-frequency in Fig.1 and potential conditions.\n\nDifferent conditions have different effects on the fusion results. In our condition bank, content and SSIM can also offer low-frequency constraints, while they also provide other information as well. Since we only select the Top-3 enhanced conditions at each step, the relatively homogeneous low-frequency condition is not selected adaptively. This further verifies our adaptivity, even on a condition bank with redundant conditions.\n\nAs mentioned in Question 2, more conditions lead to performance improved with the additional metrics such as CC, MS-SSIM, SCD, and VIFF [5-7] shown in Tab. A1 of global pdf.\n\n- Q7: The citation needs to be provided in Fig. 8.\n\nWe visualized the pre-trained DDPM model, as referred to in L95-96 in the manuscript, and we have revised it accordingly.\n\n- L2: Specify the runtime.\n\nAs mentioned in **W3&L1**, the runtime is sensitive to the number of conditions, as shown in the Tab. A1 of global pdf. With the addition of conditions, runtime linear increases but the performance improves slightly. So we empirically selected 8 conditions to **balance performance and runtime**. We will emphasize this in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear PCs, SACs, ACs, and Reviewers,



We would like to thank you for your valuable feedback and insightful reviews, which have greatly contributed to improving the paper. This is a **fluent** and **well-structured** (Reviewer spMU, y2va) manuscript with a **novel** idea (Reviewer spMU, y2va), we proposed a **new** Controllable Condition Fusion (CCF) framework (Reviewer y2va, ghie), **extensive** experiments (Reviewer spMU, PorP, y2va) on multiple tasks validate CCF’s results are **competitive** (Reviewer PorP) and **effectiveness** (Reviewer spMU, y2va, ghie) achieving **SOTA** (Reviewer y2va). Our framework is **suitable across different image fusion scenarios** (Reviewer spMU, PorP, y2va, ghie).

We hope that our responses will satisfactorily address your questions and concerns. We sincerely appreciate the time and effort you have dedicated to reviewing our submission, along with your invaluable suggestions. All the missing details will be added in the revision, and we will also release all our codes to ensure clarity and reproducibility.



Sincerely,

Authors



## Reference

[1] Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models[J]. Advances in neural information processing systems, 2020, 33: 6840-6851.

[2] Zhao Z, Bai H, Zhang J, et al. Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023: 5906-5916.

[3] Zhao Z, Bai H, Zhu Y, et al. DDFM: denoising diffusion model for multi-modality image fusion[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 8082-8093.

[4] Cheng C, Xu T, Wu X J. MUFusion: A general unsupervised image fusion network based on memory unit[J]. Information Fusion, 2023, 92: 80-92.

[5] Xu H, Ma J, Jiang J, et al. U2Fusion: A unified unsupervised image fusion network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(1): 502-518.

[6] Zhu P, Sun Y, Cao B, et al. Task-customized mixture of adapters for general image fusion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 7099-7108.

[7] Li H, Wu X J, Kittler J. RFN-Nest: An end-to-end residual fusion network for infrared and visible images[J]. Information Fusion, 2021, 73: 72-86.

[8] Ma J, Chen C, Li C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109.

[9] Liu Y, Wang L, Cheng J, et al. Multi-focus image fusion: A survey of the state of the art[J]. Information Fusion, 2020, 64: 71-91.

[10] Zhang W, Liu X, Wang W, et al. Multi-exposure image fusion based on wavelet transform[J]. International Journal of Advanced Robotic Systems, 2018, 15(2): 1729881418768939.

[11] Zhang J, Wu M, Cao W, et al. Partition-Based Image Exposure Correction via Wavelet-Based High Frequency Restoration[C]//International Conference on Intelligent Computing. Singapore: Springer Nature Singapore, 2024: 452-463.

[12] Xu Y, Li X, Jie Y, et al. Simultaneous Tri-Modal Medical Image Fusion and Super-Resolution using Conditional Diffusion Model[J]. arXiv preprint arXiv:2404.17357, 2024.

Meta-review: The paper proposes a method that enables dynamic conditional controllable fusion for image pairs with reported excellent performance.

The authors provided satisfactory responses and convinced
- Reviewer spMU to increase the score to Weak Accept
- Reviewer PorP to increase the score to Borderline Accept
- Reviewer y2va to increase the score to Accept
- while Reviewer ghie maintains Borderline Accept.

Thus, for this work there is an acceptance consensus.

After reading the paper and carefully checking the reviews and the authors' responses the ACs agree with the reviews that the present work makes important contributions and is of interest for the community.

The authors are invited to further refine their paper for the camera ready by including (part of) information/details from their responses to the reviewers' comments.
Decision: Accept (poster)