DCAFuse: Dual-Branch Diffusion-CNN Complementary Feature Aggregation Network for Multi-Modality Image Fusion

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 09:50:34

Abstract: Multi-modality image fusion (MMIF) aims to integrate the complementary features of source images into the fused image, including target saliency and texture specifics. Recently, image fusion methods leveraging diffusion models have demonstrated commendable results. Despite their strengths, diffusion models reduce the capability to perceive local features. Additionally, their inherent working mechanism, introducing noise to the inputs, consequently leads to a loss of original information. To overcome this problem, we propose a novel Diffusion-CNN feature Aggregation Fusion (DCAFuse) network that can extract complementary features from the dual branches and aggregate them effectively. Specifically, we utilize the denoising diffusion probabilistic model (DDPM) in the diffusion-based branch to construct global information, and multi-scale convolutional kernels in the CNN-based branch to extract local detailed features. Afterward, we design a novel complementary feature aggregation module (CFAM). By constructing coordinate attention maps for the concatenated features, CFAM captures long-range dependencies in both horizontal and vertical directions, thereby dynamically guiding the aggregation weights of branches. In addition, to further improve the complementarity of dual-branch features, we introduce a novel loss function based on cosine similarity and a unique denoising timestep selection strategy. Extensive experimental results show that our proposed DCAFuse outperforms other state-of-the-art methods in multiple image fusion tasks, including infrared and visible image fusion (IVF) and medical image fusion (MIF). The source code will be publicly available at https://xxx/xxx/xxx.

Main paper: 
 related works 2.1 multi-modality image fusion: 
 In recent years, multiple deep learning-based methods have been developed to address the challenges posed by MMIF and the most commonly used networks are CNNs and GANs [2, 21, 24, 54, 81] .In CNN-based methods, various frameworks and loss functions are designed for feature extraction, feature fusion, and image reconstruction [27, 58, 74, 88] . Li et al. applied dense connections to extract features [26] , and Wang et al. design multiple kernels to extract multi-scale features [67] . Besides, contrastive learning has been widely used to distinguish different modalities [33, 36, 90] [39, 66] . Additionally, some methods use prior knowledge of downstream tasks to assist in the loss function design. For example, Tang et al. [56, 58] use semantic segmentation masks, and Liu et al. [33] use salient masks to guide the training process.GAN-based methods model the global information under unsupervised conditions [32, 84, 89] . Ma et al. first introduces GAN to fusion task [41] . Later, methods such as multi-classification GAN [42] and guided filters [78] are introduced. To balance each modality, asymmetric generator-discriminator structures are proposed [40, 85] . Moreover, Li et al. introduce the attention mechanism into the GAN-based fusion network [28] .Diffusion models have demonstrated powerful capabilities in various generation tasks [9, 18, 49, 52] , including text-to-image [51, 82] , image-to-image [46, 50, 62] , image inpainting [1, 35] , etc.In addition, some works have explored the application of diffusion models, represented by DDPM [18] , to high-level vision tasks, such as semantic segmentation [70] and object detection [ 6, 14] . There are also some diffusion-based works focusing on low-level vision tasks, such as image restoration [37, 71] , image super-resolution [64] and image fusion [30, 87] . In a framework for the above tasks, a commonly used method is first to introduce noise to source images, and then extract the latent features from the denoising U-Net for the following tasks [3, 4, 79] . Although diffusion models can produce visually appealing fused images, their limited local perception capabilities and inherent noise-adding mechanism result in significant detailed information loss. 
method 3.1 overview: 
 The proposed DCAFuse utilizes a dual-branch diffusion-CNN framework for comprehensive multi-modality image fusion. Taking the IVF task for instance, the RGB-channel visible image 𝑋 𝑣𝑖𝑠 ∈ R ℎ×𝑤×3 are combined with the infrared image 𝑋 𝑖𝑟 ∈ R ℎ×𝑤×1 , forming the original input denoted as 𝑋 0 ∈ R ℎ×𝑤×4 .As shown in Fig. 3 , DCAFuse consists of diffusion-based and CNN-based branches. In the diffusion-based branch, we initially introduce noise into 𝑋 0 following our proposed timestep selection strategy, followed by intermediate feature extraction during the denoising process for global information modeling (𝐹 𝐷 ). In the CNN branch, multi-scale convolutional kernels and attention blocks are employed to extract and consolidate local detailed features (𝐹 𝐶 ). Subsequently, the Complementary Feature Aggregation Model (CFAM), a novel component of our approach, generates coordinate-aware attention maps to capture the long-range dependencies between 𝐹 𝐷 and 𝐹 𝐶 , allowing for effective aggregation. The aggregations are finally fed into the fusion head to obtain the fusion result.Through the denoising process, DDPM can encapsulate global information within intermediate features [20, 68] . In the diffusion-based branch, we first obtain noisy image 𝑋 𝑡 for specified timestep 𝑡 by introducing Gaussian Noise, denoted as 𝜖 𝑡 , to 𝑋 0 then extract intermediate features from the denoising U-Net.According to [18] , instead of progressively adding noise, we can derive 𝑋 𝑡 directly from a single operation as detailed below:EQUATIONwhere the noise 𝜖 ∼ N (0, 𝐼 ), and the variance 1 -ᾱ𝑡 is related to the predefined variance schedule. Subsequently, the noisy image 𝑋 𝑡 is fed into the DDPM for a single-step denoising (reverse diffusion) process as follows:EQUATIONwhere 𝑧 ∼ N (0, 𝐼 ), 𝜖 𝜃 (𝑋 𝑡 , 𝑡) represents the predicted noise, and 𝜎 𝑡 is related to the predefined variance schedule.Eq. 1 and Eq. 2 are performed at 𝑁 timesteps (i.e. 𝑡 1 , 𝑡 2 , . . . , 𝑡 𝑁 ) to comprehensively capture the original information [3, 79] . Then, As depicted in Fig. 3 , multi-timestep features extracted from Block 𝑖 (beginning from 𝑖 = 1) are concatenated as 𝐹 𝐷 𝑖 . Subsequently, the Cross-Timestep Feature Aggregator (CTFA) refines 𝐹 𝐷 𝑖 using a range of various convolutional and attention blocks. The refined feature is then upsampled to the same size as 𝐹 𝐷 𝑖+1 , denoted as 𝐹 𝐷 𝑖 ′ . Finally, 𝐹 𝐷 𝑖+1 = 𝐹 𝐷 𝑖+1 + 𝐹 𝐷 𝑖 ′ is obtained and then fed into the CTFA for the next iteration, continued until 𝑖 = 𝑀. The concluding output from the diffusion-based branch is represented as 𝐹 𝐷 .With superior local perception, CNN captures detailed features that serve as an effective supplement to the global information structured by DDPM [45, 55] .In the CNN-based branch, 3-stage convolutional layers along with Mixed Attention Blocks (MABs) are utilized for the extraction of multi-scale local detailed features, symbolized as 𝐹 𝐶 𝑘 where 𝑘 ∈ {1, 2, 3}.Subsequently, the Multi-Scale Feature Aggregator (MSFA) progressively merges 𝐹 𝐶 𝑘 [58] . Initially, 𝐹 𝐶 𝑘 is upscaled to match the size of 𝐹 𝐶 𝑘+1 , following which the scaling factor 𝛾 𝑘 and bias 𝛽 𝑘 are generated via MLPs to modulate 𝐹 𝐶 𝑘+1 as follows:EQUATIONwhere ⊙ denotes element-wise multiplication operation. Through Eq. 3, multi-scale local detailed features are fused into 𝐹 𝐶 .We design a novel Complementary Feature Aggregation Module (CFAM) to effectively aggregate the global information 𝐹 𝐷 ∈ R ℎ×𝑤×𝑐 and local detailed features 𝐹 𝐶 ∈ R ℎ×𝑤×𝑐 . Specifically, by generating the coordinate-aware attention maps of 𝐹 𝑐𝑎𝑡 = 𝐶𝑜𝑛𝑐𝑎𝑡 (𝐹 𝐶 , 𝐹 𝐷 ) ∈ R ℎ×𝑤×2𝑐 , CFAM can capture its longrange dependencies in multiple directions, thus dynamically adjusting the aggregation weights.Fig. 4 shows the specific workflow of the proposed CFAM. Initially, a 1 × 1 convolutional layer is utilized to adjust the number of channels (i.e. 2𝑐 → 𝑐). Then, CFAM extracts direction-aware feature maps 𝐹 𝑥 ∈ R ℎ×1×𝑐 and 𝐹 𝑦 ∈ R 1×𝑤×𝑐 by orthogonal 1-D average pooling layers as follows:EQUATIONwhere 𝑋𝐴𝑃 and 𝑌𝐴𝑃 represent performing average pooling along the horizontal and vertical directions, respectively. Given that 𝐹 𝑥 and 𝐹 𝑦 obtain the saliency information of features in corresponding directions, we concatenate them in the vertical direction and perform channel reduction by convolutional layer as:EQUATIONwhere 𝑇 denotes transposition operation, and 𝑟 represents the ratio of channel reduction. Afterward, by convolutional layers and nonlinear functions, 𝐹 𝑥 𝑦 are encoded into 1-D coordinate-aware attention vectors 𝐹 𝐶𝑑𝐴 𝑥 ∈ R ℎ×1×𝑐 and 𝐹 𝐶𝑑𝐴 𝑦 ∈ R 1×𝑤×𝑐 , which capture the long-range dependencies of input 𝐹 𝑐𝑎𝑡 along the corresponding spatial direction [19] .Subsequently, 𝐹 𝐶𝑑𝐴 𝑥 and 𝐹 𝐶𝑑𝐴 𝑦 are broadcast into (𝐻,𝑊 ) and utilized to perform element-wise multiplication, resulting in coordinateaware attention map 𝐹 𝐶𝑑𝐴 ∈ R ℎ×𝑤×𝑐 , which reflects long-range dependencies in all directions. Then CFAM aggregates 𝐹 𝐷 and 𝐹 𝐶 as follows:EQUATIONwhere 𝐹 𝑅 denotes the aggregated features. According to coordinateaware attention map 𝐹 𝐶𝑑𝐴 , CFAM fully encapsulates the complementary attributes of dual-branch features, thus effectively aggregating the global information 𝐹 𝐷 and local detailed features 𝐹 𝐶 . Finally, the aggregated features are fed into the fusion head to generate the MMIF results.To discern the timestep selection strategy that effectively complements the denoising features and features extracted by the CNNbased branch, we examine latent feature representations across multiple timesteps. Fig. 5 (a) illustrates that the CNN-based branch focuses more on salient targets and local details while paying limited attention to background information.As shown in Fig. 5 (b), although existing methods generally select early timesteps (approximately 𝑡 = 50) for feature extraction [3, 79] , aiming to diminish noise-induced distortion of the original information, this approach fails to comprehensively portray the whole global scene. Fig. 5 (c)-(d ) demonstrates that, with the progression of the sampling timestep, the diffusion model progressively captures background features, resulting in effective modeling of global information at 𝑡 = 200. Nevertheless, when the timestep exceeds 800, the intense noise seriously precludes the extraction of information within the denoising process.Overall, existing methods conventionally utilize early denoising timesteps for feature extraction, resulting in inadequate capture of global information. Stemming from our case analysis, we propose that features gathered at slightly late timesteps more effectively incorporate global information, serving as a suitable supplement to the local detailed features derived by CNN.Following our proposed strategy, we execute ablation experiments, revealing 𝑡 = [100, 200, 400] as the relatively preferable denoising timesteps combination. Further specifics will be illustrated in the experiments.The overall loss function consists of three components: intensity loss 𝐿 𝑖𝑛𝑡 , gradient loss 𝐿 𝑔𝑟𝑎𝑑 , and the proposed cosine divergence loss 𝐿 𝐶𝐷 . It can be formulated as:EQUATIONwhere 𝛼 and 𝛽 denote the balancing factors of each loss term. Specifically, 𝐿 𝑖𝑛𝑡 calculates pixel-wise intensity loss between the fused image 𝐼 𝑓 𝑢𝑠𝑒𝑑 and input image 𝐼 1 , 𝐼 2 (initial channel duplication will be applied to the single-channel input), which can be defined as:EQUATIONSimilarly, 𝐿 𝑔𝑟𝑎𝑑 to calculate the gradient loss can be defined as:EQUATIONFurthermore, we introduce a cosine divergence loss 𝐿 𝐶𝐷 ∈ [-1, 1], with 1 indicating complete similarity and -1 indicating absolute dissimilarity, which can be defined as:EQUATION)where • symbolizes the vector dot product and 𝜖 denotes a minimal constant to circumvent zero division. By minimizing the 𝐿 𝐶𝐷 between 𝐹 𝐶 and 𝐹 𝐷 , DCAFuse is encouraged to better explore the complementarity of these features, thus improving the fusion performance. 
experiments: 
 In this section, we carry out extensive experiments for infrared and visible image fusion (IVF) and medical image fusion (MIF) tasks. First, we introduce the experimental configurations and details. Subsequently, we undertake qualitative and quantitative comparisons of our proposed method with other state-of-the-art methods. Finally, various ablation studies are conducted to demonstrate the effectiveness of the proposed modules.4.1.1 Datasets. The proposed DCAFusion is trained on the MSRS [57] training set (1083 pairs). For the IVF task, we choose three datasets for testing, i.e. RoadScene [73] , MSRS [57] test set (361 pairs), and TNO [61] . As for the MIF task, we utilize three datasets collected by [86] from [44] for testing, specifically MRI-CT, MRI-PET, and MRI-SPECT. Notably, to measure the generalization performance, no additional datasets are incorporated for validation or fine-tuning.4.1.2 Metrics. Six representative evaluation metrics are employed to evaluate the fusion performance of methods quantitatively, including standard deviation (SD) [47] , entropy (EN) [48] , visual information fidelity (VIF) [16] , average gradient (AG) [7] , edge information-based 𝑄 𝐴𝐵/𝐹 [76] and spatial frequency (SF) [11] . A higher score on these metrics indicates better fusion performance. We compare the proposed DCAFusion with seven state-of-the-art image fusion methods, including U2Fusion [72] , SDNet [80] , SwinFusion [39] , TarDAL [32] , CDDFuse [86] , DDFM [87] and Dif-Fusion [79] . Methods using prior knowledge, such as [58] and [33] , are not included in comparisons.On IVF datasets, we compare the fusion performance of DCAFusion with SOTA methods, qualitatively and quantitatively.As exhibited in Fig. 6 , methods such as U2Fusion, SDNet, and DDFM appear to be under-exposed, causing the people in the red box to fade into the obscurity of the nighttime environment. In contrast, TarDAL tends to be overexposed. Of all the methods, ours distinctly outlines the shapes of people and maximizes the traffic sign's contrast in the green box, improving its legibility. Moreover, as shown in Fig. 7 , our approach Infrared Visible U2Fusion [72] SDNet [80] SwinFusion [39] TarDAL [32] CDDFuse [86] DDFM [87] Dif-Fusion [79] DCAFuse (Ours)Figure 6 : Visual comparison of "00798N" on the MSRS IVF dataset [57] .Infrared Visible U2Fusion [72]SDNet [80] SwinFusion [39] TarDAL [32] CDDFuse [86] DDFM [87] Dif-Fusion [79] DCAFuse (Ours)Figure 7 : Visual comparison of "meeting scene" on the TNO IVF dataset [61] .Table 1 : Quantitative results of the IVF task on RoadScene [73] , MSRS [57] and TNO [61] retains the most intricate details, such as the textures of the doorframes and leaves in the green box, and also distinctly separates the thermal target (people) from the background. In summary, the proposed DCAFuse effectively combines the thermal saliency information from the infrared image with the detailed texture from the visible image, generating a fusion image with the finest visual effect. SDNet [80] SwinFusion [39] TarDAL [32] CDDFuse [86] DDFM [87] Dif-Fusion [79] DCAFuse (Ours)Figure 8 : Visual comparison on MRI-PET MIF dataset [44] .VIF underscores that our method delivers the most appealing visual effects. Furthermore, the noticeable enhancements in AG and SF, by average increments of 26.36% and 30.52% respectively across the IVF datasets, validate that our results present the most detailed texture characteristics. Quantitative results prove that the proposed DCAFuse effectively integrates the saliency information in infrared images and the texture details in visible images.In this section, we evaluate the fusion performance on MIF datasets without fine-tuning, aiming to assess the generalization performance of the methods. of the visual information in our fused images, thereby providing effective assistance in medical diagnosis. Without fine-tuning, DCA-Fuse outperforms the state-of-the-art methods, demonstrating its remarkable generalization performance in diverse multi-modality image fusion tasks. Overall, quantitative evaluations demonstrate the superior performance of the proposed DCAFuse in integrating information procured from a myriad of medical imaging modalities.We conduct ablation experiments about the proposed dual-branch framework, CFAM, cosine divergence loss, and denoising timesteps, using the MSRS dataset for both training and testing procedures. , where 𝐹 𝐶 and 𝐹 𝐷 denote the output features of CNN-based and diffusion-based branch, respectively. Although 𝐿 𝑁 𝑀𝑆𝐸 amplifies the numerical difference between features, thereby boosting SD, it neglects the structural attributes of features, thus not performing optimally on other metrics. Our proposed 𝐿 𝐶𝐷 fosters feature complementarity by maximizing the cosine distance, leading to more comprehensive fusion results. 4.4.4 Denoising Timesteps. We establish five groups of denoising timesteps: earliest (5, 25, 50) , slightly early (5, 50, 100), midterm (50, 100, 200) , slightly late (100, 200, 400), and latest (200, 400, 800). Implementing the earliest timestep results in a failure of network convergence, as denoising U-Net cannot effectively comprehend the global information under extremely low-intensity noise. As demonstrated in Table . 6, enlarging the denoising timestep gradually improves the fusion effect. Upon setting the denoising timesteps to [100, 200, 400], DCAFuse yields the best performance in EN, VIF, AG, and SF, demonstrating the comprehensive preservation of both global information and local detailed features However, when the latest timesteps are employed, there's a decline in the fusion effect observed as a downturn in both SF and VIF, due to a significant reduction of the original information caused by excessive-intensity noise. The experimental results suggest that within a tolerable noise intensity range, a slight delay in denoising timesteps aids in enhancing the multi-modality image fusion effect. 
conclusion: 
 In this paper, we introduce DCAFuse, a dual-branch Diffusion-CNN framework designed for multi-modality image fusion. We propose a novel complementary feature aggregation module, based on coordinate attention, to effectively integrate the global information extracted by the diffusion model and the local detailed features captured by CNN. Moreover, the complementarity of features extracted from the dual branches is further enhanced, benefiting from our introduced cosine divergence loss and timestep selection strategy. Extensive experiments on IVF and MIF datasets demonstrate that the proposed method achieves SOTA performance in multimodality image fusion.In the future, we aim to explore the potential of diffusion models to effectively model global information across a wider scope of image fusion tasks.  
: 
 U2Fusion [72] (TPAMI '20) SDNet [80] (IJCV '21) SwinFusion [39] (JAS '22) TarDAL [32] (CVPR '22) CDDFuse [86] (CVPR'23) DDFM [87] (ICCV '23) Dif-Fusion [79] (TIP'23) DCAFuse (Ours)𝑸 𝑨𝑩/𝑭 𝑸 𝑨𝑩/𝑭 Figure 2 : Fusion evaluation metrics comparison on Road-Scene [73] and MSRS [57] datasets.infrared images and the texture details in visible images to produce results with prominent targets and clear backgrounds [27, 38, 43] , which are widely used in fields such as autonomous driving [8, 77] , drone nighttime monitoring [53, 69] , video surveillance [10, 25] , etc. On the other hand, MIF combines images obtained from various medical imaging modalities [22, 31] , such as MRI and CT, to assist in medical diagnosis and treatment [17] . Numerous deep learning-based approaches have been developed to tackle the challenges of MMIF, mainly encompassing methods founded on Convolutional Neural Networks (CNNs) and generative models [24, 54, 81] . CNN-based techniques, restrained to a limited receptive field for feature extraction, fall short of sufficiently capturing cross-modality information and long-range dependencies, thereby undermining information fidelity [26, 27, 58] . Although Generative Adversarial Networks (GANs), renowned generative models, capably model source image distribution in alignment with MMIF requirements [32, 42] , their reliance on adversarial interactions in the fusion process fosters modality imbalance and convergence issues, hence diminishing fusion quality [40, 41] . Recent developments in diffusion models, such as Denoising Diffusion Probabilistic Models (DDPM) [18] , exhibit outstanding global information modeling capabilities [20, 68] and outperform GANs in image generation quality [9, 15] . Therefore, numerous studies attempt to apply diffusion models to visual tasks such as MMIF [6, 64, 87] . The common technique involves initially infusing noise into source images, followed by extraction of latent features during the denoising process for fusion, leading to the production of visually impressive fused images [3, 4, 79] .However, existing diffusion-based fusion methods often compromise critical texture details in the source images [79, 87] . As illustrated in Fig. 1(c) , critical features such as the car's logo and contours in the Dif-Fusion fused image appear blurred. This deficiency stems from inherent limitations in existing diffusion-based MMIF methods: (i) Inability to extract local detailed features. Although latent features extracted from the denoising network can represent global information, they lack localized perception capabilities like CNNs [12, 65] . (ii) Inherent degradation of original information. The working mechanism of diffusion-based fusion methods necessitates the introduction of noise to source images [18, 23, 52] , inevitably leading to the loss of original information. (iii) Insufficient exploration of effective timestep combinations. Intermediate features at diverse denoising timesteps exhibit distinct regional perceptions [13] , requiring a tailored design of selection strategy for effective fusion. However, comprehensive explorations of this aspect are lacking in existing works [79, 87] .To address the drawbacks mentioned above, we propose a novel dual-branch Diffusion-CNN feature Aggregation Fusion (DCAFuse) network, capable of extracting complementary features in terms of perceptual range through CNN-based and diffusion-based branches, and effectively aggregating the features based on their long-range dependencies. Specifically, in the diffusion-based branch, we extract the intermediate features at multiple timesteps of the DDPM to construct global information. In the CNN-based branch, multi-scale convolutional kernels are utilized to extract local detailed features. Afterward, we propose a novel complementary feature aggregation module (CFAM) to effectively aggregate the concatenated features. By generating coordinate-aware attention maps, CFAM captures the long-range dependencies in both horizontal and vertical directions [19] , thus dynamically guiding the aggregating weights, and then the aggregated features are fed into the fusion head to output. Moreover, to further enhance the complementarity of the features extracted from each branch, we introduce a cosine divergence loss function and an innovative denoising timestep selection strategy different from the existing methods. Fig. 1(d) shows the fusion result of DCAFuse, which exhibits much clearer contour details and better contrast than the existing method. When compared with state-of-the-art methods in various evaluation metrics, our method also achieves leading performance, as shown in Fig. 2 . Overall, our contributions are summarized as follows:• We propose DCAFuse, a dual-branch diffusion  
 
 Appendix:  


Decision: Two of three reviewers tend to accept this paper. The main strengths include the novelty, extensive experiments, good paper organization.

Reviews: review: paper_summary: paper_summary: This paper presents a novel multi-modality image fusion (MMIF) method called Diffusion-CNN feature Aggregation Fusion (DCAFuse). MMIF aims to integrate complementary features from source images, such as target saliency and texture specifics, into a single fused image. Recent diffusion-based image fusion methods have shown promising results but suffer from reduced local feature perception and loss of original information due to their noise-adding mechanism.
strength_weakness: strength_weakness: Effective Feature Aggregation:
The DCAFuse method employs a novel Complementary Feature Aggregation Module (CFAM) that dynamically guides the fusion process by capturing long-range dependencies through coordinate attention maps. This enables the effective integration of global and local features, enhancing the overall quality and robustness of the fused images.

Dual-Branch Architecture:
The integration of a denoising diffusion probabilistic model (DDPM) and a convolutional neural network (CNN) allows DCAFuse to leverage the strengths of both global and local feature extraction. The DDPM-based branch excels at constructing global information, while the CNN-based branch focuses on capturing local details through multi-scale convolutional kernels, resulting in a more comprehensive and detailed fused image.

Enhanced Complementarity with Novel Loss Function:
DCAFuse introduces a novel loss function based on cosine similarity and a unique denoising timestep selection strategy. These enhancements improve the complementarity of features extracted from the dual branches, ensuring that the fused images maintain a high level of detail and accuracy, outperforming other state-of-the-art methods in multiple image fusion tasks.
limitations: limitations: 1.Despite proposing the Cross-Temporal Feature Aggregator (CTFA) and Complementary Feature Aggregation Module (CFAM), the paper lacks an in-depth explanation of the physical significance of these modules. Specifically, the paper does not adequately explain the physical basis of CTFA in aggregating features across multiple time steps and how it effectively reduces the impact of noise on feature extraction. This lack of explanation undermines the credibility and interpretability of its technical approach.
2.The study lacks innovation in the field of multimodal image fusion. Although it adopts a dual-branch architecture combining diffusion models and CNNs, this combination has already been applied in existing literature. The paper fails to clearly demonstrate significant improvements or unique advantages of its method over existing technologies.
3.In the abstract, the paper highlights the Complementary Feature Aggregation Module (CFAM), but this module is not well showcased in Figure 3. Additionally, the aggregation method employed by CFAM does not seem to offer significant innovation compared to traditional gating mechanisms.
paper_summary: paper_summary: This paper proposes a DCAFuse framework for multi-modality image fusion. It includes a novel complementary feature aggregation module, a cosine divergence loss function, and a unique denoising timestep selection strategy. Extensive experimental results show that the proposed DCAFuse outperforms other state-of-the-art methods in multiple image fusion tasks.
strength_weakness: strength_weakness: 1) This paper introduces a novel view of the diffusion model (i.e., global information extractor).
2) The proposed fusion method does not need to perform too many iterations.
3) The experiments showed the promising results of the proposed method.
limitations: limitations: 1）The Transformer is famous for its capability of modeling long-range dependencies. However, in the Related Work section, there appears to be no comparison between the diffusion model and the transformer. I recommend adding a discussion about the transformer to emphasize the advantages of the diffusion model for your specific task. Moreover, in your experiments, only one transformer-based method has been used for comparison.
2) About the Cross-Timestep Feature Aggregator, I suggest you provide a more detailed explanation and motivation. From Fig. 2, it may cause some misunderstandings, such as information redundancy.
3) In Fig. 5, I suggest you add more contents. MMIF is not a generative task. So I think it is better to add the original inputs and final results.
paper_summary: paper_summary: This paper proposes a dual-branch deep network model (named as DCAFuse) for multi-modality image fusion, which is built on DDPM and CNN and equipped with a complementary feature aggregation module. Experimental results on two fusion tasks are reported to demonstrate the effectiveness of the model.
strength_weakness: strength_weakness: The paper is well organized and easy to follow. The experimental evaluation is adequate.
limitations: limitations: 1. The proposed model is a dual-branch framework, including CNN-based branch and diffusion-based branch. The CNN-based branch aims to extract local detailed features, while the diffusion-based branch is to capture the global information. However, in the diffusion-based branch, the output of DDPM is fed into the denoising U-Net. It is known that U-Net is a CNN, which is not competent to extract global information due to the limited receptive field of convolution kernels. It is unclear how to ensure that the diffusion-based branch based on U-Net can work as desired, i.e., capturing global information. 

2. In Section 3.5, the paper claims “Fig. 5 (c)-(d) demonstrates that, with the progression of the sampling timestep, the diffusion model progressively captures background features, resulting in effective modeling of global information at 𝑡 = 200.” It is unclear to me why capturing background features can result in effective modeling of global information? How to define the global information?

3. One contribution of the paper is to introduce a cosine divergence loss, i.e., cosine similarity, which is defined by Eq. (10). However, this definition is inaccurate. Specifically, it measures the similarity between F^{C} and F^{D} by vector dot product. It is unclear how to compute the vector dot product of two tensors F^{C} and F^{D}.