Title: Revisiting Generative Infrared and Visible Image Fusion Based on Human Cognitive Laws
Venue: NeurIPS 2025
Accepted: yes
Keywords: Human cognitive law

Information bottleneck

Diffusion model

Infrared and visible image fusion

Physical law
Abstract: Existing infrared and visible image fusion methods often face the dilemma of balancing modal information. Generative fusion methods reconstruct fused images by learning from data distributions, but their generative capabilities remain limited. Moreover, the lack of interpretability in modal information selection further affects the reliability and consistency of fusion results in complex scenarios. This manuscript revisits the essence of generative image fusion under the inspiration of human cognitive laws and proposes a novel infrared and visible image fusion method, termed HCLFuse. First, HCLFuse investigates the quantification theory of information mapping in unsupervised fusion networks, which leads to the design of a multi-scale mask-regulated variational bottleneck encoder. This encoder applies posterior probability modeling and information decomposition to extract accurate and concise low-level modal information, thereby supporting the generation of high-fidelity structural details. Furthermore, the probabilistic generative capability of the diffusion model is integrated with physical laws, forming a time-varying physical guidance mechanism that adaptively regulates the generation process at different stages, thereby enhancing the ability of the model to perceive the intrinsic structure of data and reducing dependence on data quality. Experimental results show that the proposed method achieves state-of-the-art fusion performance in qualitative and quantitative evaluations across multiple datasets and significantly improves semantic segmentation metrics. This fully demonstrates the advantages of this generative image fusion method, drawing inspiration from human cognition, in enhancing structural consistency and detail quality.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces HCLFuse, a novel generative framework for infrared and visible image fusion, which is conceptually motivated by principles from human cognition. \nIt consists of two stage:\n1. In the first stage, the authors design a Variational Bottleneck Encoder (VBE) to learn a compact yet comprehensive latent representation from the source images. The design and optimization of this encoder are formally grounded in the Information Bottleneck (IB) theory.\n2. In the second stage, a conditional diffusion model synthesizes the final fused image from this latent code. The key innovation in this stage is that the reverse diffusion process is not purely data-driven. Instead, it is guided at each timestep by a set of proposed physics-based constraints, including terms that enforce heat conduction properties and structural preservation.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. The paper's primary strength is its exceptional novelty. It introduces a highly principled framework for image fusion, conceptually grounded in human cognition and physical laws. This represents a significant and refreshing departure from conventional methods that often rely on data-driven regression with generic losses (e.g., L1 or SSIM), opening a promising new direction for the field.\n2. The paper is exceptionally well-written and logically structured. The authors present a compelling narrative that connects cognitive science, information theory, and physics-based modeling. Crucially, they support their claims with both detailed theoretical derivations (Theorems 1 and 2) and strong empirical evidence from extensive experiments.\n3. The proposed method, HCLFuse, achieves state-of-the-art results. It demonstrates superior performance against a wide range of existing methods across multiple public datasets and evaluation metrics, in terms of both quantitative scores and qualitative visual quality.\n\n## Weaknesses\n\n1. The selection of baseline methods for comparison could be more contemporary and representative of the current state-of-the-art. A significant portion of the compared methods in Table 1 are from journals, which may have longer publication cycles and may not reflect the latest advancements. The comparison would be more compelling if the proposed method were benchmarked against more recent, highly relevant methods from top-tier conferences. For instance, recent works like TarDAL, DDFM, and the emerging diffusion-based or text-guided methods such as SegMIF, Text-IF, or Text-Difuse would serve as more challenging and appropriate baselines to truly validate the claimed state-of-the-art performance.\n2. Insufficient Ablation of Individual Physical Constraints: In Section 3.3, the authors propose three distinct physical constraints: Heat Conduction, Structure Preservation, and Physical Consistency. However, the ablation study in Section 4.6 only evaluates their collective impact (the \"w/o TPG\" variant). This makes it impossible to understand the individual contribution of each constraint. The paper would be much stronger if it included a more fine-grained ablation study to demonstrate which of these physical principles are most critical to the model's success.\n3. Ambiguity in Constraint Application: There is a minor ambiguity in the method's description in Section 3.3. Equation 14 applies the Heat Conduction constraint to the predicted clean data $\\hat{z}_0$. However, the subsequent Equations 15 and 16, for the Structure and Consistency constraints, are formulated with the variable $z$. It should be clarified whether all constraints are applied to $\\hat{z}_0$ or consistency, or if they are indeed applied to different $z_t$ during the whole recovery process.\n4. Lack of Hyperparameter Analysis: The model introduces a significant number of hyperparameters, particularly the initial weights $\\lambda_{i}^{0}$ and the decay factor $\\gamma$ in the Time-varying Physical Guidance (TPG) mechanism. The paper does not provide details on how these were selected or discuss the model's sensitivity to them. A discussion of the tuning strategy and a sensitivity analysis would significantly improve the paper's claims of robustness and its practical value for reproducibility.\n\nQUESTIONS:\nI thank the authors for this novel and inspiring work. To further strengthen the paper and address the potential weaknesses, I hope the authors can provide further elaboration on the following key points. My final evaluation will be heavily influenced by the responses to the **weakness 1** and **weakness 2**.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an infrared and visible image fusion framework based on generative models. This framework designs a multi-scale variational bottleneck encoder to extract structured low-level features and a physics-aware diffusion process to guide this extraction process. They provide extensive experiments to verify the effectiveness of this work across multiple fusion scenarios and in downstream semantic segmentation tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The idea of revisiting generative infrared and visible image fusion from the perspective of human cognitive principles is novel.\n2. The authors present theoretical constructs and theorems to substantiate the proposed method.\n3. The authors conduct extensive evaluations, proving the advanced performance of this work.\n\nWeaknesses：\n1. In line 162, Why are only infrared images transformed? And what are the specific transformations applied? Providing detailed explanations would improve the clarity and readability of the manuscript.\n2. In line 174, $\\M_s$ represents mask weights that determine the importance of each feature. However, it is unclear how these weights are derived. Please provide a detailed explanation.\n3. Also, the process of obtaining $\\M_{heat}$ and $\\M_{stru}$ should be described. Providing further detailed explanations would enhance the understanding of the overall method.\n4. Have the authors evaluated the necessity of the aforementioned mask weights and analyzed their impact on the overall experimental results? It would be helpful if the authors could provide additional explanations to clarify this aspect.\n5. In Table 5, the content in the last column appears to exceed the boundary. The authors may need to check the table formatting to ensure proper alignment.\n\nQUESTIONS:\na)\tTo my understanding, MSRS serves as an enhanced version of the MFNet [IEEE IROS, 2017: 5108-5115.] dataset. Have the authors considered evaluating the proposed algorithm on MFNet to assess its performance on lower-quality data?\nb)\tIn line 134, Clark's viewpoint is mentioned without a corresponding citation.\nc)\tBesides, please address the concerns in the Weakness part.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a generative infrared and visible image fusion method based on human cognitive laws, named HCLFuse. By integrating a variational bottleneck encoder and a physics-guided diffusion model, it enhances the structural consistency and detail quality of fused images.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper innovatively introduces human cognitive laws into generative image fusion, proposing a novel HCLFuse framework. Its unique innovations lie in the design of a multi-scale variational bottleneck encoder and a time-varying physics-guided mechanism, which distinguish it from both traditional methods and existing generative approaches.\n\nHCLFuse demonstrates high fusion quality: it achieves optimal performance across multiple quantitative metrics (e.g., AG, SF, DF, QSF) on datasets such as MSRS, TNO, and FMB. In qualitative evaluations, it effectively preserves complementary features, exhibits clear details, and ensures strong structural consistency, outperforming various comparative methods.\n\nHowever, the theoretical framework is overly abstruse, imposing certain requirements on readers' prior knowledge. This may hinder the accessibility of the research findings to a broader audience.\n\nQUESTIONS:\nHere are several problems and suggestion for authors to improve their paper.\n\n(1) In Eq.6, how are the mask weights $M_s$ generated? It is recommended to provide a sufficient explanation.\n\n(2) The Method section lacks an overview of the entire HCLFuse process. It is recommended that the authors add a textual description or pseudocode in the main text or appendix to outline the complete workflow from inputs (X, Y) to output Z.\n\n(3) VIF (Visual Information Fidelity) demonstrates greater advantages in perceptual quality assessment and is more widely applied. It is recommended to replace the QSF metric with VIF, which is also more relevant to the \"Human Cognitive Laws\" mentioned in the paper's title.\n\n(4) The ablation experiments are insufficiently comprehensive, as the authors do not validate the role of the heat conduction constraint. It is recommended to add relevant experiments to address this gap.\n\n(5) In the ablation experiments subsection, what is DDIM? What is \"deterministic sampling\"? It is recommended to provide clear explanations.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel infrared and visible image fusion method, HCLFuse, aiming to address the limitations of existing generative fusion techniques by incorporating \"human cognitive laws\". The paper designs a method that integrates a variational bottleneck encoder and a physics-guided diffusion model. It demonstrates excellent quantitative and qualitative performance on multiple benchmark datasets, with notable improvements in downstream semantic segmentation tasks. The writing is clear, the experimental section is well-organized, and the ablation studies are fairly comprehensive.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper clearly outlines the challenges of image fusion, the limitations of existing methods, and its proposed solutions. The presented framework structure is also intuitive and easy to follow.\n\n2. The paper presents SOTA performance on several infrared and visible image fusion datasets (MSRS, TNO, FMB). Particularly in qualitative results, HCLFuse demonstrates impressive capabilities in detail preservation, structural consistency, and cross-modal information integration.\n\nWeaknesses:\n\n1. The paper elevates mathematical tools like IB and OT to the status of \"human cognitive laws\". Applying them to image fusion, even if their functionality \"resembles\" certain aspects of human cognition (e.g., information filtering, modality alignment), does not directly equate to them being actual manifestations of \"human cognitive laws\" .\n\n2. The \"physical law\" constraints claimed in the paper (Heat Conduction, Structure Preservation, Physical Consistency), in the technical implementation, appear to be common image processing priors or regularization terms. For instance, using the Laplacian operator to promote smoothness is essentially a high-pass filtering or edge detection operation, commonly used in image denoising or enhancement.\n\n3. Despite the excellent model performance, the core components (VBE as a variant of VAE, introducing prior knowledge into diffusion models) are incremental contributions.\n\nQUESTIONS:\nPlease refer to the Weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-05"}