Title: ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image Translation

Thermal Infrared Imagery

Flow-Based Generative Model

Remote Sensing

Multi-Dataset Joint Training
Abstract: Paired RGB-thermal data is crucial for visual-thermal sensor fusion and cross-modality tasks, including important applications such as multi-modal image alignment and retrieval. However, the scarcity of synchronized and calibrated RGB-thermal image pairs presents a major obstacle to progress in these areas. To overcome this challenge, RGB-to-Thermal (RGB-T) image translation has emerged as a promising solution, enabling the synthesis of thermal images from abundant RGB datasets for training purposes. In this study, we propose ThermalGen, an adaptive flow-based generative model for RGB-T image translation, incorporating an RGB image conditioning architecture and a style-disentangled mechanism. To support large-scale training, we curated eight public satellite-aerial, aerial, and ground RGB-T paired datasets, and introduced three new large-scale satellite-aerial RGB-T datasets--DJI-day, Bosonplus-day, and Bosonplus-night--captured across diverse times, sensor types, and geographic regions. Extensive evaluations across multiple RGB-T benchmarks demonstrate that ThermalGen achieves comparable or superior translation performance compared to existing GAN-based and diffusion-based methods. To our knowledge, ThermalGen is the first RGB-T image translation model capable of synthesizing thermal images that reflect significant variations in viewpoints, sensor characteristics, and environmental conditions. Project page: http://xjh19971.github.io/ThermalGen


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce ThermalGen, a flow-based generative model designed for RGB-to-Thermal (RGB-T) image translation. ThermalGen addresses the scarcity of paired RGB-T data, which is crucial for multi-modal tasks, by synthesizing thermal images from readily available RGB inputs. The model incorporates an RGB image conditioning architecture and a style-disentangled mechanism for generating thermal images that reflect significant variations in viewpoints, sensor characteristics, and environmental conditions. In other words, they treat the unique relationship between a collection of a specific kind of imagery and its thermal counterpart as a \"style\" (in the sense of style transfer), and disentangle this relationship so that their model can adaptively generate thermal imagery based on the kind of RGB input. To support large-scale training, the authors curated eight public RGB-T datasets and introduced three new large-scale satellite-aerial RGB-T datasets: DJI-day, BosonPlus-day, and BosonPlus-night. Their evaluations demonstrate that ThermalGen achieves comparable or superior translation performance compared to existing GAN-based and diffusion-based methods. The authors state that their model is the first with such broad RGB-T synthesis capabilities.\n\nOn the specifics of their evaluation, the evaluation of ThermalGen involved experiments across various RGB-T benchmarks, including satellite-aerial, aerial, and ground datasets. The performance was assessed using four image quality metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), Fréchet Inception Distance (FID), and Learned Perceptual Image Patch Similarity (LPIPS). ThermalGen was compared against widely adopted GAN-based methods (pix2pix, CycleGAN, pix2pixHD, VQGAN) and diffusion-based methods (DDIM and DiffV2IR). Ablation studies were conducted to determine optimal architectural designs and hyperparameter configurations, examining different transformer and patch sizes, RGB image conditioning designs (cross-attention versus concatenation), and various style embedding settings. Qualitative results were also presented to visually compare ThermalGen's output with baselines across diverse datasets and conditions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Quality:\n    - Technically sound submission with well supported claims via an extensive evaluation across numerous benchmarks and competing baselines. Specifically, nine datasets, four metrics, and eight competing baselines, covering GAN-based methods and diffusion-based methods.\n    - The authors are honest in evaluating both the strengths and weaknesses of their work. For example, they point out ThermalGen's limitations when dealing with settings with homogeneous samples, low-contrast thermal features, or extreme lighting conditions.\n- Clarity:\n    - The language is clear, the narrative flows smoothly and sensibly, and the quantitative and qualitative results are presented very clearly. It was an incredibly easy and smooth read and I feel confident that I can replicate their results from reading the main paper and appendix.\n- Significance:\n    - Although the quantitative results didn't blow me away (their model is the best on average, but it wasn't a clean sweep), it wasn't until I saw the qualitative results when I was really impressed. Specifically, Figure 5. There is a very clear difference between the quality of ThermalGen's outputs and the competing models'. It's quite significant and I believe this alone would convince researchers working in this space to build off of this model and treat it as the new state of the art.\n- Originality:\n    - Although generative modelling of RGB and thermal image pairs has been done many times, this is the first paper to use a flow-based generative model, as far as I know. The authors also introduce their own unique modifications to improve the modelling to make the model adaptive to other settings via the style embedding.\n    - The authors introduce three new large-scale datasets, which is almost always a boon for the research community. Curating high quality and large scale datasets is difficult work and I commend the authors for doing so, even if it wasn't truly necessary for motivating their model.\n    - Related work has been adequately cited and the authors has made it clear how their work differs from previous contributions.\n\nWeaknesses: No significant weaknesses worth mentioning that I can think of.\n\nQUESTIONS:\nI don't have any questions since this paper was a delight to read and felt very wholly packaged in terms of problem chosen, motivation, experiments performed, and results.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-08"}

comments: SUMMARY:
This paper presents ThermalGen, a flow-based model for RGB-to-thermal image translation. The method introduces an RGB conditioning architecture and style-disentangled mechanism, supported by newly curated datasets including three large-scale satellite-aerial RGB-T pairs.

STRENGTHS AND WEAKNESSES:
Strengths:
- The contribution of three high-quality RGB-thermal datasets is particularly valuable given the scarcity of such data in this field. The planned open-source release would greatly benefit the research community.
- Experimental results demonstrate clear advantages of ThermalGen over competing methods.
- To my knowledge, this represents a comprehensive flow-based approach for RGB-thermal translation, presented in a three-in-one framework. Both novelty and technical contributions meet the standard for NIPS acceptance.

Weaknesses:
- The comparison tables include too few diffusion-based methods (which appear limited in scope). More comprehensive comparisons with general image-to-image approaches (e.g., BBDM) would better demonstrate the method's generalization capability.

QUESTIONS:
- Does the model demonstrate sufficient generalization? Could you validate its effectiveness on thermal-to-RGB tasks as well? Relevant references include:

[1] T2V-DDPM: Thermal to Visible Face Translation using Denoising Diffusion Probabilistic Models

[2] DiffTV: Identity-Preserved Thermal-to-Visible Face Translation via Feature Alignment and Dual-Stage Conditions

LIMITATIONS:
The authors have adequately illustrated limitations in the main text.
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-06-18

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a flow-based generative model for RGB-to-thermal (RGB-T) image translation, addressing the scarcity of paired RGB-thermal data. Specifically,  first, it introduces an Adaptive flow-based architecture with RGB conditioning and style-disentangled embeddings to handle variations in sensors, viewpoints, and environmental conditions.\nSecond, it proposes Three new satellite-aerial datasets (DJI-day, BosonPlus-day, BosonPlus-night) for large-scale training. Third, it achieves Superior performance over GAN/diffusion baselines (e.g., pix2pix, CycleGAN, DiffV2IR) across satellite, aerial, and ground benchmarks, validated via PSNR, SSIM, FID, and LPIPS metrics.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The flow-based model for RGB-T translation with style disentanglement is reasonable and effective for this image translation task.\n2. Unified training on 200k+ diverse samples across 13 datasets shows the potential scalability of this model.\n3. Robust performance across domains (satellite, aerial, ground) without retraining.\n4. Three new datasets are proposed to contribute to the community.\n\nWeaknesses:\n1. The motivation of this paper is unconvincing. Why need to translate RGB images into thermal images? They are within different modalities and different visual details. For example, thermal images may contain more texture details of objects than RGB images. If the proposed model works, it is better to show such differences. \n2. This paper shows significant performance drops in some extreme conditions, for example, blurred distant objects (FLIR dataset) and low-contrast thermal images (Boson-night).\n3. The Fixed per-dataset embeddings of the style embedding may not capture intra-dataset variations. However, the authors do not investigate this issue.\n\nQUESTIONS:\n1. Why use dataset-level embeddings instead of learning styles directly from image content (e.g., via attention)? Could this limit adaptation to unseen styles?\n2. The motivation of this paper is unconvincing. Why need to translate RGB images into thermal images? They are within different modalities and different visual details. For example, thermal images may contain more texture details of objects than RGB images. If the proposed model works, it is better to show such differences. \n3. Thermal images follow physical laws (e.g., heat dissipation). Could incorporating physics-informed losses (e.g., PDE-based regularization) improve accuracy?\n4. Flow-based sampling (50 denoising steps) is slower than GANs. Can distillation or quantization achieve real-time translation?\n5. Datasets focus on urban/wilderness—how would ThermalGen perform in industrial/medical thermal domains?\n\nLIMITATIONS:\n1. The motivation of this paper is unconvincing. Why need to translate RGB images into thermal images? They are within different modalities and different visual details. For example, thermal images may contain more texture details of objects than RGB images. If the proposed model works, it is better to show such differences. \n2. This paper shows significant performance drops in some extreme conditions, for example, blurred distant objects (FLIR dataset) and low-contrast thermal images (Boson-night).\n3. The Fixed per-dataset embeddings of the style embedding may not capture intra-dataset variations. However, the authors do not investigate this issue.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addressed the scarcity of RGB-thermal imaging data and proposed the ThermalGen model based on a flow generation framework. By integrating an RGB conditional architecture with a style disentanglement mechanism, this model could generate high-fidelity thermal images across different viewpoints, sensors, and environments. Additionally, three new satellite-aerial RGB-T datasets were introduced. In multiple benchmark tests, the model outperformed existing GAN and diffusion models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1) ThermalGen introduced a flow-based generative model with style-disentangled mechanism, which was the first to explicitly model variations in viewpoints, sensors, and environments in RGB-T translation. The use of adaptive layer normalization (adaLN-Zero) and dataset-specific style embeddings enabled robust generation across diverse styles without retraining.\n2) The paper released three new large-scale satellite-aerial RGB-T datasets (DJI-day, BosonPlus-day/night), covering diverse times (day/night), sensor types (Zenmuse H20T, BosonPlus), and geographic regions.\n\nWeaknesses:\n1) The style embeddings were vaguely described, as they attributed diurnal environments, sensor styles, and viewpoint variations entirely to datasets without elaborating on how to disentangle these three factors. Moreover, the study did not use the control variable method to verify whether controlling a certain variable could make the generated images change and conform to reality.\n2) The paper did not compare with other style transfer methods (such as CLIP-guided embeddings or text-based conditioning methods), which limited the universality of its conclusions. The ablation study on style embeddings (Figure 4 (c)) could further analyze the encoding methods of different styles in the latent space.\n3) ThermalGen underperformed on low-thermal-contrast datasets (Boson-night), homogeneous scenes (LLVIP), or under extreme lighting conditions (FLIR). Although the authors attributed these issues to data characteristics, they did not propose specific solutions (such as contrast enhancement modules or domain-specific data augmentation) to mitigate these problems.\nminor：\n4) The paper failed to specify the computing resources required for training ThermalGen, such as GPU types, training time, and memory usage.\n5) The flow-based model relied on the SiT framework, but the paper did not theoretically demonstrate why flow models were more suitable for RGB-T image translation than diffusion models, especially in capturing cross-modal dependencies.\n6) The evaluation was mainly limited to image alignment tasks, with fewer insights into downstream applications such as object detection.\n\nQUESTIONS:\nThe paper primarily focused on integrating existing datasets with its newly proposed datasets to enable RGB-to-IR translation across different styles, diurnal environments, and viewpoints. However, it did not mention how to disentangle these three factors (style, environment, viewpoint) nor conduct any controlled-variable experiments to verify whether manipulating a single variable could yield realistic translation results. Additionally, the study did not construct a 3D model; instead, it merely translated RGB images from different viewpoints without introducing any novel techniques for viewpoint transformation as the approach for handling viewpoint variations lacks distinctiveness beyond basic image translation. When these issues remain unexplained, the paper’s innovativeness is called into question.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}