ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation
Authors: Jiuhong Xiao
Roshan Nayak
Ning Zhang
Daniel Toertei
Giuseppe Loianno
Venue: NeurIPS 2025 poster ()
Keywords: Image Translation
Thermal Infrared Imagery
Flow-Based Generative Model
Remote Sensing
Multi-Dataset Joint Training
Primary area: deep_learning
TL;DR: An adaptive flow-based generative model for RGB-Thermal Image Translation, which can adapt to different sensors, viewpoints, and environments.

Abstract: Paired RGB-thermal data is crucial for visual-thermal sensor fusion and cross-modality tasks, including important applications such as multi-modal image alignment and retrieval. However, the scarcity of synchronized and calibrated RGB-thermal image pairs presents a major obstacle to progress in these areas. To overcome this challenge, RGB-to-Thermal (RGB-T) image translation has emerged as a promising solution, enabling the synthesis of thermal images from abundant RGB datasets for training purposes. In this study, we propose ThermalGen, an adaptive flow-based generative model for RGB-T image translation, incorporating an RGB image conditioning architecture and a style-disentangled mechanism. To support large-scale training, we curated eight public satellite-aerial, aerial, and ground RGB-T paired datasets, and introduced three new large-scale satellite-aerial RGB-T datasets--DJI-day, Bosonplus-day, and Bosonplus-night--captured across diverse times, sensor types, and geographic regions. Extensive evaluations across multiple RGB-T benchmarks demonstrate that ThermalGen achieves comparable or superior translation performance compared to existing GAN-based and diffusion-based methods. To our knowledge, ThermalGen is the first RGB-T image translation model capable of synthesizing thermal images that reflect significant variations in viewpoints, sensor characteristics, and environmental conditions. Project page: http://xjh19971.github.io/ThermalGen

ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image
Translation
Jiuhong Xiao New York University jx11900nyu.edd edu
Roshan Nayak New York University rn25888nyu.c edu edu
Ning Zhang
Technology Innovation Institute
ning. zhangetii.......
Daniel Tortei Technology Innovation Institute daniel. www.eeihhttaall..omomoommmmmo
Giuseppe Loianno
University California, Berkeley loiamnogbeerkeley. edu
Abstract
Paired RGB-thermal data is crucial for visual-thermal sensor fusion and cross- modality tasks, including important applications such as multi-modal image align- ment and retrieval. However, the scarcity of synchronized and calibrated RGB- thermal image pairs presents a major obstacle to progress these areas. To over- come this challenge, (GB-to-Thermmi (RGB-T) image translation has emerged a promising solution, enabling the synthesis of thermal images from abundant RGB datasets for training purposes. In this study, we propose ThermalGen, an adaptive flow-based generative model for RGB-T image translation, incorporating an RGB image conditioning architecture and style-disentangled mechanism. To support large-scale training, we curated eight public satellite-aerial, aerial, and ground RGB-T paired datasets, and introduced three new large-scale satellite-arria RGB- T datasets-DJlldaa Bosonplus-day, and desooplus-night-capure across diverse times, sensor types, and geographic regions. Extensive evaluations across multiple RGB-T benchmarks demonstrate that ThermalGen achieves compara- ble superior translation performance compared to existing GAN-based and diffusion-based methods. To our knowledge, ThermalGen is the first RGB-T T image translation model capable of synthesizing thermal images that reflect significant variations in viewpoints, sensor characteristics, and environmental conditions. Project page: xjh19971, 1er.i...ittu.... io/ ThermalGen
Introduction
Visual-thermal sensor fusion and cross-modality downstream tasks 29 become increasingly important for integrating visual and thermal data to achieve robust perception under challenging conditions such as low illumination and adverse weather. While deep learning methods [24) have shown promise in these areas, they typically depend on paired RGB-Thermal (RGB-T) data for training. However, such datasets are limited in availability and often difficult generalize across applications due to domain gaps. Additionally, the cost of collecting synchronized and calibrated RGB-T T data substantial. These limitations motivate the adoption of generative models to synthesize thermal images from RGB inputs as a more scalable and flexible alternative.
50
57
58 42 have
Synthesizing thermal data construct paired RGB-T datasets brings several advantages for cross- modality downstream tasks. First, this method ensures perfectly aligned details between real RGB images and their synthesized thermal counterparts, providing high-quality data for tasks that de-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Viewpoint Variation R
Day-night Change
Sensor Variationa
Zenmuse H20T BosonPlus
Environmental Change
ThermalGen exhibits robust performance in RGB-to-thermal image translation under diverse conditions. We present RGB inputs alongside generated thermal images using ThermalGen across a range challenging variations, including viewpoint variation, day-night change, sensor variation, and environmental change. Variations illustrated between the two rows in each group.
Figure 1:
mand precise cross-modaa correspondence, such as dense feature matching 45 and keypoint matching [30]. Second, it enables researchers to exploit the vast repositories of publicly available RGB data, substantially expanding the scale and diversity of training datasets beyond the limited scope of hardware-captured RGB-T pairs, which require specialized calibrated sensors [32 47
and often suffer from restricted viewpoints 19 27). Third, generative models can simulate various thermal characteristics and environmental conditions from single RGB input, thereby enhancing the robustness of downstream models to variations in thermal sensors and environmental settings.
have demonstrated promising results by leveraging synthesized RGB-T datasets to improve model performance downstream tasks significantly. In this work, we introduce an adaptive RGB-to-thermal image translation model (Fig. [. aimed high-fidelity generation across diverse viewpoints, thermal sensors, and environmental factors. The main contributions of this paper are as follows:
Benefiting from these advantages, recent studies 57
58
56
50
20
We introduce ThermalGen, an adaptive flow-based generative model for RGB-T image translation. Trained on large-scale RGB-T datasets, ThermalGen incorporates RGB image conditioning architecture with a style-disentangled mechanism, enabling robust generation of thermal images across wide spectrum of RGB-T T styles influenced by thermal sensors, viewpoints, and environmental conditions. The model's architecture facilitates joint training with additional datasets, promoting adaptability across various applications.
We release three new satellite-aerial RGB-T paired datasets- DJI-day, Bosonplus-day, and Dosooplus-nighttccommp comppi--o aligned satellite RGB and aerial thermal images with variations in time of day, sensor type, and geographic region. Additionally, curate an extensive collection of public satellite- acrial, aerial, and ground RGB-T T datasets to establish a comprehensive benchmark for large-scale training and evaluation.
Extensive experiments validate ThermalGen's effectiveness across multiple benchmarks, demonstrating comparable or superior performance against both GAN- and diffusion- baselines. We present comprehensive quantitative metrics and qualitative analyses that illus- trate the visual fidelity and style embedding influence of our approach. To our knowledge, ThermalGen represents the first model capable of generating figh-fideltty thermal imagery across diverse sensor types, viewpoints, and environmental conditions.

---

2 Related Works
Synthesized Thermal Data for Downstream Tasks. Recent studies highlight the growing trend of leveraging synthetic thermal data enhance RGB-T downstream tasks. XoFTR (50] applies randomized cosine transformations to RGB images for improved cross-modal alignment. MIN- IMA 200] introduces a versatile data engine generating paired datasets across multiple modalities. MatchAnything [12) synthesizes various modalities from inputs like videos and multi-view images for generalizable alignment. AVIID [T]] uses synthetic aerial thermal data for object detection, while STGL [57] and STHN (58) [56] boost satellite-therme retrieval and homography estimation through synthetic RGB-T datasets. These efforts highlight thermal image synthesis as a practical solution to paired data scarcity. In this work, we propose an RGB-T image translation model for general-purposs use, offering broad applicability across diverse downstream tasks.
RGB-Thermal Image Translation. Thermal image synthesis from RGB inputs can be formulated as an RGB-T image translation task-an inherently complex problem due to three primary challenges. First, the limited availability of calibrated RGB-T datasets hinders the performance of supervised learning. Unsupervised approaches. such CycleGAN [3]. also struggle to bridge this gap (25). Second, RGB images inherently lack thermal information, requiring the model to infer thermal cues solely from semantic content. Third, inconsistencies in thermal sensors and camera viewpoints introduce a considerable gap between training and testing distributions [19]
primarily used GAN-based approaches 9 to synthesize photorealistic thermal images, while recent studies [62] 38 35 explore diffusion-based methods for higher fidelity. However, these works are mostly limited by narrow training datasets, reducing generalization across diverse distributions. DIffV2IR (38 is a notable exception, combining public datasets to train semantic- or text-guided diffusion model. Our work differs in two key ways: (1) we evaluate across satellite-aerial, aerial, and ground datasets, beyond DiffV2IR's ' driving- view- focused evaluation; and (2) our style-disentangled model generates target-style thermal images by conditioning on datasct-specific style embeddings, eliminating the need for retraining as required by DiffV2IR.
Earlier works [52]
19
25
21
10 26
Diffusion- and Flow-based General Image Translation. The success of diffusion- based 15
and flow-based general image generation [3I] has enabled advanced image translation methods like Palette [41] for tasks such as colorization and editing. Latent-space models (39 4 further im-
proved conditional generation quality, leading to range of high-performance translation techniques.
BBDM (8 models domain transitions via a stochastic Brownian Bridge, while Kwon et al. [23]
use contrastive learning on DINOv2 (36] features for content-style fusion. Recent style transfer methods 44 3
61 [55 5] also explore text- and woyle-image-guicee guided transfer. However, RGB-T image translation poses unique challenges due fundamental modality differences, which prevent direct weight sharing between RGB and thermal domains. Additionally, effective translation requires learning complex cross-mows modal relationships across diverse RGB-T datasets. These challenges hinder the applicability of RGB style transfer methods for robust RGB-T image translation model.
3 Methodology
3.1 Preliminaries
We adopt flow-based generative paradigm [3] for thermal image synthesis and provide an overview of their underlying process, following the perspective introduced by the Scalable Interpolate Trans- former (SiT) 334 Let Zo ~ p(z) denote the latent variable from data, accordance with the latent diffusion paradigm [39], and let c NN I) represent standard Gaussian noise. The diffusion process is formulated as a time- dime-dependent continuous process over [0, 1], given by:
2, = a,20 + G,C.
Here, a is decreasing function with ag = 1 and a₁ O, while d; is increasing function such that do = and σ = 1. A simple illustrative example of these schedules is 0r = - and o; = t. with derivatives de -1 and d; = 1. This process can be modeled using probability flow Ordinary Differential Equation (ODE):
2, v v(zi,t). t),
(2)
3

---

RGB-T Paired Datasets
Datwonl-Srweli Style Embedding ngs
ThermalGen Framework
Sythsized Thermal imagas
Flow-Based Generative Model
Deleoei (aatiilanSsriell
Uncondirnn
Delsoee- Spcorfc Enbwlding
Datrewn-
Y1
Uncondioonal
Entowddingg
Yuu
Hetal Notarr
ng
Syca!
(Ground)
XPNB LIS
Dr
CDE
Timestop
Samplr
Roua
Noiced
$
Generative Model
Aup?
Lalent
Scale and Shill Normlizztion Farometor
Deranee C. (Aen
Xasl
Bacn
sefeas Pacm-
Froren Paremeters Turable Parameters
Rx7
MulliHoad
Seft Albertion
Mulliheedd
Gross Atlaniion
Fedd
Pointwis
AcT
%
Raua
Concatination
AQ Crose-AAth
Imagen Style Canditininng
Elumn
RGB image Encoder
RGB Image
Patasen-Speafte Style Embecding
c
Concatenation Operation
Dy:
Image
RAGS
Ym
Dataset Unconditiral Entedcing
Q:
Query Input
Witclly Prudiction Model
Reconstructed Tharmal laland
0yg
Style and Timu Conctional Embudkingg
Switch
Figure 2: Overview of TeermalGen. We sample paired RGB-T data from a diverse collection of satellite-aerial aerial, and ground datasets for training and evaluation. During thermal image synthesis, the generative model predicts the velocity for Ze,T, conditioned on the timestep t. the selected dataset- specific style embedding y, and RGB latent ZRGB. After I steps of velocity prediction and denoising. the thermal decoder Dr used to decode 20,T and generate the thermal image Xy.
where denotes the velocity function, defined as the expected time derivative of Zz:
o( Zt. 1) = 1 z] = ág 1e z] + bt E, 24 = z).
To approximate this velocity function, employ neural network vo parameterized by 0, which is trained by minimizing the following objective:
Ez.. [2),)(e, v(ze, t)]["]
Loow
Once trained, the model can obtain the denoised latent variable 2, from the terminal noise state 21 e by integrating the learned reverse flow dynamics governed by tig
3.2 www---mag--ooniiiinnee and Style-Disentangled Generative Model
The overview of ThermalGen is shown in Fig. 2 Our objective is to synthesize thermal images conditioned on an RGB image and a style embedding disentangled from the model parameters. We define the RGB-T style as the mapping relationship between RGB and thermal images, which is influenced by factors such as thermal sensor characteristics, camera viewpoints, and environmental conditions. Let p(xr) denote the distribution of thermal image Xy E ff #xxxx ThermalGen models the conditional distribution p(xr XRGB- y), which captures the likelihood of generating X1 given RGB image XRGB R xXxxx and dataset-specine style embedding y E F R Here, H and W refer to the image height and width, and / denotes the style embedding dimensionality.
Thermal Image Encoder and Decoder. To enhance computational efficiency, we adopt the latent diffusion framework [39]. Within this framework, flow and diffusion-bar operations are performed on a latent variable Zt ET(xT), where ET is the thermal image encoder. The latent representation Zt E IR 4 4 xC is a compressed form of the thermal image, with spatial dimensions reduced by factor of and C' representing the number of latent channels. The synthesized thermal image λT is reconstructed from the latent variable êT, obtained through the flow-based generative model (Section 3.I. using decoder Dr, such that X; Dr(2r).
Flow-based Latent Generation. We employ SiT 33 for flow-based latent generation (Section 3.1 leveraging its scalable diffusion transformer architecture [37] Let vy (2,,1, t, ZRGB y) denote the velocity predicted by the SiT blocks, conditioned on the noised thermal latent êG, at timestep I, RGB latent RRGB, and the style embedding y. Using the ODE sampler, we update the thermal latent to the next timestep, obtaining T. By iteratively predicting velocities and denoising over steps, progressively reconstruct the thermal latent žo, from the initially noised latent 21,T.

---

Table 1: Overview of RGB-T paired datasets. Sample numbers that correspond to each dataset's Train/validation/tess splits are provided. A total of 200k samples are used for large-scale training. "For the satellite-aerial datasets, the number of samples is computed using a sample stride of 35m, indicating the gap between adjacent samples.
Dataset Name
Environment
Resolution
Sample Number
Splits
Downstream Tasks
Satellite-errial Datasets
Boson-night 52
Wild
512 x 512
13, 011/111,1110 011 011/21, 568
train/valitstt trin truin/val train/val
Image Alignment
DJI-day' (Ours)
Wild
512 512
19,392 50, 18 329
Image Alignment
Bosonplus-day" (Ouns)
Wild
512 512
Image Alignment
Bosonplus night" (Ours)
Wild
512 x 512
50, 695/ 10, 146
Image Alignment
Aerial Dutasets
CalTech 25
Wild
960 600
282
trann train/test truin/val train/test
Semantic Segmentation
LLVIP 19
Urban
1280 x 1024
12, 025/3, 403
Human Detection
NIL-CU
Wild
2706 1980
2, 65533355 /85
Human Detection
AVIID 10
Urban
464 464
2,412/800
Object Detection
Ground Datasets
MºFD [3
Diverse
Diverse
3, 360366/844 /840
train/test train/test
Object Detection
reihuug-day
S1
Urhan
1920 x 650
12, 168/32
Semantic Segmentation
Freiburg-night
Urban
1920 650
8, 083/32
train/test
Semantic Segmentation
SMOD-day 2
Urban
640 512
5,8
truin
Object Detection
SMOD-night
Urban
640 512
3, 298
train
Object Detection
MSRS
Urban
640 480
1, 1833/361
train/test
Image Fusion
KAIST
17
Urban
6:10 512
8, 643
train
Human Detection
H.RR
8 59
Urban
64U x 512
4, 129/11013
train/test
Object Detection
Style-Disentangled Mechanism. To address the diverse RGB-T styles from different datasets, we introduce a style-disentangled mechanism that leverages dataset-specific style embeddings for condi- tional image generation. We define set of learnable style embeddings Y = (yo. Yn. Yun where n denotes the number of datasets or distinct user-defined RGB-T styles, and yun represents an unconditional style embedding used for generating thermal images without specifying style. Given a style embedding y; for the j" style and a timestep f, we adopt a-ero conditioning [37 to gen- erate corresponding condition embedding Cy,, adaLN-Zero applies adaptive layer normalization, where the scale and shift parameters are modulated based on y; and t. The selection of conditioning methods is motivated by the findings of AdaIN [16]. which demonstrated that altering normalization parameters effectively achieves style transfer by modifying feature statistics. For the unconditional style embedding Yun- we denote the resulting condition embedding as Cy During training, the model randomly selects between Cy.. and Cy enabling both Classifier-Fres Guidance (CFG) [4] and
generation. This design allows for straightforward extension additional datasets by appending new style embeddings, thus enhancing adaptability across varied domains.
RGB Image Conditioning Architecture. To incorporate RGB image information into the generative model, we utilize a pretrained KL-VAE encoder [39] as the RGB encoder EROB, which extracts the RGB latent representation ZRGB- We explore two variants for RGB image conditioning. ! Multi-head Cross-Artention (Cross-Attt): we use ZROB as the query, while 2,, serves as both the key and value within an additional cross-attention module embedded in the SiT blocks:
TrT Cross-Attn(zRee 2,,T, 2T,T),
(5)
where ž., represents the intermediate features produced by the attention operation. This module is inserted between the self-attention and pointwise feedforward layers in each SIT block. The query- key-value assignment is inspired by style transfer techniques 3 5) where the content representation
serves as the query and the style representation as the key and value. 2 Concatenation: Alternatively,
we can also concatenate ZRGB with द,, to form the input to the SiT blocks:
T Concatenate (2,,r ZRGB),
(6)
where ž denotes the input to the SiT blocks. This method enables convenient fine-tuning from pretrained SiT weights by directly extending the input representations.
4 Experiment Setup
Datasets. To train adaptive RGB-T translation model and ensure robust evaluation, we utilize over ten publicly available RGB-T paired datasets, applying a thorough process of curation, filtering, and preprocessing. This includes standardizing data formats, normalizing thermal data to the 8-bit range,
5

---

Boson-night
Dil-day (Uurs
Bosonplas-die (Ours)
Boscoplus-
Visual comparison between the Boson-night dataset [57 and our curated datasets. Each dataset illustrates differences in thermal sensors, lighting conditions, and geography. Columns show paired satellite RGB and corresponding 8-bit thermal images.
Figure 3:
aligning RGB and thermal image pairs, and removing unusable data and regions with invalid thermal readings. Additionally, we introduce three newly collected satellite-acrial acrial RGB-T datasets to enhance translation performance between remote sensing RGB and acrial thermal imagery. summary of the datasets used is presented in Table | The datasets are grouped into three categories: satellite-acrial datasets, aerial datasets, and ground datasets.
Satellite-Aerial Datasets. These datasets, such as the Boson-night dataset 5 and our curated datasets DJI-day, Bosonplus-day, and Bosonplus-night (where the prefix denotes the sensor model and the suffix indicates the time of data acquisition)-compp thermal images. Details about the data collection process are in Appendix Sec. A These aligned datasets enable range of applications, including paired image translation (57], multi-modal image alignment [58] [6). and multi- modal place recognition [57]. A comparison between the Boson-night contributions in terms of broader geographic coverage, increased diversity of thermal sensors, and the inclusion of both daytime and nighttime imagery. Figure offfer visual comparison that illustrates variations across thermal sensors, temporal differences, and the rich geographical diversity captured in our datasets.
paired satellite RGB and aerial
dataset [57 and our datasets is provided in Appendix Sec. [ highlighting
Aerial Datasets. These datasets include data captured by UAVs or surveillance cameras, featuring oblique aerial views of thermal images. We include four aerial datasets for our training and evaluation: CalTech (25 and NII-CU 44] contain thermal images captured in natural environments, while LLVIP features data from surveillance cameras in the urban or campus environment. Additionally, AVIID 10 contains acrial RGB and thermal image pairs collected via dual-light camera system in an urban environment, Most datasets were collected in daytime settings, though LLVIP and AVIID notably include nighttime RGB and thermal data.
Ground Datasets. These datasets consist of paired imagery from handheld or schicle-mountee cameras providing horizontal-vien thermal data. We incorporate six datasets for training and evaluation: The FD dataset [32] delivers RGB-T paired data across varied environments, including urban and wilderness settings, featuring broad range of objects captured in both daylight and nighttime conditions. The Freiburg [51) and SMOD 2 datasets contribute sequential street-view thermal imagery with day-night change. MSRS 48 provides aligned RGB-T image pairs with
high thermal contrast for object detection applications. KAIST Multispectral Pedestrian Detection Benchmark [17] offers regular traffic scenes captured via vehicle-mounte systems for pedestrian detection research. FLIR aligned dataset [59] provides RGB-T paired data from driving scenarios with diverse object classes, including vehicles, pedestrians, and traffic signage.
Metries. We employ four image quality metrics for evaluation: Peak Signal-to-Noiss Ratio (PSNR) quantifies image fidelity by measuring the mean-square-ervee Structural Similarity Index Measure (SSIM) (54) assesses image structural similarity. Fréchet Inception Distance (FID) [13] measures distribution similarity between predicted and ground truth imagery by comparing 2048-dimensional feature vectors extracted via the Inception v3 model [46]. Learned Perceptual Image Patch Similarity
(LPIPS) [60] quantifies perceptual similarity through features from AlexNet [22

---

Table 2: Comparison of RGB-T translation performance on satellite-aerial datasets between baseline methods and ThermalGen. Fine-tuned on FDFD; *Fine-tunee on FLIR. The best results are bold, and the second and third best are underlined. This is followed in subsequent tables.
Mathods
Categoris
PSNR1
SSIM1
Boson-nige night
[DD]
LPIPS.
PSNK
SSIME
Bosenplus-do day
FID.
LPIPS1
PSNRT
SSIM
Bescaplass night
FID.
LFIPS.
pa22pix 13157
Paired GAN
23.71
0.79
149.55
14.04
0.30
170.45
19,93
0.70
137.74
0.40
CycRGAN 03
Uppuired GAN
17:27
0.30
119.62
12.62
0.21
279.16
0.52
11.36
0.47
105.56
0.48
pa22icHD 55
106.33
12.85
0.21
657.65
0.71
207.32
14.10
185.41
DDIM
Paired Diftusion
18.31
203.05
12.50
0.20
15.23
0.72
112.35
Pained Difiusiin
141.27
12.42
DiAV2r [S]
315
Puired Diftusion
11:69
253.82
260.42
10.01
154.76
DIIVZIR
Pained Diflusiio
15:55
137.74
0.49
12.32
0.19
12.01
0.52
72.02
Thr
Psinsd Diffisiion
21.88
0.71
161.23
0.30
14.06
0.31
76.91
20.47
0.76
25,80
Table 3: Comparison of RGB-T translation performance on aerial datasets between baseline methods and Gen, 'Fine-tuned on MD + *Fine-tuned on FLIR; DDffVVIIR models use LLVIP test set for training.
Mathods
Cotegories
PSNRT
SSIM|
LLVIP
FIDL
LPIFS]
PSNR]
SSIMI
ND CU
FID.
L.PIS]]
PSNRT
SSIM1
AVIID
FID.
LPPPS]
pax2pex
Paiced GAN
12.00
0.37
526.14
0.53
17.31
168.77
0.37
21.41
0.61
146.26
CycleGAN
Unguined GAN
0.61
16.43
125.37
01.32
pis2gedH
1946
118.60
0.32
127.63
VQGAN
GIAN
0.36
273.05
0.58
15.53
175.37
DDIM
41
Paired Diffision
10:94
0:41
297.0
0.71
17.59
180.14
0.48
10.96
290.46
BBDM 23
Paired Diffision
313.54
0.67
14:36
118.99
18.53
141.06
0.31
DHrVIIR
Paired Diffisioo
22.17!
0.77
50.10'
12.38
0.63
159.99
0.50
18.17
0.53
$161
0.21
DIEV2IR
53
Paized Diffision
14.17
0.51
174.79
0.46
14.88
0.65
135.15
0.44
14.32
0.45
116.11
0.49
DIEV2IR
38
Paired Difftsiion
10.25
0,40
157.35
0.43
13.76
0.67
132.00
0.46
11.00
0.38
112.97
0.49
ThermolGen-1/2
Paired Diffision
11.12
0.34
235.00
0.51
26.44
0.92
69.30
0.21
24.89
0.75
29,85
0.13
Baselines. We evaluate ThermalGen against widely adopted GAN-based methods for RGB-T
image translation (25 57 10, including pix2pix [18], CycleGAN [33, pix2pixHD [53] and VQGAN (7). The pix2pix framework employs a U-Net -40--a generator and a patch-based discriminator, optimized with L1 loss. In comparison, piX2pixHD adopts ResNet blocks [I] and leverages LPIPS loss (60] to enhance perceptual quality. CycleGAN utilizes two generators and two discriminators to translate images into the target domain and reconstruct them back into the source domain, allowing training on unpaired data. VQGAN incorporates vector quantization with discrete codebook to improve perceptual quality. For diffusion- based baselines, we compare against DDIM [33 and DiffV2IR (8). all of which employ the latent diffusion model architecture [39] for thermal image generation. All baseline models are trained on our curated RGB-T dataset collection to evaluate cross-dataset generalization, except for Diff 38 for which use pretrained models.
Implementation Details. For joint training across multiple datasets, we randomly sample batches from all available training sets. During training, each image is randomly resized and cropped to 256 x 256 pixels. For evaluation, images are resized to 256 x 256 pixels with denoising steps T = 50. The style embedding dimensionality is 1024. Additional training details are in Appendix Sec.
5 Results
5.1 Cross-dataset RGB-T Image Translation Performance
Tables 4 present a comprehensive performance comparison between our proposed ThermalGen framework and ofate-of-the-art baseline methods across representative satellite-aerial, aerial, and ground RGB-T datasets. The results show that ThermalGen outperforms both GAN- and diffusion- based baselines across all categories. In particular, it achieves superior perceptual quality, as measured by FID and LPIPS, on datasets such as Bosonplus-day, Bosonplus-night, NII-UU, AVIID, FDFD, and MSRS. The performance advantage is especially notable when compared fine-tuned DIffV2IR models, which often perform well on isolated datasets but struggle to generalize across diverse imaging conditions. ThermalGen's 's robust adaptability stems from its architectural design and unified training strategy, which effectively captures the complex RGB-T patterns in heterogeneous datasets.
5.2 Ablation Studies
Figure dl presents the results of ablation studies conducted to identify the most effective architectural design and hyperparameter configurations for ThermalGen.
7

---

Table Comparison of RGB-T translation performance on ground datasets between baseline
methods and ThermalGen. "Fine-tunes on M MDFD; Fine-tuned on FLIR.
Calegoriss
PSNR1
SSIMI
FID]
LPIPS..
PSNK.
SSIM1
MSRS
FID.
LPIPS]
PSNRT
SSIM
FID.
LPIPS]
poc2pxx 153
Paired GAN
21.30
0.72
127.62
0.34
21.78
0.58
174.51
0.37
12.13
1154
224.11
0.44
CycleAAN
Unpuiraa GIAN
11.88
0442
171.37
0.56
11.42
0.35
99.14
0.53
11:15
0133
157.97
0.49
pa22icHD
55
Pained GAN
19.37
0.67
112.31
0.23
15.21
0.61
12105
0.35
15.63
0.49
164.53
0.37
VOGAN
1
Paired GAN
21.02
0.71
79.21
0.25
22.27
0.89
106.51
0.38
16.95
0.30
141.00
0.40
DDIM
Paired Diffusion
11:15
0.45
229.24
0.50
7.35
0.21
262.95
0.79
11.24
0.39
296
0.71
BBDM
X
Pained Diffusion
17:26
0.61
120.79
0.37
20.27
0.82
145 86
0.37
16.10
0:45
177.81
0.42
DirVIK Ex
Puired Diflusion
12:24
0.42
75.95
0.45
15.16
0.54
61.52
0.33
20.76
11.52
38.85
0.21
DirVIIX
2k
Puired Diflusion
22.76
0.79
37.74
0.13
10.16
0.30
104.01
0.57
11.29
0.44
106.1
0.45
DirV2RR
Paindl Diflusion
12.68
0145
78.45
0.43
6.80
0.17
113
0.64
22.39
0.57
37,83
0.18
ThermalGim-122
Painsi Diflusion
23.73
0.81
35,82
0.14
24.38
0.76
52.31
0.21
17.10
0.52
70.09
0.33
by
Image Canditioning
by Style Embbacinn Settings
110
100
100
300
50
31
Cross Apt
Concatentio
Uncanhlooa
Candional
acle
Madde Avrkitertue
Fmage Conkiining Aechitetre
Style Erivelding Sett
MSRS
ADD
FUR
Minh
Row MuwPer
Figure 4: Ablation studies by comparison of FID scores across different model designs.
Transformer and Patch Sizes. Figure 4 a) summarizes the performance of RGB-T image translation across various transformer sizes (SiT-B, SiT-L, SiT-XL) and patch sizes (2, 4. 8). The results clearly smaller variant (SiT-B)
demonstrate that larger transformer sizes (SiT-L and SiT-XL) outperform in FID scores across all evaluated datasets FM MSRS, and AVIID). Specifically, SiT-XL/2 achieves the lowest FID scores, indicating superior visual quality and realism. Additionally, models employing smaller patch sizes consistently yield better performance, suggesting that finer granularity for patches substantially enhances the generated image quality.
RGB Image Conditioning Design. Figure 4(b) presents an analysis of different configurations for RGB image conditioning, specifically comparing cross-attentinn and concatenation approaches over five evaluation datasets. The results indicate that concatenating RGB latents yields overall better FID performance than incorporating them as query inputs within cross-attention module.
Style Embeddings Settings. Figure 4cc) evaluates three style embedding configurations: dataset- unconditional style embedding (Unconditional), dataset-specific style embedding (Conditional), and Slassifier-FFee Guidance (CFG) with scale 2.0. For datasets with distinctive RGB-T styles (Bosonplus-day, Bosonplus-night, NIICUU), cataset-unconditional style embedding produces de- graded FID scores, while CFG (scale 2.0) outperforms using alone. This confirms that style embeddings significantly influence generation quality and encode -ppeicc- RGB-T relationships. For general-purpor purpose datasets like FD and MSRS, the improvement marginal, likely because these styles are encoded within the generative model.
style embeddings
5.3 Qualitative Results
The visualization results in Fig. demonstrate the superior performance of ThermalGen compared baseline methods across ground, acrial, and satellite-aerial datasets. GAN-based approaches exhibit notable limitations: Pix2Pix and Pix2PixHD generate distorted thermal imagery. while CycleGAN produces outputs resembling grayscale conversions of RGB images, failing accurately represent thermal distributions. VQGAN results contain prominent grid artifacts that compromise image quality. For diffusion-basee based methods, DiffV2IR tends to generate thermal images with excessively sharp boundaries, misrepresenting the gradual transitions in actual heat distributions. In contrast, ThermalGen produces high-fidelity thermal images that accurately match the distribution characteristics of ground truth thermal data. This superior performance persists across diverse thermal sensors, viewpoints, and environmental conditions, effectively bridging the substantial domain gaps between attelite-acrial acrial, and ground-level datasets. In Fig. 7 we observe that our trained DDIM

---

SAEE Vousitation DRCOx image Featurts (ooooe Cles
Table 5: FID performance across CFG scale factors. Bold values indicate optimal settings for each dataset.
CFG Scale
Boson-night
FLIR
161.22
70.09
2.0
157.57
66.54
4.0
126.50
63.43
8.0
116.46
68.24
16.0
137.95
Figure 5: t-SNE visualization of DINOv2 fea- tures for LLVIP thermal images.
often produces random samples resembling the training distribution rather than being conditioned on the RGB image. This highlights superior adaptability of ThermalGen compared to DDIM.
5.4 Limitations and Discussions
In Tables our results indicate performance degradation in specific datasets, including Boson-night, LLVIP, and FLIR datasets. Through root cause analysis of the visualized outputs and systematic investigation, we identify key limitations and propose targeted solutions for each challenging scenario:
Bishn: The underperformance stems from extremely low-contrast thermal imagery that hampers effective learning, resulting dark and blurred outputs. However, this limita- tion can be effectively mitigated by adjusting the classifier-free guidance (CFG) scale factor. Our analysis in T that varying the CFG scale significantly impacts generation quality, with optimal CFG factor of 8.0 reducing the FID from 161.22 to 116.46. This finding underscores the effectiveness our style embedding mechanism, as the CFG scale controls the influence of style embeddings during generation.
FLIR: Performance issues arise from blurred distant objects and extreme lighting conditions (overexposure and underexposure). Similarly, CFG scale adjustment proves effective in Table 5 with optimal performance achieved at CFG factor 4.0, reducing FID from 70.09 to 63.43. While this does not surpass state-of-the-ar benchmarks, the consistent trend highlights the potential of our framework to adapt under challenging conditions.
The primary challenge is limited scene diversity with predominantly static back- grounds and minimal dynamic content, creating distribution shift between training and testing conditions. Through t-SNE analysis of DINOv2 features in Fig-55 we reveal a clear distribution gap between training and testing thermal images, primarily attributed to camera differences. The most effective solution is expanding the training dataset to bridge this distribution gap and improve generalization.
LLVIP:
Overall, these results show that while dataset-specific challenges remain, CFG scale adjustment consistently improves performance and supports the effectiveness our style-disenaangeed approach. suggesting promising avenues for further enhancement of cross dross-domain generalization.
Conclusions
In summary, we presented ThermalGen, an adaptive GGGBThermal (RGB-T) image translation model built on flow-based generative architecture with RGB image conditioning and style-disemtangling mechanism. Through extensive experiments on a diverse suite of RGB-T dltasets-inncudae three newly introduced satellite-aeria datasets-Thermaan ization across different thermal sensors, viewpoints, and environmental conditions. Our analysis highlights the significance of disentangled style embeddings in enhancing translation robustness.
demonstrates strong cross-dataset general-

---

Tnyer
CyeeCAN
VOGAN
ThemalGen
Freebur Night Freibong Dag
MDD
MIRSS
AVID
NIL CU
Bosomples -ee
Bosomplos night
Figure 6: Visual Comparison of Baseline Methods and ThermalGen. A comprehensive evaluation conducted across diverse RGB-T datasets, including ground (Freiburg, FD MSRS), acrial (AVIID. NII-CU), and satellite-aerial (Bosonplus-day. Bosonplus-night) domains, demonstrating the method's adaptability and performance.
Figure 7: Failure cases DDIM. show paired RGB input and DDIM output. DDIM tends to generate random samples that align with training data, rather than reflecting the RGB input.
Future work will focus on systematically incorporating synthesized thermal data from multiple sources further advance cross-modal model training.
Broader Impacts. ThermalGen generates realistic thermal images from RGB inputs, facilitating the validation of thermal data realism. However, it also presents potential risks of misuse especially in safety-critical contexts where synthetic outputs might be misinterpreted as genuine thermal data.
Acknowledgments
This work was supported by the Technology Innovation Institute, the NSF CAREER Award 2145277, the NSF CPS Grant CNS-212139 and the NYU IT High Performance Computing resources, services, and staff expertise. We thank the support from Technology Innovation Institute ARRC ViCon team. Giuseppe Loianno serves as consultant for the Technology Innovation Institute. This arrangement has been reviewed and approved by the New York University in accordance with its policy on objectivity in research.
10

---

References
[1] Martin Brenner, Napoleon H. Reyes. Teo Susnjak, and Andre LC Barczak. Rgb-d and thermal sensor fusion: A systematic literature review. IEEE Access, 11:82410-82442, 2023.
[2] Zizhao Chen, Yegiang Qian, Xiaoxiao Yang, Chunxiang Wang, and Ming Yang. Amfd: Distillation via adaptive multimodal fusion for multispectral pedestrian detection. arXiv preprint arXiv:2405. 12944, 2024.
[3] Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injection in diffusion: training-ftee approach for adapting large-scale diffusion models for style transfer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8795-88880 8805, 2024.
[4] Quan Dao, Hao Phung, Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint arXiv:2307.06698, 2023.
[5] Yingying Deng. Xiangyu He, Fan Tang. and Weiming Dong. Z Zero-shot style transfer attention reweighting, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6934 6944, 2024.
[6] Johan Edstedt, Qiyu Sun, Georg Bökman, Märten Wadenbück, and Michael Felsberg. Roma: Robust dense feature matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19790-19800 2024.
[7] Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873-12883, 2021.
[8] TELEDYNE FLIR. Free teledyne flir thermal dataset for algorithm training. httpss//www. flir. com/ oem/ adas/ lalas-dataset.. form/ 2025. Online; accessed 21 Apr 2025.
[9] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):
139- 144,144,22 2020.
[10] Zonghao Han, Ziye Zhang, Shun Zhang, Ge Zhang, and Shaohui Mei. Aerial visible-to- to-ifraae-d image translation: Dataset, evaluation, and baseline. Journal of remote sensing, 3:0096, 2023.
[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
[12] Xingyi He, Hao Yu, Sida Peng, Dongli Tan, Zehong Shen, Hujun Bao, and Xiaowei Zhou. Matchanything: Universal cross-modality image matching with large-scale pre-training. arXiv preprint arXi::2501.075: 2025,
[13] Martin Heusel. Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by time-scale update rule converge a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[14] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12540 2022.
[15] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2020.
[16] Xun Huang and Serge Belongie. Arbitrary style transfer in cal-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision, pages 1501-1510, 2017.
[17] Soommin Hwang, Jaesik Park, Namil Kim, Yukyung Choi, and In So Kweon. Multispectral pedestrian detection: Benchmark dataset and basclines. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
[18] Phillip Isola, Jun- Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1 1134, 2017.
[19] Xinyu Jia, Chuang Zhu, Minzhen Li, Wenqi Tang, and Wenli Zhou, Livip: A visible-infrared paired dataset for low-light vision. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3496-3504. 2021.
11

---

[20] Xingyu Jiang, Jiangwei Ren, Zizhuo Li, Xin Zhou, Dingkang Liang, and Xiang Bai. Minima: Modality invariant image matching, arXiv preprint arXiv:2412.19412 2024.
[21] Vladimir V Kniaz, Vladimir A Knyaz, Jiri Hladuvka, Walter G Kropatsch, and Vladimir Mizginov. Thermalgan: Multimodal toler-lo- image translation for person re-identification in multispectral dataset. In Proceedings the European conference on computer vision (ECCV) workshops, pages 0-0, 2018.
[22] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances Neural Information Processing Systems. Curran Associates, Inc., 2012.
[23] Gihyun Kwon and Jong Chul Ye. Diffusion- image translation using disentangled style and content representation. In The Eleventh International Conference Learning Representations, 2023.
[24] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553)):33-4444 2015.
[25] Connor Lee, Matthew Anderson, Nikhil Ranganathan, Xingxing Zuo, Kevin Do. Georgia Gkioxari, and Soon-Jo Chung. Caltech acrial rgb-thermal dataset in the wild. In Eumpean Conference on Computer Vision, pages 236-256. Springer, 2024.
[26] Dong-Guw Lee, Myung-Hwan Jeon, Younggun Cho, and Ayoung Kim. Edge-guided multi-domain rgh-to-tir image image translation for training vision tasks with challenging labels. 2023 IEFE International Conference on Robotics and Automation (ICRA), pages 8291-8298, 2023.
[27] Alex Leykin, Yang Ran, and Riad Hammoud. Thermal- visible video fusion for moving target tracking and pedestrian classification. In 2007 IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8. IEEE, 2007.
[28] Bo Li, Kaitao Xue, Bin Liu, and Yu-Kun Lai. Bbdm: Image-to-image translation with brownian bridge diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern Recognition, pages 1962-1961,74 2023.
[29] Chenglong Li. Xinyan Liang. Yijuan Lu, Nan Zhao, and Jin Tang. Rgb-t object tracking: Benchmark and baseline. Pattern Recognition, 96: 96:107,77, 2019.
[30] Philipp Lindenherger, Paul-Edouard Sarlin, and Mare Pollefeys. Lightglue: Local feature matching light speed. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17627-17638, 2023.
[31] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. The Eleventh International Conference on Learning Representations, 2023.
[32] Jinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu, Risheng Liu, Wei Zhong, and Zhongxuan Luo. Target-aware dual adversarial learning and multi-scenario mudti-modality benehmark to fuse infrared and visible for object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5802-5811, 2022.
[33] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019.
[34] Nauye Ma. Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Manden-Eijnden and Saining Xic. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pages 23-40. Springer, 2024.
[35] Fangyuan Mao. Jilin Mei, Shun Lu, Fuyang Liu, Liang Chen, Fangzhou Zhao, and Yu Hu. Pid: physics- informed diffusion model for infrared image generation. Puttern Recognition, 169:116818 2026.
[36] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.00 07333, 2023.
[37] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195 2023.
[38] Lingyan Ran, Lidong Wang, Guangcong Wang, Peng Wang, and Yanning Zhang. Diffv2ir: Visible-to- infrared diffusion model via vasion-language understanding, arXiv preprint arXiv:2503. 190/2, 2025.
[39] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. Hegh-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2022.
12

---

[40] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 234-241, 2015.
[41] Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pages 1-10, 2022.
[42] Ukcheol Shin, Kyunghyun Lee. In So Kweon, and Jean Oh. Complementary random masking for rgb- thermal semantic segmentation. In 2024 IEEE International Conference on Robotics and Automation (ICRA). pages 11110-11117. IEEE, 2024.
[43] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations. 2021.
[44] Simon Speth, Artur Gonçalves, Bastien Rigault, Satoshi Suzuki, Mondher Bouazizi, Yutaka Matsuo, and Helmut Prendinger. Deep learning with rgb and thermal images onboard drone for monitoring operations. Journal of Field Robotics, 890--888-86 2022.
[45] Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector-fret local feature matching with transformers. Proceedings of the IEEF/CVF conference on computer vision and pottern recognition, pages 8922-8882888 12--893,,222 2021.
[46] Christian Szegedy, Vincent Vanhoucke, Sergey loffc, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recogmition, pages 2818-2826, 2016.
[47] Karasawa Takumi, Kohei Watanabe, Qishen Ha, Antonio Tejero-De-Pablos, Yoshitaka Ushiku, and Tatsuya Harada. Multispectral object detection autonomous vehicles. In Proceedings of the on Thematic Workshops of ACM Multimedia 2017, pages 43, 2017.
[48] Linfeng Tang, Jiteng Yuan, Hao Zhang, Xingyu Jiang, and Jiayi Ma. Piafusion: A progressive infrared and visible image fusion network based on illumination aware. Information Fusion, 83-84:79-92, 2022.
[49] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text- driven image-to-image translation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 1921-1930, 2023.
[50] Önder Tuzcuoglu, Aybora Köksal, Bugra Sofu, Sinan Kalkan, and A Aydin Alatan. Xoftr: Cross-modal feature matching transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 4275-4286, 2024.
[51] Johan Vertens, Jannik Zürn, and Wolfram Burgard. Heatnet: Bridging the day-night domain gap in semantic segmentation with thermal images. In 2020 IEEP/RSI International Conference on Intelligent Robots and Systems (IROS), pages 8461-8468, 2020.
[52] Emma Wadsworth, Advait Mahajan, Raksha Prasad, and Rajesh Menon. Deep learning for thermal-rgh image-to-image translation. Infrared Physics & Technology, 141:105442, 2024.
[53] Ting- Chun Wang, Ming- Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High- resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018
[54] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-4) 612, 2004.
[55] Zhizhong Wang, Lei Zhao, and Wei Xing. Stylediffusion: Controllable disentangled style transfer via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7677-7689, 2023.
[56] Jiuhong Xiao and Giuseppe Loianno. Uasthn: Uncertainty-awart deep homography estimation for theelit geo-localization. In 2025 JEEE International Conference on Robotics and Automation (ICRA), pages 14066-14000 14072, 2025.
[57] Jiuhong Xiao, Daniel Tortei, Eloy Roura, and Giuseppe Loianno. Long-range uav thermal geo-localization with satellite imagery. In IEEE/RSJ International Conference on Intelligent Robots and Sysiems (IROS), pages 5820 5827, 2023.
13

---

[58] Jiuhong Xiao, Ning Zhang. Daniel Tortei, and Giuseppe Loianno. Sthn: for thermal geo-localization with satellite imagery. IEEE Robotics and Automation Letters, 9(0)): 8754-8761, 2024.
Deep homography estimation
[59] Heng Zhang, Elisa Fromont, Sébastien Lefevre, and Bruno Avignon. Multispectral fusion for object detection with eyclic fuse-and-refing blocks. In 2020 IEEE International conference on image processing (ICIP), pages 276-280. IEEE, 2020.
[60] Richard Zhang, Phillip Isola, Alexci Efros, Eli Shechtman, Oliver Wang. The unreasonable effectiveness of deep features as a pereeptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595 595, 2018.
[61] Yuxin Zhang, Nisha Huang, Fan Tang. Haibin Huang, Chongyang Ma, Weiming Dong, and Changsheng Xu. Inversion-basee style transfer with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10146-1015 10156, 2023.
[62] Guoqing Zhu, Honghu Pan, Qiang Wang, Chao Tian, Chao Yang, and Zheayu He. Data generation scheme for thermal modality with edge-guided adversarial conditional diffusion model. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 10544-10553, 2024.
[63] Jun Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image to-image translation using cycle-consistent ad adversarial networks. IEEE International Conference on Computer Vision (ICCV), 2017.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claim presented in the abstract and introduction highlights our contributions to a novel KGB-T image translation model, the introduction of new datasets, and our focused scope on RGB-T image translation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
This paper has discussed the limitations.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
The paper does not include theoretical results
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper discloses the main implementation details in the main body, and additional details in the appendix. We will release the code, model, and new datasets for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We will publicly release the code, model, and datasets after review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ praips.cc/ubblicguuudee CoodeSumeeieeeeee ssi iessonPolll molyicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper discloses the main implementation details in the main body, and additional details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We don't have enough resources and time to repeat experiments and report error bars for different initialization conditions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes)
Justification:
We disclose the computer resources and the number of training steps.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We conduct research with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the potential societal impacts in the conclusions.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
The dataset will be released with safeguards.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have inspected and followed the requirements of the licenses of assets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
We introduce new datasets and describe the details in the main body and appendix.
Justification:
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20

---

A Satellite-Aerial Dataset Collection Process
Our data collection process involves the following steps: (1) deploying UAVs over selected areas to capture thermal image patches embedded with GPS metadata; (2) generating a unified thermal orthomosaic'; (3) cropping and aligning corresponding regions from satellite imagery with spatial resolution of 1m/pixcl) (4) excluding regions with invalid thermal data; and (5) applying grid sampling extract square image patches for model training and evaluation (following the setup in [57], we adopt sampling stride of 35m and a crop size of 512 x 512). This process ensures the precise alignment between satellite RGB and thermal images.
B Dataset Preprocessing Details
For large-scale training, all datasets are converted into the WebDataser format, except for the satellite-aerial datasets. These datasets consist paired, spatially aligned thermal and satellite RGB maps, from which we randomly sample regions across the entire map. We also detail the specific preprocessing steps applied to each dataset to ensure reproducibility:
NII-UU: Resized RGB images to match thermal image dimensions.
TARDAL: Created 80%/20% train/test split by random sampling since official split was provided in the dataset.
Freiburg: Removed the black padding from thermal images on both right and left sizes.
C Additional Training Details
We conduct all training and evaluation using a single NVIDIA A100 H100 GPU. For training
thermal encoder and decoder, we employ batch size of 16 and use the AdamW 3 optimizer with a learning rate of 6 x 10-5 and weight decay 1 x 10-3, over a total of 200k training steps. All other configurations follow the default settings of the Latent Diffusion Model [39] In training the flow-based generative models, we use a batch size 64 with Adam optimizer at learning rate of 1 x 10 and no weight decay, for 200k training steps.
D Dataset Parameters Comparison
comparison between the Boson-night dataset 57 and our datasets is provided Table 6 high- lighting contributions in terms of broader geographic coverage, increased diversity of thermal sensors, and the inclusion of both daytime and nighttime imagery.
Table 6: Comparison of dataset parameters between Boson-night [57 and our datasets. The differences are highlighted regarding area coverage, satellite map sources, thermal sensors used, collection years, and the number of surveyed regions.
Dataset Name
Area
Satellite Map
Thermal Sensor
Collection Year
Number of Regions
Boson-night (77
33km²
Bing
Boson
2021
DJI-day (Ours)
29km²
ESRI
DJI Zenmuse H20T
2023
1
Bosonplus-day (Ours)
85km²
ESRI
Bosonplus
2024
3
Dosonplus-night (Ours)
94km²
ESRI
Bosonplus
2024
E
Thermal Auto-encoder Training Details and Reconstruction Performance
This section details the training procedure and reconstruction performance of the thermal encoder Fr and decoder DT. Both components are trained using the KL-VAE framework (39) on our curated
used Agisoft Metashape to generate the orthomosaic: https//wwww.gissot.commm
Phttp::/gittub.c http:///wwwwwtstt////wwaaaa..aa
"FSRI satellite map: https://www.crr. wwt.:/ess..co/eesseeee com//nn-s/homee
For compliance, will release data only for regions in Bosonplus-day and Bosonplus- night.
21

---

Table 7: Reconstruction FID and PSNR performance across multiple RGB-T datasets
Method
FLIR
LIVIP
MDED
Bosonplus-day
Bosonplus-nihh
Boson-night
FID
klvae w/o GAN loss
18.51
2.94
18.16
14.37
9.07
5.50
16.63
6.48
4.24
12.20
5.33
PSNR
klvae w/o GAN loss
30.10
37.63
31.29
38.73
45.73
40.12
30.46
41.32
43.53
36.59
39.03
43.16
Table Style vs. CLIP Embeddings ThermalGen.
Method
Bosonplus-day
Bosonplus-night 4 FLIR 4
ThermalGen (Style embedding)
76.91
75.80
70.09
ThermalGen (CLIP embedding)
106.08
89.65
75.49
RGB-T dataset collection, optimized with combination of reconstruction losses (L.1 and LPIPS (00 and KI. regularization the latent space. After 300 epochs (200k steps) of training, we optionally fine-tune the final decoder layers for 75 epochs using GAN loss plus the loss mentioned above, as in the original LDM [39], to enhance FID scores without altering the latent representation, but will slightly negatively affect PSNR metries. We note that the results reported in the paper use the decoder without GAN loss.
F
Comparison of Style and CLIP Embeddings
We compare ThermalGen with two embedding strategies: (1) fixed style embeddings and (2) CLIP- based embeddings extracted from paired RGB images. As shown in Table fixed style embeddings yield substantially lower FID scores on the Bosonplus-day and Bosonplus-night datasets, while achieving comparable performance computational efficiency by removing the need to compute CLIP features, demonstrating both the effectiveness and efficiency of our approach.
FLIR. Moreover, using fixed style embeddings improves
G Evaluation Dataset Comparison
We provide detailed comparison of the evaluation datasets in Table 9 organized by key sources of variation. To further illustrate domain differences, we present t-SNE visualizations of thermal features in Fig. extracted using DINOv2 for the following dataset pairs: Bosonplus-night (sensor variation), Bosonplus-day vs. AVIID (viewpoint variation), and Bosonplus-day vs. Bosonplus- night (diurnal variation). These analyses reveal significant feature discrepancies across conditions, while demonstrating that our model effectively adapts to such shifts.
Boson-night
(a) Diurnal Variation
(b) Sensor Variation
(c) Viewpoint Variation
Figure 8: t-SNE Analysis for Diurnal, Sensor, and Viewpoint Variation of Datasets
22

---

Table 9: Evaluation Dataset Attribute Comparison. Attributes include diurnal setting, thermal
sensor configuration, viewpoint, and environment type.
Dataset
Diurnal
Sensor Configuration
Viewpoint
Environment
Boson-night
Night
FLIR Boson
Sutellite-aerie
Wild
Bosonplus-day
Day
FLIR Bosonplus
Satellite-aerial
Wild
Bosonplus-nighe
Night
FLIR Bosonplus
Satellite aerial
Wild
LLVIP
Day/Night
Vanadium Oxide Uncooled Focal Plane Arrays
Aerial
Urban
NIICU
Day
FI.IR Vue Pro
Aerial
Wild
AVIID
Day/Night
Infrared (640 4800. 8-14 pm)
Acrial
Urban
M3FD
Day/Night
Infrared (6-00x52.. 8-14 pm)
Ground
Diverse
MSRS
Day/Night
InfRec R500
Ground
Urban
FLIR
Day/Night
FLIR Tau 2
Ground
Urban
23


Reviews:
[{"review_id": "zShXXH5jfN", "reviewer": "Reviewer_Njbx", "summary": "The paper addressed the scarcity of RGB-thermal imaging data and proposed the ThermalGen model based on a flow generation framework. By integrating an RGB conditional architecture with a style disentanglement mechanism, this model could generate high-fidelity thermal images across different viewpoints, sensors, and environments. Additionally, three new satellite-aerial RGB-T datasets were introduced. In multiple benchmark tests, the model outperformed existing GAN and diffusion models.", "questions": "The paper primarily focused on integrating existing datasets with its newly proposed datasets to enable RGB-to-IR translation across different styles, diurnal environments, and viewpoints. However, it did not mention how to disentangle these three factors (style, environment, viewpoint) nor conduct any controlled-variable experiments to verify whether manipulating a single variable could yield realistic translation results. Additionally, the study did not construct a 3D model; instead, it merely translated RGB images from different viewpoints without introducing any novel techniques for viewpoint transformation as the approach for handling viewpoint variations lacks distinctiveness beyond basic image translation. When these issues remain unexplained, the paper’s innovativeness is called into question.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1) ThermalGen introduced a flow-based generative model with style-disentangled mechanism, which was the first to explicitly model variations in viewpoints, sensors, and environments in RGB-T translation. The use of adaptive layer normalization (adaLN-Zero) and dataset-specific style embeddings enabled robust generation across diverse styles without retraining.\n2) The paper released three new large-scale satellite-aerial RGB-T datasets (DJI-day, BosonPlus-day/night), covering diverse times (day/night), sensor types (Zenmuse H20T, BosonPlus), and geographic regions.\n\nWeaknesses:\n1) The style embeddings were vaguely described, as they attributed diurnal environments, sensor styles, and viewpoint variations entirely to datasets without elaborating on how to disentangle these three factors. Moreover, the study did not use the control variable method to verify whether controlling a certain variable could make the generated images change and conform to reality.\n2) The paper did not compare with other style transfer methods (such as CLIP-guided embeddings or text-based conditioning methods), which limited the universality of its conclusions. The ablation study on style embeddings (Figure 4 (c)) could further analyze the encoding methods of different styles in the latent space.\n3) ThermalGen underperformed on low-thermal-contrast datasets (Boson-night), homogeneous scenes (LLVIP), or under extreme lighting conditions (FLIR). Although the authors attributed these issues to data characteristics, they did not propose specific solutions (such as contrast enhancement modules or domain-specific data augmentation) to mitigate these problems.\nminor：\n4) The paper failed to specify the computing resources required for training ThermalGen, such as GPU types, training time, and memory usage.\n5) The flow-based model relied on the SiT framework, but the paper did not theoretically demonstrate why flow models were more suitable for RGB-T image translation than diffusion models, especially in capturing cross-modal dependencies.\n6) The evaluation was mainly limited to image alignment tasks, with fewer insights into downstream applications such as object detection.", "was_revised": true, "final_justification": "The authors have sufficiently addressed core my concerns. They clarified style embeddings via empirical validations (e.g., dataset-specific comparisons, t-SNE analyses) and demonstrated ThermalGen’s advantages over CLIP-based methods through new experiments. For weak performance on challenging datasets (e.g., low-thermal-contrast scenes), they proposed actionable solutions (CFG scale adjustment) with quantitative improvements. While minor gaps remain (e.g., limited downstream task exploration, no explicit 3D modeling for viewpoints), these are reasonable given the scope, and the authors clearly align their work to dataset - level adaptation rather than strict factor disentanglement. For this conference paper, after carefully reviewing the authors’ rebuttal, I recommend acceptance.", "rebuttal": "**Q1: The style embeddings were vaguely described, as they attributed diurnal environments, sensor styles, and viewpoint variations entirely to datasets without elaborating on how to disentangle these three factors. Moreover, the study did not use the control variable method to verify whether controlling a certain variable could make the generated images change and conform to reality.**\n\nWe emphasize that the following three dataset comparisons can represent the mentioned variations: Bosonplus-night vs. Boson-night (sensor variation), Bosonplus-day vs. AVIID (viewpoint variation), and Bosonplus-day vs. Bosonplus-night (diurnal variation). Our model consistently demonstrates robust performance across all these datasets (Boson-night issue is mitigated, see Q3), highlighting its capacity to handle such variations effectively.\n\nOur approach uses heterogeneous datasets to learn style embedding specific to each dataset through backpropagation, enabling data-driven learning. These embeddings implicitly capture variations in diurnal conditions, sensor types, and viewpoints without requiring explicit disentanglement. Although we do not enforce factor-wise separation, our empirical results show that ThermalGen generalizes well across diverse domains.\n\nWe additionally provide a comprehensive summary of the datasets, categorized for different variations. In the revised manuscript, we include t-SNE analyses of thermal features extracted via DINOv2 across the aforementioned dataset pairs. These comparisons reveal substantial feature discrepancies resulting from varying conditions, yet our model effectively adapts to them.\n\n| Dataset Name | Diurnal Environment | Thermal Sensor Configuration | Viewpoint Variation | Environment Type |\n|--------------|---------------------|-----------------------|---------------------|------------------|\n| Boson-night    | Night           | FLIR Boson | Satellite-aerial     | Wild           |\n| Bosonplus-day    | Day          | FLIR Bosonplus     | Satellite-aerial              | Wild      |\n| Bosonplus-night    | Night            | FLIR Bosonplus     | Satellite-aerial        | Wild      |\n| LLVIP    | Day/Night           | Vanadium Oxide Uncooled Focal Plane Arrays    | Aerial            | Urban          |\n| NII-CU   | Day           | FLIR Vue Pro    | Aerial             | Wild         |\n| AVIID    | Day/Night           | Infrared camera (resolution: 640x480, wavelength:  (8 – 14 μm)    | Aerial             | Urban          |\n| M3FD   | Day/Night           | Infrared camera (resolution: 640x512, wavelength:  (8 – 14 μm))    | Ground           | Diverse          |\n| MSRS   | Day/Night           | InfRec R500    | Ground             | Urban        |\n| FLIR   | Day/Night           | FLIR Tau 2    | Ground             | Urban          |\n\n\n**Q2:The paper did not compare with other style transfer methods (such as CLIP-guided embeddings or text-based conditioning methods), which limited the universality of its conclusions. The ablation study on style embeddings (Figure 4 (c)) could further analyze the encoding methods of different styles in the latent space.**\n\nWe conducted a comparative evaluation of ThermalGen using fixed style embeddings against a variant that employs CLIP-based embeddings extracted from RGB images. The fixed style embeddings demonstrate substantial improvements on the following datasets, while maintaining similar performance on other datasets. Additionally, the use of fixed style embeddings enhances computational efficiency by eliminating the need to compute CLIP-based embeddings. These findings highlight both the effectiveness and efficiency of our approach.\n| Methods       |  Bosonplus-day FID ↓| Bosonplus-night FID ↓ | FLIR FID ↓ |\n|----------------|-------------------|---------|-------|\n| ThermalGen-style-embedding | **76.91** | **75.80** | **70.09**  |\n| ThermalGen-CLIP-based-embedding | 106.08 | 89.65 | 75.49 |\n\n**Q3: ThermalGen underperformed on low-thermal-contrast datasets (Boson-night), homogeneous scenes (LLVIP), or under extreme lighting conditions (FLIR). Although the authors attributed these issues to data characteristics, they did not propose specific solutions (such as contrast enhancement modules or domain-specific data augmentation) to mitigate these problems.**\n\nBoson-night: The underperformance can be mitigated by adjusting the classifier-free guidance (CFG) scale factor. The table below illustrates how varying the CFG scale significantly impacts FID performance on this dataset. Notably, a CFG factor of 8.0 yields the best result, reducing the FID to 116.46, which represents a substantial improvement over the default setting.\n\nThis finding also underscores the effectiveness of our style embedding mechanism. Since the CFG scale controls the influence of the style embedding during generation, these results demonstrate that appropriately tuning this factor enables the model to generate clearer, higher-fidelity thermal images. Visually, we observe that ThermalGen produces more detailed and realistic outputs at the optimal CFG.  \n\n| Method          | Metric       |  CFG Factor = 1.0 (None) | 2.0   | 4.0   | 8.0   | 16.0  |\n|----------------|-------------------|---------|-------|-------|-------|-------|\n| ThermalGen-L/2 | FID ↓             | 161.22  | 157.57| 126.50| **116.46** | 137.95 |\n\nFLIR: We also observe that the underperformance on FLIR can be alleviated by adjusting the CFG scale factor. As shown in the table below, tuning the CFG scale leads to a noticeable reduction in FID, with the best performance achieved at a CFG factor of 4.0.  \n\nWhile the resulting FID does not yet surpass SOTA benchmarks, the trend highlights the potential of our style-disentangled framework to adapt under challenging conditions. The visual results further support this observation, showing improved generation quality as the influence of the style embedding is modulated. These findings suggest a promising direction for enhancing performance in extreme conditions.  \n\n| Method          | Metric | CFG Factor = 1.0 (None) | 2.0   | 4.0       | 8.0   |\n|----------------|--------|------------|-------|-----------|-------|\n| ThermalGen-L/2 | FID ↓  | 70.09      | 66.54 | **63.43** | 68.24 |\n\nLLVIP: We conduct t-SNE analysis on DINOv2 features for thermal images and reveal a clear distribution gap between training and testing data, primarily due to camera differences. Expanding the training set is the most effective solution.\n\n**Q4: Computing resources**\n\nTraining and evaluation were done on a single NVIDIA H100 or A100 GPU, with both models trained for 200,000 steps. Memory usage (~60 GB) stayed within A100/H100 capacity. Specs will be detailed in the supplementary materials.\n\n**Q5: The paper did not theoretically demonstrate why flow models were more suitable for RGB-T image translation than diffusion models, especially in capturing cross-modal dependencies.**\n\nOur work does not include theoretical proof of the advantage of the flow-based method compared with the diffusion-based method in the RGB-T context. We chose a flow-based method for its faster convergence and strong performance compared with diffusion-based methods, as supported by the Flow Matching paper [1] and SiT’s results [2] (Tables 2–4, Figures 2 and 6). While our study focuses on RGB-T image translation, we assume that these advantages of flow-based methods are generally applicable to image generation tasks, including our own.\n\n**Q6: The evaluation was mainly limited to image alignment tasks, with fewer insights into downstream applications such as object detection.**\n\nOur evaluation does not focus on image alignment tasks but image-to-image translation itself. We agree that ThermalGen has the potential to benefit downstream applications such as image alignment, object detection, and semantic segmentation in the thermal domain. While these are beyond the scope of our current study, they represent promising directions for future work.\n\n**Q7: However, it did not mention how to disentangle these three factors (style, environment, viewpoint) nor conduct any controlled-variable experiments to verify whether manipulating a single variable could yield realistic translation results. Additionally, the study did not construct a 3D model; instead, it merely translated RGB images from different viewpoints without introducing any novel techniques for viewpoint transformation as the approach for handling viewpoint variations lacks distinctiveness beyond basic image translation.**\n\nWe would like to clarify that the primary objective of our work is not to explicitly disentangle the three factors—style, environment, and viewpoint—but to learn dataset-level embeddings that capture these attributes in a unified and implicit manner. The style embedding enables our model to adapt to a wide variety of datasets with diverse characteristics, which is demonstrated in our broad evaluation on nine datasets with different attributes (refer to Q1 table).\n\nRegarding viewpoint variation, our method does not involve 3D modeling or introduce novel techniques for explicit viewpoint transformation. In our context, viewpoint variation specifically refers to differences in camera perspectives—such as aerial vs. ground views—rather than geometric transformations within a 3D space. Our results have demonstrated that ThermalGen effectively handles such variation through its style-disentangled, flow-based generative model, which is distinct from basic RGB-T image translation.\n\nReferences:  \n[1] Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., & Le, M. (2023). Flow Matching for Generative Modeling. The Eleventh International Conference on Learning Representations.  \n[2] Ma, N., Goldstein, M., Albergo, M. S., Boffi, N. M., Vanden-Eijnden, E., & Xie, S. (2024, September). Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision (pp. 23-40). Cham: Springer Nature Switzerland."}, {"review_id": "24FE9eh7JN", "reviewer": "Reviewer_ZBig", "summary": "This paper proposes a flow-based generative model for RGB-to-thermal (RGB-T) image translation, addressing the scarcity of paired RGB-thermal data. Specifically,  first, it introduces an Adaptive flow-based architecture with RGB conditioning and style-disentangled embeddings to handle variations in sensors, viewpoints, and environmental conditions.\nSecond, it proposes Three new satellite-aerial datasets (DJI-day, BosonPlus-day, BosonPlus-night) for large-scale training. Third, it achieves Superior performance over GAN/diffusion baselines (e.g., pix2pix, CycleGAN, DiffV2IR) across satellite, aerial, and ground benchmarks, validated via PSNR, SSIM, FID, and LPIPS metrics.", "questions": "1. Why use dataset-level embeddings instead of learning styles directly from image content (e.g., via attention)? Could this limit adaptation to unseen styles?\n2. The motivation of this paper is unconvincing. Why need to translate RGB images into thermal images? They are within different modalities and different visual details. For example, thermal images may contain more texture details of objects than RGB images. If the proposed model works, it is better to show such differences. \n3. Thermal images follow physical laws (e.g., heat dissipation). Could incorporating physics-informed losses (e.g., PDE-based regularization) improve accuracy?\n4. Flow-based sampling (50 denoising steps) is slower than GANs. Can distillation or quantization achieve real-time translation?\n5. Datasets focus on urban/wilderness—how would ThermalGen perform in industrial/medical thermal domains?", "limitations": "1. The motivation of this paper is unconvincing. Why need to translate RGB images into thermal images? They are within different modalities and different visual details. For example, thermal images may contain more texture details of objects than RGB images. If the proposed model works, it is better to show such differences. \n2. This paper shows significant performance drops in some extreme conditions, for example, blurred distant objects (FLIR dataset) and low-contrast thermal images (Boson-night).\n3. The Fixed per-dataset embeddings of the style embedding may not capture intra-dataset variations. However, the authors do not investigate this issue.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The flow-based model for RGB-T translation with style disentanglement is reasonable and effective for this image translation task.\n2. Unified training on 200k+ diverse samples across 13 datasets shows the potential scalability of this model.\n3. Robust performance across domains (satellite, aerial, ground) without retraining.\n4. Three new datasets are proposed to contribute to the community.\n\nWeaknesses:\n1. The motivation of this paper is unconvincing. Why need to translate RGB images into thermal images? They are within different modalities and different visual details. For example, thermal images may contain more texture details of objects than RGB images. If the proposed model works, it is better to show such differences. \n2. This paper shows significant performance drops in some extreme conditions, for example, blurred distant objects (FLIR dataset) and low-contrast thermal images (Boson-night).\n3. The Fixed per-dataset embeddings of the style embedding may not capture intra-dataset variations. However, the authors do not investigate this issue.", "was_revised": true, "final_justification": "After carefully evaluating the motivation, contribution, and response provided by the authors, I decided to increase my rating to 4. \n\nThanks to the authors' informative response and AC's organization.", "rebuttal": "**Q1: The motivation of this paper is unconvincing. Why need to translate RGB images into thermal images? They are within different modalities and different visual details.**\n\nWe acknowledge that RGB and thermal images belong to different modalities and capture distinct visual characteristics. However, translating RGB images into thermal images is highly relevant for advancing multimodal computer vision tasks that depend on cross-modality alignment.\n\nTasks such as RGB-thermal sensor fusion [1], cross-modality image alignment [2], and semantic segmentation [3] frequently require well-aligned RGB-thermal image pairs for supervised training. In practice, acquiring such paired data is challenging due to the need for precise synchronization between RGB and thermal sensors, and the resulting datasets are often limited in scale, diversity, or quality.\n\nOur work aims to address this data scarcity by synthesizing realistic thermal data from RGB inputs, thereby enabling the creation of large-scale pseudo-paired RGB-thermal datasets. These generated pairs can significantly benefit downstream multimodal tasks by enhancing model robustness and reducing dependence on costly real-world data collection. Our approach provides an efficient means for rapid prototyping and proof-of-concept development in cross-modality research. A similar motivation supports recent efforts such as Diffv2IR [6].\n\nFurthermore, we observe the potential of our model to generate multi-pattern plausible thermal outputs conditioned on RGB context by randomizing latent noises inherent in flow-based methods, rather than performing one-to-one translation. This variability helps simulate different possible real-world conditions and improves generalization in downstream applications.\n\n**Q2: This paper shows significant performance drops in some extreme conditions, for example, blurred distant objects (FLIR dataset) and low-contrast thermal images (Boson-night).**\n\nBoson-night: The underperformance on Boson-night can be mitigated by adjusting the classifier-free guidance (CFG) scale factor. The table below illustrates how varying the CFG scale significantly impacts FID performance on this dataset. Notably, a CFG factor of 8.0 yields the best result, reducing the FID to 116.46, which represents a substantial improvement over the default setting.\n\nThis finding also underscores the effectiveness of our style embedding mechanism. Since the CFG scale controls the influence of the style embedding during generation, these results demonstrate that appropriately tuning this factor enables the model to generate clearer, higher-fidelity thermal images. Visually, we observe that ThermalGen produces more detailed and realistic outputs at the optimal CFG.\n\n| Method          | Metric       |  CFG Factor = 1.0 (None) | 2.0   | 4.0   | 8.0   | 16.0  |\n|----------------|-------------------|---------|-------|-------|-------|-------|\n| ThermalGen-L/2 | FID ↓             | 161.22  | 157.57| 126.50| **116.46** | 137.95 |\n\nFLIR: Similar to the Boson-night dataset, we observe that the underperformance on FLIR can be alleviated by adjusting the classifier-free guidance (CFG) scale factor. As shown in the table below, tuning the CFG scale leads to a noticeable reduction in FID, with the best performance achieved at a CFG factor of 4.0.\n\nWhile the resulting FID does not yet surpass SOTA benchmarks, the trend highlights the potential of our style-disentangled framework to adapt under challenging conditions. The visual results further support this observation, showing improved generation quality as the influence of the style embedding is modulated. These findings suggest a promising direction for enhancing cross-domain generalization in extreme or low-quality thermal domains.\n\n| Method          | Metric | CFG Factor = 1.0 (None) | 2.0   | 4.0       | 8.0   |\n|----------------|--------|------------|-------|-----------|-------|\n| ThermalGen-L/2 | FID ↓  | 70.09      | 66.54 | **63.43** | 68.24 |\n\n\nLLVIP: We conduct t-SNE analysis on DINOv2 features for thermal images and reveal a clear distribution gap between training and testing data, primarily due to camera differences. Expanding the training set is the most effective solution. This analysis will be added to the supplementary material.\n\n**Q3: The Fixed per-dataset embeddings of the style embedding may not capture intra-dataset variations.**\n\nWe agree that intra-dataset variation can influence performance; however, we argue that such variability should primarily be addressed by the flow-based generative model rather than through the fixed style embedding. For instance, the M3FD dataset contains both daytime and nighttime scenes, yet our model maintains strong performance across this diversity, suggesting its capability to adapt to intra-dataset changes. The style embedding is designed to capture stable, dataset-wide attributes—such as camera specifications and domain characteristics—while the generative model accounts for intra-dataset variation.\n\n**Q4: Why use dataset-level embeddings instead of learning styles directly from image content (e.g., via attention)? Could this limit adaptation to unseen styles?**\n\nThe use of dataset-level embeddings in our framework is motivated by the need to encode style information that may not be inferable solely from the RGB image content. For example, consider the satellite-aerial datasets Bosonplus-day and Bosonplus-night. Although their satellite RGB inputs appear visually the same, the corresponding thermal outputs differ significantly due to day/night variation—a distinction that cannot be reliably inferred from RGB appearance alone. In such cases, image-based mechanisms may fail to capture this latent information.\n\nDataset-level style embeddings also allow the model to incorporate prior knowledge about dataset-specific characteristics. This is particularly beneficial for datasets like NII-CU, which exhibit unique thermal features. As shown in our experiments (Figure 4c), adjusting the classifier-free guidance (CFG) scale—thereby modulating the influence of the style embedding—can lead to notable performance improvements on such datasets. Furthermore, for other datasets like M3FD and MSRS, the inclusion of style embeddings consistently enhances performance, demonstrating that this mechanism improves performance without introducing degradation across diverse thermal domains.\n\nFor unseen datasets, our model utilizes an unconditional style embedding that captures a generalized thermal appearance. Alternatively, users may select an existing style embedding that closely matches the target data. For significantly different thermal data, fine-tuning on the specific dataset is recommended.\n\n**Q5: Could incorporating physics-informed losses (e.g., PDE-based regularization) improve accuracy?**\n\nWe appreciate this insightful suggestion. Indeed, physics-informed losses have the potential to enhance accuracy, particularly in scenarios involving sequential data—for example, by enforcing consistency based on principles like heat dissipation. However, our current work focuses on single-image translation, where such temporal regularization is less directly applicable. While methods such as PID loss [4] aim to predict physical quantities such as temperature, emissivity, and thermal texture, they typically require fine-tuning the underlying physical model to specific datasets, particularly to account for variations in material thermal properties and sensor configurations. Physics-informed loss represents a valuable direction for future work to enhance the adaptability of ThermalGen.\n\n**Q6: Can distillation or quantization achieve real-time translation?**\n\nWe agree that model distillation and quantization are promising strategies for enabling real-time translation. Specifically, model distillation can effectively reduce model complexity and fine-tune on specific datasets. Moreover, recent developments—such as MeanFlow [5]—highlight the feasibility of highly efficient inference through few-step denoising (e.g., 1–2 steps), significantly enhancing speed without compromising quality. These advances point to a promising path toward achieving real-time performance.\n\n**Q7: How would ThermalGen perform in industrial/medical thermal domains?**\n\nWe acknowledge the relevance and importance of extending this work to industrial and medical thermal domains. These domains, however, present distinct challenges. Publicly available datasets are often limited in scale and diversity and tend to be object-centric rather than our scene-based datasets, so that the performance in these domains may be suboptimal without adaptation. Nevertheless, ThermalGen provides a strong foundation through its pretrained weights, which capture diverse thermal characteristics across a range of conditions, and will be open-sourced for further adaptation and finetune.\n\nReferences:  \n[1] Brenner, M., Reyes, N. H., Susnjak, T., & Barczak, A. L. (2023). RGB-D and thermal sensor fusion: A systematic literature review. IEEE Access, 11, 82410-82442.  \n[2] Ren, J., Jiang, X., Li, Z., Liang, D., Zhou, X., & Bai, X. (2025). Minima: Modality invariant image matching. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 23059-23068).  \n[3] Shin, U., Lee, K., Kweon, I. S., & Oh, J. (2024, May). Complementary random masking for rgb-thermal semantic segmentation. In 2024 IEEE International Conference on Robotics and Automation (ICRA) (pp. 11110-11117). IEEE.  \n[4] Mao, F., Mei, J., Lu, S., Liu, F., Chen, L., Zhao, F., & Hu, Y. (2025). PID: physics-informed diffusion model for infrared image generation. Pattern Recognition, 111816.  \n[5] Geng, Z., Deng, M., Bai, X., Kolter, J. Z., & He, K. (2025). Mean flows for one-step generative modeling. arXiv preprint arXiv:2505.13447.\n[6] Ran, L., Wang, L., Wang, G., Wang, P., & Zhang, Y. (2025). Diffv2ir: visible-to-infrared diffusion model via vision-language understanding. arXiv preprint arXiv:2503.19012."}, {"review_id": "DjE73pRzXC", "reviewer": "Reviewer_sJDP", "summary": "This paper presents ThermalGen, a flow-based model for RGB-to-thermal image translation. The method introduces an RGB conditioning architecture and style-disentangled mechanism, supported by newly curated datasets including three large-scale satellite-aerial RGB-T pairs.", "questions": "- Does the model demonstrate sufficient generalization? Could you validate its effectiveness on thermal-to-RGB tasks as well? Relevant references include:\n\n[1] T2V-DDPM: Thermal to Visible Face Translation using Denoising Diffusion Probabilistic Models\n\n[2] DiffTV: Identity-Preserved Thermal-to-Visible Face Translation via Feature Alignment and Dual-Stage Conditions", "limitations": "The authors have adequately illustrated limitations in the main text.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The contribution of three high-quality RGB-thermal datasets is particularly valuable given the scarcity of such data in this field. The planned open-source release would greatly benefit the research community.\n- Experimental results demonstrate clear advantages of ThermalGen over competing methods.\n- To my knowledge, this represents a comprehensive flow-based approach for RGB-thermal translation, presented in a three-in-one framework. Both novelty and technical contributions meet the standard for NIPS acceptance.\n\nWeaknesses:\n- The comparison tables include too few diffusion-based methods (which appear limited in scope). More comprehensive comparisons with general image-to-image approaches (e.g., BBDM) would better demonstrate the method's generalization capability.", "was_revised": true, "final_justification": "I will keep my score.", "rebuttal": "**Q1: More comprehensive comparisons with general image-to-image approaches (e.g., BBDM) would better demonstrate the method's generalization capability.**\n\n\nThank you for this suggestion. We train the latent BBDM-f8 with our datasets and evaluate the performance and we attach the result table. The quantitative results show that ThermalGen has consistently better performance than BBDM. We also verify the visualization result showing that trained BBDM . We will add these results to our Table 1-3.\n\n| Methods         | Categories        | Boson-night PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |   | BosonPlus-day PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |   | BosonPlus-night PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |\n|----------------|------------------|-------------------|-------|--------|--------|---|----------------------|-------|--------|--------|---|------------------------|-------|--------|--------|\n|BBDM | Paired Diffusion | 17.26 | 0.61 | 120.79 | 0.37 | | 20.27 | 0.62 | 145.86 | 0.37 | | 16.10 | 0.45 | 177.81 | 0.42\n| ThermalGen-L/2 | Paired Diffusion | 21.88           | 0.71  | 161.22 | 0.32   |   | 14.66            | 0.31 | 76.91 | 0.35 |   | 20.47             | 0.76 | 75.80 | 0.34 |\n\n| Methods         | Categories        | LLVIP PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |   | NII-CU PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |   | AVIID PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |\n|----------------|------------------|-------------|-------|--------|--------|---|---------------|-------|--------|--------|---|--------------|-------|--------|--------|\n| BBDM | Paired Diffusion | 9.98 | 0.22 | 313.54 | 0.67 | | 14.36 | 0.71 | 118.59 | 0.42 | | 18.53 | 0.49 | 141.06 | 0.31 |\n| ThermalGen-L/2 | Paired Diffusion | 11.12       | 0.34  | 238.60 | 0.51   |   | 26.44     | 0.92 | 69.30 | **0.21** |   | 24.89     | 0.75 | 29.05 | 0.13 |\n\n| Methods         | Categories        | M³FD PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |   | MSRS PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |   | FLIR PSNR↑ | SSIM↑ | FID↓   | LPIPS↓ |\n|----------------|------------------|------------|-------|--------|--------|---|-------------|-------|--------|--------|---|-------------|-------|--------|--------|\n| BBDM | Paired Diffusion | 17.26 | 0.61 | 120.79 | 0.37 | | 20.27 | 0.62 | 145.86 | 0.37 | | 16.10 | 0.45 | 177.81 | 0.42 |\n| ThermalGen-L/2 | Paired Diffusion | 23.73  | 0.81 | 35.82 | 0.14 |   | 24.38   | 0.76 | 52.31 | 0.21 |   | 17.10      | 0.52 | 70.09 | 0.33 |\n\n\n**Q2: Does the model demonstrate sufficient generalization? Could you validate its effectiveness on thermal-to-RGB tasks as well?**\n\nThank you for the suggestion. We respectfully note that the Thermal-to-RGB task differs in nature from the RGB-to-Thermal direction and may involve distinct challenges. As such, its effectiveness does not directly reflect the generalization capabilities of our model within RGB-Thermal tasks. Nonetheless, the underlying principles remain consistent: our model employs separate encoders for thermal and RGB modalities, and the flow-based generative model is trained within the latent space of the thermal encoder. A thermal-to-RGB model will instead require to train the flow-based model to generate from the latent distribution from the RGB encoder."}, {"review_id": "n5Pe7iqw7f", "reviewer": "Reviewer_N5UX", "summary": "The authors introduce ThermalGen, a flow-based generative model designed for RGB-to-Thermal (RGB-T) image translation. ThermalGen addresses the scarcity of paired RGB-T data, which is crucial for multi-modal tasks, by synthesizing thermal images from readily available RGB inputs. The model incorporates an RGB image conditioning architecture and a style-disentangled mechanism for generating thermal images that reflect significant variations in viewpoints, sensor characteristics, and environmental conditions. In other words, they treat the unique relationship between a collection of a specific kind of imagery and its thermal counterpart as a \"style\" (in the sense of style transfer), and disentangle this relationship so that their model can adaptively generate thermal imagery based on the kind of RGB input. To support large-scale training, the authors curated eight public RGB-T datasets and introduced three new large-scale satellite-aerial RGB-T datasets: DJI-day, BosonPlus-day, and BosonPlus-night. Their evaluations demonstrate that ThermalGen achieves comparable or superior translation performance compared to existing GAN-based and diffusion-based methods. The authors state that their model is the first with such broad RGB-T synthesis capabilities.\n\nOn the specifics of their evaluation, the evaluation of ThermalGen involved experiments across various RGB-T benchmarks, including satellite-aerial, aerial, and ground datasets. The performance was assessed using four image quality metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), Fréchet Inception Distance (FID), and Learned Perceptual Image Patch Similarity (LPIPS). ThermalGen was compared against widely adopted GAN-based methods (pix2pix, CycleGAN, pix2pixHD, VQGAN) and diffusion-based methods (DDIM and DiffV2IR). Ablation studies were conducted to determine optimal architectural designs and hyperparameter configurations, examining different transformer and patch sizes, RGB image conditioning designs (cross-attention versus concatenation), and various style embedding settings. Qualitative results were also presented to visually compare ThermalGen's output with baselines across diverse datasets and conditions.", "questions": "I don't have any questions since this paper was a delight to read and felt very wholly packaged in terms of problem chosen, motivation, experiments performed, and results.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- Quality:\n    - Technically sound submission with well supported claims via an extensive evaluation across numerous benchmarks and competing baselines. Specifically, nine datasets, four metrics, and eight competing baselines, covering GAN-based methods and diffusion-based methods.\n    - The authors are honest in evaluating both the strengths and weaknesses of their work. For example, they point out ThermalGen's limitations when dealing with settings with homogeneous samples, low-contrast thermal features, or extreme lighting conditions.\n- Clarity:\n    - The language is clear, the narrative flows smoothly and sensibly, and the quantitative and qualitative results are presented very clearly. It was an incredibly easy and smooth read and I feel confident that I can replicate their results from reading the main paper and appendix.\n- Significance:\n    - Although the quantitative results didn't blow me away (their model is the best on average, but it wasn't a clean sweep), it wasn't until I saw the qualitative results when I was really impressed. Specifically, Figure 5. There is a very clear difference between the quality of ThermalGen's outputs and the competing models'. It's quite significant and I believe this alone would convince researchers working in this space to build off of this model and treat it as the new state of the art.\n- Originality:\n    - Although generative modelling of RGB and thermal image pairs has been done many times, this is the first paper to use a flow-based generative model, as far as I know. The authors also introduce their own unique modifications to improve the modelling to make the model adaptive to other settings via the style embedding.\n    - The authors introduce three new large-scale datasets, which is almost always a boon for the research community. Curating high quality and large scale datasets is difficult work and I commend the authors for doing so, even if it wasn't truly necessary for motivating their model.\n    - Related work has been adequately cited and the authors has made it clear how their work differs from previous contributions.\n\nWeaknesses: No significant weaknesses worth mentioning that I can think of.", "was_revised": true, "final_justification": "Nothing's changed in my opinion of the paper and I had no questions I could come up with. The authors have done a great job, in my opinion.", "rebuttal": "Thank you for the encouraging and thoughtful review. We appreciate your recognition of our technical contributions, including the flow-based architecture, style-disentangled mechanism, and the curated RGB-T datasets. We're especially grateful for your positive assessment of ThermalGen's quality, clarity, significance, and originality. While the model may not significantly outperform all baselines on every metric, its consistent performance across most datasets and strong qualitative results highlight its robustness and potential. Your feedback affirms our direction and inspires further development of ThermalGen."}]

Author rebuttal:

Meta-review: This paper is on translating an RGB image into the thermal domain using flow-based generative models trained on paired RGB–thermal images. The contributions span dataset collection, task-specific algorithm development, and systematic experimental evaluation. After extensive author–reviewer interactions, three reviewers are positive (Accept). Although reviewer ZBig initially raised concerns about the meaningfulness of translating RGB images into thermal images, ZBig raised the score to Borderline Accept. Given the unanimous recommendations by all four reviewers, the AC believes this paper lies above the acceptance bar and recommends acceptance as a poster. Yet it is hard to recommend a spotlight or oral, since the AC believes that the concern raised by reviewer ZBig is quite significant. Considering that thermal imaging is widely used for temperature measurement, the proposed technique will hardly help in this important field, since minor changes in temperature cannot be captured by RGB imaging.
Decision: Accept (poster)