GbrlOl/thermal-fr-lora

🔥 Thermal Face LoRA — FLUX.1 Kontext (I2I)

Visible → Thermal (RGB-to-TIR) facial translation that preserves subject identity

---

A LoRA adapter that specializes FLUX.1 Kontext [dev] for visible-to-thermal facial translation under an explicit image-to-image (I2I) scheme. A real visible face acts as the structural and identity anchor while a fixed text prompt steers the thermal appearance — so the generated thermal face keeps the subject's identity instead of merely looking realistic.

These weights were used to generate PUCV-TER-S, a synthetic thermal face dataset, and accompany the code released at 👉 github.com/GbrlOl/thermal-fr-lora.

⚠️ Gated, non-commercial research release. These weights are a Derivative of FLUX.1 Kontext [dev]; the upstream FLUX.1 [dev] Non-Commercial License and its Acceptable Use Policy apply in addition to the cc-by-nc-4.0 label on this repo. See License & intended use.

📦 Model summary

Type	LoRA adapter for FLUX.1 Kontext [dev]
Task	Image-to-image, visible → thermal (RGB-to-TIR) facial translation
Base model	black-forest-labs/FLUX.1-Kontext-dev
Backbone	Multimodal diffusion transformer (flow matching)
Adapter target	All linear projections (attention + feed-forward) of the double-stream and single-stream blocks
LoRA rank / alpha	r = 128 / α = 128
Training pairs	50 synchronized visible–thermal pairs (PUCV-VTF)
Weights file	Flux_ter_50_redo_seed42.safetensors (≈1.37 GB, bf16)

🎯 Intended use

• Generating synthetic thermal face datasets from consented visible-face images, for research on data scarcity in thermal-face recognition.
• Studying synthetic-to-real transfer and cross-sensor robustness in thermal biometrics.

Out of scope / prohibited: any commercial or production use; generating thermal representations of individuals without consent; impersonation; fabrication of biometric samples; any unlawful, defamatory, or abusive use.

🚀 How to use

You must be authenticated on the Hugging Face Hub and have accepted the terms of both this repository and the base model black-forest-labs/FLUX.1-Kontext-dev (which is itself gated).

huggingface-cli login
pip install -U diffusers transformers accelerate peft safetensors facenet-pytorch

import torch
from diffusers import FluxKontextPipeline
from diffusers.utils import load_image

# 1) Base FLUX.1 Kontext model
pipe = FluxKontextPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-Kontext-dev",
    torch_dtype=torch.bfloat16,
).to("cuda")

# 2) Attach this LoRA adapter
pipe.load_lora_weights(
    "GbrlOl/thermal-fr-lora",
    weight_name="Flux_ter_50_redo_seed42.safetensors",
)

# 3) Preprocessed visible face (MTCNN-cropped, 512x512) as the I2I condition
visible = load_image("visible_face_512.png")

prompt = (
    "convert this image to a high-contrast thermal (8-12 µm) infrared view, "
    "monochrome grayscale, black background, preserve face structure, "
    "no visible-spectrum colours"
)

# Deterministic generation settings used to build PUCV-TER-S
thermal = pipe(
    image=visible,
    prompt=prompt,
    guidance_scale=4.0,
    num_inference_steps=20,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]

thermal.save("thermal_face.png")

Input preprocessing

For results consistent with the paper, preprocess visible images as in training: detect the face with MTCNN (first / highest-confidence detection, 10-pixel margin), crop, and resize to 512×512. The prompt above is fixed and was used for all generations. The full generation and evaluation pipeline is on GitHub.

🧪 Training details

• Base model: FLUX.1 Kontext [dev] (flow-matching, I2I conditioning).
• Adaptation: LoRA (rank 128, α 128) on all attention and feed-forward linear projections of the double-stream and single-stream blocks; base weights frozen.
• Optimization: learning rate 1e-4, batch size 1, 3000 steps, single fixed prompt, seed 42.
• Data: 50 synchronized visible–thermal pairs from PUCV-VTF; thermal captured with a FLIR Tau 2 640 (LWIR, 14-bit), contrast-normalized to 8-bit via per-image min–max scaling.
• Hardware: 2× NVIDIA H100 (80 GB).

📊 Evaluation (summary)

Synthetic thermal faces from this adapter preserve subject identity at 99.87% rank-1 (d′ = 13.6) against real-thermal templates, versus 11.4% (Pix2Pix) and 3.5% (CycleGAN) under an identical 50-pair budget. An ArcFace recognizer trained only on the synthetic data transfers to real thermal benchmarks, including unseen-identity cross-sensor settings. Full protocol, confidence intervals and open-set results are in the code repository.

🔁 Reproducibility & code

Training, generation and evaluation code, plus the generation prompt and configuration: github.com/GbrlOl/thermal-fr-lora

The synthetic PUCV-TER-S dataset and the source PUCV-VTF recordings are available from the corresponding author upon reasonable request, subject to the consent conditions of the original captures. With the released code and these weights, the synthetic dataset can be regenerated exactly from the source visible images.

📜 License & intended use

This repository is labeled cc-by-nc-4.0 (attribution, non-commercial). Because the weights are a Derivative of FLUX.1 Kontext [dev], the upstream FLUX.1 [dev] Non-Commercial License and its Acceptable Use Policy also apply, and its restrictions (including non-commercial use only) propagate to these weights. In case of any conflict, the more restrictive terms govern. Commercial use of the base model requires a separate license from Black Forest Labs.

🔒 Ethical considerations

The synthetic images derive from consented captures (PUCV-VTF) and are intended solely for research on data scarcity in thermal biometrics. Identity-preserving generative models can, in principle, be misused to fabricate biometric samples; access is therefore gated, redistribution is non-commercial and attribution-bound, and downstream users are expected to adopt equivalent safeguards. No model capable of impersonating individuals outside the enrolled cohort is distributed.

🙏 Acknowledgements

Funded by ANID (FONDECYT Grant 1240573; ANID Magíster Nacional 2025-22251171; ANID Doctorado Nacional 2025-21251472), Pontificia Universidad Católica de Valparaíso (PUCV), Chile.