Byuns5/RamiGen_Pretrained_checkpoint

RamiGen: Region-Aware Controllable Diffusion with Self-Adaptive Conditional Evolution for Instance-Centric Remote Sensing Object Generation

Official pretrained checkpoints for RamiGen
ISPRS Journal of Photogrammetry and Remote Sensing, 2026

Yueheng Yang, Bin Wang, Wenyan Ran, Ning Zhou, Minting Zhou, Jindi Wang, and Haigang Sui

📄 Paper | 💻 GitHub Repository | 🤗 Model Weights

🌟 Overview

RamiGen is a region-aware controllable diffusion framework for instance-centric object generation in optical and synthetic aperture radar (SAR) remote sensing imagery. It is designed to synthesize objects with improved spatial placement, geometric structure, contours, and local appearance, especially in scenes with dense layouts, small objects, and complex backgrounds.

Unlike approaches that rely only on textual descriptions or coarse bounding-box constraints, RamiGen uses annotated object regions to construct fine-grained, complementary conditions. A Region-Aware Adaptive Condition Evolution Module (RA²CEM) produces and refines spatial guidance, while an SSAFormer-based diffusion backbone and conditional guidance network use the resulting information for controllable generation.

🔍 Key Features

• Instance-centric generation: Uses object bounding boxes as regional prompts to guide the location and appearance of generated remote sensing objects.
• Multi-dimensional spatial guidance: Combines mask, edge, and depth information to capture object semantics, boundaries, and structural details.
• Self-adaptive condition evolution: Refines region-specific control signals to support more detailed object generation.
• Transformer-based diffusion: Employs the RamiGen SSAFormer architecture to build the denoising backbone and conditional guidance network.
• Optical and SAR imagery: Evaluated on DIOR, FRSMOD, and SARDet-100K in the associated paper.

📦 Released Checkpoints

This repository hosts the pretrained RamiGen DIOR backbone and its matching ControlNet checkpoint, along with the HunyuanDiT-v1.2 base-model assets required by the native RamiGen inference code (including the VAE, text encoders, and tokenizers).

The files are intended for the custom RamiGen PyTorch inference scripts in our GitHub repository. They are not a drop-in diffusers.DiffusionPipeline.from_pretrained() model.

Relevant paths within this Hugging Face repository:

t2i/
├── model/pytorch_model_ema.pt
├── clip_text_encoder/
├── tokenizer/
├── mt5/
└── sdxl-vae-fp16-fix/

Pretrained_weights/
└── DIOR_weights/
    ├── RamiGen_Backbone_DIOR.pt
    └── DIOR_ControlNet_0029300.pt

Note: The results for FRSMOD and SARDet-100K below are reported in the paper. They do not imply that pretrained checkpoints for those datasets are included in this release.

⬇️ Download

Install the Hugging Face CLI and download all files to the model directory:

python -m pip install -U huggingface_hub

hf download Byuns5/RamiGen_Pretrained_checkpoint \
  --local-dir ckpts/HunyuanDiT-v1.2

To download only the files required for the DIOR inference example:

hf download Byuns5/RamiGen_Pretrained_checkpoint \
  --include "t2i/*" "Pretrained_weights/DIOR_weights/*" \
  --local-dir ckpts/HunyuanDiT-v1.2

Both commands preserve the original folder hierarchy. No separate download of the HunyuanDiT-v1.2 base weights is required when using these assets.

🚀 Inference

Use the native RamiGen implementation from GitHub. Follow the repository's installation and RACEM preprocessing instructions to prepare reference images and edge/depth/mask condition maps before running conditional inference.

From the root of the RamiGen project, after downloading the checkpoints and preparing the inference JSONL:

MODEL_ROOT=ckpts/HunyuanDiT-v1.2

CUDA_VISIBLE_DEVICES=0 PYTHONPATH=. python sample_controlnet_multi.py \
  --model-root "$MODEL_ROOT" \
  --dit-weight "$MODEL_ROOT/Pretrained_weights/DIOR_weights/RamiGen_Backbone_DIOR.pt" \
  --controlnet-weight "$MODEL_ROOT/Pretrained_weights/DIOR_weights/DIOR_ControlNet_0029300.pt" \
  --use-condition-fusion --condition-fusion-in-channels 7 \
  --fusion-control-types edge depth mask --control-weight 1.0 \
  --index-file dataset/porcelain_DIOR/inference_jsonl/test.jsonl \
  --results-dir results/inference/DIOR \
  --image-size 1024 1024 --rope-img base512 --rope-real \
  --sampler dpmms --infer-steps 30 --cfg-scale 6.5 --batch-size 1 \
  --seed 42 --infer-mode fa --no-enhance

Important: This is a conditional inference example, not an unconditional text-to-image example. Its JSONL input must reference existing condition maps, and the base512 RoPE option must match the released checkpoint pair. The example uses FlashAttention (--infer-mode fa); see the project README for the required environment and alternatives.

The standalone RACEM preprocessing models (HED, SAM2.1, and MiDaS) are described in the project README.

📊 Results

Selected results from the published paper (not new measurements generated by the example above):

Dataset	FID ↓	KID ↓	YOLO Score ↑
DIOR	37.586	0.011	0.3212
FRSMOD	45.761	0.008	0.7319
SARDet-100K	55.738	0.034	0.2371

See the paper for complete comparisons, ablation studies, and experimental protocols.

📢 Release Status

• ✅ Paper published in ISPRS Journal of Photogrammetry and Remote Sensing (2026).
• ✅ RamiGen pretrained DIOR checkpoints released on Hugging Face.
• ✅ Initial inference and offline RACEM code available in the GitHub project.
• ⏳ Training code and FRSMOD dataset are planned for a later release.

📚 Citation

If you use RamiGen or its released checkpoints in your research, please cite:

@article{yang2026ramigen,
  title   = {RamiGen: region-aware controllable diffusion with self-adaptive conditional evolution for instance-centric remote sensing object generation},
  author  = {Yang, Yueheng and Wang, Bin and Ran, Wenyan and Zhou, Ning and Zhou, Minting and Wang, Jindi and Sui, Haigang},
  journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
  volume  = {242},
  pages   = {1039--1066},
  year    = {2026},
  doi     = {10.1016/j.isprsjprs.2026.09.048}
}

🙏 Acknowledgements and Licensing

RamiGen builds upon HunyuanDiT, and its conditional preprocessing uses components from Paddle HED, SAM2, and MiDaS. We thank the authors of these projects for their contributions.

Licensing: The metadata above preserves the Hunyuan community license associated with the included base-model assets. Users should also consult the licenses and usage terms of the upstream components and the RamiGen source code. Redistribution of third-party assets does not eliminate their original licensing requirements.