SAR ControlNet

Overview

ControlNet for spatially controlled synthesis of Sentinel-1 SAR images.
Published alongside the upcoming paper "Diffusion-Based SAR Training Data Synthesis Controlled by Spatial Annotations" (Hochstuhl et al., 2026; accepted for GCPR conference 2026).

This model must be used together with the corresponding Stable Diffusion backbone:
sylviaHoch/SAR-StableDiffusion

Spatial control is provided via three-channel one-hot masks (water / ship / land), enabling precise placement of ships in the generated SAR images.

Database

The ControlNet was trained on the training set of OpenSARShip Ship Detection Dataset (OSSDD) [dataset], a Sentinel-1 dataset with 15,161 patches (700 × 700 px) and 64,640 ships, providing rotated bounding boxes (RBBs) and land–water masks. Text captions used during ControlNet training depend on land coverage: open-water patches are captioned "An aerial view of ships on open water.", patches with land (> 5 % of pixels) "An aerial view of a coastal area with ships on the water." During training, 512 × 512 crops are extracted, amplitude values converted to dB and linearly mapped to [-1, 1].

Architecture

Component	Details
Type	ControlNet (based on SD 1.5 UNet)
Control input — ship + land	Three-channel one-hot mask `(3, H, W)` — ch0: background, ch1: ship, ch2: land
Required backbone	`sylviaHoch/SAR-StableDiffusion`

Repository Structure

SAR-ControlNet/
├── config.json
└── diffusion_pytorch_model.safetensors

Usage

This model can be used directly with the 🤗 diffusers ControlNet pipeline. It requires the corresponding Stable Diffusion backbone (sylviaHoch/SAR-StableDiffusion).

Requirements:

torch
diffusers
peft

Control input: The control image must be a three-channel, one-hot encoded mask of size 512 × 512, i.e. a tensor of shape (3, 512, 512) with values in {0, 1}. Each pixel is assigned to exactly one class across the three channels:

Channel 0: water
Channel 1: ship (represented as rotated bounding box/ minimal enclosing rectangle)
Channel 2: land

Output: image is a torch.Tensor of shape (N, C, H, W) with values in the range [0, 1] and dtype float32, where N is the batch size (here 1), C the number of channels (here 1, single-channel SAR amplitude), and H and W the native model resolution.

import torch
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline, UniPCMultistepScheduler
from peft import PeftModel

SD_MODEL_ID = "sylviaHoch/SAR-StableDiffusion"
CN_MODEL_ID = "sylviaHoch/SAR-ControlNet"

# control_image: torch.Tensor of shape (3, 512, 512), one-hot encoded
#   channel 0: water, channel 1: ship, channel 2: land

controlnet = ControlNetModel.from_pretrained(CN_MODEL_ID, torch_dtype=torch.float16)

pipeline = StableDiffusionControlNetPipeline.from_pretrained(
    SD_MODEL_ID,
    controlnet=controlnet,
    torch_dtype=torch.float16
).to("cuda")

# Load LoRA text-encoder adapter
pipeline.text_encoder = PeftModel.from_pretrained(
    pipeline.text_encoder,
    SD_MODEL_ID,
    subfolder="adapter_text_encoder"
)

pipeline.scheduler = UniPCMultistepScheduler.from_config(pipeline.scheduler.config)

# Generate
image = pipeline(
    "An aerial view of ships on open water.",
    control_image,
    num_inference_steps=50,
    guidance_scale=7.5,
    controlnet_conditioning_scale=1.0,
    output_type="pt"
).images

Intended Use

Spatially controlled synthesis of Sentinel-1 SAR training data.
Ship placement guided by spatial annotations.
Training data generation or data augmentation for ship detection models.

Limitations

Tuned specifically to Sentinel-1 sensor characteristics (in particular Sentinel-1 GRD products in VH polarization).
Training scenes are limited to harbour, coastal, and open water scenes.
Generated images are synthetic and may not fully capture all real SAR image properties.

License

This model is released under CC BY-NC-SA 4.0.
Commercial use is not permitted. Derivatives must be shared under the same license.
See LICENSE for details. ```

Downloads last month: 2

Safetensors

Model size

0.4B params

Tensor type

F32