BiliSakura/Text2Earth-ControlNet-xBD

Text2Earth ControlNet for xBD

ControlNet fine-tuned on xBD post-event imagery. It takes a building-damage mask and samples a matching satellite image from Text2Earth.

The ControlNet is the only trained module. Text2Earth (UNet, VAE, text encoder) stays frozen.

Conditioning

Item	Value
Base model	lcybuaa/Text2Earth
Conditioning	3-channel binary damage mask, RGB order, values in {0, 1}
Prompt	18_GOOGLE_LEVEL_a high resolution satellite image
Scheduler	Euler discrete
Steps / guidance	50 / 1.0

Binary channels follow the xBD hues: background black, no-damage green, minor blue, major yellow, destroy red, un-classified white.

Usage

import torch
from diffusers import (
    ControlNetModel,
    EulerDiscreteScheduler,
    StableDiffusionControlNetPipeline,
)
from PIL import Image

repo = "BiliSakura/Text2Earth-ControlNet-xBD"
base = "lcybuaa/Text2Earth"
dtype = torch.float16

controlnet = ControlNetModel.from_pretrained(
    repo, subfolder="controlnet", torch_dtype=dtype
)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    base,
    controlnet=controlnet,
    scheduler=EulerDiscreteScheduler.from_pretrained(base, subfolder="scheduler"),
    torch_dtype=dtype,
    safety_checker=None,
).to("cuda")

image = pipe(
    "18_GOOGLE_LEVEL_a high resolution satellite image",
    image=Image.open("mask.png"),
    num_inference_steps=50,
    guidance_scale=1.0,
    controlnet_conditioning_scale=1.0,
).images[0]
image.save("post_event.png")

Weights are under controlnet/.

License

CC BY-NC-SA 4.0, the license of the xBD dataset these weights were trained on.