Instructions to use JiayiWuLeo/WaterGen with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use JiayiWuLeo/WaterGen with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("JiayiWuLeo/WaterGen") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
WaterGen: Decoupling Scene and Medium in Underwater Image Generation
Jiayi Wu1,, Tianfu Wang1,, Tianyi Xiong1, Dehao Yuan1, Xiaomin Lin2, Md Jahidul Islam3, Cornelia Fermüller1, Christopher Metzler1, Yiannis Aloimonos1
1University of Maryland · 2University of South Florida · 3University of Florida
*Equal contribution
TL;DR
WaterGen generates diverse underwater images while independently controlling scene content and physical water-medium effects. A LoRA-adapted diffusion backbone first synthesizes a clean underwater scene latent; a medium-conditioned decoder then applies attenuation, scattering, and background light. The same scene can be rendered under many water conditions without changing geometry, producing paired clean/degraded images for downstream restoration and segmentation.
Overview
WaterGen treats underwater generation as two separate controls: what is in the image and what the water does to the image. Existing methods typically entangle these factors, so changing water color or turbidity often alters object layout. WaterGen avoids this by decoupling the two stages. A LoRA-adapted SDXL backbone generates a clean underwater scene latent from a text prompt, and a medium-conditioned decoder applies physically meaningful attenuation and backscattering according to specified water parameters.
This decoupled design lets a single generated scene be rendered under many water types without changing the underlying geometry. The resulting aligned clean/degraded pairs can be used as synthetic training data for underwater restoration and semantic segmentation.
Model Files
This repository contains the inference checkpoints used to reproduce the results shown on the project page.
stage1/
└── pytorch_lora_weights.safetensors # SDXL LoRA for clean scene generation (rank 32, α 16)
stage2/
└── model.pth # medium-conditioned decoder
Refer to the code repository for loading and inference instructions.
Citation
@inproceedings{wu2026watergen,
title={WaterGen: Decoupling Scene and Medium in Underwater Image Generation},
author={Wu, Jiayi and Wang, Tianfu and Xiong, Tianyi and Yuan, Dehao and Lin, Xiaomin and Islam, Md Jahidul and Fermuller, Cornelia and Metzler, Christopher and Aloimonos, Yiannis},
booktitle={European Conference on Computer Vision (ECCV)},
year={2026}
}
- Downloads last month
- -
Model tree for JiayiWuLeo/WaterGen
Base model
stabilityai/stable-diffusion-xl-base-1.0