license: mit
tags:
- stable-diffusion
- stable-diffusion-diffusers
inference: false
Watermarked (extreme) VAE for SDXL
You can use this watermarked VAE for your existing SDXL image generation pipelines to make sure generated images are watermarked.
Usage
from diffusers.models import AutoencoderKL
from diffusers import StableDiffusionXLPipeline
model = "stabilityai/sdxl-turbo"
vae = AutoencoderKL.from_pretrained("imatag/stable-signature-bzh-sdxl-vae-extreme")
pipe = StableDiffusionXLPipeline.from_pretrained(model, vae=vae)
For more information, please have a look at the official demo
Decoder Finetuning
This model was finetuned from the original StableDiffusion-XL autoencoder with the additional objective of including a extreme invisible watermark, following the procedure of StableSignature. This watermark is detectable by IMATAG's demo BZH decoder, available via API.
Detecting the watermark
For security reasons we cannot release the weights of the detector. Instead, you may check for the presence of a watermark with the detect_api.py script like this:
python detect_api.py test.png
Visual
Visualization of watermark impact on 512x512 images from the COCO2017 validation dataset prompts with SDXL-turbo.
512x512: original (left), watermarked (middle), difference (right)
Robustness
The watermark is robust to most editorial changes to an image, such as cropping, changing brightness or contrast, resizing or JPEG compression:
from left to right: original, watermarked, difference, 'crop 50% + brigthen 50% + jpeg 80%' attack, 'downscale 2x + crop 50% + brigthen 50% + jpeg 50%' attack
The Receiver Operating Characteristic curve shows its performance under no modification and when modified by the two scenarios illustrated above:
More details available in our announcement and our lab's blog post. For watermarked models with a different key, support for payload, other perceptual compromises, robustness to other attacks, or faster detection, please contact IMATAG.