File size: 4,133 Bytes
7add7b1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
caafa2f
 
7b97ec7
caafa2f
7b97ec7
 
 
 
 
 
 
caafa2f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
---
license: mit
tags:
- stable-diffusion
- stable-diffusion-diffusers
inference: false
---

# Watermarked (extreme) VAE for SDXL

You can use this watermarked VAE for your existing SDXL image generation pipelines 
to make sure generated images are watermarked.

## Usage

```py
from diffusers.models import AutoencoderKL
from diffusers import StableDiffusionXLPipeline

model = "stabilityai/sdxl-turbo"

vae = AutoencoderKL.from_pretrained("imatag/stable-signature-bzh-sdxl-vae-extreme")

pipe = StableDiffusionXLPipeline.from_pretrained(model, vae=vae)
```

For more information, please have a look at [the official demo](https://huggingface.co/spaces/imatag/stable-signature-bzh)

## Purpose

This model was finetuned from the original StableDiffusion-XL autoencoder with the additional objective of including an extremely strong watermark, following the procedure of [StableSignature](https://ai.meta.com/blog/stable-signature-watermarking-generative-ai/). This watermark is detectable by [IMATAG](https://www.imatag.com/)'s demo BZH decoder, available via API.

Compared to other watermarking solutions such as [invisible-watermark](https://github.com/ShieldMnt/invisible-watermark):
- the watermark is quite robust to many unintentional editing transforms such as cropping, resizing, changing contrast or brightness, and JPEG compression
- the watermark has zero overhead in terms of computational power
- the watermark is embedded in the model weights and not as a post-processing of the non-watermarked image
- the probability of detecting the watermark by chance on non-watermarked images may be computed analytically and limited to a specific threshold

## Detecting the watermark

For security reasons we cannot release the weights of the detector. Instead, you may check for the presence of a watermark with the [detect_api.py](https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/detect_api.py) script like this:
```shell
python detect_api.py test.png
```

### Visual
_Visualization of watermark impact on 512x512 images from the COCO2017 validation dataset prompts with SDXL-turbo._ 

<p align="center">
  <br>
  <b>
512x512: original (left), watermarked (middle), difference (right)</b>
</p>

<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/extreme/0001.png />
</p>

<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/extreme/0003.png />
</p>

<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/extreme/0004.png />
</p>

<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/extreme/0005.png />
</p>

<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/extreme/0006.png />
</p>

### Robustness

The watermark is robust to most editorial changes to an image, such as cropping, changing brightness or contrast, resizing or JPEG compression:

<p align="center">
  <br>
  <b>
from left to right: original, watermarked, difference, 'crop 50% + brigthen 50% + jpeg 80%' attack, 'downscale 2x + crop 50% + brigthen 50% + jpeg 50%' attack</b>
</p>

<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/extreme/0000.png />
</p>

The Receiver Operating Characteristic curve shows its performance under no modification and when modified by the two scenarios illustrated above:
<p align="center">
<img src=https://huggingface.co/spaces/imatag/stable-signature-bzh/resolve/main/eval/extreme/roc.png />
</p>

More details available in our [announcement](https://www.imatag.com/blog/unlocking-the-future-of-content-authentication-imatags-breakthrough-in-ai-generated-image-watermarking) and our lab's [blog post](https://imatag-lab.medium.com/stable-signature-meets-bzh-53ad0ba13691).
For watermarked models with a different key, support for payload, other perceptual compromises, robustness to other attacks, or faster detection, please [contact IMATAG](https://pages.imatag.com/contact-us-imatag).