laurent commited on
Commit
dbb3278
1 Parent(s): bb395cb

Update the readme.

Browse files
Files changed (1) hide show
  1. README.md +73 -0
README.md CHANGED
@@ -1,3 +1,76 @@
1
  ---
2
  license: creativeml-openrail-m
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: creativeml-openrail-m
3
+ tags:
4
+ - stable-diffusion
5
+ - stable-diffusion-diffusers
6
+ - text-to-image
7
+ - rust
8
+ inference: true
9
+ extra_gated_prompt: |-
10
+ This model is open access and available to all, with a CreativeML OpenRAIL-M license further specifying rights and usage.
11
+ The CreativeML OpenRAIL License specifies:
12
+
13
+ 1. You can't use the model to deliberately produce nor share illegal or harmful outputs or content
14
+ 2. CompVis claims no rights on the outputs you generate, you are free to use them and are accountable for their use which must not go against the provisions set in the license
15
+ 3. You may re-distribute the weights and use the model commercially and/or as a service. If you do, please be aware you have to include the same use restrictions as the ones in the license and share a copy of the CreativeML OpenRAIL-M to all your users (please read the license entirely and carefully)
16
+ Please read the full license carefully here: https://huggingface.co/spaces/CompVis/stable-diffusion-license
17
+
18
+ extra_gated_heading: Please read the LICENSE to access this model
19
  ---
20
+
21
+ This repository hosts weights for a Rust based version of Stable Diffusion.
22
+ These weights have been directly adapted from the
23
+ [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5)
24
+ weights, they can be used with the
25
+ [diffusers-rs](https://github.com/LaurentMazare/diffusers-rs) crate.
26
+
27
+ The license is unchanged, see the
28
+ [original version](https://huggingface.co/spaces/CompVis/stable-diffusion-license).
29
+ In line with paragraph 4, the original copyright is preserved:
30
+ Copyright (c) 2022 Robin Rombach and Patrick Esser and contributors
31
+
32
+ The model details section below is copied from the runwayml version, refer to
33
+ the [original repo](https://huggingface.co/runwayml/stable-diffusion-v1-5) for
34
+ use restrictions, limitations, bias discussion etc.
35
+
36
+ ## Model Details
37
+ - **Developed by:** Robin Rombach, Patrick Esser
38
+ - **Model type:** Diffusion-based text-to-image generation model
39
+ - **Language(s):** English
40
+ - **License:** [The CreativeML OpenRAIL M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license) is an [Open RAIL M license](https://www.licenses.ai/blog/2022/8/18/naming-convention-of-responsible-ai-licenses), adapted from the work that [BigScience](https://bigscience.huggingface.co/) and [the RAIL Initiative](https://www.licenses.ai/) are jointly carrying in the area of responsible AI licensing. See also [the article about the BLOOM Open RAIL license](https://bigscience.huggingface.co/blog/the-bigscience-rail-license) on which our license is based.
41
+ - **Model Description:** This is a model that can be used to generate and modify images based on text prompts. It is a [Latent Diffusion Model](https://arxiv.org/abs/2112.10752) that uses a fixed, pretrained text encoder ([CLIP ViT-L/14](https://arxiv.org/abs/2103.00020)) as suggested in the [Imagen paper](https://arxiv.org/abs/2205.11487).
42
+ - **Resources for more information:** [GitHub Repository](https://github.com/CompVis/stable-diffusion), [Paper](https://arxiv.org/abs/2112.10752).
43
+ - **Cite as:**
44
+
45
+ @InProceedings{Rombach_2022_CVPR,
46
+ author = {Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bj\"orn},
47
+ title = {High-Resolution Image Synthesis With Latent Diffusion Models},
48
+ booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
49
+ month = {June},
50
+ year = {2022},
51
+ pages = {10684-10695}
52
+ }
53
+
54
+ ## Weight Extraction
55
+
56
+ The weights have been converted by downloading them from the runwayml/stable-diffusion-v1.5 repo,
57
+ and then running the following commands in the
58
+ [diffusers-rs repo](https://github.com/LaurentMazare/diffusers-rs).
59
+
60
+ After downloading the files, use Python to convert them to `npz` files.
61
+
62
+ ```python
63
+ import numpy as np
64
+ import torch
65
+ model = torch.load("./vae.bin")
66
+ np.savez("./vae.npz", **{k: v.numpy() for k, v in model.items()})
67
+ model = torch.load("./unet.bin")
68
+ np.savez("./unet.npz", **{k: v.numpy() for k, v in model.items()})
69
+ ```
70
+
71
+ Convert these `.npz` files to `.ot` files via `tensor-tools`.
72
+
73
+ ```bash
74
+ cargo run --release --example tensor-tools cp ./data/vae.npz ./data/vae.ot
75
+ cargo run --release --example tensor-tools cp ./data/unet.npz ./data/unet.ot
76
+ ```