multimodalart HF staff dn6 HF staff commited on
Commit
7ca32c2
1 Parent(s): 621fc2d
README.md CHANGED
@@ -10,13 +10,13 @@ license_link: LICENSE
10
  <!-- Provide a quick summary of what the model is/does. -->
11
  <img src="figures/collage_1.jpg" width="800">
12
 
13
- This model is built upon the [Würstchen](https://openreview.net/forum?id=gU58d5QeGv) architecture and its main
14
- difference to other models like Stable Diffusion is that it is working at a much smaller latent space. Why is this
15
- important? The smaller the latent space, the **faster** you can run inference and the **cheaper** the training becomes.
16
- How small is the latent space? Stable Diffusion uses a compression factor of 8, resulting in a 1024x1024 image being
17
- encoded to 128x128. Stable Cascade achieves a compression factor of 42, meaning that it is possible to encode a
18
- 1024x1024 image to 24x24, while maintaining crisp reconstructions. The text-conditional model is then trained in the
19
- highly compressed latent space. Previous versions of this architecture, achieved a 16x cost reduction over Stable
20
  Diffusion 1.5. <br> <br>
21
  Therefore, this kind of model is well suited for usages where efficiency is important. Furthermore, all known extensions
22
  like finetuning, LoRA, ControlNet, IP-Adapter, LCM etc. are possible with this method as well.
@@ -41,65 +41,182 @@ For research purposes, we recommend our `StableCascade` Github repository (https
41
  ### Model Overview
42
  Stable Cascade consists of three models: Stage A, Stage B and Stage C, representing a cascade to generate images,
43
  hence the name "Stable Cascade".
44
- Stage A & B are used to compress images, similar to what the job of the VAE is in Stable Diffusion.
45
- However, with this setup, a much higher compression of images can be achieved. While the Stable Diffusion models use a
46
- spatial compression factor of 8, encoding an image with resolution of 1024 x 1024 to 128 x 128, Stable Cascade achieves
47
- a compression factor of 42. This encodes a 1024 x 1024 image to 24 x 24, while being able to accurately decode the
48
- image. This comes with the great benefit of cheaper training and inference. Furthermore, Stage C is responsible
49
  for generating the small 24 x 24 latents given a text prompt. The following picture shows this visually.
50
 
51
  <img src="figures/model-overview.jpg" width="600">
52
 
53
- For this release, we are providing two checkpoints for Stage C, two for Stage B and one for Stage A. Stage C comes with
54
- a 1 billion and 3.6 billion parameter version, but we highly recommend using the 3.6 billion version, as most work was
55
- put into its finetuning. The two versions for Stage B amount to 700 million and 1.5 billion parameters. Both achieve
56
- great results, however the 1.5 billion excels at reconstructing small and fine details. Therefore, you will achieve the
57
- best results if you use the larger variant of each. Lastly, Stage A contains 20 million parameters and is fixed due to
58
  its small size.
59
 
60
  ## Evaluation
61
  <img height="300" src="figures/comparison.png"/>
62
- According to our evaluation, Stable Cascade performs best in both prompt alignment and aesthetic quality in almost all
63
- comparisons. The above picture shows the results from a human evaluation using a mix of parti-prompts (link) and
64
- aesthetic prompts. Specifically, Stable Cascade (30 inference steps) was compared against Playground v2 (50 inference
65
  steps), SDXL (50 inference steps), SDXL Turbo (1 inference step) and Würstchen v2 (30 inference steps).
66
 
67
  ## Code Example
 
 
 
 
 
68
  ```shell
69
- #install `diffusers` from this branch while the PR is WIP
70
- pip install git+https://github.com/kashif/diffusers.git@wuerstchen-v3
71
  ```
72
 
73
  ```python
74
  import torch
75
  from diffusers import StableCascadeDecoderPipeline, StableCascadePriorPipeline
76
 
77
- device = "cuda"
78
- dtype = torch.bfloat16
79
- num_images_per_prompt = 2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
80
 
81
- prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior", torch_dtype=dtype).to(device)
82
- decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", torch_dtype=dtype).to(device)
83
 
84
- prompt = "Anthropomorphic cat dressed as a pilot"
 
 
 
 
 
 
 
 
 
 
85
  negative_prompt = ""
86
 
87
- with torch.cuda.amp.autocast(dtype=dtype):
88
- prior_output = prior(
89
- prompt=prompt,
90
- height=1024,
91
- width=1024,
92
- negative_prompt=negative_prompt,
93
- guidance_scale=4.0,
94
- num_images_per_prompt=num_images_per_prompt,
95
- )
96
- decoder_output = decoder(
97
- image_embeddings=prior_output.image_embeddings,
98
- prompt=prompt,
99
- negative_prompt=negative_prompt,
100
- guidance_scale=0.0,
101
- output_type="pil",
102
- ).images
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
103
  ```
104
 
105
  ## Uses
@@ -118,7 +235,7 @@ Excluded uses are described below.
118
 
119
  ### Out-of-Scope Use
120
 
121
- The model was not trained to be factual or true representations of people or events,
122
  and therefore using the model to generate such content is out-of-scope for the abilities of this model.
123
  The model should not be used in any way that violates Stability AI's [Acceptable Use Policy](https://stability.ai/use-policy).
124
 
@@ -135,4 +252,4 @@ The model is intended for research purposes only.
135
 
136
  ## How to Get Started with the Model
137
 
138
- Check out https://github.com/Stability-AI/StableCascade
 
10
  <!-- Provide a quick summary of what the model is/does. -->
11
  <img src="figures/collage_1.jpg" width="800">
12
 
13
+ This model is built upon the [Würstchen](https://openreview.net/forum?id=gU58d5QeGv) architecture and its main
14
+ difference to other models like Stable Diffusion is that it is working at a much smaller latent space. Why is this
15
+ important? The smaller the latent space, the **faster** you can run inference and the **cheaper** the training becomes.
16
+ How small is the latent space? Stable Diffusion uses a compression factor of 8, resulting in a 1024x1024 image being
17
+ encoded to 128x128. Stable Cascade achieves a compression factor of 42, meaning that it is possible to encode a
18
+ 1024x1024 image to 24x24, while maintaining crisp reconstructions. The text-conditional model is then trained in the
19
+ highly compressed latent space. Previous versions of this architecture, achieved a 16x cost reduction over Stable
20
  Diffusion 1.5. <br> <br>
21
  Therefore, this kind of model is well suited for usages where efficiency is important. Furthermore, all known extensions
22
  like finetuning, LoRA, ControlNet, IP-Adapter, LCM etc. are possible with this method as well.
 
41
  ### Model Overview
42
  Stable Cascade consists of three models: Stage A, Stage B and Stage C, representing a cascade to generate images,
43
  hence the name "Stable Cascade".
44
+ Stage A & B are used to compress images, similar to what the job of the VAE is in Stable Diffusion.
45
+ However, with this setup, a much higher compression of images can be achieved. While the Stable Diffusion models use a
46
+ spatial compression factor of 8, encoding an image with resolution of 1024 x 1024 to 128 x 128, Stable Cascade achieves
47
+ a compression factor of 42. This encodes a 1024 x 1024 image to 24 x 24, while being able to accurately decode the
48
+ image. This comes with the great benefit of cheaper training and inference. Furthermore, Stage C is responsible
49
  for generating the small 24 x 24 latents given a text prompt. The following picture shows this visually.
50
 
51
  <img src="figures/model-overview.jpg" width="600">
52
 
53
+ For this release, we are providing two checkpoints for Stage C, two for Stage B and one for Stage A. Stage C comes with
54
+ a 1 billion and 3.6 billion parameter version, but we highly recommend using the 3.6 billion version, as most work was
55
+ put into its finetuning. The two versions for Stage B amount to 700 million and 1.5 billion parameters. Both achieve
56
+ great results, however the 1.5 billion excels at reconstructing small and fine details. Therefore, you will achieve the
57
+ best results if you use the larger variant of each. Lastly, Stage A contains 20 million parameters and is fixed due to
58
  its small size.
59
 
60
  ## Evaluation
61
  <img height="300" src="figures/comparison.png"/>
62
+ According to our evaluation, Stable Cascade performs best in both prompt alignment and aesthetic quality in almost all
63
+ comparisons. The above picture shows the results from a human evaluation using a mix of parti-prompts (link) and
64
+ aesthetic prompts. Specifically, Stable Cascade (30 inference steps) was compared against Playground v2 (50 inference
65
  steps), SDXL (50 inference steps), SDXL Turbo (1 inference step) and Würstchen v2 (30 inference steps).
66
 
67
  ## Code Example
68
+
69
+ **Note:** In order to use the `torch.bfloat16` data type with the `StableCascadeDecoderPipeline` you need to have PyTorch 2.2.0 or higher installed. This also means that using the `StableCascadeCombinedPipeline` with `torch.bfloat16` requires PyTorch 2.2.0 or higher, since it calls the StableCascadeDecoderPipeline internally.
70
+
71
+ If it is not possible to install PyTorch 2.2.0 or higher in your environment, the `StableCascadeDecoderPipeline` can be used on its own with the torch.float16 data type. You can download the full precision or bf16 variant weights for the pipeline and cast the weights to torch.float16.
72
+
73
  ```shell
74
+ pip install diffusers
 
75
  ```
76
 
77
  ```python
78
  import torch
79
  from diffusers import StableCascadeDecoderPipeline, StableCascadePriorPipeline
80
 
81
+ prompt = "an image of a shiba inu, donning a spacesuit and helmet"
82
+ negative_prompt = ""
83
+
84
+ prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior", variant="bf16", torch_dtype=torch.bfloat16)
85
+ decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", variant="bf16", torch_dtype=torch.float16)
86
+
87
+ prior.enable_model_cpu_offload()
88
+ prior_output = prior(
89
+ prompt=prompt,
90
+ height=1024,
91
+ width=1024,
92
+ negative_prompt=negative_prompt,
93
+ guidance_scale=4.0,
94
+ num_images_per_prompt=1,
95
+ num_inference_steps=20
96
+ )
97
+
98
+ decoder.enable_model_cpu_offload()
99
+ decoder_output = decoder(
100
+ image_embeddings=prior_output.image_embeddings.to(torch.float16),
101
+ prompt=prompt,
102
+ negative_prompt=negative_prompt,
103
+ guidance_scale=0.0,
104
+ output_type="pil",
105
+ num_inference_steps=10
106
+ ).images[0]
107
+ decoder_output.save("cascade.png")
108
+ ```
109
+
110
+ ### Using the Lite Version of the Stage B and Stage C models
111
+
112
+ ```python
113
+ import torch
114
+ from diffusers import (
115
+ StableCascadeDecoderPipeline,
116
+ StableCascadePriorPipeline,
117
+ StableCascadeUNet,
118
+ )
119
+
120
+ prompt = "an image of a shiba inu, donning a spacesuit and helmet"
121
+ negative_prompt = ""
122
+
123
+ prior_unet = StableCascadeUNet.from_pretrained("stabilityai/stable-cascade-prior", subfolder="prior_lite")
124
+ decoder_unet = StableCascadeUNet.from_pretrained("stabilityai/stable-cascade", subfolder="decoder_lite")
125
+
126
+ prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior", prior=prior_unet)
127
+ decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", decoder=decoder_unet)
128
+
129
+ prior.enable_model_cpu_offload()
130
+ prior_output = prior(
131
+ prompt=prompt,
132
+ height=1024,
133
+ width=1024,
134
+ negative_prompt=negative_prompt,
135
+ guidance_scale=4.0,
136
+ num_images_per_prompt=1,
137
+ num_inference_steps=20
138
+ )
139
+
140
+ decoder.enable_model_cpu_offload()
141
+ decoder_output = decoder(
142
+ image_embeddings=prior_output.image_embeddings,
143
+ prompt=prompt,
144
+ negative_prompt=negative_prompt,
145
+ guidance_scale=0.0,
146
+ output_type="pil",
147
+ num_inference_steps=10
148
+ ).images[0]
149
+ decoder_output.save("cascade.png")
150
+ ```
151
 
152
+ ### Loading original checkpoints with `from_single_file`
 
153
 
154
+ Loading the original format checkpoints is supported via `from_single_file` method in the StableCascadeUNet.
155
+
156
+ ```python
157
+ import torch
158
+ from diffusers import (
159
+ StableCascadeDecoderPipeline,
160
+ StableCascadePriorPipeline,
161
+ StableCascadeUNet,
162
+ )
163
+
164
+ prompt = "an image of a shiba inu, donning a spacesuit and helmet"
165
  negative_prompt = ""
166
 
167
+ prior_unet = StableCascadeUNet.from_single_file(
168
+ "https://huggingface.co/stabilityai/stable-cascade/resolve/main/stage_c_bf16.safetensors",
169
+ torch_dtype=torch.bfloat16
170
+ )
171
+ decoder_unet = StableCascadeUNet.from_single_file(
172
+ "https://huggingface.co/stabilityai/stable-cascade/blob/main/stage_b_bf16.safetensors",
173
+ torch_dtype=torch.bfloat16
174
+ )
175
+
176
+ prior = StableCascadePriorPipeline.from_pretrained("stabilityai/stable-cascade-prior", prior=prior_unet, torch_dtype=torch.bfloat16)
177
+ decoder = StableCascadeDecoderPipeline.from_pretrained("stabilityai/stable-cascade", decoder=decoder_unet, torch_dtype=torch.bfloat16)
178
+
179
+ prior.enable_model_cpu_offload()
180
+ prior_output = prior(
181
+ prompt=prompt,
182
+ height=1024,
183
+ width=1024,
184
+ negative_prompt=negative_prompt,
185
+ guidance_scale=4.0,
186
+ num_images_per_prompt=1,
187
+ num_inference_steps=20
188
+ )
189
+
190
+ decoder.enable_model_cpu_offload()
191
+ decoder_output = decoder(
192
+ image_embeddings=prior_output.image_embeddings,
193
+ prompt=prompt,
194
+ negative_prompt=negative_prompt,
195
+ guidance_scale=0.0,
196
+ output_type="pil",
197
+ num_inference_steps=10
198
+ ).images[0]
199
+ decoder_output.save("cascade-single-file.png")
200
+ ```
201
+
202
+ ### Using the `StableCascadeCombinedPipeline`
203
+
204
+ ```python
205
+ from diffusers import StableCascadeCombinedPipeline
206
+
207
+ pipe = StableCascadeCombinedPipeline.from_pretrained("stabilityai/stable-cascade", variant="bf16", torch_dtype=torch.bfloat16)
208
+
209
+ prompt = "an image of a shiba inu, donning a spacesuit and helmet"
210
+ output = pipe(
211
+ prompt=prompt,
212
+ negative_prompt="",
213
+ num_inference_steps=10,
214
+ prior_num_inference_steps=20,
215
+ prior_guidance_scale=3.0,
216
+ width=1024,
217
+ height=1024,
218
+ )
219
+ output.images[0].save("cascade-combined.png")
220
  ```
221
 
222
  ## Uses
 
235
 
236
  ### Out-of-Scope Use
237
 
238
+ The model was not trained to be factual or true representations of people or events,
239
  and therefore using the model to generate such content is out-of-scope for the abilities of this model.
240
  The model should not be used in any way that violates Stability AI's [Acceptable Use Policy](https://stability.ai/use-policy).
241
 
 
252
 
253
  ## How to Get Started with the Model
254
 
255
+ Check out https://github.com/Stability-AI/StableCascade
image_encoder/config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "StableCascade-prior/image_encoder",
3
  "architectures": [
4
  "CLIPVisionModelWithProjection"
5
  ],
@@ -19,5 +19,5 @@
19
  "patch_size": 14,
20
  "projection_dim": 768,
21
  "torch_dtype": "bfloat16",
22
- "transformers_version": "4.38.0.dev0"
23
  }
 
1
  {
2
+ "_name_or_path": "openai/clip-vit-large-patch14",
3
  "architectures": [
4
  "CLIPVisionModelWithProjection"
5
  ],
 
19
  "patch_size": 14,
20
  "projection_dim": 768,
21
  "torch_dtype": "bfloat16",
22
+ "transformers_version": "4.38.2"
23
  }
image_encoder/model.bf16.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4b33d864f89a793357a768cb07d0dc18d6a14e6664f4110a0d535ca9ba78da8
3
+ size 607980488
image_encoder/model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e4b33d864f89a793357a768cb07d0dc18d6a14e6664f4110a0d535ca9ba78da8
3
- size 607980488
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77b33d2a3a643650857672e880ccf73adbaf114fbbadec36d142ee9d48af7e20
3
+ size 1215912728
model_index.json CHANGED
@@ -1,7 +1,6 @@
1
  {
2
  "_class_name": "StableCascadePriorPipeline",
3
- "_diffusers_version": "0.26.0.dev0",
4
- "_name_or_path": "StableCascade-prior/",
5
  "feature_extractor": [
6
  "transformers",
7
  "CLIPImageProcessor"
@@ -11,8 +10,8 @@
11
  "CLIPVisionModelWithProjection"
12
  ],
13
  "prior": [
14
- "stable_cascade",
15
- "StableCascadeUnet"
16
  ],
17
  "resolution_multiple": 42.67,
18
  "scheduler": [
 
1
  {
2
  "_class_name": "StableCascadePriorPipeline",
3
+ "_diffusers_version": "0.27.0.dev0",
 
4
  "feature_extractor": [
5
  "transformers",
6
  "CLIPImageProcessor"
 
10
  "CLIPVisionModelWithProjection"
11
  ],
12
  "prior": [
13
+ "diffusers",
14
+ "StableCascadeUNet"
15
  ],
16
  "resolution_multiple": 42.67,
17
  "scheduler": [
prior/config.json CHANGED
@@ -1,61 +1,64 @@
1
  {
2
- "_class_name": "StableCascadeUnet",
3
- "_diffusers_version": "0.26.0.dev0",
4
- "_name_or_path": "StableCascade-prior/prior",
5
- "block_repeat": [
6
- [
7
- 1,
8
- 1
9
- ],
10
- [
11
- 1,
12
- 1
13
- ]
14
  ],
15
- "blocks": [
16
  [
17
- 8,
18
- 24
 
19
  ],
20
  [
21
- 24,
22
- 8
 
23
  ]
24
  ],
25
- "c_clip_img": 768,
26
- "c_clip_seq": 4,
27
- "c_clip_text": 1280,
28
- "c_clip_text_pooled": 1280,
29
- "c_cond": 2048,
30
- "c_effnet": null,
31
- "c_hidden": [
32
- 2048,
33
- 2048
 
 
 
34
  ],
35
- "c_in": 16,
36
- "c_out": 16,
37
- "c_pixels": null,
38
- "c_r": 64,
39
  "dropout": [
40
  0.1,
41
  0.1
42
  ],
 
 
43
  "kernel_size": 3,
44
- "level_config": [
45
- "CTA",
46
- "CTA"
47
- ],
48
- "nhead": [
49
  32,
50
  32
51
  ],
 
52
  "patch_size": 1,
 
53
  "self_attn": true,
54
  "switch_level": [
55
  false
56
  ],
57
- "t_conds": [
58
  "sca",
59
  "crp"
 
 
 
 
 
 
 
 
 
60
  ]
61
  }
 
1
  {
2
+ "_class_name": "StableCascadeUNet",
3
+ "_diffusers_version": "0.27.0.dev0",
4
+ "block_out_channels": [
5
+ 2048,
6
+ 2048
 
 
 
 
 
 
 
7
  ],
8
+ "block_types_per_layer": [
9
  [
10
+ "SDCascadeResBlock",
11
+ "SDCascadeTimestepBlock",
12
+ "SDCascadeAttnBlock"
13
  ],
14
  [
15
+ "SDCascadeResBlock",
16
+ "SDCascadeTimestepBlock",
17
+ "SDCascadeAttnBlock"
18
  ]
19
  ],
20
+ "clip_image_in_channels": 768,
21
+ "clip_seq": 4,
22
+ "clip_text_in_channels": 1280,
23
+ "clip_text_pooled_in_channels": 1280,
24
+ "conditioning_dim": 2048,
25
+ "down_blocks_repeat_mappers": [
26
+ 1,
27
+ 1
28
+ ],
29
+ "down_num_layers_per_block": [
30
+ 8,
31
+ 24
32
  ],
 
 
 
 
33
  "dropout": [
34
  0.1,
35
  0.1
36
  ],
37
+ "effnet_in_channels": null,
38
+ "in_channels": 16,
39
  "kernel_size": 3,
40
+ "num_attention_heads": [
 
 
 
 
41
  32,
42
  32
43
  ],
44
+ "out_channels": 16,
45
  "patch_size": 1,
46
+ "pixel_mapper_in_channels": null,
47
  "self_attn": true,
48
  "switch_level": [
49
  false
50
  ],
51
+ "timestep_conditioning_type": [
52
  "sca",
53
  "crp"
54
+ ],
55
+ "timestep_ratio_embedding_dim": 64,
56
+ "up_blocks_repeat_mappers": [
57
+ 1,
58
+ 1
59
+ ],
60
+ "up_num_layers_per_block": [
61
+ 24,
62
+ 8
63
  ]
64
  }
prior/diffusion_pytorch_model.bf16.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44a4cd9540f327f2fb4ac09179e4e87912a01cdb1b3b86c79f0f853976fb4c98
3
+ size 7178377816
prior/diffusion_pytorch_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:44a4cd9540f327f2fb4ac09179e4e87912a01cdb1b3b86c79f0f853976fb4c98
3
- size 7178377816
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a2c7aa62c503780b85f74fd513b1b99c12ea4f83422bdbad5ac264aa68efb4b
3
+ size 14356584672
prior_lite/config.json ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_class_name": "StableCascadeUNet",
3
+ "_diffusers_version": "0.27.0.dev0",
4
+ "block_out_channels": [
5
+ 1536,
6
+ 1536
7
+ ],
8
+ "block_types_per_layer": [
9
+ [
10
+ "SDCascadeResBlock",
11
+ "SDCascadeTimestepBlock",
12
+ "SDCascadeAttnBlock"
13
+ ],
14
+ [
15
+ "SDCascadeResBlock",
16
+ "SDCascadeTimestepBlock",
17
+ "SDCascadeAttnBlock"
18
+ ]
19
+ ],
20
+ "clip_image_in_channels": 768,
21
+ "clip_seq": 4,
22
+ "clip_text_in_channels": 1280,
23
+ "clip_text_pooled_in_channels": 1280,
24
+ "conditioning_dim": 1536,
25
+ "down_blocks_repeat_mappers": [
26
+ 1,
27
+ 1
28
+ ],
29
+ "down_num_layers_per_block": [
30
+ 4,
31
+ 12
32
+ ],
33
+ "dropout": [
34
+ 0.1,
35
+ 0.1
36
+ ],
37
+ "effnet_in_channels": null,
38
+ "in_channels": 16,
39
+ "kernel_size": 3,
40
+ "num_attention_heads": [
41
+ 24,
42
+ 24
43
+ ],
44
+ "out_channels": 16,
45
+ "patch_size": 1,
46
+ "pixel_mapper_in_channels": null,
47
+ "self_attn": true,
48
+ "switch_level": [
49
+ false
50
+ ],
51
+ "timestep_conditioning_type": [
52
+ "sca",
53
+ "crp"
54
+ ],
55
+ "timestep_ratio_embedding_dim": 64,
56
+ "up_blocks_repeat_mappers": [
57
+ 1,
58
+ 1
59
+ ],
60
+ "up_num_layers_per_block": [
61
+ 12,
62
+ 4
63
+ ]
64
+ }
prior_lite/diffusion_pytorch_model.bf16.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1f1e7f429fe290bead3b044734a4aa21ad7e6ae4ed709fc85f65d8d7460190e
3
+ size 2061655280
prior_lite/diffusion_pytorch_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61a01756dcaecda654074624fd5f993dbe22c7f5cb0d08887416e6c594179a6a
3
+ size 4123225040
scheduler/scheduler_config.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "_class_name": "DDPMWuerstchenScheduler",
3
- "_diffusers_version": "0.26.0.dev0",
4
  "s": 0.008,
5
  "scaler": 1.0
6
  }
 
1
  {
2
  "_class_name": "DDPMWuerstchenScheduler",
3
+ "_diffusers_version": "0.27.0.dev0",
4
  "s": 0.008,
5
  "scaler": 1.0
6
  }
text_encoder/config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "StableCascade-prior/text_encoder",
3
  "architectures": [
4
  "CLIPTextModelWithProjection"
5
  ],
@@ -20,6 +20,6 @@
20
  "pad_token_id": 1,
21
  "projection_dim": 1280,
22
  "torch_dtype": "bfloat16",
23
- "transformers_version": "4.38.0.dev0",
24
  "vocab_size": 49408
25
  }
 
1
  {
2
+ "_name_or_path": "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k",
3
  "architectures": [
4
  "CLIPTextModelWithProjection"
5
  ],
 
20
  "pad_token_id": 1,
21
  "projection_dim": 1280,
22
  "torch_dtype": "bfloat16",
23
+ "transformers_version": "4.38.2",
24
  "vocab_size": 49408
25
  }
text_encoder/model.bf16.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:260e0127aca3c89db813637ae659ebb822cb07af71fedc16cbd980e9518dfdcd
3
+ size 1389382688
text_encoder/model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:260e0127aca3c89db813637ae659ebb822cb07af71fedc16cbd980e9518dfdcd
3
- size 1389382688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa5b2e6f4c2efc2d82e4b8312faec1a5540eabfc6415126c9a05c8436a530ef4
3
+ size 2778702264
tokenizer/tokenizer.json CHANGED
@@ -1,21 +1,7 @@
1
  {
2
  "version": "1.0",
3
- "truncation": {
4
- "direction": "Right",
5
- "max_length": 77,
6
- "strategy": "LongestFirst",
7
- "stride": 0
8
- },
9
- "padding": {
10
- "strategy": {
11
- "Fixed": 77
12
- },
13
- "direction": "Right",
14
- "pad_to_multiple_of": null,
15
- "pad_id": 49407,
16
- "pad_type_id": 0,
17
- "pad_token": "<|endoftext|>"
18
- },
19
  "added_tokens": [
20
  {
21
  "id": 49406,
 
1
  {
2
  "version": "1.0",
3
+ "truncation": null,
4
+ "padding": null,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  "added_tokens": [
6
  {
7
  "id": 49406,