Lawrence-cj commited on
Commit
d6e1198
ยท
verified ยท
1 Parent(s): 8c79a9d

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +108 -0
README.md ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - text-to-image
4
+ - Sana
5
+ - 512px_based_image_size
6
+ - Multi-language
7
+ language:
8
+ - en
9
+ - zh
10
+ base_model:
11
+ - Efficient-Large-Model/Sana_1600M_512px_MultiLing
12
+ pipeline_tag: text-to-image
13
+ ---
14
+
15
+ <p align="center" style="border-radius: 10px">
16
+ <img src="https://raw.githubusercontent.com/NVlabs/Sana/refs/heads/main/asset/logo.png" width="35%" alt="logo"/>
17
+ </p>
18
+
19
+ <div style="display:flex;justify-content: center">
20
+ <a href="https://huggingface.co/collections/Efficient-Large-Model/sana-673efba2a57ed99843f11f9e"><img src="https://img.shields.io/static/v1?label=Demo&message=Huggingface&color=yellow"></a> &ensp;
21
+ <a href="https://github.com/NVlabs/Sana"><img src="https://img.shields.io/static/v1?label=Code&message=Github&color=blue&logo=github"></a> &ensp;
22
+ <a href="https://nvlabs.github.io/Sana/"><img src="https://img.shields.io/static/v1?label=Project&message=Github&color=blue&logo=github-pages"></a> &ensp;
23
+ <a href="https://hanlab.mit.edu/projects/sana/"><img src="https://img.shields.io/static/v1?label=Page&message=MIT&color=darkred&logo=github-pages"></a> &ensp;
24
+ <a href="https://arxiv.org/abs/2410.10629"><img src="https://img.shields.io/static/v1?label=Arxiv&message=Sana&color=red&logo=arxiv"></a> &ensp;
25
+ <a href="https://nv-sana.mit.edu/"><img src="https://img.shields.io/static/v1?label=Demo&message=MIT&color=yellow"></a> &ensp;
26
+ <a href="https://discord.gg/rde6eaE5Ta"><img src="https://img.shields.io/static/v1?label=Discuss&message=Discord&color=purple&logo=discord"></a> &ensp;
27
+ </div>
28
+
29
+ # Model card
30
+
31
+ We introduce **Sana**, a text-to-image framework that can efficiently generate images up to 4096 ร— 4096 resolution.
32
+ Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU.
33
+
34
+ Source code is available at https://github.com/NVlabs/Sana.
35
+
36
+
37
+ ## Compare with base model
38
+
39
+ | Model | Language |
40
+ |----------------------------------------------------------------------------------------|----------------------------|
41
+ | [Sana_1600M_512px](https://huggingface.co/Efficient-Large-Model/Sana_1600M_512px) | English |
42
+ | Sana_1600M_512px_MultiLing | English, Chinese, Emoji |
43
+
44
+
45
+ | Model | Sample-1 | Sample-2 | Sample-3 | Sample-4 |
46
+ |----------|-------------------|-----------------|-----------------------------|-------------------|
47
+ | [Sana_1600M_512px](https://huggingface.co/Efficient-Large-Model/Sana_1600M_512px) | <img src="" width=256> | <img src="" width=256> | <img src="" width=256> | |
48
+ | Sana_1600M_512px_MultiLing | <img src="" width=256> | <img src="" width=256> | <img src="" width=256> |
49
+ | Prompt | ๐Ÿฏ ็ฉฟ็€ ๐Ÿ‘• ๅน ๐ŸŽท | ้‡‘่‰ฒ ๐ŸŒ… ไธ‹็š„้•ฟๅŸŽ, traditional Chinese style | ็Œซ Wearing ๐Ÿ•ถ flying on the ๅฝฉ่™น with ๐ŸŒน in the โ„๏ธ | ๐Ÿฆ teaching ๐Ÿฏ to catch ๐Ÿฆ‹
50
+
51
+
52
+ ### Model Description
53
+
54
+ - **Developed by:** NVIDIA, Sana
55
+ - **Model type:** Linear-Diffusion-Transformer-based text-to-image generative model
56
+ - **Model size:** 1648M parameters
57
+ - **Model resolution:** This model is developed to generate 512px based images with multi-scale heigh and width.
58
+ - **License:** [CC BY-NC-SA 4.0 License](./LICENSE.txt)
59
+ - **Model Description:** This is a model that can be used to generate and modify images based on text prompts.
60
+ It is a Linear Diffusion Transformer that uses one fixed, pretrained text encoders ([Gemma2-2B-IT](https://huggingface.co/google/gemma-2-2b-it))
61
+ and one 32x spatial-compressed latent feature encoder ([DC-AE](https://hanlab.mit.edu/projects/dc-ae)).
62
+ - **Special:** This model is fine-tuned from the base model [Efficient-Large-Model/Sana_1600M_512px](https://huggingface.co/Efficient-Large-Model/Sana_1600M_512px) and it supports Emoji, Chinese and English and all mixed prompts.
63
+ - **Resources for more information:** Check out our [GitHub Repository](https://github.com/NVlabs/Sana) and the [Sana report on arXiv](https://arxiv.org/abs/2410.10629).
64
+
65
+ ### Model Sources
66
+
67
+ For research purposes, we recommend our `generative-models` Github repository (https://github.com/NVlabs/Sana),
68
+ which is more suitable for both training and inference and for which most advanced diffusion sampler like Flow-DPM-Solver is integrated.
69
+ [MIT Han-Lab](https://nv-sana.mit.edu/) provides free Sana inference.
70
+ - **Repository:** ttps://github.com/NVlabs/Sana
71
+ - **Demo:** https://nv-sana.mit.edu/
72
+
73
+ ### ๐Ÿงจ Diffusers
74
+
75
+ PR developing: [Sana](https://github.com/huggingface/diffusers/pull/9982) and [DC-AE](https://github.com/huggingface/diffusers/pull/9708)
76
+
77
+
78
+ ## Uses
79
+
80
+ ### Direct Use
81
+
82
+ The model is intended for research purposes only. Possible research areas and tasks include
83
+
84
+ - Generation of artworks and use in design and other artistic processes.
85
+ - Applications in educational or creative tools.
86
+ - Research on generative models.
87
+ - Safe deployment of models which have the potential to generate harmful content.
88
+
89
+ - Probing and understanding the limitations and biases of generative models.
90
+
91
+ Excluded uses are described below.
92
+
93
+ ### Out-of-Scope Use
94
+
95
+ The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
96
+
97
+ ## Limitations and Bias
98
+
99
+ ### Limitations
100
+
101
+
102
+ - The model does not achieve perfect photorealism
103
+ - The model cannot render complex legible text
104
+ - fingers, .etc in general may not be generated properly.
105
+ - The autoencoding part of the model is lossy.
106
+
107
+ ### Bias
108
+ While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.