haodongli commited on
Commit
42bfae1
1 Parent(s): fdb7a14

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -68
README.md CHANGED
@@ -9,71 +9,5 @@ python_version: 3.9
9
  app_file: gradio_demo.py
10
  pinned: false
11
  license: mit
12
- tags: [arXiv:2311.11284, Diffusion, 3D Generation, Gaussian Splatting]
13
- ---
14
-
15
-
16
- # LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching
17
-
18
- [Yixun Liang](https://yixunliang.github.io/)\* [Xin Yang](https://abnervictor.github.io/2023/06/12/Academic-Self-Intro.html)\*, [Jiantao Lin](https://ltt-o.github.io/), [Haodong Li](https://haodong-li.com/), [Xiaogang Xu](https://xiaogang00.github.io), [Yingcong Chen](https://www.yingcong.me)\**
19
-
20
- \*: Equal contribution.
21
- \**: Corresponding author.
22
-
23
- [Paper PDF (Arxiv)](https://arxiv.org/abs/2311.11284) | [Project Page (Coming Soon)]()
24
-
25
- ---
26
-
27
- <div align=center>
28
- <img src="resources/gif/demo-1.gif" width="47.5%"/><img src="resources/gif/demo-2.gif" width="47.5%"/>
29
-
30
- Note: we compress these motion pictures for faster previewing.
31
- </div>
32
-
33
- <div align=center>
34
- <img src="resources/teaser.jpg" width="95%"/>
35
-
36
- Examples of text-to-3D content creations with our framework, the *LucidDreamer*, within **~35mins** on A100.
37
- </div>
38
-
39
-
40
- ## 🎏 Abstract
41
-
42
- We present a text-to-3D generation framework, named the *LucidDreamer*, to distill high-fidelity textures and shapes from pretrained 2D diffusion models.
43
-
44
- <details><summary>CLICK for the full abstract</summary>
45
-
46
- > The recent advancements in text-to-3D generation mark a significant milestone in generative models, unlocking new possibilities for creating imaginative 3D assets across various real-world scenarios. While recent advancements in text-to-3D generation have shown promise, they often fall short in rendering detailed and high-quality 3D models. This problem is especially prevalent as many methods base themselves on Score Distillation Sampling (SDS). This paper identifies a notable deficiency in SDS, that it brings inconsistent and low-quality updating direction for the 3D model, causing the over-smoothing effect. To address this, we propose a novel approach called Interval Score Matching (ISM). ISM employs deterministic diffusing trajectories and utilizes interval-based score matching to counteract over-smoothing. Furthermore, we incorporate 3D Gaussian Splatting into our text-to-3D generation pipeline. Extensive experiments show that our model largely outperforms the state-of-the-art in quality and training efficiency.
47
-
48
- </details>
49
-
50
- ## 🔧 Training Instructions
51
-
52
- Our code is now released! Please refer to this [**link**](resources/Training_Instructions.md) for detailed training instructions.
53
-
54
- ## 🚧 Todo
55
-
56
- - [x] Release the basic training codes
57
- - [x] Release the guidance documents
58
- - [ ] Release the training codes for more applications
59
-
60
-
61
- ## 📍 Citation
62
- ```
63
- @misc{EnVision2023luciddreamer,
64
- title={LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching},
65
- author={Yixun Liang and Xin Yang and Jiantao Lin and Haodong Li and Xiaogang Xu and Yingcong Chen},
66
- year={2023},
67
- eprint={2311.11284},
68
- archivePrefix={arXiv},
69
- primaryClass={cs.CV}
70
- }
71
- ```
72
-
73
- ## Acknowledgement
74
- This work is built on many amazing research works and open-source projects:
75
- - [gaussian-splatting](https://github.com/graphdeco-inria/gaussian-splatting) and [diff-gaussian-rasterization](https://github.com/graphdeco-inria/diff-gaussian-rasterization)
76
- - [Stable-Dreamfusion](https://github.com/ashawkey/stable-dreamfusion)
77
- - [Point-E](https://github.com/openai/point-e)
78
-
79
- Thanks for their excellent work and great contribution to 3D generation area.
 
9
  app_file: gradio_demo.py
10
  pinned: false
11
  license: mit
12
+ tags: [arXiv:2311.11284, Stable Diffusion, 3D Generation, Gaussian Splatting, ISM]
13
+ ---