boris commited on
Commit
db5a22a
1 Parent(s): 5a390e8

doc: update README

Browse files
Files changed (1) hide show
  1. README.md +20 -20
README.md CHANGED
@@ -24,43 +24,48 @@ You can create your own pictures with [the demo](https://huggingface.co/spaces/f
24
 
25
  Refer to [our report](https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-mini--Vmlldzo4NjIxODA).
26
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  ## Development
28
 
29
  ### Dependencies Installation
30
 
31
  For inference only, use `pip install git+https://github.com/borisdayma/dalle-mini.git`.
32
 
33
- For development, clone the repo and use `pip install -e ".[dev]"`. Check style with `make style`.
 
34
 
35
- ### Training of VQGAN
36
 
37
- The VQGAN was trained using [taming-transformers](https://github.com/CompVis/taming-transformers).
38
 
39
- We recommend using the latest version available.
40
 
41
- ### Conversion of VQGAN to JAX
42
 
43
- Use [patil-suraj/vqgan-jax](https://github.com/patil-suraj/vqgan-jax).
44
-
45
- ### Training of Seq2Seq
46
 
47
  Use [`tools/train/train.py`](tools/train/train.py).
48
 
49
  You can also adjust the [sweep configuration file](https://docs.wandb.ai/guides/sweeps) if you need to perform a hyperparameter search.
50
 
51
- ### Inference Pipeline
52
-
53
- To generate sample predictions and understand the inference pipeline step by step, refer to [`tools/inference/inference_pipeline.ipynb`](tools/inference/inference_pipeline.ipynb).
54
-
55
- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/borisdayma/dalle-mini/blob/main/tools/inference/inference_pipeline.ipynb)
56
-
57
  ## FAQ
58
 
59
  ### Where to find the latest models?
60
 
61
  Trained models are on 🤗 Model Hub:
62
 
63
- - [VQGAN-f16-16384](https://huggingface.co/flax-community/vqgan_f16_16384) for encoding/decoding images
64
  - [DALL·E mini](https://huggingface.co/flax-community/dalle-mini) for generating images from a text prompt
65
 
66
  ### Where does the logo come from?
@@ -91,11 +96,6 @@ Many thanks to the people who helped make it better:
91
  - the [DALLE-Pytorch](https://discord.gg/xBPBXfcFHd) and [EleutherAI](https://www.eleuther.ai/) communities for testing and exchanging cool ideas
92
  - [Rohan Anil](https://github.com/rohan-anil) for adding Distributed Shampoo optimizer
93
 
94
- ### Contributing
95
-
96
- Join the community on the [DALLE-Pytorch Discord](https://discord.gg/xBPBXfcFHd).
97
- Any contribution is welcome, from reporting issues to proposing fixes/improvements or testing the model with cool prompts!
98
-
99
  ## Citing DALL·E mini
100
 
101
  If you find DALL·E mini useful in your research or wish to refer, please use the following BibTeX entry.
 
24
 
25
  Refer to [our report](https://wandb.ai/dalle-mini/dalle-mini/reports/DALL-E-mini--Vmlldzo4NjIxODA).
26
 
27
+ ## Inference Pipeline
28
+
29
+ To generate sample predictions and understand the inference pipeline step by step, refer to [`tools/inference/inference_pipeline.ipynb`](tools/inference/inference_pipeline.ipynb).
30
+
31
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/borisdayma/dalle-mini/blob/main/tools/inference/inference_pipeline.ipynb)
32
+
33
+ ## Contributing
34
+
35
+ Join the community on the [DALLE-Pytorch Discord](https://discord.gg/xBPBXfcFHd).
36
+ Any contribution is welcome, from reporting issues to proposing fixes/improvements or testing the model with cool prompts!
37
+
38
+
39
  ## Development
40
 
41
  ### Dependencies Installation
42
 
43
  For inference only, use `pip install git+https://github.com/borisdayma/dalle-mini.git`.
44
 
45
+ For development, clone the repo and use `pip install -e ".[dev]"`.
46
+ Before making a PR, check style with `make style`.
47
 
48
+ ### Image Encoder
49
 
50
+ We use a VQGAN from [taming-transformers](https://github.com/CompVis/taming-transformers), which can also be fine-tuned.
51
 
52
+ Use [patil-suraj/vqgan-jax](https://github.com/patil-suraj/vqgan-jax) if you want to convert a checkpoint to JAX (does not support Gumbel).
53
 
54
+ Any image encoder that turns an image into a fixed sequence of tokens can be used.
55
 
56
+ ### Training of DALL·E mini
 
 
57
 
58
  Use [`tools/train/train.py`](tools/train/train.py).
59
 
60
  You can also adjust the [sweep configuration file](https://docs.wandb.ai/guides/sweeps) if you need to perform a hyperparameter search.
61
 
 
 
 
 
 
 
62
  ## FAQ
63
 
64
  ### Where to find the latest models?
65
 
66
  Trained models are on 🤗 Model Hub:
67
 
68
+ - [VQGAN-f16-16384](https://huggingface.co/dalle-mini/vqgan_imagenet_f16_16384) for encoding/decoding images
69
  - [DALL·E mini](https://huggingface.co/flax-community/dalle-mini) for generating images from a text prompt
70
 
71
  ### Where does the logo come from?
 
96
  - the [DALLE-Pytorch](https://discord.gg/xBPBXfcFHd) and [EleutherAI](https://www.eleuther.ai/) communities for testing and exchanging cool ideas
97
  - [Rohan Anil](https://github.com/rohan-anil) for adding Distributed Shampoo optimizer
98
 
 
 
 
 
 
99
  ## Citing DALL·E mini
100
 
101
  If you find DALL·E mini useful in your research or wish to refer, please use the following BibTeX entry.