justinpinkney commited on
Commit
66dc58a
1 Parent(s): 2cc94e0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +33 -1
README.md CHANGED
@@ -4,4 +4,36 @@ datasets:
4
  - ChristophSchuhmann/LAION-5B-EN-Aesthetics-Subset_above_6
5
  ---
6
 
7
- ![banner-large.jpeg](https://s3.amazonaws.com/moonup/production/uploads/1673968679262-62bd5f951e22ec84279820e8.jpeg)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  - ChristophSchuhmann/LAION-5B-EN-Aesthetics-Subset_above_6
5
  ---
6
 
7
+ ![banner-large.jpeg](https://s3.amazonaws.com/moonup/production/uploads/1674039767068-62bd5f951e22ec84279820e8.jpeg)
8
+
9
+ Image Mixer is a model that lets you combine the concepts, styles, and compositions from multiple images (and text prompts too) and generate new images.
10
+
11
+ It was trained by [Justin Pinkney](https://www.justinpinkney.com) at [Lambda Labs](https://lambdalabs.com/).
12
+
13
+ ## Training details
14
+
15
+ This model is a fine tuned version of [Stable Diffusion Image Variations](https://huggingface.co/lambdalabs/sd-image-variations-diffusers)
16
+ it has been trained to accept multiple CLIP embedding concatenated along the sequence dimension (as opposed to 1 in the original model).
17
+ During training up to 5 crops of the training images are taken and CLIP embeddings are extracted, these are concatenated and used as the conditioning for the model.
18
+ At inference time, CLIP embeddings from multiple images can be used to generate images which are influence by multiple inputs.
19
+
20
+ Training was done at 640x640 on a subset of LAION improved aesthetics, using 8xA100 from [Lambda GPU Cloud](https://cloud.lambdalabs.com).
21
+
22
+ ## Usage
23
+
24
+ The model is available on [huggingface spaces](https://huggingface.co/spaces/lambdalabs/image-mixer-demo) or to run locally do the following:
25
+
26
+ ```bash
27
+ git clone https://github.com/justinpinkney/stable-diffusion.git
28
+ cd stable-diffusion
29
+ git checkout 1c8a598f312e54f614d1b9675db0e66382f7e23c
30
+ python -m venv .venv --prompt sd
31
+ . .venv/bin/activate
32
+ pip install -U pip
33
+ pip install -r requirements.txt
34
+ python scripts/gradio_image_mixer.py
35
+ ```
36
+
37
+ Then navigate to the gradio demo link printed in the terminal.
38
+
39
+ For details on how to use the model outside the app refer to the [`run` function](https://github.com/justinpinkney/stable-diffusion/blob/c1963a36a4f8ce23784c8247fa1af0e34e02b766/scripts/gradio_image_mixer.py#L79) in `gradio_image_mixer.py`