Hugging Face's logo

sshh12
/
sdxl-lora-pokemon

Text-to-Image
Diffusers
stable-diffusion-xl
stable-diffusion-xl-diffusers
lora
Model card Files Community
sshh12 commited on
Commit
c71c54b
•
1 Parent(s): 0038a7e

update

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +1 -0
  2. README.md +50 -1
  3. checkpoint-1000/optimizer.bin +3 -0
  4. checkpoint-1000/pytorch_lora_weights.bin +3 -0
  5. checkpoint-1000/random_states_0.pkl +3 -0
  6. checkpoint-1000/scaler.pt +3 -0
  7. checkpoint-1000/scheduler.bin +3 -0
  8. checkpoint-1500/optimizer.bin +3 -0
  9. checkpoint-1500/pytorch_lora_weights.bin +3 -0
  10. checkpoint-1500/random_states_0.pkl +3 -0
  11. checkpoint-1500/scaler.pt +3 -0
  12. checkpoint-1500/scheduler.bin +3 -0
  13. checkpoint-2000/optimizer.bin +3 -0
  14. checkpoint-2000/pytorch_lora_weights.bin +3 -0
  15. checkpoint-2000/random_states_0.pkl +3 -0
  16. checkpoint-2000/scaler.pt +3 -0
  17. checkpoint-2000/scheduler.bin +3 -0
  18. checkpoint-2500/optimizer.bin +3 -0
  19. checkpoint-2500/pytorch_lora_weights.bin +3 -0
  20. checkpoint-2500/random_states_0.pkl +3 -0
  21. checkpoint-2500/scaler.pt +3 -0
  22. checkpoint-2500/scheduler.bin +3 -0
  23. checkpoint-3000/optimizer.bin +3 -0
  24. checkpoint-3000/pytorch_lora_weights.bin +3 -0
  25. checkpoint-3000/random_states_0.pkl +3 -0
  26. checkpoint-3000/scaler.pt +3 -0
  27. checkpoint-3000/scheduler.bin +3 -0
  28. checkpoint-3500/optimizer.bin +3 -0
  29. checkpoint-3500/pytorch_lora_weights.bin +3 -0
  30. checkpoint-3500/random_states_0.pkl +3 -0
  31. checkpoint-3500/scaler.pt +3 -0
  32. checkpoint-3500/scheduler.bin +3 -0
  33. checkpoint-4000/optimizer.bin +3 -0
  34. checkpoint-4000/pytorch_lora_weights.bin +3 -0
  35. checkpoint-4000/random_states_0.pkl +3 -0
  36. checkpoint-4000/scaler.pt +3 -0
  37. checkpoint-4000/scheduler.bin +3 -0
  38. checkpoint-4500/optimizer.bin +3 -0
  39. checkpoint-4500/pytorch_lora_weights.bin +3 -0
  40. checkpoint-4500/random_states_0.pkl +3 -0
  41. checkpoint-4500/scaler.pt +3 -0
  42. checkpoint-4500/scheduler.bin +3 -0
  43. checkpoint-500/optimizer.bin +3 -0
  44. checkpoint-500/pytorch_lora_weights.bin +3 -0
  45. checkpoint-500/random_states_0.pkl +3 -0
  46. checkpoint-500/scaler.pt +3 -0
  47. checkpoint-500/scheduler.bin +3 -0
  48. checkpoint-5000/optimizer.bin +3 -0
  49. checkpoint-5000/pytorch_lora_weights.bin +3 -0
  50. checkpoint-5000/random_states_0.pkl +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ *.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -8,9 +8,58 @@ tags:
8
  - text-to-image
9
  - diffusers
10
  - lora
11
- inference: true
12
  ---
13
 
14
  These are LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0. The weights were fine-tuned on the lambdalabs/pokemon-blip-captions dataset.
15
 
16
  Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  - text-to-image
9
  - diffusers
10
  - lora
11
+ inference: false
12
  ---
13
 
14
  These are LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0. The weights were fine-tuned on the lambdalabs/pokemon-blip-captions dataset.
15
 
16
  Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.
17
+
18
+ | | | | |
19
+ | ------------------------------------------------------- | ---------------------------------------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------- |
20
+ | ![img_1](./imgs/img_aspider_ckpt8000_gs5.0_seed100.png) | ![img_2](./imgs/img_roboticcatwithwingsabrahamlincoln_ckpt3000_gs10.0_seed100.png) | ![img_3](./imgs/img_yoda_ckpt3000_gs10.0_seed100.png) | ![img_4](./imgs/img_abrahamlincoln_ckpt3000_gs10.0_seed0.png) |
21
+
22
+ ## 🧨 Diffusers Usage
23
+
24
+ ```py
25
+ import torch
26
+ from diffusers import DiffusionPipeline, AutoencoderKL
27
+
28
+ vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)
29
+ pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", vae=vae, torch_dtype=torch.float16, variant="fp16", use_safetensors=True)
30
+ pipe.load_lora_weights("sshh12/sdxl-lora-pokemon")
31
+ pipe.to("cuda")
32
+
33
+ prompt = "..."
34
+
35
+ image = pipe(prompt=prompt).images[0]
36
+ image
37
+ ```
38
+
39
+ ## Training
40
+
41
+ ```py
42
+ MODEL_NAME="stabilityai/stable-diffusion-xl-base-1.0"
43
+ DATASET_NAME="lambdalabs/pokemon-blip-captions"
44
+
45
+ !accelerate launch train_text_to_image_lora_sdxl.py \
46
+ --pretrained_model_name_or_path="$MODEL_NAME" \
47
+ --pretrained_vae_model_name_or_path="madebyollin/sdxl-vae-fp16-fix" \
48
+ --dataset_name="$DATASET_NAME" \
49
+ --caption_column="text" \
50
+ --resolution=1024 \
51
+ --random_flip \
52
+ --mixed_precision="fp16" \
53
+ --use_8bit_adam \
54
+ --train_batch_size=1 \
55
+ --gradient_accumulation_steps=8 \
56
+ --num_train_epochs=200 \
57
+ --checkpointing_steps=500 \
58
+ --learning_rate=1e-04 \
59
+ --lr_scheduler="constant" \
60
+ --lr_warmup_steps=0 \
61
+ --seed=0 \
62
+ --validation_prompt="cute dragon creature" \
63
+ --enable_xformers_memory_efficient_attention \
64
+ --report_to="wandb"
65
+ ```
checkpoint-1000/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f0c3e5c2b6f9c4a5ecce0f29594d13ddead9e6905be484f83503b3b0b5e1d0a
3
+ size 14990599
checkpoint-1000/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d4c3f111e6e6e1f1b1ac344216aefe7e7c9905a7b851b93e8283a93a21fa072
3
+ size 23664755
checkpoint-1000/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8fc0d2410d336a82dcbe775ca697ea3176a1e34afa31ff8e67d1691c2c407d3b
3
+ size 14663
checkpoint-1000/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68cff80b680ddf6e7abbef98b5f336b97f9b5963e2209307f639383870e8cc71
3
+ size 557
checkpoint-1000/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:963c3085720a3a3ca1d178559a0f2dc5d5b36fa395a4663f0194bfac4a754038
3
+ size 563
checkpoint-1500/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8c99a096240609bd62854899aaba8d8fc531f9c7e8dd087dd097ce39b02dda0
3
+ size 14990599
checkpoint-1500/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e4e15d316b891ce618f357061f949bbd5901e593f98c036482584a540b06c42
3
+ size 23664755
checkpoint-1500/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a662631b07adadebf172fae7a26f5b9ed4cad57ebfa6a6eca93270954ee9cb1
3
+ size 14599
checkpoint-1500/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:203a72d6c29f42a0e2964fdddc8d7a98df1eccee78fea9de0fa416613390f5c6
3
+ size 557
checkpoint-1500/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6ab1339da08e3fa015ca19e5a001977bd59cee7c7573120ae85950b1c7964a0
3
+ size 563
checkpoint-2000/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7906e272c0f573576adfa7c5891e495dc6786b912947bf224e3e867c710e7623
3
+ size 14990599
checkpoint-2000/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:300babdb2d6277df3fe7fd563b4a498297c6bdfa56b09d467561772a6ad12d07
3
+ size 23664755
checkpoint-2000/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64c8adb0f93e12d41d91f95ff37663ffed89010ea02d5a1bdfeabddc517598b1
3
+ size 14663
checkpoint-2000/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd2de9749828adacdf103bf6e9592702bb7067a2c1df27dd62ab38c1eb8c070f
3
+ size 557
checkpoint-2000/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5a99a60215bbd5dc45ef331980a49be61c64d9af979413113bdfdd196321f239
3
+ size 563
checkpoint-2500/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ec10193e2a2c98d96d9521833d38fd008b000a6e8248341a9942909c620f30e
3
+ size 14990599
checkpoint-2500/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f934ccc0a06a1256450553837cd09e4b6f785a9fbbdf0403f8918f1d6a061cd
3
+ size 23664755
checkpoint-2500/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:793202eedd8e0443c265914656d6418ffee391c8009861aba2213a8522bac424
3
+ size 14727
checkpoint-2500/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0fbcebc8f5487b0c117b5dd47f2ea304af3eebf408d297118d9307e1223927e1
3
+ size 557
checkpoint-2500/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93ffb3304c18d08989aa03bfdd71e5f5bf18f5c6c28053891b95749795419abf
3
+ size 563
checkpoint-3000/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b342b7eb5d85421f4e7b001cbbf28179de17b3d45c9d8a0a064d202aa499a171
3
+ size 14990599
checkpoint-3000/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93f4098e9a585a3597f735c6d401eb88b5985087b0e6c129c29d71bc0170461f
3
+ size 23664755
checkpoint-3000/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:724a67b98c59ab3eb04afb3157bb239d5d4b537f255587ff827d10c10b898fb2
3
+ size 14663
checkpoint-3000/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fb1f9398b77268202e8e1465734a63d123b1ef11c27f20f2473677e9883a6869
3
+ size 557
checkpoint-3000/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f09f1701655fb0b46f613fd0e247c71854a63f39d00ff846a65f60d677f52186
3
+ size 563
checkpoint-3500/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1dcc0e25ae7b0b5c2218ed2d0cdee186b2f3c50df0c0fef7aef08a7475c44494
3
+ size 14990599
checkpoint-3500/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3417359c5408da1a67a8e88f13835f319c7bc9a2e59d5d1d251c55aad5f30de
3
+ size 23664755
checkpoint-3500/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88c6be8fa7d8fe628acfd139d54f627cec3ad34ffe6233d38d6a0c71116ae80f
3
+ size 14663
checkpoint-3500/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4cfe72973031c5a482f6e9ed0cb664a6aa7f63116e294e2b1f72d360d9b033c
3
+ size 557
checkpoint-3500/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d51deca79223f6727d91e7c80b69bf9259b3a3b7de246db779b2348b630c3492
3
+ size 563
checkpoint-4000/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6912a99c36ecd791e2201a325a5386d992f83cb4d5f94030706308fc3afaadf3
3
+ size 14990599
checkpoint-4000/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9fdd0e0af65927f032082c40f984ec9b5548d94a03a48a205dae498952d8b0f3
3
+ size 23664755
checkpoint-4000/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:765f2808b33ee5b6bcaf9f8ea9cb90174b41445c3055e75171a9b5ef692935cb
3
+ size 14663
checkpoint-4000/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:09a61ecdaa4a69ba865e0b9e19169660549c34c60a83537ab5ece413678520ab
3
+ size 557
checkpoint-4000/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0680b1515e44be86ffadd4b90d1307cf5b2489fb6d3258dd184ae2b014367dda
3
+ size 563
checkpoint-4500/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d74697c6e830a8d90c374662c54c74042565f59aa307c35f7c3e6cd33738f19
3
+ size 14990599
checkpoint-4500/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:381dfb0f9ec2ed02893c5abfa2758d3144b2ff53c25f6bd60a09930703e2b1fa
3
+ size 23664755
checkpoint-4500/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68df039a1f6e964d84d92935aee62df1a9ae1ec456b85ce9027fef0e525b74af
3
+ size 14663
checkpoint-4500/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:945e2cc9e717c41b0b1286d550bee32d1e1a6cb3d9a42935c0294b123372c89f
3
+ size 557
checkpoint-4500/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4a4898304ffaab5f4f469b6893e7f931b44058e3444c46443edc3d83fb45507f
3
+ size 563
checkpoint-500/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1cf3cb0fc98d57ba61ab512d62c4d87e0be4bcdf2fe5e13dc286669865903bb
3
+ size 14990599
checkpoint-500/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e29080f73c9d9eb38c1dc89767469d5326d8ab522314048de3245db893b2138
3
+ size 23664755
checkpoint-500/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:472fc814a62f3ce1c2f714f9ef4818f9df7493fbc831ccc947b2cdb6964d45ed
3
+ size 14727
checkpoint-500/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a3f196a54202bb4ba1220e8c59f42f9cda0702d68ea83147d814c2fb2f36b8f2
3
+ size 557
checkpoint-500/scheduler.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62783ff6f0ae2c2ccc177b097e430c03940c5e70c38a191df273c5cf7ab1227c
3
+ size 563
checkpoint-5000/optimizer.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ecef9a5705157b066ac35e8623e0c1b961f3a6155ac62dde3ebb92bedcdc9a36
3
+ size 14990599
checkpoint-5000/pytorch_lora_weights.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4bfee37058ceaefd50ab9536193f68d9736556306cfc9f7a72b9dd1c1a7a894
3
+ size 23664755
checkpoint-5000/random_states_0.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:955e7c1803295f70874381287674b7e10cf5449593dc150d1e60cc651a9268b4
3
+ size 14663