hhks commited on
Commit
ee02afd
1 Parent(s): 0d064a1

Upload ponyxl_loras_n_stuff.md

Browse files
Files changed (1) hide show
  1. ponyxl_loras_n_stuff.md +44 -39
ponyxl_loras_n_stuff.md CHANGED
@@ -1,4 +1,7 @@
1
  #PonyXL Notes
 
 
 
2
  ##Get PonyXL here
3
  https://civitai.com/models/257749?modelVersionId=290640
4
 
@@ -17,11 +20,12 @@ An anon also created a LoRA extract of it if you want to experiment with it on b
17
  https://civitai.com/models/312010/4th-tail-lora-extract
18
 
19
  ##Reverse Engineered Hashed Tokens
20
- It seems like the obfuscated/hashed tokens in the model are only mapped to three letter strings. There's only about 17k possible 3 letter combinations so it might be possible to just reverse engineer the remapped tokens through brute force, here's a few that are already known, hopefully this list will be expanded.
21
 
22
- **Update 3/11/2024**
23
- An anon has been generating images for each 3 letter combination and noting which ones seem to change the image significantly. Here's his output:
24
 
 
25
  aaa to bzm:
26
  https://files.catbox.moe/c0rl1r.jpg
27
  cad to eum:
@@ -34,12 +38,26 @@ jkv to lek:
34
  https://files.catbox.moe/tj2aeq.jpg
35
  lgu to mkb:
36
  https://files.catbox.moe/p7qqaz.jpg
 
 
 
 
37
  pyh to rwy:
38
  https://files.catbox.moe/ws5643.jpg
39
  rxb to tir:
40
  https://files.catbox.moe/yhg6tr.jpg
41
-
42
- Less consistent:
 
 
 
 
 
 
 
 
 
 
43
  aav to frw:
44
  https://files.catbox.moe/1z4efd.jpg
45
  fsp to klm:
@@ -48,32 +66,19 @@ kmq to ojn:
48
  https://files.catbox.moe/6qpxyi.jpg
49
  oka to rrg:
50
  https://files.catbox.moe/hvy7re.jpg
 
 
 
 
51
 
52
- There's an effort underway to identify the triggers.
53
-
54
- ###Hashed Styles
55
- arb - superbusty
56
- bnp - konpeto?
57
- aqe - melon22
58
- dhg - nyantcha?
59
- fqx - camonome?
60
- grt - dikko?
61
- mjm
62
- wfk
63
- lbp
64
- zkm
65
- qse
66
- ert
67
- hrt
68
- rer
69
- rty
70
- zzg - jvteemo
71
 
 
72
 
73
- ###Hashed Characters
74
- aua - houshou_marine
75
-
76
 
 
 
 
77
 
78
  ##List of Ponyxl LoRAs from /h/
79
  NEW NOTE 1/24/2024: I've noticed that some LoRA bakers are using different negatives when testing their LoRAs, which can lead to discrepancies when using them. I've mostly noticed that some LoRAs have a significantly reduced effect if you don't have source_pony or some combination of the source_* tags other than source anime in the negatives. This is unfortunate but a byproduct of not having a standardized process for creating the LoRAs. There's another section down below that has more info about similar issues with PonyXL that's worth reading.
@@ -127,7 +132,7 @@ https://files.catbox.moe/nvc1m8.safetensors ndc
127
  https://files.catbox.moe/kbaz4k.safetensors trogfor
128
  https://files.catbox.moe/4b97sc.safetensors nephlite
129
  https://files.catbox.moe/swjqw4.safetensors yd
130
- https://files.catbox.moe/qyreix.safetensors rustle
131
  https://files.catbox.moe/ojbvr4.safetensors afrobull
132
  https://files.catbox.moe/rqqk5m.safetensors akihiko_yoshida
133
  https://files.catbox.moe/cuzgnf.safetensors aoi_nagia just using stuff from Elf Mori
@@ -274,10 +279,10 @@ https://files.catbox.moe/6mq6hn.safetensors wamudraws
274
  https://files.catbox.moe/dkjzm3.safetensors gattles
275
  https://files.catbox.moe/6nnfzi.safetensors norza
276
  https://files.catbox.moe/4z5ngk.safetensors ter
277
- https://litter.catbox.moe/cburag.safetensors juno really only trained on paizuri since that's pretty much all the artist draws, has activation trigger junopai, 1girl, solo focus, 1boy, penis, paizuri, large breasts will get the typical image layout
278
  https://files.catbox.moe/5oa88d.safetensors shadman
279
  https://files.catbox.moe/tccjiu.safetensors kylin
280
  https://files.catbox.moe/7sgo6g.safetensors uno_makoto exmaple gen: https://files.catbox.moe/yt0ip7.png
 
281
 
282
  **These are from the megas below, unmodified, if you want good example gens you can check in their respective megas.**
283
  https://files.catbox.moe/9qqq1s.safetensors suga_hideo
@@ -329,6 +334,9 @@ https://rentry.org/shitty_loras#styles-pony-diffusion-v6-xl made_in_abyss, moshi
329
  https://mega.nz/folder/1yRSXQCR#Ziu9Q9Is5BCTYn53tnionw akatsuki_giken, alkemanubism, annin, aoin, chabo_fuketsudan, collagen, go_robots, ishida_umi, kafun, kaomin, kenya_suzuki, kink_(tortoiseshell), lucient, m_k, marui, miche, nezumi, nnf, pantsu_ruipper, RA, shizuoka, simzarts, testsuwan_woopie
330
  https://mega.nz/folder/NegX3DDL#NX3YguN8GSrEbOXuN918BA inhoso
331
  https://mega.nz/folder/fdoG2QDQ#6ZS_K55fY3g4NottW2NJng thanuki
 
 
 
332
  ###Characters
333
 
334
  https://files.catbox.moe/1wdl00.safetensors character_momoka helps to prompt features like hair, clothes
@@ -360,9 +368,11 @@ https://files.catbox.moe/ydfkfj.safetensors character_seele_starchasm_nyx activa
360
  https://files.catbox.moe/0escz4.safetensors character_sindel activation trigger is sindel, helps to have multicolored_hair in the prompt to get the black streak in her hair to show up
361
  https://files.catbox.moe/pagyj3.safetensors character_agnes_tachyon activation trigger is agnes_tachyon_\(umamusume\)
362
  https://files.catbox.moe/ou15oe.safetensors character_rishiah_ivyred activation trigger is rishia ivyred, seemd like you need to prompt her features as well like green hair, green eyes, twin braids, very long hair, hair bow. example gen: https://files.catbox.moe/5t64vg.png
 
363
 
364
  **Mega uploads with files too large for catbox**
365
  https://mega.nz/folder/uuI2DCjA#JKGPF0uYT-1Zo0dE3wCPJQ sorceress_sellen
 
366
 
367
  ##Innate artists/styles
368
  List of tags innate to ponyxl that change the style significantly, usually artists but sometimes other strange behavior some of these could just be getting activated by character names in the artist name or just misfires in the AI network ¯\_(ツ)_/¯
@@ -703,6 +713,9 @@ koikatsu
703
  [![no scores](https://files.catbox.moe/ymxn5j.png)](https://files.catbox.moe/ssmhux.png)[![using 3d in neg](https://files.catbox.moe/bp3ebs.png)](https://files.catbox.moe/g17f51.png)[![using score_9](https://files.catbox.moe/lui5ad.png)](https://files.catbox.moe/nvy14k.png)[![using all score tags](https://files.catbox.moe/ctx00a.png)](https://files.catbox.moe/ro2z3l.png)
704
  mamimi
705
 
 
 
 
706
  **strong and doesn't look anything like the artist imo but still possibly useful through varying strength and combining with other artists**
707
  [![no scores](https://files.catbox.moe/usoc6a.png)](https://files.catbox.moe/h6q88a.png)[![using 3d in neg](https://files.catbox.moe/svb2wy.png)](https://files.catbox.moe/izu8y9.png)[![using score_9](https://files.catbox.moe/29h794.png)](https://files.catbox.moe/jt3val.png)[![using all score tags](https://files.catbox.moe/kqnti6.png)](https://files.catbox.moe/rm24aw.png)
708
  geo_(yukishitadou)
@@ -804,16 +817,9 @@ mana_kakkowarai
804
  last tried was yanyan_\(shinken_gomi\)
805
 
806
  ##Innate Characters
807
- An anon went through the trouble of compiling a list of characters known to the model, these are not comprehensive it seems to know hundreds if not thousands of characters. Any popular female character from a popular franchise has a good chance of being known to the model, movies, video games, and tv shows are a good source.
808
-
809
- **I've gotten some complaints that these lists aren't accurate, I'm keeping them around for posterity but don't count on a character showing up just because it's in these lists.**
810
 
811
- Females: https://files.catbox.moe/oklpz8.txt
812
- Males: https://files.catbox.moe/1lptzn.txt
813
- Female Pokemon: https://files.catbox.moe/sljtm8.txt
814
- Male Pokemon: https://files.catbox.moe/ka331j.txt
815
- Western Characters: https://files.catbox.moe/15bayo.txt
816
- More Western Characters: https://files.catbox.moe/bhjnpb.txt
817
 
818
  ##Tricks with the model for more anime style
819
  There's some evidence that the model has a bit of a bias towards western style art, particularly when using the score tags (it is made by furries after all). It might be beneficial to tag your image sets with the score tags, the laziest way would be to tag every image with score_9, source_anime, this may affect the overall "quality" of images the lora generates since some of the knowledge in the model about "high quality" might be overwritten but it will look more like the artist's style.
@@ -860,7 +866,6 @@ https://files.catbox.moe/m861nn.json
860
  ####My Current Settings
861
  My current settings are pretty similar to the prodigy one above, a couple times I've had a hard time getting prodigy to get that last bit of distance to really recreate the artist and have used AdamW in those cases, which takes a bit more tweaking with the learning rates. Usually I'll resize the resulting LoRA down to 16ish dims (for a style) and make sure it still looks good.
862
 
863
- --network_module = networks.lora
864
  --train_data_dir = "$IMAGES_PATH$"
865
  --output_dir = "$OUTPUT_FOLDER$"
866
  --output_name = "$LORA_NAME$"
 
1
  #PonyXL Notes
2
+
3
+ [TOC2]
4
+
5
  ##Get PonyXL here
6
  https://civitai.com/models/257749?modelVersionId=290640
7
 
 
20
  https://civitai.com/models/312010/4th-tail-lora-extract
21
 
22
  ##Reverse Engineered Hashed Tokens
23
+ It seems like the obfuscated/hashed tokens in the model can often times be exposed through three letter strings (probably substrings of the full hash). There's only about 17k possible 3 letter combinations so it's possible to just reverse engineer the remapped tokens through brute force, an amazing anon went through the trouble of genning every 3 letter combination and noting which combinations seemed to affect the style. Here's his latest update:
24
 
25
+ All tokens I found: https://files.catbox.moe/41sbn0.txt
26
+ Zip with all images (small) and tokens: https://files.catbox.moe/tqb4go.zip
27
 
28
+ Compilations:
29
  aaa to bzm:
30
  https://files.catbox.moe/c0rl1r.jpg
31
  cad to eum:
 
38
  https://files.catbox.moe/tj2aeq.jpg
39
  lgu to mkb:
40
  https://files.catbox.moe/p7qqaz.jpg
41
+ mkg to nyj:
42
+ https://files.catbox.moe/365n8h.jpg
43
+ and nyp to pyb:
44
+ https://files.catbox.moe/64zi6v.jpg
45
  pyh to rwy:
46
  https://files.catbox.moe/ws5643.jpg
47
  rxb to tir:
48
  https://files.catbox.moe/yhg6tr.jpg
49
+ tjt to ulx:
50
+ https://files.catbox.moe/5pipmv.jpg
51
+ ulz to viv:
52
+ https://files.catbox.moe/pj5qsd.jpg
53
+ vix to wgv:
54
+ https://files.catbox.moe/niiwxt.jpg
55
+ wha to xph:
56
+ https://files.catbox.moe/w00hdt.jpg
57
+ xpk to zzj
58
+ https://files.catbox.moe/pjt03o.jpg
59
+
60
+ Some more repetitive ones:
61
  aav to frw:
62
  https://files.catbox.moe/1z4efd.jpg
63
  fsp to klm:
 
66
  https://files.catbox.moe/6qpxyi.jpg
67
  oka to rrg:
68
  https://files.catbox.moe/hvy7re.jpg
69
+ rzj to xlv:
70
+ https://files.catbox.moe/jrk1xl.jpg
71
+ xmj to zzr:
72
+ https://files.catbox.moe/882h83.jpg
73
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
 
75
+ There's an effort underway to identify the triggers.
76
 
77
+ ###Hashed Styles/Characters
 
 
78
 
79
+ **Update 3/21/2024**
80
+ Here's a csv of the latest version of a collaborative spreadsheet with some identified styles.
81
+ https://files.catbox.moe/j7g22l.csv
82
 
83
  ##List of Ponyxl LoRAs from /h/
84
  NEW NOTE 1/24/2024: I've noticed that some LoRA bakers are using different negatives when testing their LoRAs, which can lead to discrepancies when using them. I've mostly noticed that some LoRAs have a significantly reduced effect if you don't have source_pony or some combination of the source_* tags other than source anime in the negatives. This is unfortunate but a byproduct of not having a standardized process for creating the LoRAs. There's another section down below that has more info about similar issues with PonyXL that's worth reading.
 
132
  https://files.catbox.moe/kbaz4k.safetensors trogfor
133
  https://files.catbox.moe/4b97sc.safetensors nephlite
134
  https://files.catbox.moe/swjqw4.safetensors yd
135
+ https://files.catbox.moe/ecebk5.safetensors rustle
136
  https://files.catbox.moe/ojbvr4.safetensors afrobull
137
  https://files.catbox.moe/rqqk5m.safetensors akihiko_yoshida
138
  https://files.catbox.moe/cuzgnf.safetensors aoi_nagia just using stuff from Elf Mori
 
279
  https://files.catbox.moe/dkjzm3.safetensors gattles
280
  https://files.catbox.moe/6nnfzi.safetensors norza
281
  https://files.catbox.moe/4z5ngk.safetensors ter
 
282
  https://files.catbox.moe/5oa88d.safetensors shadman
283
  https://files.catbox.moe/tccjiu.safetensors kylin
284
  https://files.catbox.moe/7sgo6g.safetensors uno_makoto exmaple gen: https://files.catbox.moe/yt0ip7.png
285
+ https://files.catbox.moe/vlmuy2.safetensors pixzapix
286
 
287
  **These are from the megas below, unmodified, if you want good example gens you can check in their respective megas.**
288
  https://files.catbox.moe/9qqq1s.safetensors suga_hideo
 
334
  https://mega.nz/folder/1yRSXQCR#Ziu9Q9Is5BCTYn53tnionw akatsuki_giken, alkemanubism, annin, aoin, chabo_fuketsudan, collagen, go_robots, ishida_umi, kafun, kaomin, kenya_suzuki, kink_(tortoiseshell), lucient, m_k, marui, miche, nezumi, nnf, pantsu_ruipper, RA, shizuoka, simzarts, testsuwan_woopie
335
  https://mega.nz/folder/NegX3DDL#NX3YguN8GSrEbOXuN918BA inhoso
336
  https://mega.nz/folder/fdoG2QDQ#6ZS_K55fY3g4NottW2NJng thanuki
337
+ https://mega.nz/folder/L7Q3XaQC#jg7_M_gCun-aQenQmLISIQ banpai_akira, cluseller, tangobat, toi8, bonnie, everyntrge, gbf splashes (I assume some sort of gran blue style?), kaptivate
338
+ https://mega.nz/folder/TZElCCIL#OEutCsJPD0G5lXPXLGgfcQ/folder/qMUFgTrR 2b213, Aestheticc-meme, Avotoma3, Blurent, Colodraws, Dustbinrat, Healthyman, Ibuo, Inkterton-kun, Jlullaby, Kairunoburogu, Kevbot, Korotsuke, Lakilolom03, Leftlewdz, Lone, Mantis-x, Marui Maru, Mctamagomaki, Murata Renge, Namespace, Possum Machine, Randomboobguy, Roly, Sandcavern, Sano-br, Sarikyou, Sweetdynamite, Takato Kurosuke, Vammzu, Wamudraws, Woofycakes, Yume Yoroi, Akai Sashimi, Fluffydango, Jujunaught, Jun (Rojiura Jack), Kanabun, Kanden-ki, Mystical, Spring2013, Bunybunyboi, Captainjerkpants, Citrusdoodles, Cloudxmoe, Coffeelove68, Doppel, Eigaka, Forastero, Gats, Gloss, Grayimpact, Greatm8, Kabos, Kanikama, Krekkov, Lk149, Loliconder, Marthedog, Naga U, Neo Hajime, Nuclearwasabi, Oekakizuki, Peculiart, Rizdraws, Ryusei Hashida, Sokonasu, Suzuki Kenya, Tod0231, Tomu (Tomubobu), Torathi, Zako
339
+
340
  ###Characters
341
 
342
  https://files.catbox.moe/1wdl00.safetensors character_momoka helps to prompt features like hair, clothes
 
368
  https://files.catbox.moe/0escz4.safetensors character_sindel activation trigger is sindel, helps to have multicolored_hair in the prompt to get the black streak in her hair to show up
369
  https://files.catbox.moe/pagyj3.safetensors character_agnes_tachyon activation trigger is agnes_tachyon_\(umamusume\)
370
  https://files.catbox.moe/ou15oe.safetensors character_rishiah_ivyred activation trigger is rishia ivyred, seemd like you need to prompt her features as well like green hair, green eyes, twin braids, very long hair, hair bow. example gen: https://files.catbox.moe/5t64vg.png
371
+ https://files.catbox.moe/62g6g1.safetensors character_uni seems like un1 is the activation trigger, example gens: https://files.catbox.moe/3ijdtz.png, https://files.catbox.moe/rgndp3.png, https://files.catbox.moe/t306wb.png
372
 
373
  **Mega uploads with files too large for catbox**
374
  https://mega.nz/folder/uuI2DCjA#JKGPF0uYT-1Zo0dE3wCPJQ sorceress_sellen
375
+ https://mega.nz/folder/L7Q3XaQC#jg7_M_gCun-aQenQmLISIQ characters from nikke, hsr, yugioh, azure lane, blue archive, gran blue
376
 
377
  ##Innate artists/styles
378
  List of tags innate to ponyxl that change the style significantly, usually artists but sometimes other strange behavior some of these could just be getting activated by character names in the artist name or just misfires in the AI network ¯\_(ツ)_/¯
 
713
  [![no scores](https://files.catbox.moe/ymxn5j.png)](https://files.catbox.moe/ssmhux.png)[![using 3d in neg](https://files.catbox.moe/bp3ebs.png)](https://files.catbox.moe/g17f51.png)[![using score_9](https://files.catbox.moe/lui5ad.png)](https://files.catbox.moe/nvy14k.png)[![using all score tags](https://files.catbox.moe/ctx00a.png)](https://files.catbox.moe/ro2z3l.png)
714
  mamimi
715
 
716
+ [![1girl, cowboy shot, wide hips, hand on hip, contrapposto, one-piece-swimsuit, beach, sky, cloud NEG: monochrome, greyscale](https://files.catbox.moe/xlr9r6.png)](https://files.catbox.moe/aoeusb.png)[![chun-li, upper body, looking at viewer, crossed_arms, annoyed, city, sky NEG: 3d, monochrome, greyscale](https://files.catbox.moe/qevsx5.png)](https://files.catbox.moe/vd6lg0.png)[![source_anime, score_9, from behind, cowboy_shot, 2b \(nier:automata\), ass_focus, leotard, hands on hips, looking back NEG: monochrome, greyscale](https://files.catbox.moe/c3rsm6.png)](https://files.catbox.moe/sh86y8.png)[![source_anime, score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, tatsumaki, sitting, on_bed, leg_up, leaning back, arm_support, smug NEG: source_pony, monochrome, greyscale, topless](https://files.catbox.moe/54ojqr.png)](https://files.catbox.moe/3ggenf.png)
717
+ mdf_an
718
+
719
  **strong and doesn't look anything like the artist imo but still possibly useful through varying strength and combining with other artists**
720
  [![no scores](https://files.catbox.moe/usoc6a.png)](https://files.catbox.moe/h6q88a.png)[![using 3d in neg](https://files.catbox.moe/svb2wy.png)](https://files.catbox.moe/izu8y9.png)[![using score_9](https://files.catbox.moe/29h794.png)](https://files.catbox.moe/jt3val.png)[![using all score tags](https://files.catbox.moe/kqnti6.png)](https://files.catbox.moe/rm24aw.png)
721
  geo_(yukishitadou)
 
817
  last tried was yanyan_\(shinken_gomi\)
818
 
819
  ##Innate Characters
820
+ I spent some time generating images of the characters on danbooru in descending order by post count, here's about 500 of the ones I saw that pony recognized.
 
 
821
 
822
+ https://files.catbox.moe/r29xho.txt
 
 
 
 
 
823
 
824
  ##Tricks with the model for more anime style
825
  There's some evidence that the model has a bit of a bias towards western style art, particularly when using the score tags (it is made by furries after all). It might be beneficial to tag your image sets with the score tags, the laziest way would be to tag every image with score_9, source_anime, this may affect the overall "quality" of images the lora generates since some of the knowledge in the model about "high quality" might be overwritten but it will look more like the artist's style.
 
866
  ####My Current Settings
867
  My current settings are pretty similar to the prodigy one above, a couple times I've had a hard time getting prodigy to get that last bit of distance to really recreate the artist and have used AdamW in those cases, which takes a bit more tweaking with the learning rates. Usually I'll resize the resulting LoRA down to 16ish dims (for a style) and make sure it still looks good.
868
 
 
869
  --train_data_dir = "$IMAGES_PATH$"
870
  --output_dir = "$OUTPUT_FOLDER$"
871
  --output_name = "$LORA_NAME$"