alea31415 commited on
Commit
ef083f6
1 Parent(s): 6efc174

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -3
README.md CHANGED
@@ -306,17 +306,21 @@ On the other hand, styles trained into LoHas would be less effective if applying
306
  ![xyz_grid-0030-20230330090242](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0030-20230330090242.jpg)
307
 
308
 
309
- **Implication for Making Cosplay Images**
310
 
311
  To summarize, if you want to have style of some model X, instead of training directly on X it would be better to train on an ancestor of X that does not contain this style.
312
  Therefore, if you want to get cosplay of characters, you can do either of the following
313
  - Train on NED and add a decent amount of photos in regularization set
314
- - Train on AnyLora and transfer to NED
315
 
316
  Lykon did show some successful results by only training with anime images on NED, but I doubt this is really optimal. Actually, he uses again a doll LoRa to reinforce the photo-realistic concept. It may be simper to just do what I suggest above.
317
 
 
318
 
319
- **Some Myths**
 
 
 
320
 
321
  Clearly, the thing that really matters is how the model is made, and not how the model looks like. A model that is versatile in style does not make it a good base model for whatever kind of training. In fact, VBP2-2 has around 300 styles trained in but LoHa trained on top of it does not transfer well to other models.
322
  Similarly, two models that produce similar style do not mean they transfer well to each other. Both MFB and Salt-Mix have strong anime screenshot style but a LoHa trained on MFB does not transfer well to Salt-Mix.
@@ -372,6 +376,22 @@ However, you may also need to worry about some bad hand in such a model
372
  ![00032-20230330225216](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/00032-20230330225216.png)
373
 
374
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
375
 
376
  #### Training Speed
377
 
 
306
  ![xyz_grid-0030-20230330090242](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0030-20230330090242.jpg)
307
 
308
 
309
+ *Implication for Making Cosplay Images*
310
 
311
  To summarize, if you want to have style of some model X, instead of training directly on X it would be better to train on an ancestor of X that does not contain this style.
312
  Therefore, if you want to get cosplay of characters, you can do either of the following
313
  - Train on NED and add a decent amount of photos in regularization set
314
+ - Train on NAI and transfer to NED
315
 
316
  Lykon did show some successful results by only training with anime images on NED, but I doubt this is really optimal. Actually, he uses again a doll LoRa to reinforce the photo-realistic concept. It may be simper to just do what I suggest above.
317
 
318
+ For illustration, here is what we get by applying the LoHas trained on different base models on NeverEndingDream. As we can see, the one trained on NAI retains the most photo-realistic style (of course you can always adjusting various weight to improve things but I still get the best result with the one trained on NAI).
319
 
320
+ ![xyz_grid-0017-20230331061354](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0017-20230331061354.jpg)
321
+
322
+
323
+ *Some Myths*
324
 
325
  Clearly, the thing that really matters is how the model is made, and not how the model looks like. A model that is versatile in style does not make it a good base model for whatever kind of training. In fact, VBP2-2 has around 300 styles trained in but LoHa trained on top of it does not transfer well to other models.
326
  Similarly, two models that produce similar style do not mean they transfer well to each other. Both MFB and Salt-Mix have strong anime screenshot style but a LoHa trained on MFB does not transfer well to Salt-Mix.
 
376
  ![00032-20230330225216](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/00032-20230330225216.png)
377
 
378
 
379
+ **A Case Study on Training on vanilla SD Model**
380
+
381
+ Someone may believe that training on vanilla SD model would give better result. I however not advice it for anime training because you can hardly make your model usable with a small dataset. In turn you get a model that does not understand booru tags and often produces bad anatomy and bad hands.
382
+
383
+ Moreover, as we see below, all the vanilla SD models are so far away from NAI that even character cannot be transfer. It would only transfer to more photo-realistic model such as ChilloutMix (as they have a more important vanilla SD component) but this kills their photo-realistic style and make something terrible so this really there is really no interests in so doing in general.
384
+
385
+ The trend we see here is also coherent as what we have observed so far
386
+ - When training on vanilla SD and applying to descendant model we get mostly the style of descendant model
387
+ - The only exception is ChilloutMix for which it becomes less photo-realistic, which makes sense because it is much closer to vanilla SD
388
+ - Training on NAI and applying to vanilla SD gives a diluted anime style or no style at all
389
+
390
+ ![xyz_grid-0016-20230331054835](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0016-20230331054835.jpg)
391
+ ![xyz_grid-0014-20230331053913](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0014-20230331053913.jpg)
392
+ ![xyz_grid-0015-20230331054327](https://huggingface.co/alea31415/LyCORIS-experiments/resolve/main/generated_samples_0329/xyz_grid-0015-20230331054327.jpg)
393
+
394
+
395
 
396
  #### Training Speed
397