hollowstrawberry
commited on
Commit
•
66846fc
1
Parent(s):
7669666
Update README.md
Browse files
README.md
CHANGED
@@ -421,10 +421,11 @@ To train a [Lora ▲](#lora) yourself is an achievement. It's certainly doable,
|
|
421 |
You can do it on your own computer if you have at least 8 GB of VRAM. However, I will be using a Google Colab document for educational purposes.
|
422 |
|
423 |
Here are some classic resources if you want to read about the topic in depth. Rentry may be blocked by your internet provider, in which case you may use a VPN or try putting it through [Google Translate](https://translate.google.cl/?op=websites).
|
424 |
-
* [Lora
|
425 |
-
* [
|
426 |
* [Original Kohya Trainer (Dreambooth method)](https://colab.research.google.com/github/Linaqruf/kohya-trainer/blob/main/kohya-LoRA-dreambooth.ipynb)
|
427 |
* [List of trainer parameters](https://github.com/derrian-distro/LoRA_Easy_Training_Scripts#list-of-arguments)
|
|
|
428 |
|
429 |
With those way smarter resources out of the way, I'll try to produce a simple guide for you to make your very own Lora for a character, concept, or artstyle.
|
430 |
|
@@ -440,17 +441,20 @@ With those way smarter resources out of the way, I'll try to produce a simple gu
|
|
440 |
|
441 |
This is the largest part of Lora training. You will need to create a "dataset" of images to train with, along with corresponding text files containing descriptions for those images (tags in the case of anime).
|
442 |
|
443 |
-
1. Find some images online representing the character/artstyle/concept you want to convey, possibly on sites such as [gelbooru](https://gelbooru.com/). You will need at least 10 images, I'd recommend at least 20
|
|
|
|
|
444 |
|
445 |
-
1. If you only have a few images you may tag them yourself, but it may be slow and inaccurate.
|
|
|
446 |
|
447 |
-
1. Once your images and their
|
448 |
|
449 |
![Folder structure for training](images/trainfolder.png)
|
450 |
|
451 |
-
1. At this point when naming your folders you get to choose the number of repetitions for your dataset. I recommend that your amount of images multiplied by their repetitions does not exceed a total of 400. So, if you have 20 images, I'd recommend 10 to 20 repetitions. More files should mean less repetitions. Then, your inner folder should be called `
|
452 |
|
453 |
-
1. Upload the entire parent folder (the one with your project's name) into your Google Drive's `lora_training/datasets
|
454 |
|
455 |
1. **Training Parameters** <a name="trainparams"></a>[▲](#index)
|
456 |
|
@@ -478,17 +482,16 @@ With those way smarter resources out of the way, I'll try to produce a simple gu
|
|
478 |
|
479 |
* **Additional tips** <a name="trainchars"></a>[▲](#index)
|
480 |
|
481 |
-
The most important thing for characters and concepts is the tags. You
|
482 |
|
483 |
-
When training a character or concept you should place an **activation tag** at the start of every text file, and set `keep_tokens` to 1. An activation tag is how we'll invoke your Lora to work. Having done that,
|
|
|
484 |
|
485 |
-
|
486 |
|
487 |
-
Style Loras meanwhile don't really need an activation tag, as we want them to always be active. They will absorb the artstyle naturally, and will work at varying weights.
|
488 |
|
489 |
-
|
490 |
-
|
491 |
-
Another tip is having multiple folders with different number of repetitions. Give your highest quality images more repetitions, while some unique concepts but with worse style/quality can have less repetitions such as 1.
|
492 |
|
493 |
|
494 |
|
|
|
421 |
You can do it on your own computer if you have at least 8 GB of VRAM. However, I will be using a Google Colab document for educational purposes.
|
422 |
|
423 |
Here are some classic resources if you want to read about the topic in depth. Rentry may be blocked by your internet provider, in which case you may use a VPN or try putting it through [Google Translate](https://translate.google.cl/?op=websites).
|
424 |
+
* [Classic Lora training guide](https://rentry.org/lora_train)
|
425 |
+
* [Lora training science](https://rentry.org/lora-training-science)
|
426 |
* [Original Kohya Trainer (Dreambooth method)](https://colab.research.google.com/github/Linaqruf/kohya-trainer/blob/main/kohya-LoRA-dreambooth.ipynb)
|
427 |
* [List of trainer parameters](https://github.com/derrian-distro/LoRA_Easy_Training_Scripts#list-of-arguments)
|
428 |
+
* [Angry Lora training guide by ao](https://rentry.org/tohoaifaq#opinionated-lora-guide-for-colab)
|
429 |
|
430 |
With those way smarter resources out of the way, I'll try to produce a simple guide for you to make your very own Lora for a character, concept, or artstyle.
|
431 |
|
|
|
441 |
|
442 |
This is the largest part of Lora training. You will need to create a "dataset" of images to train with, along with corresponding text files containing descriptions for those images (tags in the case of anime).
|
443 |
|
444 |
+
1. Find some images online representing the character/artstyle/concept you want to convey, possibly on sites such as [gelbooru](https://gelbooru.com/). You will need at least 10 images, I'd recommend at least 20, but more is almost always better.
|
445 |
+
* Optionally, you can get hundreds of them using [Grabber](https://github.com/Bionus/imgbrd-grabber/releases).
|
446 |
+
* If you want to do a character, I recommend filtering tags like this: `1girl solo character_name score:>10 -rating:explicit` (the explicit rating may include weird images, so it's fine to exclude them)
|
447 |
|
448 |
+
1. If you only have a few images you may tag them yourself, but it may be slow and inaccurate. If you're using photographs you should describe each one in detail using simple sentences.
|
449 |
+
* Optionally, add the [Tagger extension](https://github.com/toriato/stable-diffusion-webui-wd14-tagger) to your webui, through which you can automatically analyze all your training images and generate accurate anime tags for them. Instructions are as follows: First add and enable the extension, and restart your entire webui. Then go to the new **Tagger** tab, then *Batch from directory*, and select the folder with your images. Set the output name to `[name].txt` and the threshold at or above 0.2 (this is how closely each tag must match an image to be included). Then **Interrogate** and it will start generating your text files.
|
450 |
|
451 |
+
1. Once your images and their text files are ready, put them all in a folder following this structure: A folder with your project name, containing at least 1 folder in the format `repetitions_name`, which each contain some images and their tags. Like this:
|
452 |
|
453 |
![Folder structure for training](images/trainfolder.png)
|
454 |
|
455 |
+
1. At this point when naming your folders you get to choose the number of repetitions for your dataset. I recommend that your amount of images multiplied by their repetitions does not exceed a total of 400. So, if you have 20 images, I'd recommend 10 to 20 repetitions. (More files should mean less repetitions). Then, your inner folder should be called `10_repetitions` (you can replace the word "repetitions" with a custom name).
|
456 |
|
457 |
+
1. Upload the entire parent folder (the one with your project's name) into your Google Drive's `lora_training/datasets/` folder. Now it's ready.
|
458 |
|
459 |
1. **Training Parameters** <a name="trainparams"></a>[▲](#index)
|
460 |
|
|
|
482 |
|
483 |
* **Additional tips** <a name="trainchars"></a>[▲](#index)
|
484 |
|
485 |
+
The most important thing for characters and concepts is the tags. You want a dataset with varying poses and scenarios, but if they're tagged incorrectly it's not gonna work.
|
486 |
|
487 |
+
When training a character or concept you should place an **activation tag** at the start of every text file, and set `keep_tokens` to 1 before training. An activation tag is how we'll invoke your Lora to work. Having done that, **some people reccommend** removing or "pruning" tags that are intrinsic to your character or concept, such as hair color and eye color. For example, if a character always has cat ears, you would remove tags such as `animal ears, animal ear fluff, cat ears`, etc. This way they become "absorbed" by your activation tag, and makes your Lora easier to use, but less flexible.
|
488 |
+
* You can use the [Tag Editor extension](https://github.com/toshiaki1729/stable-diffusion-webui-dataset-tag-editor) to add an activation tag. Instructions are as follows: After adding the extension and restarting your webui, go to the new **Dataset Tag Editor** tab then *Batch Edit Captions*. Write your activation tag under *Edit Tags* and turn on *Prepend additional tags*. Then apply your changes, scroll up and save your changes.
|
489 |
|
490 |
+
This "absorption" of details not provided by tags is also how Loras work at all, by representing things normally imperceptible or hard to describe like faces, accessories, brushstrokes, etc. If you desire you may also prune redundant clothing tags, such as removing "red tie" but keeping "tie". You can even define an additional activation tag for different outfits your character may wear regularly, eg. character-default, character-bikini, etc. But there's more than one way to do it. In any case, with the correct usage of tags, your character should easily be able to change clothes.
|
491 |
|
492 |
+
Style Loras meanwhile don't really need an activation tag, as we want them to always be active. They will absorb the artstyle naturally, and will work at varying weights. Tagging different elements in the training images is still important, though.
|
493 |
|
494 |
+
Another tip is having multiple folders with different numbers of repetitions. Give your highest quality images more repetitions, while some unique concepts but with worse style/quality can have less repetitions such as 1.
|
|
|
|
|
495 |
|
496 |
|
497 |
|