--- language: - en tags: - stable-diffusion - text-to-image license: creativeml-openrail-m inference: false --- # ReFT: Fine Tuned models by AI Generated Images ReFT is a series of fine-tuned models from SD 1.5 for producing illustration style images with high resolution or different aspect ratio images. ## Model Description | Model Name | Base Model | Typical Resolusion | lr | | ----------------- | ---------------- | ------------------ | ---- | | ReFT ReWaifu 512p | SD1.5 | 512x512 | 4e-6 | | ReFT ReWaifu 768p | ReWaifu_512p_e12 | 768x768 | 4e-6 | | ReFT ReWaifu 1K | ReWaifu_768p_e5 | 1024x1024 | 2e-6 | | ReFT Rellust 512p | SD1.5 | 512x512 | 4e-6 | | ReFT Rellust 768p | Rellust_512p_e10 | 768x768 | 4e-6 | | ReFT Rellust 1K | Rellust_768p_e3 | 1024x1024 | 4e-6 | **CLIP Skip 1 is recommended for all models!!** ### ReFT ReWaifu Each model is fine-tuned from SD1.5 with AI-Illustration tagged images of various authors published on the web and [NijiJourney-Prompt-Pairs]([/NijiJourney-Prompt-Pairs](https://huggingface.co/datasets/Korakoe/NijiJourney-Prompt-Pairs)) using Aspect Ratio Bucketing based on either 512p, 768p, or 1K resolution. For 512p, a 370k image data set was used, and for 768p and 1K a subset of 20k images selected by [ImageReward](https://huggingface.co/THUDM/ImageReward) score with BLIP was used. Image captions are made by BLIP and WD1.4-tagger. The batch size is 16 for 512p only and 8 for the rest. Fine tuning is also performed at fp16, using mutires noise. ### ReFT Rellust Each model is fine-tuned from SD1.5 with ~16k nijijourneyv5 tagged images of various authors published on the web with a learning rate of 4e-6 using Aspect Ratio Bucketing based on either 512p, 768p, or 1K resolution. Image captions are made by BLIP and WD1.4-tagger. The batch size is 16 for 512p only and 8 for the rest. Fine tuning is also performed at fp16, using mutires noise. ### Usage Since these models are generic illustration models, the generated images can be in a variety of styles. If the style is not to your liking, please specify a style such as "anime" or "wator colors" as appropriate. The format of the caption suggests that a short natural language sentence followed by a comma-separated tags is the most natural way to describe the prompt. Using more tags that are well-estimated by the tagger in the trained images may lead to more preferable generation. Tag semantics may be inappropriate for automatic tagging, so please emphasize appropriately. The models do not require underscores for tags in the prompts. CLIP Skip 1 is recommended. ### Examples #### ReFT ReWafu 1K - v2 (Improved quality degradation with short prompts, as well as little better detail. If multiple persons appear unintentionally or the anatomy is bad, the first (t2i) resolution should be reduced slightly.) ![](images/00434-276042060.png) ``` solo, 1girl, school uniform, white shirt, sailor collar, tie, brown hair, bob cut, scenery, outdoors, solo Negative prompt: bad anatomy, text, frame Steps: 30, Sampler: UniPC, CFG scale: 6, Seed: 276042060, Size: 1280x720, Model hash: 5b383d445c, Model: ReWaifu_v2_1K_e4, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1 ``` ![](images/00510-3916112453.png) ``` wide shot, grapes, muscat, white background, simple background, full body, solo, 1girl, small head, light-brown hair, medium hair, wavy hair, brown eyes, simple dress, casual, slim, shoes, water colors, pastel colors, light coloring Negative prompt: worst quality, low quality, lowres, blurry, text, frame, light particle, water, vivid color, black, messy hair, frills Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 3916112453, Size: 1024x1024, Model hash: 5b383d445c, Model: ReWaifu_v2_1K_e4, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1 ``` - v1 ![](images/00608-2310821.png) ``` wide shot, solo, 1girl, full body, black hair, twintails, red eyes, white blouse, Peter Pan collar, dress, pinafore dress, strawberry print skirt, waist bow, frilled dress, pink skirt, long sleeves, puffy long sleeves, food print, pink footwear, strap shoes, closed mouth, fruit, white background, strawberry, pastel Negative prompt: bad anatomy, text, frame, blurry, realistic Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 2310821, Size: 1024x1024, Model hash: 7716dbfa7b, Model: ReWaifu_1K_e5, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1 ``` ![](images/00620-4266538265.png) ``` solo, 1girl, cowboy shot, magical girl, white hair, medium hair, hair ornament, blue eyes, black dress, frilled dress, short sleeves, puffy sleeves, holding staff of ruby, starry sky, milky way Negative prompt: bad anatomy, text, frame, blurry, realistic, skirt hold Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 4266538265, Size: 960x1152, Model hash: 7716dbfa7b, Model: SDAXa3c, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: FWR_TEfixed2(539136a8cf23), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Version: v1.5.1 ``` ![](images/00616-2887952460.png) ``` wide shot, grapes, muscat, white background, simple background, full body, solo, 1girl, small head, light-brown hair, medium hair, wavy hair, brown eyes, simple dress, casual, slim, shoes, water colors, pastel colors, light coloring Negative prompt: worst quality, low quality, lowres, blurry, text, frame, light particle, water, vivid color, black, messy hair, frills Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 2887952460, Size: 1024x1024, Model hash: 7716dbfa7b, Model: ReWaifu_1K_e5, Variation seed: 1970649291, Variation seed strength: 0.1, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1 ``` - 2d <-> 3d: Use "realistic" tag. - 2d ![](images/grid-0102-1783332202.png) ``` solo, 1girl, from side, anime girl is holding a maple leaf, red hair, own hands together , medium hair, wavy hair, yellow eyes, white sweater, long sleeves, skirt, mini skirt, plaid skirt, strap shoes, out doors, maple hair ornament, scenery, depth of field, tree, maple, look into hands Negative prompt: bad anatomy, text, frame, blurry, school uniform, realistic, long neck Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 1783332202, Size: 1024x1024, Model hash: 7716dbfa7b, Model: ReWaifu_1K_e5, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: FWR_TEfixed2(539136a8cf23), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Version: v1.5.1 ``` - 3d ![](images/grid-0103-1783332202.png) ``` solo, 1girl, from side, anime girl is holding a maple leaf, red hair, own hands together , medium hair, wavy hair, yellow eyes, white sweater, long sleeves, skirt, mini skirt, plaid skirt, strap shoes, out doors, maple hair ornament, scenery, depth of field, tree, maple, look into hands, realistic Negative prompt: bad anatomy, text, frame, blurry, school uniform Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 1783332202, Size: 1024x1024, Model hash: 7716dbfa7b, Model: ReWaifu_1K_e5, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, AddNet Enabled: True, AddNet Module 1: LoRA, AddNet Model 1: FWR_TEfixed2(539136a8cf23), AddNet Weight A 1: 0.5, AddNet Weight B 1: 0.5, Version: v1.5.1 ``` #### ReFT Rellust 1K ![](images/00056-4203630207.png) ``` solo, 1girl, full body, white background, indoors, from side, short hair, bangs, long sleeves, closed mouth, white hair, jewelry, blush, standing, earrings dress, flower, shirt, blue eyes, hair ornament, skirt, bow Negative prompt: worst quality, low quality, bad anatomy blurry, text Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 4203630207, Size: 1280x720, Model hash: 7d610c9f1e, Model: Rellust_1K_e3, Denoising strength: 0.55, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1 ``` ![](images/00052-122211893.png) ``` wide shot, cosmos, white background, simple background, full body, solo, 1girl, small head, light-brown hair, medium hair, wavy hair, brown eyes, simple dress, casual, slim, strap shoes, cosmos, water colors, pastel colors, light coloring Negative prompt: worst quality, low quality, logo, blurry, text, frame, light particle, water, vivid color, black, messy hair, frills Steps: 30, Sampler: UniPC, CFG scale: 6, Seed: 122211893, Size: 1024x1024, Model hash: 7d610c9f1e, Model: Rellust_1K_e3, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1 ``` ![](images/00040-3764144276.png) ``` wide shot, grapes, white background, simple background, full body, solo, 1girl, small head, light-brown hair, medium hair, wavy hair, brown eyes, simple dress, casual, slim, shoes, muscat, water colors, pastel colors, light coloring, grapes Negative prompt: worst quality, low quality, lowres, blurry, text, frame, light particle, water, vivid color, black, messy hair, frills Steps: 30, Sampler: UniPC, CFG scale: 7, Seed: 3764144276, Size: 1024x1024, Model hash: 7d610c9f1e, Model: Rellust_1K_e3, Denoising strength: 0.6, FreeU Stages: "[{\"backbone_factor\": 1.2, \"skip_factor\": 0.9}, {\"backbone_factor\": 1.4, \"skip_factor\": 0.2}]", CFG Rescale phi: 0, Hires upscale: 1.5, Hires steps: 20, Hires upscaler: Latent, Version: v1.5.1 ``` ## License This model is open access and available to all, with a CreativeML OpenRAIL-M license further specifying rights and usage. The CreativeML OpenRAIL License specifies: 1. You can't use the model to deliberately produce nor share illegal or harmful outputs or content 2. The authors claims no rights on the outputs you generate, you are free to use them and are accountable for their use which must not go against the provisions set in the license 3. You may re-distribute the weights and use the model commercially and/or as a service. If you do, please be aware you have to include the same use restrictions as the ones in the license and share a copy of the CreativeML OpenRAIL-M to all your users (please read the license entirely and carefully) [Please read the full license here](https://huggingface.co/spaces/CompVis/stable-diffusion-license) ## Acknowledgements These Models build on the excellent works: SD1.5, developed by [CompVis](https://ommer-lab.com/) and [RUNWAY](https://runwayml.com), and [ImageReward](https://huggingface.co/THUDM/ImageReward) (Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong).