|
--- |
|
license: openrail++ |
|
base_model: diffusers/stable-diffusion-xl-base-1.0 |
|
tags: |
|
- stable-diffusion-xl |
|
- stable-diffusion-xl-diffusers |
|
- stable-diffusion |
|
- text-to-image |
|
- lora |
|
- loraxl |
|
- di.FFusion.ai |
|
language: |
|
- en |
|
thumbnail: >- |
|
https://cdn-uploads.huggingface.co/production/uploads/6380cf05f496d57325c12194/p54u7dEP1u8en0--NMEjS.png |
|
library_name: diffusers |
|
--- |
|
|
|
![ffusion1.jpg](https://cdn-uploads.huggingface.co/production/uploads/6380cf05f496d57325c12194/yS5z9x4AOxLilW17ZZaiV.jpeg) |
|
|
|
# FFusion LoRA Extracted Models for Hugging Face π |
|
|
|
Welcome to the FFusion LoRA extracted models repository on Hugging Face! Here, we present a collection of models extracted using the Low-Rank Adaptation (LoRA) technique to provide a rich dataset for research and further exploration. |
|
|
|
## π¦ Base Models |
|
|
|
The following models served as the foundation for our extractions: |
|
|
|
- [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) |
|
- [stabilityai/stable-diffusion-xl-base-0.9](https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9) |
|
- [FFusion/FFusionXL-BASE](https://huggingface.co/FFusion/FFusionXL-BASE) |
|
|
|
## π Extraction Details |
|
|
|
- **Variants:** Each base model was extracted into 4-5 distinct variants. |
|
- **Extraction Depth:** The models uploaded here contain approximately 70% of extracted data. These extractions yield a dataset size of around 400 GB. |
|
- **Precision:** We experimented with both `float32` and `float64` for optimal extraction results. |
|
- **Differences Measurement:** Singular Value Decomposition (SVD) was utilized to measure differences between the original and the tuned models. A threshold of \(1 \times 10^{-3}\) was commonly used, although in some cases, \(1 \times 10^{-5}\) and \(1 \times 10^{-2}\) were tested. |
|
- **Demonstration Parameters:** For our demonstration, we employed `"conv_dim": 256` and `"conv_alpha": 256`. |
|
|
|
## βοΈ Technical Notes |
|
|
|
- Most SDXL models in this collection are not traditionally "trained." Instead, they are merged from previous SDXL 0.9 versions or created using other methods with the help of Comfy UI. |
|
- An important note for users: all models saved with Comfy add an extra key `text_model.encoder.text_model.embeddings.position_ids`. We've made necessary adjustments to ensure compatibility with the current scripts from Kohoya. |
|
|
|
## π Use Cases |
|
|
|
These extracted models are intended for research and testing. They can be particularly useful for: |
|
|
|
- Investigating the potential of merging multiple LoRAs. |
|
- Weighting experiments with 1-5 LoRAs simultaneously. |
|
- Exploring the differences and similarities between LoRAs extracted from different base models. |
|
|
|
## π Acknowledgements & Citations |
|
|
|
A huge shoutout to the community for their continued support and feedback. Together, we are pushing the boundaries of what's possible with machine learning! |
|
|
|
We would also like to acknowledge and give credit to the following projects and authors: |
|
|
|
- **ComfyUI**: We've used and modified portions of [ComfyUI](https://github.com/comfyanonymous/ComfyUI) for our work. |
|
- **kohya-ss/sd-scripts and bmaltais**: Our work also incorporates modifications from [kohya-ss/sd-scripts](https://github.com/kohya-ss/sd-scripts). |
|
- **lora-inspector**: We've benefited from the [lora-inspector](https://github.com/rockerBOO/lora-inspector) project. |
|
- **KohakuBlueleaf**: Special mention to KohakuBlueleaf for their invaluable contributions. |
|
|
|
|
|
# FFusion LoRA Extracted Models - How to Use Guide π§ |
|
|
|
Welcome to the technical guide for using the FFusion LoRA extracted models. This document will walk you through the steps required to fuse LoRA parameters, load checkpoints, and perform inference. |
|
|
|
## Fusing LoRA Parameters π |
|
|
|
To merge the LoRA parameters with the original parameters of the underlying model(s), leading to a potential speedup in inference latency: |
|
|
|
```python |
|
pipe.fuse_lora() |
|
``` |
|
|
|
### Unfusing LoRA Parameters βοΈ |
|
|
|
To revert the effects of `fuse_lora()`: |
|
|
|
```python |
|
pipe.unfuse_lora() |
|
``` |
|
|
|
### Working with Different LoRA Scales ποΈ |
|
|
|
To control the influence of the LoRA parameters on the outputs: |
|
|
|
```python |
|
pipe.fuse_lora(lora_scale=0.5) |
|
``` |
|
|
|
## Working with FFusion Models π |
|
|
|
Here's how to load and utilize our FFusion models: |
|
|
|
```python |
|
from diffusers import DiffusionPipeline |
|
import torch |
|
|
|
pipeline_id = "FFusion/FFusionXL-BASE" |
|
|
|
pipe = DiffusionPipeline.from_pretrained(pipeline_id, torch_dtype=torch.float16) |
|
pipe.enable_model_cpu_offload() |
|
|
|
lora_model_id = "FFusion/400GB-LoraX" |
|
lora_filename = "FFai.0038.Realitycheckxl_Alpha11.lora.safetensors" |
|
pipe.load_lora_weights(lora_model_id, weight_name=lora_filename) |
|
|
|
prompt = "papercut sonic" |
|
image = pipe(prompt=prompt, num_inference_steps=20, generator=torch.manual_seed(0)).images[0] |
|
image |
|
``` |
|
## Running Inference πΌοΈ |
|
|
|
After loading the desired model, you can perform inference as follows: |
|
|
|
```python |
|
generator = torch.manual_seed(0) |
|
images_fusion = pipe( |
|
"masterpiece, best quality, mountain", output_type="np", generator=generator, num_inference_steps=2 |
|
).images |
|
``` |
|
|
|
|
|
## Library of Available LoRA Models π |
|
|
|
You can choose any of the models from our repository on Hugging Face or the upcoming repository on CivitAI. Here's a list of available models with `lora_model_id = "FFusion/400GB-LoraXL"`: |
|
|
|
``` |
|
lora_filename = |
|
- FFai.0001.4Guofeng4xl_V1125d.lora_Dim64.safetensors |
|
- FFai.0002.4Guofeng4xl_V1125d.lora_Dim8.safetensors |
|
- FFai.0003.4Guofeng4xl_V1125d.loraa.safetensors |
|
- FFai.0004.Ambiencesdxl_A1.lora.safetensors |
|
- FFai.0005.Ambiencesdxl_A1.lora_8.safetensors |
|
- FFai.0006.Angrasdxl10_V22.lora.safetensors |
|
- FFai.0007.Animaginexl_V10.lora.safetensors |
|
- FFai.0008.Animeartdiffusionxl_Alpha3.lora.safetensors |
|
- FFai.0009.Astreapixiexlanime_V16.lora.safetensors |
|
- FFai.0010.Bluepencilxl_V010.lora.safetensors |
|
- FFai.0011.Bluepencilxl_V021.lora.safetensors |
|
- FFai.0012.Breakdomainxl_V03d.lora.safetensors |
|
- FFai.0013.Canvasxl_Bfloat16v002.lora.safetensors |
|
- FFai.0014.Cherrypickerxl_V20.lora.safetensors |
|
- FFai.0015.Copaxtimelessxlsdxl1_V44.lora.safetensors |
|
- FFai.0016.Counterfeitxl-Ffusionai-Alpha-Vae.lora.safetensors |
|
- FFai.0017.Counterfeitxl_V10.lora.safetensors |
|
- FFai.0018.Crystalclearxl_Ccxl.lora.safetensors |
|
- FFai.0019.Deepbluexl_V006.lora.safetensors |
|
- FFai.0020.Dream-Ffusion-Shaper.lora.safetensors |
|
- FFai.0021.Dreamshaperxl10_Alpha2xl10.lora.safetensors |
|
- FFai.0022.Duchaitenaiartsdxl_V10.lora.safetensors |
|
- FFai.0023.Dynavisionxlallinonestylized_Beta0371bakedvae.lora.safetensors |
|
- FFai.0024.Dynavisionxlallinonestylized_Beta0411bakedvae.lora.safetensors |
|
- FFai.0025.Fantasticcharacters_V55.lora.safetensors |
|
- FFai.0026.Fenrisxl_V55.lora.safetensors |
|
- FFai.0027.Fudukimix_V10.lora.safetensors |
|
- FFai.0028.Infinianimexl_V16.lora.safetensors |
|
- FFai.0029.Juggernautxl_Version1.lora_1.safetensors |
|
- FFai.0030.Lahmysterioussdxl_V330.lora.safetensors |
|
- FFai.0031.Mbbxlultimate_V10rc.lora.safetensors |
|
- FFai.0032.Miamodelsfwnsfwsdxl_V30.lora.safetensors |
|
- FFai.0033.Morphxl_V10.lora.safetensors |
|
- FFai.0034.Nightvisionxlphotorealisticportrait_Beta0681bakedvae.lora_1.safetensors |
|
- FFai.0035.Osorubeshialphaxl_Z.lora.safetensors |
|
- FFai.0036.Physiogenxl_V04.lora.safetensors |
|
- FFai.0037.Protovisionxlhighfidelity3d_Beta0520bakedvae.lora.safetensors |
|
- FFai.0038.Realitycheckxl_Alpha11.lora.safetensors |
|
- FFai.0039.Realmixxl_V10.lora.safetensors |
|
- FFai.0040.Reproductionsdxl_V31.lora.safetensors |
|
- FFai.0041.Rundiffusionxl_Beta.lora.safetensors |
|
- FFai.0042.Samaritan3dcartoon_V40sdxl.lora.safetensors |
|
- FFai.0043.Sdvn6realxl_Detailface.lora.safetensors |
|
- FFai.0044.Sdvn7realartxl_Beta2.lora.safetensors |
|
- FFai.0045.Sdxl10arienmixxlasian_V10.lora.safetensors |
|
- FFai.0046.Sdxlbasensfwfaces_Sdxlnsfwfaces03.lora.safetensors |
|
- FFai.0047.Sdxlfaetastic_V10.lora.safetensors |
|
- FFai.0048.Sdxlfixedvaefp16remove_Basefxiedvaev2fp16.lora.safetensors |
|
- FFai.0049.Sdxlnijiv4_Sdxlnijiv4.lora.safetensors |
|
- FFai.0050.Sdxlronghua_V11.lora.safetensors |
|
- FFai.0051.Sdxlunstablediffusers_V5unchainedslayer.lora.safetensors |
|
- FFai.0052.Sdxlyamersanimeultra_Yamersanimev2.lora.safetensors |
|
- FFai.0053.Shikianimexl_V10.lora.safetensors |
|
- FFai.0054.Spectrumblendx_V10.lora.safetensors |
|
- FFai.0055.Stablediffusionxl_V30.lora.safetensors |
|
- FFai.0056.Talmendoxlsdxl_V11beta.lora.safetensors |
|
- FFai.0057.Wizard_V10.lora.safetensors |
|
- FFai.0058.Wyvernmix15xl_Xlv11.lora.safetensors |
|
- FFai.0059.Xl13asmodeussfwnsfw_V17bakedvae.lora.safetensors |
|
- FFai.0060.Xl3experimentalsd10xl_V10.lora.safetensors |
|
- FFai.0061.Xl6hephaistossd10xlsfw_V21bakedvaefp16fix.lora.safetensors |
|
- FFai.0062.Xlperfectdesign_V2ultimateartwork.lora.safetensors |
|
- FFai.0063.Xlyamersrealistic_V3.lora.safetensors |
|
- FFai.0064.Xxmix9realisticsdxl_Testv20.lora.safetensors |
|
- FFai.0065.Zavychromaxl_B2.lora.safetensors |
|
|
|
``` |
|
|
|
π Text Encoder Difference Overview |
|
Based on the extraction process, we observed the following differences in the text encoder across various models: |
|
|
|
## 4Guofeng4XL_v1125D |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## AmbienceSDXL_a1 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## angraSDXL10_v22 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## animagineXL_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## animeArtDiffusionXL_alpha3 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## AstreaPixieXLAnime_v16 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## bluePencilXL_v010 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## bluePencilXL_v021 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## breakdomainXL_v03d |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## canvasXL_bFloat16v002 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## cherryPickerXL_v20 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## copaXTimelessXLSDXL1_v44 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## counterfeitXL-FFusionAI-alpha-VAE |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## counterfeitXL_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## crystalclearXL_ccxl |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## deepblueXL_v006 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## dream-FFusion-shaper |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## dreamShaperXL10_alpha2xl10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## duchaitenaiartsSDXL_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## dynavisionXLallInOneStylized_beta0371bakedVAE |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## dynavisionXLallInOneStylized_beta0411bakedVAE |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## fantasticCharacters_v55 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## fenrisXL_v55 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## fudukimix_v10 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## infiniaNimeXL_v16 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## juggernautXL_version1 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## lahMysteriousSDXL_v330 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## mbBXLUltimate_v10rc |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## miaModelsFWNsfwSDXL_v30 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## morphXL_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## nightvisionXLphotorealisticPortrait_beta0681bakedVAE |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## osorubeshiAlphaXL_z |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## physiogenXL_v04 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## protoVisionXLhighFidelity3D_beta0520bakedVAE |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## realityCheckXL_alpha11 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## realmixXL_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## reproductionSDXL_v31 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## runDiffusionXL_beta |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## samaritan3DCartoon_v40sdxl |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## sdvn6realXL_detailFace |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## sdvn7realArtXL_beta2 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## sdxl10arienMixXLasian_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## sdxlBaseNSFWFaces_sdxlNSFWFaces03 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## sdxlFaeTastic_v10 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## sdxlFixedVAEfp16Remove_baseFxiedVAEv2fp16 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## sdxlNijiV4_sdxlNijiV4 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## sdxlRonghua_v11 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## sdxlUnstableDiffusers_v5UnchainedSlayer |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## sdxlYamersAnimeUltra_yamersAnimeV2 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## shikiAnimeXL_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## spectrumBlendX_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## stableDiffusionXL_v30 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## talMendoXlsdXL_v11beta |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## wizard_v10 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## wyvernMix15XL_XLv11 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## xl13asmodeusSFWNSFW_v17bakedVAEfp16fix |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## xl3experimentalSD10XL_v10 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## xl6hephaistosSD10XLsfw_v21bakedVAEfp16fix |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## xlPerfectDesign_v2UltimateArtwork |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## xlYamersRealistic_v3 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|
|
## xxMix9RealisticSDXL_Testv20 |
|
β
Text encoder available. Difference by 0.001953125 |
|
|
|
## zavyChromaXL_b2 |
|
π΄ Text encoder unavailable. Same as SDXL 1.0 Base |
|
|