Update README.md

README.md

@@ -1,3 +1,388 @@
 ---
-license: other
+license: openrail++
+base_model: diffusers/stable-diffusion-xl-base-1.0
+tags:
+- stable-diffusion-xl
+- stable-diffusion-xl-diffusers
+- stable-diffusion
+- text-to-image
+- lora
+- loraxl
+- di.FFusion.ai
+language:
+- en
+thumbnail: >-
+  https://cdn-uploads.huggingface.co/production/uploads/6380cf05f496d57325c12194/p54u7dEP1u8en0--NMEjS.png
+library_name: diffusers
 ---
+
+![ffusion1.jpg](https://cdn-uploads.huggingface.co/production/uploads/6380cf05f496d57325c12194/yS5z9x4AOxLilW17ZZaiV.jpeg)
+
+# FFusion LoRA Extracted Models for Hugging Face 🚀
+
+Welcome to the FFusion LoRA extracted models repository on Hugging Face! Here, we present a collection of models extracted using the Low-Rank Adaptation (LoRA) technique to provide a rich dataset for research and further exploration.
+
+## 📦 Base Models
+
+The following models served as the foundation for our extractions:
+
+- [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
+- [stabilityai/stable-diffusion-xl-base-0.9](https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9)
+- [FFusion/FFusionXL-BASE](https://huggingface.co/FFusion/FFusionXL-BASE)
+
+## 🔍 Extraction Details
+
+- **Variants:** Each base model was extracted into 4-5 distinct variants.
+- **Extraction Depth:** The models uploaded here contain approximately 70% of extracted data. These extractions yield a dataset size of around 400 GB.
+- **Precision:** We experimented with both `float32` and `float64` for optimal extraction results.
+- **Differences Measurement:** Singular Value Decomposition (SVD) was utilized to measure differences between the original and the tuned models. A threshold of \(1 \times 10^{-3}\) was commonly used, although in some cases, \(1 \times 10^{-5}\) and \(1 \times 10^{-2}\) were tested.
+- **Demonstration Parameters:** For our demonstration, we employed `"conv_dim": 256` and `"conv_alpha": 256`.
+
+## ⚙️ Technical Notes
+
+- Most SDXL models in this collection are not traditionally "trained." Instead, they are merged from previous SDXL 0.9 versions or created using other methods with the help of Comfy UI.
+- An important note for users: all models saved with Comfy add an extra key `text_model.encoder.text_model.embeddings.position_ids`. We've made necessary adjustments to ensure compatibility with the current scripts from Kohoya.
+
+## 📈 Use Cases
+
+These extracted models are intended for research and testing. They can be particularly useful for:
+
+- Investigating the potential of merging multiple LoRAs.
+- Weighting experiments with 1-5 LoRAs simultaneously.
+- Exploring the differences and similarities between LoRAs extracted from different base models.
+
+## 🎉 Acknowledgements & Citations
+
+A huge shoutout to the community for their continued support and feedback. Together, we are pushing the boundaries of what's possible with machine learning!
+
+We would also like to acknowledge and give credit to the following projects and authors:
+
+- **ComfyUI**: We've used and modified portions of [ComfyUI](https://github.com/comfyanonymous/ComfyUI) for our work.
+- **kohya-ss/sd-scripts and bmaltais**: Our work also incorporates modifications from [kohya-ss/sd-scripts](https://github.com/kohya-ss/sd-scripts).
+- **lora-inspector**: We've benefited from the [lora-inspector](https://github.com/rockerBOO/lora-inspector) project.
+- **KohakuBlueleaf**: Special mention to KohakuBlueleaf for their invaluable contributions.
+
+
+# FFusion LoRA Extracted Models - How to Use Guide 🧠
+
+Welcome to the technical guide for using the FFusion LoRA extracted models. This document will walk you through the steps required to fuse LoRA parameters, load checkpoints, and perform inference.
+
+## Fusing LoRA Parameters 🔗
+
+To merge the LoRA parameters with the original parameters of the underlying model(s), leading to a potential speedup in inference latency:
+
+```python
+pipe.fuse_lora()
+```
+
+### Unfusing LoRA Parameters ⛓️
+
+To revert the effects of `fuse_lora()`:
+
+```python
+pipe.unfuse_lora()
+```
+
+### Working with Different LoRA Scales 🎚️
+
+To control the influence of the LoRA parameters on the outputs:
+
+```python
+pipe.fuse_lora(lora_scale=0.5)
+```
+
+## Working with FFusion Models 🔍
+
+Here's how to load and utilize our FFusion models:
+
+```python
+from diffusers import DiffusionPipeline
+import torch
+
+pipeline_id = "FFusion/FFusionXL-BASE"
+
+pipe = DiffusionPipeline.from_pretrained(pipeline_id, torch_dtype=torch.float16)
+pipe.enable_model_cpu_offload()
+
+lora_model_id = "FFusion/400GB-LoraX"
+lora_filename = "FFai.0038.Realitycheckxl_Alpha11.lora.safetensors"
+pipe.load_lora_weights(lora_model_id, weight_name=lora_filename)
+
+prompt = "papercut sonic"
+image = pipe(prompt=prompt, num_inference_steps=20, generator=torch.manual_seed(0)).images[0]
+image
+```
+## Running Inference 🖼️
+
+After loading the desired model, you can perform inference as follows:
+
+```python
+generator = torch.manual_seed(0)
+images_fusion = pipe(
+    "masterpiece, best quality, mountain", output_type="np", generator=generator, num_inference_steps=2
+).images
+```
+
+
+## Library of Available LoRA Models 📚
+
+You can choose any of the models from our repository on Hugging Face or the upcoming repository on CivitAI. Here's a list of available models with `lora_model_id = "FFusion/400GB-LoraXL"`:
+
+```
+lora_filename = 
+    - FFai.0001.4Guofeng4xl_V1125d.lora_Dim64.safetensors
+    - FFai.0002.4Guofeng4xl_V1125d.lora_Dim8.safetensors
+    - FFai.0003.4Guofeng4xl_V1125d.loraa.safetensors
+    - FFai.0004.Ambiencesdxl_A1.lora.safetensors
+    - FFai.0005.Ambiencesdxl_A1.lora_8.safetensors
+    - FFai.0006.Angrasdxl10_V22.lora.safetensors
+    - FFai.0007.Animaginexl_V10.lora.safetensors
+    - FFai.0008.Animeartdiffusionxl_Alpha3.lora.safetensors
+    - FFai.0009.Astreapixiexlanime_V16.lora.safetensors
+    - FFai.0010.Bluepencilxl_V010.lora.safetensors
+    - FFai.0011.Bluepencilxl_V021.lora.safetensors
+    - FFai.0012.Breakdomainxl_V03d.lora.safetensors
+    - FFai.0013.Canvasxl_Bfloat16v002.lora.safetensors
+    - FFai.0014.Cherrypickerxl_V20.lora.safetensors
+    - FFai.0015.Copaxtimelessxlsdxl1_V44.lora.safetensors
+    - FFai.0016.Counterfeitxl-Ffusionai-Alpha-Vae.lora.safetensors
+    - FFai.0017.Counterfeitxl_V10.lora.safetensors
+    - FFai.0018.Crystalclearxl_Ccxl.lora.safetensors
+    - FFai.0019.Deepbluexl_V006.lora.safetensors
+    - FFai.0020.Dream-Ffusion-Shaper.lora.safetensors
+    - FFai.0021.Dreamshaperxl10_Alpha2xl10.lora.safetensors
+    - FFai.0022.Duchaitenaiartsdxl_V10.lora.safetensors
+    - FFai.0023.Dynavisionxlallinonestylized_Beta0371bakedvae.lora.safetensors
+    - FFai.0024.Dynavisionxlallinonestylized_Beta0411bakedvae.lora.safetensors
+    - FFai.0025.Fantasticcharacters_V55.lora.safetensors
+    - FFai.0026.Fenrisxl_V55.lora.safetensors
+    - FFai.0027.Fudukimix_V10.lora.safetensors
+    - FFai.0028.Infinianimexl_V16.lora.safetensors
+    - FFai.0029.Juggernautxl_Version1.lora_1.safetensors
+    - FFai.0030.Lahmysterioussdxl_V330.lora.safetensors
+    - FFai.0031.Mbbxlultimate_V10rc.lora.safetensors
+    - FFai.0032.Miamodelsfwnsfwsdxl_V30.lora.safetensors
+    - FFai.0033.Morphxl_V10.lora.safetensors
+    - FFai.0034.Nightvisionxlphotorealisticportrait_Beta0681bakedvae.lora_1.safetensors
+    - FFai.0035.Osorubeshialphaxl_Z.lora.safetensors
+    - FFai.0036.Physiogenxl_V04.lora.safetensors
+    - FFai.0037.Protovisionxlhighfidelity3d_Beta0520bakedvae.lora.safetensors
+    - FFai.0038.Realitycheckxl_Alpha11.lora.safetensors
+    - FFai.0039.Realmixxl_V10.lora.safetensors
+    - FFai.0040.Reproductionsdxl_V31.lora.safetensors
+    - FFai.0041.Rundiffusionxl_Beta.lora.safetensors
+    - FFai.0042.Samaritan3dcartoon_V40sdxl.lora.safetensors
+    - FFai.0043.Sdvn6realxl_Detailface.lora.safetensors
+    - FFai.0044.Sdvn7realartxl_Beta2.lora.safetensors
+    - FFai.0045.Sdxl10arienmixxlasian_V10.lora.safetensors
+    - FFai.0046.Sdxlbasensfwfaces_Sdxlnsfwfaces03.lora.safetensors
+    - FFai.0047.Sdxlfaetastic_V10.lora.safetensors
+    - FFai.0048.Sdxlfixedvaefp16remove_Basefxiedvaev2fp16.lora.safetensors
+    - FFai.0049.Sdxlnijiv4_Sdxlnijiv4.lora.safetensors
+    - FFai.0050.Sdxlronghua_V11.lora.safetensors
+    - FFai.0051.Sdxlunstablediffusers_V5unchainedslayer.lora.safetensors
+    - FFai.0052.Sdxlyamersanimeultra_Yamersanimev2.lora.safetensors
+    - FFai.0053.Shikianimexl_V10.lora.safetensors
+    - FFai.0054.Spectrumblendx_V10.lora.safetensors
+    - FFai.0055.Stablediffusionxl_V30.lora.safetensors
+    - FFai.0056.Talmendoxlsdxl_V11beta.lora.safetensors
+    - FFai.0057.Wizard_V10.lora.safetensors
+    - FFai.0058.Wyvernmix15xl_Xlv11.lora.safetensors
+    - FFai.0059.Xl13asmodeussfwnsfw_V17bakedvae.lora.safetensors
+    - FFai.0060.Xl3experimentalsd10xl_V10.lora.safetensors
+    - FFai.0061.Xl6hephaistossd10xlsfw_V21bakedvaefp16fix.lora.safetensors
+    - FFai.0062.Xlperfectdesign_V2ultimateartwork.lora.safetensors
+    - FFai.0063.Xlyamersrealistic_V3.lora.safetensors
+    - FFai.0064.Xxmix9realisticsdxl_Testv20.lora.safetensors
+    - FFai.0065.Zavychromaxl_B2.lora.safetensors
+
+```
+
+📊 Text Encoder Difference Overview
+Based on the extraction process, we observed the following differences in the text encoder across various models:
+
+## 4Guofeng4XL_v1125D
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## AmbienceSDXL_a1
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## angraSDXL10_v22
+✅ Text encoder available. Difference by 0.001953125
+
+## animagineXL_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## animeArtDiffusionXL_alpha3
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## AstreaPixieXLAnime_v16
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## bluePencilXL_v010
+✅ Text encoder available. Difference by 0.001953125
+
+## bluePencilXL_v021
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## breakdomainXL_v03d
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## canvasXL_bFloat16v002
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## cherryPickerXL_v20
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## copaXTimelessXLSDXL1_v44
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## counterfeitXL-FFusionAI-alpha-VAE
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## counterfeitXL_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## crystalclearXL_ccxl
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## deepblueXL_v006
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## dream-FFusion-shaper
+✅ Text encoder available. Difference by 0.001953125
+
+## dreamShaperXL10_alpha2xl10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## duchaitenaiartsSDXL_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## dynavisionXLallInOneStylized_beta0371bakedVAE
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## dynavisionXLallInOneStylized_beta0411bakedVAE
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## fantasticCharacters_v55
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## fenrisXL_v55
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## fudukimix_v10
+✅ Text encoder available. Difference by 0.001953125
+
+## infiniaNimeXL_v16
+✅ Text encoder available. Difference by 0.001953125
+
+## juggernautXL_version1
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## lahMysteriousSDXL_v330
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## mbBXLUltimate_v10rc
+✅ Text encoder available. Difference by 0.001953125
+
+## miaModelsFWNsfwSDXL_v30
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## morphXL_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## nightvisionXLphotorealisticPortrait_beta0681bakedVAE
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## osorubeshiAlphaXL_z
+✅ Text encoder available. Difference by 0.001953125
+
+## physiogenXL_v04
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## protoVisionXLhighFidelity3D_beta0520bakedVAE
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## realityCheckXL_alpha11
+✅ Text encoder available. Difference by 0.001953125
+
+## realmixXL_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## reproductionSDXL_v31
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## runDiffusionXL_beta
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## samaritan3DCartoon_v40sdxl
+✅ Text encoder available. Difference by 0.001953125
+
+## sdvn6realXL_detailFace
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## sdvn7realArtXL_beta2
+✅ Text encoder available. Difference by 0.001953125
+
+## sdxl10arienMixXLasian_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## sdxlBaseNSFWFaces_sdxlNSFWFaces03
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## sdxlFaeTastic_v10
+✅ Text encoder available. Difference by 0.001953125
+
+## sdxlFixedVAEfp16Remove_baseFxiedVAEv2fp16
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## sdxlNijiV4_sdxlNijiV4
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## sdxlRonghua_v11
+✅ Text encoder available. Difference by 0.001953125
+
+## sdxlUnstableDiffusers_v5UnchainedSlayer
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## sdxlYamersAnimeUltra_yamersAnimeV2
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## shikiAnimeXL_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## spectrumBlendX_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## stableDiffusionXL_v30
+✅ Text encoder available. Difference by 0.001953125
+
+## talMendoXlsdXL_v11beta
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## wizard_v10
+✅ Text encoder available. Difference by 0.001953125
+
+## wyvernMix15XL_XLv11
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## xl13asmodeusSFWNSFW_v17bakedVAEfp16fix
+✅ Text encoder available. Difference by 0.001953125
+
+## xl3experimentalSD10XL_v10
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## xl6hephaistosSD10XLsfw_v21bakedVAEfp16fix
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## xlPerfectDesign_v2UltimateArtwork
+✅ Text encoder available. Difference by 0.001953125
+
+## xlYamersRealistic_v3
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base
+
+## xxMix9RealisticSDXL_Testv20
+✅ Text encoder available. Difference by 0.001953125
+
+## zavyChromaXL_b2
+🔴 Text encoder unavailable. Same as SDXL 1.0 Base