metadata

license: other
base_model: black-forest-labs/FLUX.1-dev
tags:
  - flux
  - flux-diffusers
  - text-to-image
  - diffusers
  - simpletuner
  - safe-for-work
  - lora
  - template:sd-lora
  - lycoris
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: >-
      In the style of an Albert Bierstadt oil painting, A vast mountain lake
      reflects snow-capped peaks under a dramatic sky with golden light breaking
      through storm clouds. In the foreground, a small deer drinks from the
      crystal-clear water while pine trees frame the scene.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png
  - text: >-
      In the style of an Albert Bierstadt oil painting, A majestic waterfall
      cascades down rocky cliffs surrounded by towering pine trees. Morning mist
      rises from the valley below while early sunlight catches the spray of the
      falls, creating a rainbow effect.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_2_0.png
  - text: >-
      In the style of an Albert Bierstadt oil painting, A sweeping valley vista
      with a winding river catching the last rays of sunset. Massive clouds turn
      pink and gold above distant mountains while two figures on horseback
      appear tiny against the epic landscape.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_3_0.png
  - text: >-
      In the style of an Albert Bierstadt oil painting, A lone Range Rover
      appears dwarfed by towering granite cliffs and ancient sequoias. Golden
      afternoon light streams through the trees while a waterfall tumbles in the
      distance. Mist partially shrouds the vehicle, making it seem like a
      mystical chariot.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_4_0.png
  - text: >-
      In the style of an Albert Bierstadt oil painting, A pristine mountain
      spring bubbles up through rocks, its crystal waters mixing with the
      distinctive red of natural cola springs. Native wildlife gather to drink
      while steam rises in the crisp mountain air. A massive peak looms in the
      background, partially hidden by dramatic clouds.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_5_0.png
  - text: >-
      In the style of an Albert Bierstadt oil painting, A giant golden hamster
      stands majestically atop a rocky outcrop overlooking a vast valley, scaled
      to the size of a buffalo. Morning light catches its fur while smaller
      hamsters graze peacefully in the meadow below. Storm clouds gather
      dramatically above snow-capped peaks in the distance.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_6_0.png
  - text: >-
      In the style of an Albert Bierstadt oil painting, A line of electric
      vehicles winds through a mountain pass, dwarfed by massive cliffs and
      ancient trees. Late afternoon light creates dramatic shadows while eagles
      soar overhead. A campsite with solar panels glints in the distance like a
      modern wagon train.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_7_0.png
  - text: >-
      In the style of an Albert Bierstadt oil painting, Half Dome towers over
      Yosemite Valley at sunset, while advanced flying vehicles appear as tiny
      specks against the epic landscape. A crystalline research station nestled
      among the trees catches the golden light, while traditional hikers on the
      trail below emphasize the timeless scale of nature.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_8_0.png

AlbertBierstadt-Flux-LoKr

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

CFG: 3.0
CFG Rescale: 0.0
Steps: 20
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 1024x1280
Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)

Negative Prompt
blurry, cropped, ugly

Prompt
In the style of an Alfred Edmund Brehm illustration, Three stag beetles on oak bark, with one near green leaves at the top, another climbing vertically in the middle, and a third at the base amid fallen leaves and moss.

Negative Prompt
blurry, cropped, ugly

Prompt
In the style of an Alfred Edmund Brehm illustration, Four large moths around green leaves, one cream-colored, two brown with circular wing patterns, and one white moth in flight, with a pale caterpillar climbing on a leaf above.

Negative Prompt
blurry, cropped, ugly

Prompt
In the style of an Alfred Edmund Brehm illustration, A golden hamster sits upright on desert sand, its cheek pouches full of seeds. Three small scarab beetles move across the sand nearby, while a scorpion rests in the lower right corner.

Negative Prompt
blurry, cropped, ugly

Prompt
In the style of an Alfred Edmund Brehm illustration, A Range Rover in an African savanna setting, with two rhinoceros beetles on its front tire. Three dung beetles roll balls past its tracks in the dirt, while acacia trees stand in the background.

Negative Prompt
blurry, cropped, ugly

Prompt
In the style of an Alfred Edmund Brehm illustration, A glass Coca-Cola bottle lying sideways on brown leaves and soil. A line of black ants traverses its red label, two iridescent beetles explore the metal cap, and a pale moth rests on the glass neck.

Negative Prompt
blurry, cropped, ugly

Prompt
In the style of an Alfred Edmund Brehm illustration, Black over-ear headphones on a wooden table. Three small beetles crawl along the ear cushions, while a spider hangs between the headband adjusters, its web gleaming in the light.

Negative Prompt
blurry, cropped, ugly

Prompt
In the style of an Alfred Edmund Brehm illustration, A white athletic shoe on packed earth. Carpenter ants march through its eyelets, a beetle rests under the loosened tongue, and a cricket perches on the heel.

Negative Prompt
blurry, cropped, ugly

Prompt
In the style of an Alfred Edmund Brehm illustration, Three wooden pencils lying across a blank paper sheet. A praying mantis stands on one pencil tip, while two ladybugs explore graphite shavings scattered below.

Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 4
Training steps: 6000
Learning rate: 0.0001
- Learning rate schedule: constant
- Warmup steps: 200
Max grad norm: 2.0
Effective batch size: 3
- Micro-batch size: 3
- Gradient accumulation steps: 1
- Number of GPUs: 1
Gradient checkpointing: True
Prediction type: flow-matching (extra parameters=['shift=3', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flow_matching_loss=compatible'])
Optimizer: adamw_bf16
Trainable parameter precision: Pure BF16
Caption dropout probability: 10.0%
SageAttention: Enabled inference

LyCORIS Config:

{
    "algo": "lokr",
    "multiplier": 1.0,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 16,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "Attention": {
                "factor": 16
            },
            "FeedForward": {
                "factor": 8
            }
        }
    }
}

Datasets

ab-512

Repeats: 12
Total number of images: 74
Total number of aspect buckets: 4
Resolution: 0.262144 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ab-768

Repeats: 8
Total number of images: 74
Total number of aspect buckets: 4
Resolution: 0.589824 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ab-1024

Repeats: 5
Total number of images: 74
Total number of aspect buckets: 3
Resolution: 1.048576 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ab-1536

Repeats: 2
Total number of images: 73
Total number of aspect buckets: 4
Resolution: 2.359296 megapixels
Cropped: False
Crop style: None
Crop aspect: None
Used for regularisation data: No

ab-crops-512

Repeats: 8
Total number of images: 74
Total number of aspect buckets: 1
Resolution: 0.262144 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

ab-1024-crop

Repeats: 6
Total number of images: 74
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights


def download_adapter(repo_id: str):
    import os
    from huggingface_hub import hf_hub_download
    adapter_filename = "pytorch_lora_weights.safetensors"
    cache_dir = os.environ.get('HF_PATH', os.path.expanduser('~/.cache/huggingface/hub/models'))
    cleaned_adapter_path = repo_id.replace("/", "_").replace("\\", "_").replace(":", "_")
    path_to_adapter = os.path.join(cache_dir, cleaned_adapter_path)
    path_to_adapter_file = os.path.join(path_to_adapter, adapter_filename)
    os.makedirs(path_to_adapter, exist_ok=True)
    hf_hub_download(
        repo_id=repo_id, filename=adapter_filename, local_dir=path_to_adapter
    )

    return path_to_adapter_file
    
model_id = 'black-forest-labs/FLUX.1-dev'
adapter_repo_id = 'davidrd123/AlbertBierstadt-Flux-LoKr'
adapter_filename = 'pytorch_lora_weights.safetensors'
adapter_file_path = download_adapter(repo_id=adapter_repo_id)
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_file_path, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1280,
    guidance_scale=3.0,
).images[0]
image.save("output.png", format="PNG")

Exponential Moving Average (EMA)

SimpleTuner generates a safetensors variant of the EMA weights and a pt file.

The safetensors file is intended to be used for inference, and the pt file is for continuing finetuning.

The EMA model may provide a more well-rounded result, but typically will feel undertrained compared to the full model as it is a running decayed average of the model weights.