README.md

metadata

license: other
base_model: black-forest-labs/FLUX.1-dev
tags:
  - flux
  - flux-diffusers
  - text-to-image
  - diffusers
  - simpletuner
  - safe-for-work
  - lora
  - template:sd-lora
  - lycoris
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: >-
      In the style of a c4ss4tt oil painting, A child wearing an elaborate blue
      silk dress with tiered ruffles, pin-tucked bodice, and white valenciennes
      lace trim sits near a tall window. The afternoon light catches each fold
      and pleat, revealing the fabric's subtle sheen and varied textures. A
      white satin bow adorns the child's collar.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png
  - text: >-
      In the style of a c4ss4tt oil painting, A close portrait of a young
      child's face with rosy apple cheeks, naturally flushed complexion, and
      delicate features typical of the artist. Soft directional light from a
      nearby window creates subtle shadows under the child's rounded chin. Their
      brown hair falls in loose natural curls around their face.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_2_0.png
  - text: >-
      In the style of a c4ss4tt oil painting, Strong afternoon light streams
      through a partially curtained window, falling dramatically across a
      child's face and shoulder, casting deep shadows on their cornflower blue
      dress. The contrast highlights the child's profile and the crisp white
      collar at their neck. The background fades to muted, warm tones.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_3_0.png
  - text: >-
      In the style of a c4ss4tt oil painting, A child in a wide-brimmed navy
      blue hat with a white ribbon stands in profile by a tall window. The
      background is intentionally sparse with soft, neutral tones. The child's
      face is partially shadowed by the hat's brim.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_4_0.png
  - text: >-
      In the style of a c4ss4tt oil painting, A woman in a cream-colored dress
      holds her sleeping baby close to her shoulder, their faces touching. The
      composition is deliberately simple, set against a plain wall with subtle
      blue undertones. The baby's hand rests gently on the mother's collar.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_5_0.png
  - text: >-
      In the style of a c4ss4tt oil painting, A woman in an intricately detailed
      white lace dress with high collar reads while seated in a
      damask-upholstered chair by a window with gauzy muslin curtains. A
      patterned Oriental rug lies beneath, a brass lamp with crystal drops
      stands nearby, and a Chinese porcelain vase with camellias rests on a
      mahogany side table.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_6_0.png
  - text: >-
      In the style of a c4ss4tt oil painting, A mother in a chunky cream
      cable-knit sweater checks her smartphone while her baby sleeps against her
      shoulder. The device's blue light reflects subtly on her face, while warm
      lamplight illuminates the scene. The baby wears contemporary striped
      pajamas.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_7_0.png
  - text: >-
      In the style of a c4ss4tt oil painting, A mother Persian cat with long
      silver fur grooms her cream-colored kitten by a sunlit window with lace
      curtains. Their fur catches the morning light, creating a soft halo
      effect. A blue and white porcelain bowl sits nearby on the window sill.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_8_0.png

Cassatt-AdjustedCrops-Flux-LoKr-4e-4

This is a LyCORIS adapter derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

• CFG: 3.0
• CFG Rescale: 0.0
• Steps: 20
• Sampler: FlowMatchEulerDiscreteScheduler
• Seed: 42
• Resolution: 1024x1024
• Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt

unconditional (blank prompt)

Negative Prompt

blurry, cropped, ugly

Prompt

In the style of a c4ss4tt oil painting, A child wearing an elaborate blue silk dress with tiered ruffles, pin-tucked bodice, and white valenciennes lace trim sits near a tall window. The afternoon light catches each fold and pleat, revealing the fabric's subtle sheen and varied textures. A white satin bow adorns the child's collar.

Negative Prompt

blurry, cropped, ugly

Prompt

In the style of a c4ss4tt oil painting, A close portrait of a young child's face with rosy apple cheeks, naturally flushed complexion, and delicate features typical of the artist. Soft directional light from a nearby window creates subtle shadows under the child's rounded chin. Their brown hair falls in loose natural curls around their face.

Negative Prompt

blurry, cropped, ugly

Prompt

In the style of a c4ss4tt oil painting, Strong afternoon light streams through a partially curtained window, falling dramatically across a child's face and shoulder, casting deep shadows on their cornflower blue dress. The contrast highlights the child's profile and the crisp white collar at their neck. The background fades to muted, warm tones.

Negative Prompt

blurry, cropped, ugly

Prompt

In the style of a c4ss4tt oil painting, A child in a wide-brimmed navy blue hat with a white ribbon stands in profile by a tall window. The background is intentionally sparse with soft, neutral tones. The child's face is partially shadowed by the hat's brim.

Negative Prompt

blurry, cropped, ugly

Prompt

In the style of a c4ss4tt oil painting, A woman in a cream-colored dress holds her sleeping baby close to her shoulder, their faces touching. The composition is deliberately simple, set against a plain wall with subtle blue undertones. The baby's hand rests gently on the mother's collar.

Negative Prompt

blurry, cropped, ugly

Prompt

In the style of a c4ss4tt oil painting, A woman in an intricately detailed white lace dress with high collar reads while seated in a damask-upholstered chair by a window with gauzy muslin curtains. A patterned Oriental rug lies beneath, a brass lamp with crystal drops stands nearby, and a Chinese porcelain vase with camellias rests on a mahogany side table.

Negative Prompt

blurry, cropped, ugly

Prompt

In the style of a c4ss4tt oil painting, A mother in a chunky cream cable-knit sweater checks her smartphone while her baby sleeps against her shoulder. The device's blue light reflects subtly on her face, while warm lamplight illuminates the scene. The baby wears contemporary striped pajamas.

Negative Prompt

blurry, cropped, ugly

Prompt

In the style of a c4ss4tt oil painting, A mother Persian cat with long silver fur grooms her cream-colored kitten by a sunlit window with lace curtains. Their fur catches the morning light, creating a soft halo effect. A blue and white porcelain bowl sits nearby on the window sill.

Negative Prompt

blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

• Training epochs: 11
• Training steps: 4500
• Learning rate: 0.0003
  • Learning rate schedule: polynomial
  • Warmup steps: 200
• Max grad norm: 2.0
• Effective batch size: 3
  • Micro-batch size: 3
  • Gradient accumulation steps: 1
  • Number of GPUs: 1
• Gradient checkpointing: True
• Prediction type: flow-matching (extra parameters=['flux_schedule_auto_shift', 'shift=0.0', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flux_beta_schedule_alpha=2.0', 'flux_beta_schedule_beta=5.0', 'flow_matching_loss=compatible'])
• Optimizer: adamw_bf16
• Trainable parameter precision: Pure BF16
• Caption dropout probability: 10.0%

LyCORIS Config:

{
    "algo": "lokr",
    "multiplier": 1.0,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 16,
    "apply_preset": {
        "target_module": [
            "Attention",
            "FeedForward"
        ],
        "module_algo_map": {
            "Attention": {
                "factor": 16
            },
            "FeedForward": {
                "factor": 8
            }
        }
    }
}

Datasets

cassatt-512

• Repeats: 10
• Total number of images: 16
• Total number of aspect buckets: 6
• Resolution: 0.262144 megapixels
• Cropped: False
• Crop style: None
• Crop aspect: None
• Used for regularisation data: No

cassatt-1024

• Repeats: 10
• Total number of images: 16
• Total number of aspect buckets: 8
• Resolution: 1.048576 megapixels
• Cropped: False
• Crop style: None
• Crop aspect: None
• Used for regularisation data: No

cassatt-1536

• Repeats: 5
• Total number of images: 16
• Total number of aspect buckets: 10
• Resolution: 2.359296 megapixels
• Cropped: False
• Crop style: None
• Crop aspect: None
• Used for regularisation data: No

cassatt-crops-512

• Repeats: 10
• Total number of images: 16
• Total number of aspect buckets: 1
• Resolution: 0.262144 megapixels
• Cropped: True
• Crop style: random
• Crop aspect: square
• Used for regularisation data: No

cassatt-crops-1024

• Repeats: 10
• Total number of images: 16
• Total number of aspect buckets: 1
• Resolution: 1.048576 megapixels
• Cropped: True
• Crop style: random
• Crop aspect: square
• Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights


def download_adapter(repo_id: str):
    import os
    from huggingface_hub import hf_hub_download
    adapter_filename = "pytorch_lora_weights.safetensors"
    cache_dir = os.environ.get('HF_PATH', os.path.expanduser('~/.cache/huggingface/hub/models'))
    cleaned_adapter_path = repo_id.replace("/", "_").replace("\\", "_").replace(":", "_")
    path_to_adapter = os.path.join(cache_dir, cleaned_adapter_path)
    path_to_adapter_file = os.path.join(path_to_adapter, adapter_filename)
    os.makedirs(path_to_adapter, exist_ok=True)
    hf_hub_download(
        repo_id=repo_id, filename=adapter_filename, local_dir=path_to_adapter
    )

    return path_to_adapter_file
    
model_id = 'black-forest-labs/FLUX.1-dev'
adapter_repo_id = 'davidrd123/Cassatt-AdjustedCrops-Flux-LoKr-4e-4'
adapter_filename = 'pytorch_lora_weights.safetensors'
adapter_file_path = download_adapter(repo_id=adapter_repo_id)
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_file_path, pipeline.transformer)
wrapper.merge_to()

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
image = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]
image.save("output.png", format="PNG")