Zaytron40k/Qwen-Image-Edit-2511-StyleTransfer-LoRA

Qwen-Image-Edit-2511 LoRA - Style Transfer (flat | realistic -> target style)

Converts an input image (either flat-color art OR a realistic render) into the target illustrated art style, preserving composition and content. Trained with DiffSynth-Studio on 362 pixel-aligned triplets (722 input->target pairs; SFW + explicit).

Prompt format (same as training), trigger word t01nstyle: Convert image 1 to t01nstyle. <optional scene description> The bare instruction Convert image 1 to t01nstyle. also works (50% of training was trigger-only).

Usage

import torch
from PIL import Image
from diffsynth.pipelines.qwen_image import QwenImagePipeline, ModelConfig

pipe = QwenImagePipeline.from_pretrained(
    torch_dtype=torch.bfloat16, device="cuda",
    model_configs=[
        ModelConfig(model_id="Qwen/Qwen-Image-Edit-2511", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="text_encoder/model*.safetensors"),
        ModelConfig(model_id="Qwen/Qwen-Image", origin_file_pattern="vae/diffusion_pytorch_model.safetensors"),
    ],
    tokenizer_config=None,
    processor_config=ModelConfig(model_id="Qwen/Qwen-Image-Edit", origin_file_pattern="processor/"),
)
pipe.load_lora(pipe.dit, "checkpoints/epoch-4.safetensors")
src = Image.open("input.jpeg")  # flat-color OR realistic
img = pipe("Convert image 1 to t01nstyle.",
           edit_image=[src], seed=0, num_inference_steps=40,
           height=1328, width=1024, zero_cond_t=True)  # zero_cond_t is REQUIRED for 2511

Training config

") + 50% trigger-only

base	Qwen/Qwen-Image-Edit-2511 (DiT only)
rank / lr	32 / 7e-5
epochs x steps	6 x 1444 (722 pairs, dual-caption, repeat 1)
resolution	target ~1328^2, input ~1024^2 (pre-resized), max_pixels 1763584
precision	bf16 + gradient checkpointing
special	--zero_cond_t (2511-specific, also required at inference)
captions	dual: 50% full ("Convert image 1 to t01nstyle.

Loss

epoch	step	EMA loss min
0	815	0.0395
1	1460	0.0428
2	4331	0.0389 <- global min
3	4334	0.0393
4	6026	0.0414
5	8521	0.0398

Loss is flat/noisy (normal for diffusion edit LoRA) - pick the checkpoint by visual eval on HELD-OUT inputs, not by loss. Watch for appearance-leakage (copying the target instead of restyling).

Dataset sample

One training group in dataset_example/: the target (target.jpeg) plus its two pixel-aligned inputs (input_flat.jpeg, input_realistic.jpeg) and the prompts (pair.json). Each input->target is a separate training pair; both inputs map to the same target.