SimpleTuner/Qwen-Image-2.1-training-assistant-v1

SimpleTuner Training Assistant v1 for Qwen Image 2.1

Built with Qwen. An experimental training assistant LoRA for Qwen Image 2.1, trained for 1,000 updates across 512, 1024, 1536 and 2048 base resolutions, including portrait and landscape buckets.

The intended use is to keep this adapter frozen and active during downstream LoRA training, then disable it when generating images with the downstream adapter. Its downstream benefit has not been demonstrated in the initial paired test described below. This checkpoint also predates a confirmed optimizer arithmetic correction.

Use in SimpleTuner

Use a SimpleTuner checkout that includes Qwen Image 2.1 and Qwen assistant-LoRA support. Add these fields to an otherwise complete concept-training configuration:

{
  "model_family": "qwen_image",
  "model_flavour": "v2.1",
  "model_type": "lora",
  "lora_type": "standard",
  "assistant_lora_path": "SimpleTuner/Qwen-Image-2.1-training-assistant-v1",
  "assistant_lora_weight_name": "pytorch_lora_weights.safetensors",
  "disable_assistant_lora": false,
  "assistant_lora_strength": 1.0,
  "assistant_lora_inference_strength": 0.0
}

SimpleTuner loads and freezes the assistant alongside the trainable adapter. Use ordinary concept training; distillation_method: assistant_lora is for creating a new assistant, not for using this one. At inference, use the base model plus your resulting concept LoRA with this assistant disabled. This release has no trigger word and is not a character adapter or a few-step generation adapter. Compatibility with older Qwen Image flavours has not been tested.

Training

The assistant learned from fresh outputs of the adapter-disabled base model, following the general training-adapter approach described by Ostris. Teacher outputs were generated directly as latents with 40 native inference steps and CFG 1, then used as denoising targets after restoring the trainable adapter. No VAE decoding was used to create training targets, and terminal latent outputs were not pre-cached.

Captions came from a local pool of 4,096 long_caption entries from CC12M structured captions. The run generated 1,000 training samples with batch size 1. Dataset images were not used as the target images.

Setting	Value
GPU	NVIDIA L40S
Updates / batch / accumulation	1,000 / 1 / 1
Precision	BF16
LoRA rank / alpha	32 / 32
Attention projections	to_q, to_k, to_v, to_out.0
Optimizer	adamw_bf16, pre-fix implementation described below
Learning rate	1e-5, constant after 25 warmup steps
Gradient clipping	Individual values at 0.01
Gradient checkpointing	Enabled, interval 2
Generation seed	42, advancing for each generated sample
Trainable parameters	33,554,432

Each batch used the next width × height bucket in this sequence:

Base resolution	Square	Portrait	Landscape
512	512 × 512	384 × 672	672 × 384
1024	1024 × 1024	768 × 1344	1344 × 768
1536	1536 × 1536	1152 × 2016	2016 × 1152
2048	2048 × 2048	1536 × 2688	2688 × 1536

All 12 buckets completed successfully. The safetensors header retains the original nominal 1024x1024 configuration field; the actual training sizes are listed above and in training_details.json.

Evaluation and limitations

Final assistant-validation fox and portrait images at 1024 and 2048 pixels remained coherent. Tiled VAE decoding produced visible colour seams in previews; it was not part of teacher-target generation.

A paired downstream test trained two fresh Domokun adapters for 250 updates each at 2048 pixels, batch size 1, learning rate 1e-5, and value clipping at 1. The assistant had training strength 0 in the control and 1 in the assisted branch; it was disabled during inference in both. Initial adapter weights and validation images matched exactly. At 40 inference steps, CFG 1 and 1024 pixels, both final branches still produced people for the character prompts, while fox and portrait priors remained coherent. This test demonstrated no downstream benefit. It was a limited experiment, not a conclusion about all assistant-training recipes.

Optimizer provenance

Assistant preparation and both downstream branches used a stochastic-add helper that computed other + alpha * input instead of input + alpha * other. With beta1 = 0.9, this made the first-moment recurrence approximately m = 0.09 * m_previous + gradient, rather than m = 0.9 * m_previous + 0.1 * gradient, before rounding effects. The reversal is also present in upstream adamw-bf16 at revision 0a6c09a.

The arithmetic issue is reproducible independently of model training. It substantially changes momentum smoothing, but does not establish the cause of the downstream failure. This v1 artifact preserves the completed checkpoint; it has not been retrained with the corrected helper. Use the corrected implementation for new experiments and retune settings as needed.

Artifact and license

pytorch_lora_weights.safetensors contains 256 BF16 tensors and is byte-identical to the completed multi-resolution checkpoint at update 1,000.

SHA-256: 9985b6983ddf03ba44b5b9e9e16d63e7ed1dad274d937c80c46c11215c7e99b0.

Modification notice: this is a separately trained LoRA that changes Qwen Image 2.1 transformer attention projections when loaded. Distribution and use are subject to the Qwen Research License Agreement, including its non-commercial terms. See Notice for the upstream attribution.