Iceky/AuK-French-LoRA

AuK French LoRA

LoRA adapters for tencent/AuK, merged into the generation backbone for French language adaptation. The merged checkpoint keeps the layout of auk_base.safetensors, so the standard AuK inference path loads it without changes.

The Qwen2.5-Omni-3B text encoder is not included. It is loaded separately at runtime and remains under its own license.

Files

File	Description
model_last.safetensors	Adapters folded into the base weights, lora_* and text_encoder.* removed
config.yaml	Model config used for training and inference

Training

• Base checkpoint: tencent/AuK (auk_base.safetensors)
• Adapters: rank 32, alpha 32, dropout 0.0; attention QKV and output projections, DiT feed-forward, txt_proj, proj_out, audio_embed.linear, adaLN modulation, timestep MLP
• Fully trained alongside the adapters: layer_weights, layer_scale
• Trainable parameters: 41.3M / 5.6B (0.74%)
• Data: VoxPopuli (CC0) and MLS (CC BY 4.0), 327k utterances
• Hardware: 4 GPUs, bf16, DeepSpeed ZeRO-3, one epoch

Evaluation

French subset of X-Voice-Testset, 500 prompts. WER is computed with whisper-large-v3 and text normalisation, SIM with WavLM-ECAPA, UTMOS with SpeechMOS. Generation used 32 NFE, CFG 2.0, and a fixed seed.

Model	WER	SIM	UTMOS
This checkpoint	9.028	0.7464	3.4752

The adapter lowers WER and slightly lowers speaker similarity; that trade-off is expected for a language adaptation run.

Usage

auk-infer \
  --model model_last.safetensors \
  --config config.yaml \
  --ref_audio reference.wav \
  --ref_text "Text spoken in the reference clip." \
  --gen_text "Texte à synthétiser en français."

License

MIT, following the base checkpoint. The Qwen2.5-Omni-3B encoder is distributed by its own authors under a separate license.