audio-cpp/SoulX-FlashHead-GGUF

SoulX-FlashHead Lite GGUF

Follow or star audio.cpp for the C++ runtime and model integration updates.

GGUF packaging of Soul-AILab/SoulX-FlashHead-1_3B. The model generates streaming talking-portrait video from a reference image and speech audio. This repository packages Lite only, including its DiT, LTX video VAE and Wav2Vec2 speech encoder. Upstream checkpoint revision: 5fbd2e44ba08ab2deb28bd55cff2c15523e51ba5. Model architecture and weights are by Soul-AILab; the C++ integration is in audio.cpp.

cuDNN is strongly recommended for CUDA: about 2× faster warm inference, with lower peak VRAM. Prebuilt audio.cpp binaries do not include cuDNN support. Build from source with cuDNN and -DGGML_CUDA_CUDNN_CONV3D=ON, then select --session-option 'soulx_flashhead.exec_plan={"vae":{"impl":"cudnn"}}'. The default GGML path remains available without cuDNN, including in prebuilts.

English streaming demo

Generated from a single reference portrait and 42.24 seconds of English narration using audio.cpp: Lite Q8, 512×512, four denoising steps, 25 FPS, Vulkan backend. Audio is processed in 0.96-second chunks with continuous motion between chunks.

Inputs: English audio, transcript, and portrait. The portrait is from the upstream example. The English narration was generated with Qwen3-TTS.

audio.cpp CUDA performance

Measured on an RTX 5090 with the Q8 package, 512×512 output, four denoising steps, and the 42.24-second English demo above (44 streaming chunks). Debug build, eight CPU threads; the two modes were run sequentially.

VAE implementation	Warm RTF ↓	Full-run RTF ↓	Peak VRAM
cuDNN (recommended)	0.215	0.430	5,168 MiB
GGML (default)	0.490	0.706	9,152 MiB

RTF is generation time divided by video duration; lower is faster. Warm RTF covers the full audio-encoder, DiT, VAE, color-correction and RGB-output pipeline, excluding the first two chunks and CLI artifact serialization. Full-run RTF also includes initialization, cold chunks and CLI artifact serialization. Both exclude model loading and MP4 encoding. Peak VRAM is sampled per-process GPU memory. The cuDNN VAE uses F16 activations with F32 accumulation.

Weights and usage

File	Components
SoulX-FlashHead-Lite-Q8_0.gguf	Q8 DiT, BF16 LTX VAE, F32 Wav2Vec2, embedded configuration and model contract
SoulX-FlashHead-Lite-BF16.gguf	BF16 DiT and LTX VAE, F32 Wav2Vec2, embedded configuration and model contract

audiocpp_cli --family soulx_flashhead --task gen \
  --model SoulX-FlashHead-Lite-Q8_0.gguf --backend cuda \
  --audio speech.wav --request-option reference_image_file=portrait.png \
  --mode streaming --out-dir outputs/flashhead \
  --log --log-file outputs/flashhead/run.log

For a cuDNN-enabled source build, add the following option to the command above:

--session-option 'soulx_flashhead.exec_plan={"vae":{"impl":"cudnn"}}'

Lite emits 24 RGB24 frames per chunk at 25 FPS. Streaming input is mono 16 kHz; offline input is mixed down and resampled when needed. Default output is 512×512 with four denoising steps. Chunks retain motion continuity from the preceding output.

See the usage and conversion documentation.