HirCoir/yue2-multigpu-kaggle

YuE2 Multi-GPU for Kaggle (stable C++ build)

A reproducible Kaggle package for YuE2-3B Q8_0 using the stable audio.cpp C++ runtime with CUDA, context-preserving NAR scheduling, and multi-GPU VAE tiles. This repository intentionally excludes the experimental AR/NAR streaming branch.

Access: this is a public Hugging Face model repository with manual approval enabled. You can see the model card, but you must request access and wait for approval before the files can be downloaded. After approval, create a Hugging Face token and save it in Kaggle Secrets under the exact name HF_TOKEN.

Included

• bin/audiocpp_server — stable Linux x86-64 CUDA server binary, built from 0xShug0/audio.cpp commit 12341c53.
• bin/audiocpp_cli — command-line inference binary from the same build.
• models/Yue2-3B-GGUF/yue2-3b-q8_0.gguf — Q8_0 YuE2-3B weights.
• models/Yue2-3B-GGUF/yue2-vae-f16.gguf — F16 VAE weights.
• models/Yue2-3B-GGUF/sidecars/ — model, generation, VAE configs and tokenizer.
• model_specs/yue2.json — runtime model specification.
• notebooks/yue2_multigpu_kaggle.ipynb — ready-to-run Kaggle notebook that reads HF_TOKEN from Kaggle Secrets, downloads this gated repo, starts the C++ server and launches a Gradio UI.

Start in Kaggle

1. Open the notebook at notebooks/yue2_multigpu_kaggle.ipynb and import it into Kaggle.
2. On this model page, submit Request access and wait for manual approval.
3. In Kaggle, turn Internet on and enable a GPU accelerator. Two CUDA GPUs are needed for the multi-GPU path; with one GPU, the notebook falls back to single-GPU mode.
4. Under Add-ons → Secrets, create a secret named exactly HF_TOKEN containing a Hugging Face read token for the account that was approved.
5. Run the notebook cells in order. The final cell provides a temporary Gradio link.

The notebook never contains or prints your token. Hugging Face approval is checked by the Hub when snapshot_download accesses this gated repository; having a token alone does not bypass a pending or rejected request.

Quality-first multi-GPU settings

The notebook sets these environment variables:

• YUE2_MULTI_GPU=on
• YUE2_MULTI_GPU_DEVICES=1
• YUE2_MULTI_GPU_NAR_SPLIT=off
• YUE2_MULTI_GPU_VAE_TILES=auto

AR semantic-token generation remains sequential on primary GPU 0. The default NAR_SPLIT=off preserves the original NAR chunk boundaries instead of creating an extra split across the middle of a song. This avoids resetting local NAR context at an artificial GPU boundary and prioritizes consistent timbre across the full track. VAE_TILES=auto still allows independent VAE decode tiles to run across GPU 0 and GPU 1 when the request uses serial NAR. If you explicitly prefer speed over this continuity safeguard, set YUE2_MULTI_GPU_NAR_SPLIT=auto; long songs may then be split into parallel NAR chunks and can sound different across chunk boundaries. AR semantic-token generation remains sequential on primary GPU 0. This is not AR/NAR live streaming or tensor-parallel AR.

See the audio.cpp model implementation and server logs for the actual per-request scheduler decisions.

License and attribution

• Model weights: YuE2-3B and its GGUF conversion are under CC BY-NC 4.0. Base model: https://huggingface.co/m-a-p/YuE2-3B. GGUF conversion: https://huggingface.co/audio-cpp/Yue2-3B-GGUF. The model card and original license terms are authoritative; non-commercial restrictions remain in force. Manual approval does not grant commercial rights.
• Inference engine: https://github.com/0xShug0/audio.cpp, commit 12341c53, Apache-2.0. A copy of the engine license is in licenses/LICENSE-AUDIOCPP.txt.
• The compiled binaries are for Linux x86-64 with CUDA 12.x, built for NVIDIA compute capability 7.5 (Tesla T4). Other GPU architectures may require rebuilding.

Model weights are redistributed with their original attribution and license. This repository does not claim ownership of YuE2 or the model weights.