LARA-Gen

Inference release for LARA-Gen: Enabling Continuous Emotion Control for Music Generation Models via Latent Affective Representation Alignment.

model.safetensors and av_encoder.safetensors together contain the final-paper main checkpoint: experiment 59dc26ab, epoch 30, embedded best_state. Both files are required. The included laragen/ code implements the corresponding streaming Transformer and continuous VA conditioning.

Install and generate

Use Python 3.9 or 3.10, install FFmpeg with your system package manager, then run from this directory:

python -m pip install -r requirements.txt
python generate.py --arousal 7.4 --valence 5.8 --seed 204000 --output outputs/example.wav

Use matched PyTorch/torchaudio wheels for your platform. CUDA is recommended; --device cpu supports slow CPU inference. The WAV is 30 seconds, 32 kHz, mono, with loudness normalization. --duration accepts 0.5–30 seconds. A JSON sidecar records the inputs and seed.

Attention uses native PyTorch SDPA with backend='torch' and memory_efficient=true. No xformers installation is required. Streaming generation retains its KV cache, including support for decoding multiple new tokens per call.

Arousal and valence are continuous values in [1, 9]. Numeric conditioning uses [arousal, valence], normalized by (x-5)/4. The original prompt spelling is preserved: Genarate an instrumental music with arousal 7.40 and valence 5.80. Sampling uses temperature 1, top-k 250, top-p 0 and CFG 3. Paper samples use seeds 204000 + sample index.

from laragen import LaraGen
model = LaraGen('.', device='cuda')
audio = model.generate(arousal=7.4, valence=5.8, seed=204000)
# [1, 1, 960000], before WAV loudness normalization

Frozen dependencies

On first use the loader downloads the original frozen components from their public repositories:

• T5-base: t5-base, revision a9723ea7f1b39c1eae772870f3b547bf6ef7e6c1.
• EnCodec: facebook/encodec_32khz, revision d0c45384f6c44db055f78200cfdcb9c1c8706727.

For offline inference, download these snapshots on a connected machine:

hf download t5-base --revision a9723ea7f1b39c1eae772870f3b547bf6ef7e6c1 --include config.json model.safetensors spiece.model tokenizer.json tokenizer_config.json special_tokens_map.json --local-dir dependencies/t5-base
hf download facebook/encodec_32khz --revision d0c45384f6c44db055f78200cfdcb9c1c8706727 --include config.json model.safetensors --local-dir dependencies/encodec
export LARAGEN_T5_PATH="$PWD/dependencies/t5-base"
export LARAGEN_CODEC_PATH="$PWD/dependencies/encodec"
export HF_HUB_OFFLINE=1
export TRANSFORMERS_OFFLINE=1

Then run generate.py as above. The loader uses FP32 modules with FP16 LM autocast on CUDA. Validated without xformers using PyTorch 2.1.1: CPU attention/streaming checks, CUDA FP16 streaming checks, and full 30-second generation on an NVIDIA A100 (32 kHz, mono, 960,000 samples).

License

Model weights: CC BY-NC 4.0, inherited from MusicGen. Code: MIT, with Meta/AudioCraft notices retained in adapted sources. External T5 and EnCodec retain their own licenses.