gopalanj/maya-voice-agent

Maya voice agent: LoRA adapters for MiniCPM-o 4.5

LoRA adapters that turn MiniCPM-o 4.5 into a phone agent for businesses that calls tools. It looks things up, books and cancels only after reading the details back, and hands emergencies to a person. Only the language model (Qwen3-8B inside MiniCPM-o) is adapted. The audio encoder, its projector and the speech decoder are unchanged.

Folder	Trained on	Steps
round1/	3,536 synthetic spoken conversations: a travel agent, plus general questions	442
round2/	8,807: round 1's data, new cities, a clinic receptionist, and generic tool use from ToolACE and When2Call	1,100

Exact tool turns (%)	Base	Round 1	Round 2
Round 1 test (seen businesses)	35	88	88
Round 2 test (held-out cities and clinics)	25	35	67
Car service centre (never trained)	25	53	79

LoRA rank 16, alpha 32, on all linear layers of the language model (43.6M parameters), trained with ms-swift for two epochs. Each folder holds the adapter, its ms-swift args.json and the trainer state. To use one, merge it with ms-swift (swift export --adapters round2 --merge_lora true), or use train/export.sh in the code repository, which also builds the GGUF for the patched llama.cpp-omni engine the paper uses.

• Code, generator and all conversation text: github.com/gopalanj/maya-voice-agent
• Test sets with audio: gopalanj/maya-voice-agent-tests
• Paper: link to follow.

Caveats. This is a research prototype. The tools are mocks, the businesses are fictional, and the training callers are synthetic speech. The models are not safety-tuned: they call tools on some harmful requests. Do not use them for real emergencies.