Akkimetry/milkledger-voice

MilkLedger voice: offline speech recognition and speaking voices for Hindi, Gujarati and Marathi

The speech models used by the MilkLedger app's voice assistant (Bolo), on the phone and offline.

Each language folder holds AI4Bharat's IndicConformer hybrid CTC-RNNT model for that language, converted with export_engine.py (in this repository) into three ONNX files for ONNX Runtime:

• encoder.int8.onnx: 16 kHz audio → encoder states and the CTC branch's log-probabilities (the log-mel front end is inside the model; network weights int8, front end in float)
• decoder.int8.onnx, joiner.int8.onnx: the RNNT prediction and joint networks for that language
• tokens.txt: the language's 256 word pieces, then the blank

voices/<lang>/ holds the assistant's speaking voice for that language (SYSPIN VITS: Hindi male, Gujarati female, Marathi female), converted with export_voice.py into model.onnx + tokens.txt for sherpa-onnx (Coqui character models). Each passed a round trip in MilkLedger's voice-eval: the voice says 150 test sentences and the speech engine above understands 90 % (Hindi), 84 % (Gujarati) and 94 % (Marathi) of them right first time (computer voices reading test sentences, not real speech).

en/moonshine-tiny/ and en/moonshine-base/ hold Useful Sensors' Moonshine English recognisers (MIT; int8, as exported by sherpa-onnx), used on English-language phones beside the Hindi model: tiny on phones with 3 GB of memory or less, base on bigger ones.

manifest.json lists every file's SHA-256 and size; the app checks them after downloading.

Credits and licence

The models are © 2024 AI4Bhārat, released under the MIT licence (see LICENSE): AI4Bharat/IndicConformerASR, ai4bharat/indicconformer_stt_*_hybrid_ctc_rnnt_large. This repository redistributes them, converted, under the same MIT licence; only the format changed.

The speaking voices are SYSPIN's VITS models (MIT; SYSPIN), trained on the SYSPIN dataset (SPIRE Lab, IISc Bengaluru and ARTPARK, CC BY 4.0); see LICENSE-SYSPIN. They are redistributed here converted, under the same MIT licence.

The English recognisers are Useful Sensors' Moonshine (MIT, © 2024 Useful Sensors; usefulsensors/moonshine), int8 as exported by sherpa-onnx; see LICENSE-MOONSHINE. Redistributed unchanged under the same MIT licence.