AKYLAI

Multilingual speech generation model.

Akylai is a Kyrgyz and Kazakh fine-tuned version of OpenMOSS-Team/MOSS-TTS-v1.5, released by nineninesix as standalone merged BF16 weights. Russian is also evaluated. The LoRA adapter from checkpoint 14500 is already merged; inference loads this repository directly. The external MOSS audio tokenizer is downloaded separately by the inference code.

Model details

Item	Value
Base model	OpenMOSS-Team/MOSS-TTS-v1.5
Distribution	Four Safetensors shards, merged BF16 weights
Adaptation	LoRA rank 32, alpha 64, dropout 0.05; 252 projection modules
Output	Mono speech at 24 kHz
Evaluated languages	Kyrgyz, Kazakh, Russian
License	Apache 2.0, matching the base model

Inference

The example follows the MOSS-TTS-v1.5 inference API, adapted to this merged checkpoint and the tested runtime. Always pass the known language explicitly: Kyrgyz, Kazakh or Russian. Use a local reference recording for voice cloning, or omit it for direct TTS.

Environment

Python 3.12 and a CUDA 12.8-compatible GPU runtime were used for validation. The pinned requirements include inference dependencies. The evaluated runtime was an RTX 5090 with 32 GB VRAM; the BF16 model and FP32 audio tokenizer occupied about 22.4 GiB before generation. Longer sequences need additional memory.

python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Command line

From a downloaded checkout of this repository:

python run_model/inference.py \
  --text "Саламатсызбы! Бүгүн биз кыргыз тилинде сүйлөйбүз." \
  --language Kyrgyz \
  --reference /path/to/reference.wav \
  --output /path/to/generated.wav

For Kazakh, set --language Kazakh; for Russian, set --language Russian. The default model path is the checkout itself. --model nineninesix/moss-tts-ky-kk loads the Hub model. The reference is read in full, converted to mono and resampled directly to 24 kHz. The CLI reports whether generation reached EOS. --max-new-tokens defaults to 4096; the benchmark used 512. A token-limit output may need a larger allowance or shorter input text.

Python API

The audio tokenizer is pinned to the same revision used for evaluation. The explicit waveform path below uses SoundFile and soxr and does not require TorchCodec.

import numpy as np
import soundfile as sf
import soxr
import torch
from huggingface_hub import snapshot_download
from transformers import AutoModel, AutoProcessor, set_seed

MODEL_ID = "nineninesix/akylai"  # Or a local checkout path.
CODEC_ID = "OpenMOSS-Team/MOSS-Audio-Tokenizer"
CODEC_REVISION = "3cd226ba2947efa357ef453bcad111b6eafba782"
device = "cuda"
torch.backends.cuda.enable_cudnn_sdp(False)
set_seed(42)

codec_path = snapshot_download(
    CODEC_ID, revision=CODEC_REVISION,
    allow_patterns=["*.json", "*.py", "*.safetensors", "*.txt"],
)
processor = AutoProcessor.from_pretrained(
    MODEL_ID, codec_path=codec_path, trust_remote_code=True,
)
processor.audio_tokenizer = processor.audio_tokenizer.to(device).eval()
model = AutoModel.from_pretrained(
    MODEL_ID, trust_remote_code=True, dtype=torch.bfloat16,
    device_map=device, attn_implementation="sdpa",
).eval()

wave, sr = sf.read("reference.wav", dtype="float32", always_2d=True)
wave = wave.mean(axis=1)
if sr != 24000:
    wave = soxr.resample(wave, sr, 24000, quality="HQ").astype(np.float32)

with torch.inference_mode():
    codes = processor.encode_audios_from_wav(
        [torch.from_numpy(wave).unsqueeze(0)], sampling_rate=24000, n_vq=32,
    )
    message = processor.build_user_message(
        text="Саламатсызбы! Бүгүн биз кыргыз тилинде сүйлөйбүз.",
        reference=codes, language="Kyrgyz",
    )
    batch = processor([[message]], mode="generation")
    outputs = model.generate(
        input_ids=batch["input_ids"].to(device),
        attention_mask=batch["attention_mask"].to(device),
        max_new_tokens=4096,
        text_temperature=1.5, text_top_p=1.0, text_top_k=50,
        audio_temperature=1.7, audio_top_p=0.8, audio_top_k=25,
        audio_repetition_penalty=1.0,
    )
    messages = processor.decode(outputs)
    audio = torch.cat([a.reshape(-1) for a in messages[0].audio_codes_list])
sf.write("generated.wav", audio.float().cpu().numpy(), 24000, subtype="FLOAT")

Evaluation against human ASR anchors

This checkpoint was evaluated on 2,600 utterances from Multilingual TTS Benchmark v3.1: 700 Kyrgyz, 400 Kazakh and 1,500 Russian. Every row uses its own native restored prompt_audio_orig, resampled directly from 48 to 24 kHz, and an explicit language tag. External reference recordings were not used.

ASR also makes errors on human recordings. The main comparison therefore reports human GT WER, synthesis WER on the same target utterances, and the difference in percentage points. The anchor is an empirical ASR baseline on this corpus, rather than a strict error floor or a causal separation of recognition errors from TTS errors.

Language	ASR	Matched N	Human GT WER %	Synthesis WER %	WER gap pp
Kyrgyz	GigaAM CTC	694	9.98	13.33	3.35
Kazakh	GigaAM CTC	400	11.79	16.26	4.48
Russian	Whisper large-v3	1500	5.43	7.23	1.8

Kyrgyz has 694 targets with human GT, so the matched comparison uses 694 rows on each side. The full 700-row Kyrgyz synthesis WER is 13.46%. Kazakh and Russian have complete GT coverage. Primary ASR follows the benchmark: GigaAM-Multilingual CTC for Kyrgyz/Kazakh and Whisper large-v3 for Russian.

Full synthesis metrics

Language	Primary ASR	N	WER %	WER 95% CI %	CER %	Exact transcript %
Kyrgyz	GigaAM CTC	700	13.46	11.86–15.23	5.63	57.0
Kazakh	GigaAM CTC	400	16.26	14.04–18.66	4.96	53.0
Russian	Whisper large-v3	1500	7.23	6.43–8.05	3.16	68.1

WER/CER use corpus edit-count aggregation, with benchmark text normalization and diacritics preserved. Intervals use 2,000 bootstrap resamples of prompt speakers, seed 20260725. They describe sampling uncertainty conditional on the selected ASR; they do not account for recognizer bias.

Additional ASR comparisons

Language	ASR	Matched N	Human GT WER %	Synthesis WER %	Difference pp
Kyrgyz	GigaAM CTC	694	9.98	13.33	3.35
Kyrgyz	MMS CTC	694	24.25	30.86	6.61
Kazakh	GigaAM CTC	400	11.79	16.26	4.48
Kazakh	MMS CTC	400	29.03	35.61	6.57
Russian	GigaAM CTC	1500	4.81	7.4	2.58
Russian	MMS CTC	1500	22.59	21.89	-0.7
Russian	Whisper large-v3	1500	5.43	7.23	1.8

Matched CER comparisons:

Language	ASR	Matched N	Human GT CER %	Synthesis CER %	CER difference pp
Kyrgyz	GigaAM CTC	694	2.93	5.53	2.59
Kyrgyz	MMS CTC	694	5.8	9.47	3.67
Kazakh	GigaAM CTC	400	2.63	4.96	2.32
Kazakh	MMS CTC	400	5.77	9.27	3.5
Russian	GigaAM CTC	1500	1.16	3.18	2.02
Russian	MMS CTC	1500	4.67	5.95	1.28
Russian	Whisper large-v3	1500	1.48	3.16	1.68

MMS CTC provides an independent reading for all three languages. GigaAM is an additional Russian diagnostic. Russian synthesis has slightly lower MMS WER than its human GT; this illustrates why the anchor is not a hard lower bound. Cross-ASR raw WER values are not interchangeable.

Speaker similarity

Language	Encoder	N	SIM	95% CI	Human anchor	Normalized SIM	Too short
Kyrgyz	ECAPA-TDNN	700	0.5683	0.5558–0.5804	0.5641	1.012	0
Kyrgyz	WavLM fine-tuned	700	0.5813	0.5680–0.5942	0.5795	1.005	0
Kyrgyz	WavLM-SV	700	0.9239	0.9141–0.9329	0.9267	0.985	0
Kazakh	ECAPA-TDNN	399	0.5423	0.5273–0.5566	0.5628	0.943	1
Kazakh	WavLM fine-tuned	399	0.5566	0.5440–0.5693	0.5796	0.938	1
Kazakh	WavLM-SV	399	0.9277	0.9171–0.9362	0.9333	0.971	1
Russian	ECAPA-TDNN	1499	0.6352	0.6285–0.6417	0.6348	1.001	1
Russian	WavLM fine-tuned	1499	0.6662	0.6597–0.6724	0.6684	0.995	1
Russian	WavLM-SV	1499	0.9433	0.9388–0.9473	0.9457	0.988	1

SIM compares generated speech with canonical 16 kHz prompt_audio from the same benchmark prompt speaker. All encoders use complete prepared waveforms and batch=1. The separate restored sim_ref_audio supplies the published same-speaker human anchor. Normalized SIM is (SIM − impostor floor mean) / (human anchor − impostor floor mean). Values near 1 indicate proximity to that anchor and can exceed 1; they are not naturalness MOS. The anchor comes from rows with second-reference audio, whereas synthesis SIM includes every embeddable output. One Kazakh and one Russian output are below the 0.4-second embedding threshold and are explicitly counted.

Reliability and limitations

Language	WER > 50%	Delivered duration > 2× GT	Delivered duration < 0.5× GT	Median duration ratio	Max duration ratio
Kyrgyz	54/700 (7.7%)	73/694 (10.5%)	4/694 (0.6%)	1.16	18.75
Kazakh	35/400 (8.8%)	12/400 (3.0%)	5/400 (1.2%)	1.06	12.63
Russian	41/1500 (2.7%)	72/1500 (4.8%)	10/1500 (0.7%)	1.12	23.08

All 2,600 outputs remain in the evaluation. Twelve outputs reached the 512-token cap: 5 Kyrgyz, 3 Kazakh and 4 Russian (0.46% overall). EOS alone does not guarantee a correct reading. Duration ratios above use delivered WAV duration and human GT duration on rows with GT. The ASR long-audio path was repaired to cover all samples; all 6,700 configured utterance–recognizer pairs are scored, with zero missing ASR cells. Previously valid transcripts were unchanged.

This run evaluates only the selected checkpoint. It does not quantify improvements over the unadapted base, establish superiority over other systems or audit training/evaluation overlap. Native-speaker listening evaluation is needed for naturalness, prosody and pronunciation claims beyond the reported ASR/SIM metrics.

Auxiliary acoustic diagnostics

Language	N	DNSMOS OVRL	DNSMOS SIG	DNSMOS BAK	UTMOS first 3 s	NISQA acoustic MOS
Kyrgyz	700	3.157	3.402	4.009	2.394	4.503
Kazakh	400	3.158	3.414	4.022	2.403	4.486
Russian	1500	3.25	3.491	4.067	2.573	4.599

These are diagnostic predictions, not validated human MOS for Kyrgyz, Kazakh or Russian. DNSMOS measures noise-suppression-related acoustic quality. UTMOSv2 in this run receives only the first three seconds; random inference windows were not seeded and late errors can be missed. NISQA uses nisqa.tar for transmitted-speech acoustic quality, rather than nisqa_tts.tar for TTS naturalness. These predictions do not support cross-language naturalness rankings.

Reproducibility

Aggregate measurements and evaluation settings are saved in evaluation/metrics.json. Dataset revision: bb8fa64c0a1732f1e8d2e1c73fb5e2933d8c6869, split main. The benchmark uses 24 kHz FLOAT synthesis, seed 42 with deterministic per-utterance seeds, and a 512-token generation cap. Text sampling: temperature 1.5, top-p 1.0, top-k 50. Audio sampling: temperature 1.7, top-p 0.8, top-k 25, repetition penalty 1.0. Evaluation synthesis preprocessing is mono, soxr HQ to 16 kHz, Silero VAD with 50 ms padding, and peak −1 dBFS. Canonical stored references are used without another VAD pass.

License and attribution

This fine-tuned model is released under Apache License 2.0, the same license as MOSS-TTS-v1.5. See LICENSE and NOTICE. Model architecture, tokenizer, audio codec and inference API originate from OpenMOSS. Fine-tuning, checkpoint selection and these evaluation measurements are by nineninesix. The original Apache headers remain in the custom model code.