You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

HyperneuronAI Text-to-Speech Model

Multilingual Open-Source Text-to-Speech for Indian Languages

Model Details

Model Description

This is an open-source Text-to-Speech model developed by HyperneuronAI. The model is designed to generate natural speech from text and now supports all 22 officially recognized languages of India: Assamese, Bengali, Bodo, Dogri, Gujarati, Hindi, Kannada, Kashmiri, Konkani, Maithili, Malayalam, Manipuri, Marathi, Nepali, Odia, Punjabi, Sanskrit, Santali, Sindhi, Tamil, Telugu, and Urdu.

What's new: this release expands the original 4-language model (Hindi, Assamese, Punjabi, Kannada) to full coverage of all 22 scheduled Indian languages, using a replay-based fine-tuning strategy so quality on the original 4 languages is preserved while adding 18 new ones. See Training Details below.

Quipus is 2x Faster than realtime and lower latency TTFB ~120ms.^†

⚡Quipus Performance

Metric	Value
First Audio	~120 ms^†
Generation Speed	2× Realtime^†
Languages	22

^† Inherited from the prior 4-language release — architecture and generation loop are unchanged, so these should still hold, but they have not been re-benchmarked specifically against the 22-language checkpoint yet. Treat as provisional until re-verified.

Sneak Peak

In model Inference performance

🗺️ Roadmap

😊 Emotion-aware Speech
🌍 Arabic & English support as part of worldwide contribution
⚡ Streaming Backend Optimizations
Training and Inference Code

The model uses a Qwen3 backbone and is intended for research, experimentation, and building voice AI applications. Users are free to fine-tune the model for custom voices and additional languages.

Voice cloning capabilities are not provided with this release to encourage responsible AI usage.

Developed by: HyperneuronAI
Funded by: HyperneuronAI
Shared by: HyperneuronAI
Model type: Text-to-Speech
Backbone: Qwen3
Languages: Assamese, Bengali, Bodo, Dogri, Gujarati, Hindi, Kannada, Kashmiri, Konkani, Maithili, Malayalam, Manipuri, Marathi, Nepali, Odia, Punjabi, Sanskrit, Santali, Sindhi, Tamil, Telugu, Urdu
License: MIT

Audio Samples

Language	Sample
🇮🇳 Kannada
🇮🇳 Hindi
🇮🇳 Punjabi
🇮🇳 Assamese
🇮🇳 Tamil
🇮🇳 Telugu
🇮🇳 Bengali
🇮🇳 Bodo
🇮🇳 Dogri
🇮🇳 Gujarati
🇮🇳 Kashmiri
🇮🇳 Konkani
🇮🇳 Maithili
🇮🇳 Malayalam
🇮🇳 Manipuri
🇮🇳 Marathi
🇳🇵 Nepali
🇮🇳 Odia
🇮🇳 Sanskrit
🇮🇳 Santali
🇮🇳 Sindhi
🇮🇳 Urdu

Samples for the 18 newly added languages are not yet uploaded to this repo — the table above only has the original 4. Upload new .wav files to artifacts/ and add a row per language (or generate them fresh with the code in "How to Get Started" below) before this table reflects full 22-language coverage.

🎙️ Available Voices

Language	Speakers
🇮🇳 Assamese	Dipankar(Male) • Kavita(Female)
🇮🇳 Bengali	Arindam(Male) • Rupa(Female)
🇮🇳 Bodo	Daimalu(Male) • Mainao(Female)
🇮🇳 Dogri	Balwant(Male) • Shakti(Female)
🇮🇳 Gujarati	Nirav(Male) • Hetal(Female)
🇮🇳 Hindi	Raman(Male) • Anvita(Female)
🇮🇳 Kannada	Manjunath(Male) • Sindhura(Female)
🇮🇳 Kashmiri	Farooq(Male) • Habba(Female)
🇮🇳 Konkani	Ramnath(Male) • Shweta(Female)
🇮🇳 Maithili	Vidyapati(Male) • Mythili(Female)
🇮🇳 Malayalam	Unnikrishnan(Male) • Lakshmi(Female)
🇮🇳 Manipuri	Ibomcha(Male) • Chaoba(Female)
🇮🇳 Marathi	Sachin(Male) • Smita(Female)
🇮🇳 Nepali	Prakash(Male) • Sunita(Female)
🇮🇳 Odia	Jagannath(Male) • Sunanda(Female)
🇮🇳 Punjabi	Amanjit(Male) • Supreet(Female)
🇮🇳 Sanskrit	Aryan(Male) • Devavani(Female)
🇮🇳 Santali	Sido(Male) • Phulmani(Female)
🇮🇳 Sindhi	Hari(Male) • Kajal(Female)
🇮🇳 Tamil	Murugan(Male) • Kamala(Female)
🇮🇳 Telugu	Ravi(Male) • Padma(Female)
🇮🇳 Urdu	Salman(Male) • Zoya(Female)

44 speaker voices total (2 per language × 22 languages).

Model Sources

Repository: Realtime optimized streaming inference code is planned for a future release.
Demo: Coming soon.

Uses

Direct Use

This model can be used for Text-to-Speech generation in voice AI applications, including:

✅
Conversational AI
AI Calling
Voice Assistants
Accessibility
Customer Support
IVR
Edge Devices
Research

Fine-Tuning

Users may fine-tune this model for:

New speakers
Domain-specific speech styles
Additional languages beyond the current 22
Custom application-specific voices

Out-of-Scope Use

This model should not be used for unethical, harmful, deceptive, or illegal purposes, including but not limited to:

Impersonation without consent
Fraudulent voice generation
Misinformation or manipulation
Harassment or abuse
Any use that violates applicable laws or platform policies

HyperneuronAI is not responsible for misuse of this model by third parties.

How to Get Started

vLLM code examples.

#Install libraries if not installed
# Download vlllm based on cuda version
#!pip install vllm==0.19.0
#!pip install -q soundfile

# ==========================================
# vLLM SERVING - Colab-safe audio saving
# ==========================================
"""
You can use vllm for inferencing as it provides optimised control over cuda profiling and inturn better response time.
"""
import time
import re
import torch
import numpy as np
import soundfile as sf
from IPython.display import Audio, display

from vllm import LLM, SamplingParams
from snac import SNAC

VLLM_MODEL_PATH = "hyperneuronAILabs/quipus-0.6-speechv1"
DEFAULT_SPEAKER = "Anvita"  # any of the 44 voices in "Available Voices" above
FRAME_LAYER_PATTERN = [0, 1, 2, 2, 1, 2, 2]
SAMPLE_RATE = 24000

llm = LLM(
    model=VLLM_MODEL_PATH,
    dtype="bfloat16",
    max_model_len=2048,
    gpu_memory_utilization=0.20,
    enforce_eager=False,
)

tok = llm.get_tokenizer()

AUDIO_END_ID = tok.convert_tokens_to_ids("<audio_end>")

snac_decoder = (
    SNAC.from_pretrained("hubertsiuzdak/snac_24khz")
    .eval()
    .cuda()
)

def build_prompt_prefix(speaker, text):
    return f"{speaker}: {text} <audio_start> "

def quipus_tts(
    text,
    speaker=DEFAULT_SPEAKER,
    out="vllm_out.wav",
    temperature=0.7,
    top_p=0.9,
):
    prompt = build_prompt_prefix(speaker, text)

    sampling_params = SamplingParams(
        temperature=temperature,
        top_p=top_p,
        repetition_penalty=1.1,
        max_tokens=1024,
        stop_token_ids=[AUDIO_END_ID],
    )

    t0 = time.time()
    result = llm.generate([prompt], sampling_params)
    gen_time = time.time() - t0

    out_ids = list(result[0].outputs[0].token_ids)
    toks = tok.convert_ids_to_tokens(out_ids)

    parsed = []
    for token in toks:
        match = re.fullmatch(r"<snac_l(\d+)_c(\d+)>", token)
        if match:
            parsed.append((int(match.group(1)), int(match.group(2))))

    l0, l1, l2 = [], [], []
    i = 0

    while i + 7 <= len(parsed):
        window = parsed[i:i + 7]

        if [layer for layer, _ in window] == FRAME_LAYER_PATTERN:
            codes = [code for _, code in window]

            l0.append(codes[0])
            l1.extend([codes[1], codes[4]])
            l2.extend([codes[2], codes[3], codes[5], codes[6]])

            i += 7
        else:
            i += 1

    if not l0:
        print("No valid SNAC frames from vLLM output.")
        return None

    with torch.inference_mode():
        wav = snac_decoder.decode(
            [
                torch.tensor([l0], dtype=torch.long, device="cuda"),
                torch.tensor([l1], dtype=torch.long, device="cuda"),
                torch.tensor([l2], dtype=torch.long, device="cuda"),
            ]
        )

    audio_sec = wav.shape[-1] / SAMPLE_RATE

    pcm = wav.detach().squeeze().float().cpu().numpy()
    pcm = np.clip(pcm, -1.0, 1.0)

    sf.write(out, pcm, SAMPLE_RATE)

    n = len(out_ids)

    display(Audio(out, rate=SAMPLE_RATE))

    return out

quipus_tts(
    "यह मॉडल स्पीच-टू-टेक्स्ट मॉडल है , जिसे निखिल ने विकसित किया है। এই মডেলটো হৈছে স্পিচ টু টেক্সট মডেল, নিখিলে বিকশিত কৰিছে",
    speaker=DEFAULT_SPEAKER,
)

Training Details

Training Data

The model was fine-tuned on the AI4Bharat Rasa dataset, covering all 22 officially recognized languages of India:

Training Procedure

Base checkpoint: this repo's prior release (4-language: Hindi, Assamese, Punjabi, Kannada).
Method: full parameter fine-tuning — all 608,362,496 parameters trained.
Audio representation: SNAC neural codec at 24kHz, hierarchical 3-layer codebook, flattened into a 7-token-per-frame sequence (Orpheus-style interleave) so all three resolution layers stay time-aligned.
Prompt format: "{speaker}: {text} <audio_start> {snac_tokens} <audio_end>" — loss is computed only on the audio-token portion; the text prompt is masked out.
Tokenizer: unchanged from the base checkpoint — the SNAC control tokens were already present in its vocabulary, so no vocabulary resize was needed this round.

Training Hyperparameters

Training hyperparameters will be added in a future update.

Evaluation

Testing Data

Informal qualitative listening checks were run across all 22 languages using fixed test phrases and each language's paired speakers. A formal held-out evaluation set has not yet been built for this release.

Metrics

Formal evaluation metrics such as MOS, speaker similarity, intelligibility, word error rate, and latency benchmarks are not yet published for the 22-language checkpoint.

Results

Evaluation results will be added after broader testing and benchmarking.

Technical Specifications

Model Architecture and Objective

This is a Text-to-Speech model with a Qwen3 backbone. The model is optimized to generate speech from input text in supported Indian languages.

Further architecture details will be added in future documentation.

Compute Infrastructure

Compute details will be added in a future update.

Software

Software and inference dependencies will be added with the official inference code.

Limitations

Output quality may vary depending on language, text normalization, punctuation, and input style.
The model may struggle with code-mixed text, rare words, abbreviations, numerals, and domain-specific terminology.
Voice cloning is not included in this release.
Realtime streaming inference code is planned but not included yet.

Ethical Considerations

This model is released to support open-source development of Indian language voice AI. Users should ensure responsible deployment, obtain consent where required, and avoid deceptive or harmful applications.