ashishkblink/vakya-asr

Vakya

Speech to text, and speech to decisions, in one open model.

Vakya is a rebranded and redistributed copy of Hima-ASR by Prahaladha (the Hima authors), published by Hustle. The model architecture, code, and weights are unchanged from the original Hima-ASR release, except for the README and the added name. The text decoder is a derivative of Gemma 3 270M and is subject to the Gemma Terms of Use. See LICENSE and NOTICE for the full terms.

Hima-ASR is a compact 0.46B model for 11 Indian languages and English. It writes down what was said, and its decision head answers questions about how it was said: the speaker's tone and emotion, what a caller wants, what they're talking about, how urgent it is, which language it is and who is speaking. You ask in plain English, and the answers come from the audio itself.

Speech to text	Assamese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu, English
Speech to decisions	any question you write in plain English, answered with a probability for every option
One call	transcript and decisions together from a single listen, under half a second per clip on a laptop GPU
Size	0.46B ASR (920 MB) + 17.5M decision head
Runs on	GPU, CPU (ONNX Runtime) and Mac (MLX), with 8-bit and 4-bit versions
Input	16 kHz mono audio, 1 to 30 seconds per clip

Quick start

pip install "transformers>=5.13" torch torchaudio soundfile safetensors huggingface_hub

Speech to text

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained("Prahaladha/hima-asr", trust_remote_code=True, dtype=torch.float32)
model = model.to("cuda").eval()                                     # or leave on CPU

print(model.transcribe("call.wav", language="hi"))                 # one file -> one string
print(model.transcribe(["a.wav", "b.wav"], language="ta"))          # list -> list
print(model.transcribe("call.wav"))                                 # the model detects the language
print(model.transcribe("call.wav", language="hi", style="mixed"))   # English words in Latin letters

Argument	Values	Default
language	as bn en gu hi kn ml mr or pa ta te, or None	None (detected)
style	native (own script), mixed (English words in Latin letters)	native
batch_size	clips decoded together	16

Trimmed or full piece table. The default (main) uses a piece table trimmed to the 63,335 pieces Hima writes: 0.46B parameters, the same transcripts, a bit faster. The original full table (262,144 pieces, 0.59B) is on the full-vocab branch:

model = AutoModel.from_pretrained("Prahaladha/hima-asr", revision="full-vocab", trust_remote_code=True, dtype=torch.float32)

Speech to text and decisions, in one call

import sys
from huggingface_hub import snapshot_download
sys.path.insert(0, snapshot_download("Prahaladha/hima-asr", allow_patterns=["decide/*"]) + "/decide")
from decide import Hima

hima = Hima("both")          # "asr" = transcripts only, "decide" = decisions only, "both" = both from one listen

result = hima.run("call.wav", [
    {"q": "How does the speaker sound?", "options": ["calm", "polite", "angry", "confused", "rushed", "cheerful"]},
    {"q": "What is the caller doing?", "options": ["will pay", "already paid", "cannot pay now", "call me later", "wrong person"]},
    {"q": "Does the speaker ask to talk to a human?", "type": "yesno"},
    {"q": "How urgent is it?", "type": "scale", "options": ["not urgent", "somewhat urgent", "very urgent"]},
], language="hi")

result["text"]               # the transcript
result["decisions"]          # one {option: probability} per question

The audio encoder runs once per clip, and the writer and the decision head both read its output. run also takes a list of clips, and picks CUDA, then a Mac GPU, then CPU on its own.

Fast versions: ONNX and MLX

Hima-ASR also ships ready-made for CPUs (ONNX Runtime) and Apple Silicon GPUs (MLX), in full precision and quantized, each in its own repo. They need no PyTorch: just numpy, the runtime and tokenizers. Each one includes the decision head too: model.run(clip) gives the transcript and the decisions from one listen, and model.decide(clip, questions) answers your own questions.

Runs on	Version	Repo	Size	Speed (times realtime)
CPU	PyTorch	this repo	920 MB	████░░░░░░░░░░░░░░░░ 16x
CPU	ONNX fp32	hima-asr-ONNX	1.8 GB	███████░░░░░░░░░░░░░ 25x
CPU	ONNX int8	hima-asr-ONNX	487 MB	██████████░░░░░░░░░░ 35x
CPU	ONNX q4	hima-asr-ONNX	360 MB	█████████░░░░░░░░░░░ 31x
Mac GPU	PyTorch	this repo	920 MB	██████░░░░░░░░░░░░░░ 21x
Mac GPU	MLX fp16	hima-asr-mlx	1.0 GB	█████████████████░░░ 63x
Mac GPU	MLX 8-bit	hima-asr-mlx-8bit	536 MB	████████████████████ 73x
Mac GPU	MLX 4-bit	hima-asr-mlx-4bit	379 MB	████████████████████ 73x

Measured on a MacBook Pro (M4 Pro).

CPU, with ONNX Runtime

pip install onnxruntime numpy tokenizers soundfile huggingface_hub
hf download Prahaladha/hima-asr-ONNX hima_onnx.py --local-dir .

from hima_onnx import HimaONNX

model = HimaONNX("Prahaladha/hima-asr-ONNX", precision="q4")   # or "int8", "fp32"
print(model.transcribe("call.wav", language="hi"))

Mac, with MLX

pip install mlx mlx-lm numpy tokenizers soundfile huggingface_hub
hf download Prahaladha/hima-asr-mlx-8bit hima_mlx.py --local-dir .

from hima_mlx import HimaMLX

model = HimaMLX("Prahaladha/hima-asr-mlx-8bit")   # or hima-asr-mlx-4bit, hima-asr-mlx
print(model.transcribe("call.wav", language="hi"))

Both take 16 kHz audio files or arrays (pip install librosa to pass other sample rates), and language=None lets the model detect the language.

Ask anything

A decision is any question with a set of answers:

Type	Example
choice (default)	{"q": "Which emotion best describes the speaker?", "options": ["calm", "happy", "frustrated", "worried", "apologetic"]}
yesno	{"q": "Is the speaker complaining about something?", "type": "yesno"}
scale	{"q": "How formal is the speaker's language?", "type": "scale", "options": ["very casual", "neutral", "very formal"]}

Write the question the way you'd ask a person who just heard the clip. Options can carry a short description: {"options": {"callback": "asks to talk later", "wrong_person": "says this is the wrong number"}}.

Call hima.run("call.wav") with no questions to get a ready-made set of 16 (tone, emotion, sentiment, intent, topic, urgency, formality, language, speaker and more), listed in decide/questions.json.

Decisions in action

Click to see 10 examples with audio: speakers in Hindi, Telugu, Tamil, Marathi, Gujarati, Malayalam and English, and every decision the model made

Every example below is a held-out clip the model never trained on, with the probabilities straight from the model.

Hindi · an angry caller

“Don't mess with my head this early in the morning. I told you, no Rahul lives here! I'm hanging up!”

Decision	Answer	Confidence	Next best
Sentiment	negative	████████████████████ 100%	neutral 0%
Emotion	frustrated	████████████████████ 99%	worried 1%
Complaining?	yes	███████████████████░ 97%	no 3%
Agrees or disagrees?	disagrees	█████████████████░░░ 85%	neither 11%
Asking you to act?	yes	██████████████████░░ 90%	no 10%
Talks about money?	no	████████████████████ 98%	yes 2%
Formality	very casual	████████████████████ 99%	neutral 1%
Wants a human agent?	no	███████████████████░ 95%	yes 5%

Hindi · read speech: who is speaking, and about what

“Most of the work was theoretical, but the program was written to simulate observations of the Sagittarius galaxy.”

Decision	Answer	Confidence	Next best
Language	Hindi	████████████████████ 100%	Bengali 0%
Speaker	man	████████████████████ 100%	woman 0%
Topic	science / nature	█████████████████░░░ 86%	other 6%
What the caller is doing	giving information	██████████████████░░ 92%	other 8%
Emotion	calm	███████████████████░ 94%	frustrated 3%
Urgency	not urgent	████████████████████ 98%	somewhat urgent 1%
Asking a question?	no	████████████████████ 98%	yes 2%
Talks about money?	no	███████████████████░ 97%	yes 3%

Telugu · a caller in distress

“Why do you call every day and torment me? I had a road accident and I'm stuck in bed. Where am I supposed to bring the money from?”

Decision	Answer	Confidence	Next best
Tone of voice	angry	███████████████████░ 95%	frustrated 5%
Sentiment	negative	████████████████████ 100%	neutral 0%
Emotion	frustrated	████████████████████ 98%	worried 2%
Asking a question?	yes	███████████████████░ 94%	no 6%
Complaining?	yes	████████████████████ 98%	no 2%
Formality	very casual	██████████████████░░ 92%	neutral 8%
Wants a human agent?	no	███████████████████░ 95%	yes 5%
Language	Telugu	████████████████████ 100%	Urdu 0%

Telugu + English · a caller in a hurry

“Hello, I'm driving... when is the due date?”

Decision	Answer	Confidence	Next best
Tone of voice	rushed	████████████████████ 100%	cheerful 0%
What the caller is doing	asking for details	████████████████████ 99%	other 1%
Asking a question?	yes	████████████████████ 98%	no 2%
Agrees or disagrees?	neither	████████████████████ 100%	disagrees 0%
Sentiment	neutral	███████████████████░ 97%	negative 3%
Complaining?	no	█████████████████░░░ 87%	yes 13%
Wants a human agent?	no	███████████████████░ 96%	yes 4%
Language	Telugu	████████████████████ 100%	Urdu 0%

Tamil · “I've already paid”

“Sir, I think you've called by mistake. I paid the ₹7,000 credit card bill on the 15th itself.”

Decision	Answer	Confidence	Next best
What the caller is doing	already paid / disputes	██████████████████░░ 91%	will pay 8%
Talks about money?	yes	████████████████████ 100%	no 0%
Mentions a time?	yes	███████████████████░ 95%	no 5%
Topic	money	████████████████████ 100%	service 0%
Tone of voice	calm	███████████████████░ 95%	polite 4%
Formality	very casual	███████████████████░ 96%	neutral 4%
Wants a human agent?	no	███████████████████░ 97%	yes 3%
Language	Tamil	████████████████████ 100%	Hindi 0%

Tamil + English · a frustrated caller

“Hello, what does it mean to call ten times a day? Don't you have any manners?”

Decision	Answer	Confidence	Next best
Tone of voice	frustrated	████████████████████ 99%	angry 1%
Asking a question?	yes	███████████████████░ 95%	no 5%
Mentions a time?	yes	██████████████████░░ 90%	no 10%
Sentiment	negative	████████████████░░░░ 79%	neutral 20%
Complaining?	yes	████████████████░░░░ 81%	no 19%
Formality	very casual	███████████████████░ 96%	neutral 4%
Talks about money?	no	██████████████████░░ 90%	yes 10%
Language	Tamil	████████████████████ 100%	Telugu 0%

Marathi · paid, and angry about the calls

“What is going on? I've already paid and you still keep calling to collect? Now I'll drag you to court!”

Decision	Answer	Confidence	Next best
Tone of voice	angry	████████████████████ 98%	frustrated 2%
Sentiment	negative	████████████████████ 100%	neutral 0%
Emotion	frustrated	████████████████████ 99%	worried 1%
What the caller is doing	already paid / disputes	██████████████░░░░░░ 70%	other 10%
Complaining?	yes	████████████████████ 99%	no 1%
Talks about money?	yes	████████████████████ 98%	no 2%
Agrees or disagrees?	disagrees	█████████████████░░░ 83%	neither 13%
Formality	very casual	███████████████████░ 96%	neutral 4%

Gujarati · a confused caller disputing a charge

“But I pay the full amount every month, so where did this extra interest come from? I don't understand at all.”

Decision	Answer	Confidence	Next best
Tone of voice	confused	████████████████████ 100%	anxious 0%
What the caller is doing	already paid / disputes	█████████████████░░░ 85%	cannot pay now 6%
Asking a question?	yes	███████████████████░ 95%	no 5%
Talks about money?	yes	████████████████████ 100%	no 0%
Complaining?	yes	███████████████████░ 96%	no 4%
Topic	money	████████████████████ 100%	service 0%
Sentiment	negative	████████████████████ 99%	neutral 1%
Wants a human agent?	no	███████████████████░ 97%	yes 3%

Malayalam · “call me later”

“I'm in a big meeting. Call at four.”

Decision	Answer	Confidence	Next best
What the caller is doing	call me later	███████████████████░ 97%	other 1%
Mentions a time?	yes	██████████████████░░ 92%	no 8%
Asking you to act?	yes	██████████████████░░ 91%	no 9%
Tone of voice	calm	███████████████████░ 95%	rushed 4%
Sentiment	neutral	██████████████████░░ 91%	negative 8%
Complaining?	no	████████████████░░░░ 81%	yes 19%
Talks about money?	no	███████████████████░ 95%	yes 5%
Language	Malayalam	████████████████████ 100%	Urdu 0%
Wants a human agent?	no	█████████████████░░░ 85%	yes 15%

English · a loan question

“Good morning! Can we talk in Malayalam? Ramesh here, regarding my home loan EMI.”

Decision	Answer	Confidence	Next best
Language	English	████████████████████ 99%	Hindi 1%
Asking a question?	yes	███████████████████░ 97%	no 3%
Topic	money	████████████████████ 100%	daily life 0%
Talks about money?	yes	███████████████████░ 97%	no 3%
Emotion	calm	█████████████████░░░ 87%	worried 8%
Complaining?	no	██████████████████░░ 91%	yes 9%
Agrees or disagrees?	neither	███████████████████░ 96%	agrees 3%
Urgency	not urgent	█████████████████░░░ 83%	somewhat urgent 17%

How often it's right

On held-out test clips in 12 Indian languages and English that the model never trained on:

Decision	Right
Speaker is a man or a woman	12 / 12
Wants a human agent	34 / 34
Talks about money	34 / 35
Language (out of 13)	35 / 36
Mentions a date or time	33 / 35
Urgency	27 / 29
Asking a question	30 / 32
Asking the listener to do something	31 / 34

The caller examples are acted replies in synthetic voices; the read-speech examples are real recordings (FLEURS, CC BY 4.0).

Benchmarks

Hima-ASR is the best of the four at South Indian languages. It has the lowest error in Telugu, Tamil, Kannada and Malayalam, and on 9 of their 11 test sets, at well under half the size of Flex and Core.

	Telugu	Tamil	Kannada	Malayalam
Hima-ASR (0.46B)	13.4	17.4	11.1	15.0
Flex (1.2B)	15.0	20.9	13.5	16.8
Core (1.2B)	14.6	19.8	13.0	16.5
Sarvam Saaras v3	15.5	20.5	11.7	16.7

Word error rate (WER, %, lower is better) against Indic-Transcribe-Flex, Indic-Transcribe-Core and Sarvam Saaras v3 (API). Every model is scored on the same clips with the same scorer, with the language given and text compared after standard normalisation (punctuation removed, Unicode NFC). Best result per row in bold.

Language	Test set	Hima-ASR (0.46B)	Flex (1.2B)	Core (1.2B)	Sarvam Saaras v3
All 12 languages	Overall average	12.95	13.23	12.84	12.79
All 12 languages	FLEURS average	12.7	12.7	12.3	12.0
Telugu	Overall	13.4	15.0	14.6	15.5
	Kathbath	12.9	12.8	11.6	13.7
Tamil	Overall	17.4	20.9	19.8	20.5
	Kathbath	15.4	14.1	13.8	15.0
	MILE	14.0	18.8	17.3	18.7
	SPRING-INX round 1	36.4	46.5	44.7	39.6
	SPRING-INX round 2	37.5	50.5	48.1	40.9
Kannada	Overall	11.1	13.5	13.0	11.7
	Kathbath	9.5	11.7	10.4	11.4
	MILE	13.8	15.8	15.0	15.5
	SPRING-INX round 1	49.0	58.0	57.5	53.8
Malayalam	Overall	15.0	16.8	16.5	16.7
	Kathbath	19.9	26.8	23.6	25.2
	SPRING-INX round 1	47.3	49.0	47.9	47.8
	SPRING-INX round 2	50.8	52.9	52.4	51.9
Hindi	Overall	14.2	13.2	13.1	16.6
	FLEURS	5.7	4.3	4.2	4.9
	Gram Vaani (phone-quality, rural)	17.2	16.5	16.4	20.1
	Kathbath	3.4	2.5	2.6	3.4
	Lahaja (many accents)	8.8	8.1	8.0	12.1
	Monsoon (conversational)	11.1	8.0	7.6	9.9
Bengali	Overall	12.3	10.9	10.0	10.5
	Kathbath	8.3	5.4	5.1	7.2
	SPRING-INX round 1	26.2	26.9	26.4	26.8
	SPRING-INX round 2	27.3	27.8	27.3	28.1
Gujarati	Overall	10.2	10.5	10.1	10.6
	Kathbath	7.3	5.4	5.2	6.7
	SPRING-INX round 1	35.9	37.2	35.9	36.4
	SPRING-INX round 2	39.6	41.1	39.9	39.9
Marathi	Overall	13.9	13.4	13.4	12.0
	Kathbath	11.0	8.8	8.7	9.9
	SPRING-INX round 1	38.7	39.6	38.3	37.1
	SPRING-INX round 2	34.2	38.6	34.0	33.0
Odia	Overall	12.5	12.9	12.3	11.1
	Kathbath	8.3	10.0	8.7	8.6
	SPRING-INX round 1	35.3	40.0	38.8	37.3
Punjabi	Overall	10.9	9.4	9.1	9.7
	Kathbath	8.1	7.1	6.9	7.7
	SPRING-INX round 1	30.3	32.8	30.9	30.6
	SPRING-INX round 2	33.1	34.7	33.4	33.3
Assamese	Overall	17.0	15.8	15.2	14.7
	SPRING-INX round 1	35.6	38.5	37.9	38.9
	SPRING-INX round 2	33.4	35.6	34.9	37.2
English	Overall	7.5	6.4	6.9	4.0
	Monsoon (conversational)	4.2	6.1	6.3	3.5
	Svarah (Indian-accented)	11.4	6.8	7.8	4.7

Architecture

Hima-ASR has four parts:

1. Audio encoder: turns 16 kHz speech into acoustic frames (12.5 per second). Initialised from the audio encoder of Qwen3-ASR-1.7B.
2. Projector: maps the audio encoder's frames into the text decoder's input space.
3. Text decoder: writes the transcript piece by piece (greedy). Initialised from Gemma 3 270M. Its piece table keeps only the 63,335 pieces Hima writes (Indian scripts and English), out of Gemma's 262,144.
4. Decision head (decide/, 17.5M): reads a learned mix of every encoder and decoder layer over the audio, together with the question and its options, and scores each option. Its attention runs over the audio frames, so it hears tone, pauses and voice as well as content.

Files

File	What it is
model.safetensors	weights (bf16, 920 MB, trimmed piece table)
config.json, configuration_hima_asr.py	configuration
modeling_hima_asr.py	model code (loaded with trust_remote_code=True)
preprocessor_config.json	audio feature settings
tokenizer.json, tokenizer_config.json	text decoder tokenizer
decide/	decision head: head.safetensors, decide.py (the Hima class), questions.json, demo clips

License

• Hima-ASR's own parts (projector, training changes, inference code): Apache 2.0, see LICENSE.
• Audio encoder: initialised from Qwen3-ASR-1.7B, Apache 2.0.
• Text decoder: initialised from Gemma 3 270M. Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms, including its use restrictions. Hima-ASR is a Model Derivative of Gemma, so any use or redistribution of Hima-ASR must follow those terms. See NOTICE.