Qwen3-ASR-Nepali

Fine-tuned Qwen3-ASR-1.7B for Nepali automatic speech recognition.

Best average WER among 8 tested open-source models across 3 cross-dataset benchmarks. Trained on 157 hours of public Nepali speech data for approximately $7 compute.

Benchmark Results

Cross-dataset evaluation on 3 diverse Nepali speech benchmarks, 100 samples each. None of these datasets were used during training.

Model	FLEURS	IndicVoices-R	OpenSLR-43	Average
Qwen3-ASR-Nepali (ours)	37.0%	55.8%	31.4%	41.4%
Meta MMS-1B (npi)	33.6%	62.4%	40.5%	45.5%
Whisper large-v3	94.0%	96.7%	105.8%	98.8%
Whisper-small-Nepali (amitpant7)	64.5%	77.7%	2.3%*	48.2%
wav2vec2-xlsr-300m (shniranjan)	43.3%	59.5%	33.9%*	45.5%
wav2vec2-nepali (anish)	54.3%	73.7%	4.6%*	44.2%
wav2vec2-xlsr (gagan)	70.8%	86.1%	5.0%*	54.0%
Qwen3-ASR-0.6B Base	116.0%	112.5%	100.4%	109.6%

Models marked with * show anomalously low OpenSLR-43 WER despite high WER on other datasets, suggesting dataset-specific overfitting or training-set overlap.

Datasets:

• FLEURS — Clean read speech (Google)
• IndicVoices-R — Spontaneous conversational speech, 2060 speakers (AI4Bharat, NeurIPS 2024)
• OpenSLR-43 — TTS-generated synthetic speech

Key Results

• #1 on spontaneous speech — beats MMS-1B by 6.6 points on IndicVoices-R
• #1 on synthetic speech — beats MMS-1B by 9.1 points on OpenSLR-43
• #2 on clean read speech — 3.4 points behind MMS-1B on FLEURS
• Best macro-average WER (41.4%) among all tested models

How to Use

from qwen_asr import Qwen3ASRModel
import torch

model = Qwen3ASRModel.from_pretrained(
    "sidskarki/Qwen3-ASR-Nepali",
    dtype=torch.float16,
    device_map="cuda"
)

result = model.transcribe("audio.wav")
print(result[0].text)

Training Details

Base model	Qwen3-ASR-1.7B
Training data	OpenSLR-54 (157h Nepali read speech, ~37K utterances)
Hardware	A100 80GB, single GPU
Best checkpoint	Step 2000
Batch size	16 (effective 128 with gradient accumulation)
Learning rate	2e-5, 3 epochs
Compute cost	~$7

Whisper Evaluation Bug

During benchmarking, we discovered that prior Whisper evaluations for Nepali were affected by a float16 dtype bug. Loading Whisper in float16 causes a dtype mismatch with the float32 processor output, and benchmark scripts using except: pass silently produce empty predictions, giving 100% WER. After fixing this (loading in float32), Whisper large-v3 produces Nepali text but still has high WER due to word boundary and spelling issues.

Links

• Code: github.com/sidskarkii/nepali-asr
• Case study: siddhantskarki.com/case-studies/nepali-asr
• Portfolio: siddhantskarki.com

Citation

@misc{karki2026nepali_asr,
  author = {Karki, Siddhant Singh},
  title = {Nepali ASR: Fine-tuning Qwen3-ASR with Cross-Dataset Evaluation},
  year = {2026},
  url = {https://github.com/sidskarkii/nepali-asr}
}