osmapi/tamil-asr-qwen3

Tamil ASR (Qwen3-ASR) — தமிழ் பேச்சு-to-எழுத்து

State-of-the-art Tamil Automatic Speech Recognition. An LLM-based speech-to-text model fine-tuned from Qwen/Qwen3-ASR-1.7B for the Tamil language (தமிழ்). On the public FLEURS-Tamil benchmark it outperforms AI4Bharat's IndicConformer (the prior Indic SOTA) on both WER and CER, and beats OpenAI's Whisper-large-v3 by a wide margin.

Keywords: Tamil ASR · Tamil speech to text · Tamil STT · தமிழ் speech recognition · Indic ASR · Dravidian ASR

---

🏆 Results — FLEURS-Tamil test set (586 utterances)

Model	WER ↓	CER ↓
Tamil-ASR-Qwen3 (this model)	25.27%	7.95%
IndicConformer-600M (AI4Bharat, prior SOTA)	25.49%	9.33%
Whisper-large-v3 (OpenAI)	48.58%	13.75%
Qwen3-ASR-1.7B (base, no Tamil support)	153.29%	113.24%

• 🥇 #1 on FLEURS-Tamil — beats IndicConformer on both WER (25.27 vs 25.49) and CER (7.95 vs 9.33).
• ✅ ~23 WER points better than Whisper-large-v3.
• ✅ An LLM-based ASR beating a purpose-built Conformer (CTC/RNNT) SOTA — with far less Tamil audio.
• All numbers use an identical text normalizer across every model, computed by us on the same 586 held-out clips (see Evaluation).

---

📌 Model description

• Task: Automatic Speech Recognition (speech → Tamil text)
• Language: Tamil (ta) only
• Architecture: Qwen3-ASR — audio encoder + Qwen3 LLM decoder (generative ASR), 1.7B parameters
• Base model: Qwen/Qwen3-ASR-1.7B
• Audio input: 16 kHz mono
• Decoding: beam search (num_beams=5) recommended

🗂️ Training data (~1,475 hours of Tamil)

Source	Hours	Style
IndicVoices (AI4Bharat)	803	natural / conversational
Shrutilipi (AI4Bharat)	462	read / news
Kathbath (AI4Bharat)	172	read
FLEURS-Tamil train + Tamil TTS	~38	read

🧪 Training procedure

A two-stage recipe:

1. Language adaptation — fine-tune the base on the large conversational corpus (IndicVoices) to teach Tamil; merge into the base weights.
2. Read-speech domain adaptation — LoRA (rank 64, α 128) on the combined read corpora (Shrutilipi + Kathbath + FLEURS-train + TTS), 2 epochs, lr 5e-5, bf16, then merged.

Trained with ms-swift on 4× NVIDIA H100. (Note: a full-parameter fine-tune was also tried and underperformed LoRA — LoRA r64 is the released configuration.)

📏 Evaluation

• Benchmark: google/fleurs, config ta_in, test split — 586 clips, fully held out (never seen in training; training corpora are separate from FLEURS-test, so there is no leakage).
• Metrics: WER and CER via jiwer.
• Fairness: the same punctuation-stripping / whitespace normalizer is applied to every model (this model, IndicConformer, Whisper, base) on the same clips. Decoding: beam search (num_beams=5).

🚀 How to use

With ms-swift:

pip install ms-swift transformers==4.57.6 qwen-asr soundfile
swift infer \
  --model osmapi/tamil-asr-qwen3 \
  --val_dataset your_data.jsonl \
  --infer_backend pt --max_new_tokens 256 --num_beams 5

Dataset JSONL format (one line per clip):

{"messages": [{"role": "user", "content": "<audio>"}, {"role": "assistant", "content": ""}], "audios": ["/path/to/clip.wav"]}

Audio should be 16 kHz mono.

✅ Intended uses & limitations

Intended: transcribing Tamil speech (read and conversational), Tamil voice interfaces, captioning, transcription pipelines.

Limitations:

• Tamil only — not trained for other languages.
• Benchmarked on read speech (FLEURS); very noisy/far-field/heavy-code-mixed audio may degrade.
• May reflect biases (domain, dialect, speaker) present in the training corpora.

🙏 Acknowledgements

• Base model: Qwen/Qwen3-ASR-1.7B (Alibaba Qwen team)
• Training data: AI4Bharat (IndicVoices, Shrutilipi, Kathbath) and Google FLEURS
• Reference/comparison: AI4Bharat IndicConformer, OpenAI Whisper-large-v3

📄 License

Apache-2.0 (inherits the base model's license). Please also respect the licenses of the training datasets.

✍️ Citation

@misc{tamil_asr_qwen3_2026,
  title  = {Tamil ASR (Qwen3-ASR): an LLM-based Tamil speech recognizer},
  author = {osmapi},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/osmapi/tamil-asr-qwen3}}
}