sajilck/whisper-small-malayalam

Whisper Small Malayalam

Fine-tuned version of openai/whisper-small on a multi-corpus Malayalam speech dataset. This is the first publicly available Whisper model fine-tuned specifically for Malayalam ASR.

Model Description

• Base model: openai/whisper-small (244M parameters)
• Language: Malayalam (ml)
• Task: Automatic Speech Recognition (transcription)
• Training steps: 3500
• Best WER: 37.64% on CommonVoice 25 Malayalam test set

Training Data

The model was trained on an aggregated corpus of 5 Malayalam speech datasets, combined and published as sajilck/malayalam-asr-corpus.

Corpus	Source	Domain	Access
IMaSC	thennal/imasc	TTS / Read speech	HuggingFace
SMC Malayalam Speech Corpus	sajilck/smc-malayalam-speech-corpus	Read speech	Kaggle
IndicTTS Malayalam	kavyamanohar/indic-tts-malayalam-speech-corpus	TTS / Read speech	Kaggle
OpenSLR 63	sajilck/openslr63	Crowdsourced	Kaggle
CommonVoice 25 Malayalam	sajilck/common-voice-malayalam	Crowdsourced	Kaggle

Total: ~86,000 samples across TTS-recorded, read speech, and crowdsourced domains.

Benchmark Results

Model	Params	WER ↓	Notes
openai/whisper-small (base)	244M	~85%	No Malayalam fine-tuning
smcproject/Malwhisper-v1-medium	769M	61.84%	Single corpus (IMaSC only)
sajilck/whisper-small-malayalam	244M	37.64%	Multi-corpus fine-tuning

Key advantages over prior work:

• 3× smaller model than Malwhisper-v1-medium, better WER
• 5 corpora vs 1 — better speaker and domain diversity
• Multi-domain training — TTS, read speech, and crowdsourced audio

Usage

from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="sajilck/whisper-small-malayalam",
    generate_kwargs={"language": "malayalam", "task": "transcribe"},
)

result = pipe("your_audio.wav")
print(result["text"])

For longer audio files:

pipe = pipeline(
    "automatic-speech-recognition",
    model="sajilck/whisper-small-malayalam",
    generate_kwargs={"language": "malayalam", "task": "transcribe"},
    chunk_length_s=30,
    stride_length_s=5,
)
result = pipe("long_audio.wav")
print(result["text"])

Training Details

Parameter	Value
Base model	openai/whisper-small
Training steps	3500
Effective batch size	16 (batch=4, grad_accum=4)
Learning rate	1e-5
Warmup steps	500
Precision	fp16
Hardware	NVIDIA Tesla P100 16GB
Framework	HuggingFace Transformers + Seq2SeqTrainer

Training Data Preprocessing

Audio from all 5 corpora was:

• Resampled to 16kHz mono
• Filtered to 0.5–30 second clips
• Converted to log-mel spectrograms (80 mel bins)
• Tokenized using Whisper's multilingual tokenizer with language token <|ml|>

Limitations

• Trained on read speech and crowdsourced audio — may perform worse on spontaneous conversational Malayalam
• Foreign proper nouns (English names, place names) may be transcribed with Malayalam phonetic approximations
• Performance may vary across Malayalam dialects

Future Work

• v2: Adding Shrutilipi (AI4Bharat broadcast news corpus) for improved domain coverage and lower WER
• whisper-medium-malayalam and whisper-tiny-malayalam variants
• GGUF export for CPU inference via whisper.cpp

Citation

@misc{sajilck2026whispermalayalam,
  author = {Sajil C.K.},
  title = {Whisper Small Malayalam: Multi-Corpus Fine-Tuning of Whisper for Malayalam ASR},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/sajilck/whisper-small-malayalam}
}

License

This model is released under the Apache 2.0 license, consistent with the base Whisper model. Training corpora retain their individual licenses — please refer to each source dataset for usage terms.