whisper-base-sk

Slovak fine-tune of openai/whisper-base by the KInIT team. Full parameter fine-tuning on a curated Slovak speech corpus, with noise augmentation for robustness to real-world recording conditions. Listed in SLAIH, a catalog of Slovak NLP resources.

Model Details

Property	Value
Base model	openai/whisper-base
Parameters	~73M
Architecture	Whisper encoder-decoder
Fine-tuning method	Full fine-tuning
Language	Slovak (sk)
Task	Automatic Speech Recognition
License	Apache 2.0

Intended Use

This model is intended for Slovak automatic speech recognition across a range of domains and recording conditions.

Out-of-scope: Non-Slovak audio, real-time streaming without appropriate chunking, safety-critical transcription without human review.

Evaluation

Word Error Rate (WER) and Character Error Rate (CER), lower is better. Measured on two Slovak eval sets:

• CV24 - Common Voice 24.0 test split (5,239 samples, public)
• Internal - held-out KInIT set (9,317 samples, stratified by domain and speaker gender, one third noise-augmented; not public)

Fine-tuned vs. base model

Model	CV24 WER ↓	CV24 CER ↓	Internal WER ↓	Internal CER ↓
kinit/whisper-base-sk	27.22%	9.78%	14.01%	6.32%
openai/whisper-base	87.61%	28.50%	76.38%	31.53%

Slovak fine-tuning reduces WER by 69% on Common Voice and 82% on the internal eval set, at the cost of multilingual performance (see Limitations).

Choosing a model size

All six Slovak fine-tunes, scored on the same eval sets with the same pipeline:

Model	Params	CV24 WER ↓	CV24 CER ↓	Internal WER ↓	Internal CER ↓
whisper-large-v3-sk	1.55B	7.68%	2.27%	5.67%	3.01%
whisper-large-v3-turbo-sk	809M	9.29%	2.55%	5.60%	2.65%
whisper-medium-sk	769M	10.57%	3.14%	6.28%	3.11%
whisper-small-sk	244M	14.15%	4.27%	8.64%	4.25%
whisper-base-sk	74M	27.22%	9.78%	14.01%	6.32%
whisper-tiny-sk	39M	38.24%	13.53%	20.04%	8.86%

Training Data

Fine-tuned on an internal curated Slovak speech corpus compiled at KInIT. The corpus combines public datasets with internal KInIT recordings. Recordings containing personal data were anonymised prior to use. Samples were quality-filtered using a CER-based threshold validated against multiple ASR models. During training, 75% of batches were additionally augmented on the fly with synthetic background noise for robustness to real-world recording conditions (see Training Procedure below).

Data sources
SloPalSpeech
Municipal council session recordings
Read literature
Mozilla Common Voice
TEDxSK and JumpSK Lecture Speech Corpus
FLEURS read speech
Internal KInIT recordings

Training Procedure

Hyperparameter	Value
Epochs	2
Learning rate	5e-5
LR scheduler	Linear with warmup
Optimizer	AdamW
Effective batch size	64
Precision	fp16
Noise augmentation	Applied on the fly to 75% of training batches: phone noise, background speech, background noise, white noise, and packet loss
Framework	HuggingFace Transformers Seq2SeqTrainer

Training was performed on the Devana HPC cluster.

Usage

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

model_id = "kinit/whisper-base-sk"

device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    dtype=dtype,
    use_safetensors=True,
).to(device)

processor = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    dtype=dtype,
    device=device,
)

result = pipe("audio.wav", generate_kwargs={"language": "slovak"})
print(result["text"])

License

Apache 2.0

Limitations

• Catastrophic forgetting: Fine-tuning exclusively on Slovak data significantly degrades performance on other languages. Use the base openai/whisper-base if multilingual transcription is required.
• Performance may degrade on strongly accented, dialectal, or domain-specific speech not represented in the training data.
• Maximum reliable single-segment length is 30 seconds without chunking.

Acknowledgements

Public datasets used in training: SloPalSpeech, Mozilla Common Voice, FLEURS, and the TEDxSK and JumpSK Lecture Speech Corpus (KEMT NLP).

(Part of the) Research results was obtained using the computational resources procured in the national project National competence centre for high performance computing (project code: 311070AKF2) funded by European Regional Development Fund, EU Structural Funds Informatization of society, Operational Program Integrated Infrastructure.