DarlingCat/whisper-small-ru-100h

Whisper Small Russian (100h Fine-Tuned)

This is a fine-tuned version of openai/whisper-small (241.7M parameters) adapted specifically for Russian Automatic Speech Recognition (ASR).

It was trained on ~109 hours of Russian speech composed of Mozilla Common Voice 27.0 (100h) and Sber Golos (9h) using a strictly speaker-disjoint split. The fine-tuned model delivers dramatic accuracy gains over the base model, reducing the overall Word Error Rate (WER) on Russian speech from 21.83% down to 9.39% (>55% relative error reduction).

---

Benchmark & Evaluation Results

All evaluations were performed with greedy decoding (num_beams=1) and standard Cyrillic normalization (lowercased, ё replaced by е, punctuation removed, whitespace collapsed).

Test Set Comparison: Baseline vs Fine-Tuned

Evaluated on the held-out test splits (2,000 clips / ~2.55 hours total):

Benchmark / Dataset	Audio Hours	Test Samples	Baseline WER (whisper-small)	Fine-Tuned WER	Relative WER Reduction	Baseline CER	Fine-Tuned CER
Mozilla Common Voice (ru)	1.43 h	1,000	14.24%*	6.49%	-54.4%	3.88%*	1.69%
Sber Golos	1.12 h	1,000	35.21%*	14.49%	-58.8%	18.25%*	3.79%
Overall (Macro Average)	2.55 h	2,000	24.72%*	10.49%	-57.6%	—	—
Overall (Micro Average)	2.55 h	2,000	21.83%*	9.39%	-57.0%	8.89%*	2.41%

* Baseline metrics measured on a matched 1,000-clip sample (503 Common Voice + 497 Golos).

Validation / Dev Set Performance

Evaluated on 800 validation clips (1.06 hours):

• Overall WER: 8.22%
• Overall CER: 2.16%
• Common Voice Dev WER: 6.51% (CER 1.84%)
• Sber Golos Dev WER: 13.09% (CER 3.17%)

---

Training Details

• Base Model: openai/whisper-small (241.7M parameters)
• Training Data: 78,033 clips (~108.95 hours)
  • Mozilla Common Voice 27.0 Russian: 70,040 clips (100.0 hours), speaker-disjoint partition
  • Sber Golos (crowd/audio): 7,993 clips (8.95 hours)
• Audio Preprocessing:
  • Sampling rate: 16 kHz mono
  • Clip duration filter: 0.5s min, 30.0s max
  • Log-mel spectrogram: 80 frequency bins
• Hyperparameters:
  • Optimizer: AdamW (weight decay: 0.01)
  • Learning Rate: 1e-5 with linear warmup (warmup ratio 0.1) and linear decay
  • Batch Size: 16 per device × 2 gradient accumulation steps = effective batch size 32
  • Mixed Precision: bfloat16 (bf16)
  • Gradient Checkpointing: Enabled
  • Training Duration: 3 epochs (7,317 optimizer steps)
  • Evaluation Interval: Every 1,000 steps on dev set
  • Best Model Selection: Step 7,317 selected based on lowest dev macro WER

---

Usage Examples

1. Fast Transcription using Hugging Face Pipeline (Recommended)

import torch
from transformers import pipeline

# Pick your device: GPU if available, otherwise CPU
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

# Load pipeline
pipe = pipeline(
    "automatic-speech-recognition",
    model="DarlingCat/whisper-small-ru-100h",
    torch_dtype=torch_dtype,
    device=device,
)

# Transcribe audio file (supports wav, flac, mp3, etc.)
result = pipe(
    "path/to/russian_audio.wav",
    generate_kwargs={"language": "ru", "task": "transcribe"},
    return_timestamps=False,
)

print("Transcription:", result["text"])

For long audio files (>30s), enable chunking:

result = pipe(
    "path/to/long_audio.mp3",
    chunk_length_s=30,
    stride_length_s=5,
    generate_kwargs={"language": "ru", "task": "transcribe"},
)
print("Transcription:", result["text"])

---

2. Direct Model & Processor Usage

import torch
import torchaudio
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor

model_id = "DarlingCat/whisper-small-ru-100h"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

# Load model & processor
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch_dtype,
    low_cpu_mem_usage=True,
).to(device)

# Load audio (16 kHz mono required)
waveform, sample_rate = torchaudio.load("path/to/russian_audio.wav")
if sample_rate != 16000:
    waveform = torchaudio.functional.resample(waveform, sample_rate, 16000)
if waveform.shape[0] > 1:
    waveform = waveform.mean(dim=0, keepdim=True)

# Prepare input features
input_features = processor(
    waveform.squeeze(0).numpy(),
    sampling_rate=16000,
    return_tensors="pt"
).input_features.to(device, dtype=torch_dtype)

# Generate transcription
with torch.inference_mode():
    predicted_ids = model.generate(
        input_features,
        language="ru",
        task="transcribe",
    )

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print("Transcription:", transcription)

---

3. Usage with faster-whisper (CTranslate2)

This model can be converted to CTranslate2 for ultra-fast CPU/GPU inference:

pip install ctranslate2 faster-whisper

# Convert checkpoint
ct2-transformers-converter --model DarlingCat/whisper-small-ru-100h --output_dir whisper-small-ru-ct2 --quantization float16

Then transcribe:

from faster_whisper import WhisperModel

model = WhisperModel("whisper-small-ru-ct2", device="cuda", compute_type="float16")
segments, info = model.transcribe("path/to/russian_audio.wav", language="ru")

for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

---

Intended Use & Limitations

• Domain: General conversational Russian speech, podcasts, voice commands, interviews.
• Limitations:
  • Accuracy may decrease on heavy regional dialects, specialized medical or legal terminology, or extreme background noise.
  • Whisper's standard hallucinations on pure silence or background music can occur; using Voice Activity Detection (VAD) like Silero VAD before feeding audio is recommended.

---

License

This model inherits the Apache 2.0 license from OpenAI's Whisper architecture. Training data conforms to Mozilla Common Voice (CC0/Terms) and Sber Golos (Apache 2.0) guidelines.