TalTechNLP/Voxtral-Mini-3B-2507-estonian

Voxtral Mini 3B 2507 Estonian

This model is a full-parameter fine-tune of mistralai/Voxtral-Mini-3B-2507 for Estonian speech recognition and audio understanding. It was trained on a mixture of transcription, subtitle and stenogram generation, news writing, speech translation, summarization, and question-answering tasks.

Model details

Property	Value
Base model	mistralai/Voxtral-Mini-3B-2507
Architecture	VoxtralForConditionalGeneration
Primary language	Estonian (et/est)
Additional output language	English (en)
Fine-tuning method	Full-parameter supervised fine-tuning
Selected checkpoint	4,500 steps
Weight precision	bfloat16
License	Apache 2.0, following the base model

Intended uses

The model is intended for research and development involving Estonian speech, including:

• verbatim automatic speech recognition;
• readable subtitle and parliamentary stenogram generation;
• summaries and news stories based on broadcast audio;
• answering questions about spoken content; and
• translation of Estonian speech into English.

The output style depends on the instruction. Prompts should state the desired task and format explicitly. This is not a certified system for legal, medical, safety-critical, or high-stakes transcription.

Usage with Transformers

The following example uses the selected local checkpoint for Estonian transcription:

import torch
from transformers import AutoProcessor, VoxtralForConditionalGeneration

model_id = "exp/models/voxtral-mini-3b-2507-full/1c/checkpoint-4500"
device = "cuda"

processor = AutoProcessor.from_pretrained(model_id)
model = VoxtralForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="sdpa",
    low_cpu_mem_usage=True,
).to(device)
model.eval()

inputs = processor.apply_transcription_request(
    language="et",
    audio="example.wav",
    model_id=model_id,
    return_tensors="pt",
)
inputs = {
    key: value.to(device=device, dtype=torch.bfloat16)
    if value.is_floating_point()
    else value.to(device)
    for key, value in inputs.items()
}

with torch.inference_mode():
    output_ids = model.generate(**inputs, do_sample=False, max_new_tokens=8192)

prompt_length = inputs["input_ids"].shape[1]
text = processor.batch_decode(
    output_ids[:, prompt_length:], skip_special_tokens=True
)[0]
print(text.strip())

Use 16 kHz mono audio where possible. For long recordings, segmenting at natural speech boundaries is recommended.

Training data

The training data was combined from seven task datasets.

Task	Input	Retained	Filtered out
Verbatim Estonian ASR	13,434	12,972	462
Edited broadcast subtitles	5,639	5,412	227
Estonian news stories from audio	50,000	49,998	2
English news stories from Estonian audio	10,000	9,999	1
Broadcast summarization and structured extraction	49,803	49,797	6
Questions about broadcast audio	10,000	5,264	4,736
Edited parliamentary stenograms	17,556	15,955	1,601
Total	156,432	149,397	7,035

Of the 7,035 filtered examples, 5,500 exceeded the task-specific audio-duration limit and 1,535 exceeded the maximum combined audio/text sequence length. Transcription examples were limited to 30 minutes and audio-understanding examples to 40 minutes. The final prepared manifest used a maximum sequence length of 28,000 tokens.

Combined sequence lengths in the retained manifest were:

Statistic	Tokens
Minimum	419
Median	6,845
90th percentile	19,163
95th percentile	19,734
99th percentile	24,925
Maximum	27,999

Some instructions and answers in trainining data and the English news-story targets were generated with google/gemini-3.1-flash-lite. Consequently, the training targets may contain errors or stylistic biases inherited from that model.

The source collections include ERR broadcasts and subtitles, an Estonian ASR manifest, ERR news video/audio and article text, Päevakaja programmes, Riigikogu recordings and stenograms, and TalTechNLP/qa_broadcast_conv_et. Users are responsible for checking the terms, copyright, privacy constraints, and permitted uses of the underlying data for their application.

Training procedure

Training used four NVIDIA A100-SXM4 80 GB GPUs. The per-device batch size was 1 with 8 gradient-accumulation steps, giving an effective global batch size of 32. Both the audio encoder and aligner were unfrozen.

Hyperparameter	Value
Maximum training steps	6,000
Selected checkpoint step	4,500
Effective global batch size	32
Optimizer	Fused AdamW
Learning rate	5e-5
Adam betas	(0.9, 0.95)
Adam epsilon	1e-8
Weight decay	0.01
Maximum gradient norm	1.0
Scheduler	Cosine
Warmup ratio	0.03
Training precision	bfloat16 with TF32 enabled
Gradient checkpointing	Enabled for language and vision/audio components
Sequence packing	Disabled
Truncation strategy	Delete over-length examples
Evaluation/save interval	500 steps
Random and data seeds	13

The run was resumed from checkpoints during development, and the configured maximum sequence length was reduced from 32,768 to 30,000 and finally 28,000. The data statistics above describe the final prepared 28,000-token manifest; they should not be interpreted as an exact record of the manifest state at every earlier resumed stage.

Evaluation

Checkpoints were ranked by word error rate on an internal held-out Estonian ASR set. Generation used greedy decoding (temperature=0) with up to 8,192 new tokens.

Checkpoint	WER	Reference words
4,500	5.05%	9,085

The validation set contains only 10 recordings. These figures are useful for checkpoint selection but should not be treated as a broad estimate of Estonian ASR quality or as directly comparable with public benchmark results.

Toolkits and environment

The recorded environment for the final training stage included:

Toolkit	Version or role
ms-swift	4.4.2; supervised fine-tuning and evaluation loop
Transformers	4.57.6; Voxtral model and processor
PyTorch	2.10.0
Datasets	3.6.0
Accelerate	1.12.0
Mistral Common	1.11.7
kaldiio	2.18.1; reading FLAC ark audio
SoundFile	0.13.1
Safetensors	0.7.0
Weights & Biases	0.27.2; experiment tracking
CUDA	13.2
Slurm	Distributed job launch and resource allocation

A local ms-swift plugin supplied Voxtral dataset integration and support for the repository's ark:<path>:<offset> audio references.

Limitations

• The model can hallucinate, omit, repeat, or incorrectly normalize speech.
• Performance may degrade for dialects, code-switching, overlapping speakers, noise, distant microphones, music, or domains underrepresented in training.
• The English outputs were trained on machine-generated references and may inherit translation errors.
• Generated summaries and answers are not guaranteed to be factually faithful to the recording.
• Training data may contain personal names, opinions, and other information present in public broadcasts or parliamentary speech. Outputs should be reviewed before publication or consequential use.

Acknowledgements

The model builds on Mistral AI's Voxtral Mini 3B 2507. See the upstream model card for architecture details, original capabilities, usage requirements, and base-model limitations.