Feature request: distilled per-language "Voxtral Micro" models for edge/on-device use Context
Hi,
I build Voxtral Transcribe, an Obsidian dictation plugin on the Voxtral API (realtime + batch, 13 languages). I recently explored a fully local mode for my users and hit a wall that I suspect many integrators are hitting.
The gap
Voxtral-Mini-4B-Realtime is excellent, and the model card explicitly targets "laptops, phones, smartwatches" — but in practice the on-device story stops at 16 GB GPUs today: llama.cpp declined realtime support, quantized runtimes are immature, and 4B (2.5 GB at Q4) is out of reach for phone webviews and ordinary laptops. Meanwhile the demand side is loud: my users ask for offline/private dictation, and the common trigger is dissatisfaction with OS-level dictation quality in non-English languages.
The ask
A line of small, per-language distilled models — "Voxtral Micro", say 100–250M params per language — designed for streaming edge inference. The per-language-specialization thesis is well proven at this scale (Moonshine's monolingual models beat Whisper Large-v3 on English at 1/25th the parameters), but the existing small-model vendors have deliberately skipped the European languages. That's exactly where Mistral is uniquely positioned: you own the teacher models and data pipeline for distillation, the ExecuTorch export path already exists for the 4B, and "the European languages Silicon Valley skips, on-device and GDPR-friendly" fits your positioning like a glove.
Even 2–3 languages to start (French, German, Dutch?) with open weights would let integrators like me ship a genuinely private dictation mode on ordinary hardware — with the cloud API remaining the quality tier for everything heavier (batch V2, diarization).
Happy to help validate: I have a Dutch-language test setup and a user base that would exercise this immediately.
Kind regards,
Max