Maxdorger29/whisper-large-v3-hungarian-lora

Whisper Large-v3 — Hungarian LoRA (CTranslate2)

  If this Hungarian ASR model helps your project, a coffee keeps the GPUs training. 🙏

A LoRA fine-tuned Whisper large-v3 model optimized for the Hungarian language. This model reduces common recognition inaccuracies and hallucinations of the base model on Hungarian speech (special Hungarian characters, native names, silence hallucinations).

This repository contains the CTranslate2-converted FP16 version for use with the popular faster-whisper library, plus the original PEFT LoRA adapter (in adapter/) for anyone who wants to merge onto the base model themselves — e.g. for MLX, vanilla transformers, or further fine-tuning.

Note: this model was previously published as whisper-large-v3-turbo-hungarian-lora and was renamed to reflect its true base — openai/whisper-large-v3 (32 decoder layers, 128 mel bins), not the 4-layer turbo variant. The old name still redirects here, so existing links and downloads keep working.

---

Highlights

• Optimized for Hungarian: LoRA fine-tuned (r=32, alpha=64, q_proj/v_proj) on Hungarian speech
• Fast inference: CTranslate2 engine; use compute_type="float16" (or "int8_float16" for lower VRAM)
• Two formats in one repo: ready-to-run CT2 weights + the raw PEFT adapter for custom workflows
• Perfect match for F5-TTS: recommended for generating accurate reference text (ref_text) for Maxdorger29/f5-tts-hungarian voice cloning

Quick Start (faster-whisper)

pip install faster-whisper

from faster_whisper import WhisperModel

model_id = "Maxdorger29/whisper-large-v3-hungarian-lora"
model = WhisperModel(model_id, device="cuda", compute_type="float16")

segments, info = model.transcribe("magyar_hang_minta.wav", language="hu", beam_size=5)

for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

Lower VRAM: compute_type="int8_float16".

Using the LoRA adapter (transformers / PEFT)

The adapter/ folder contains the original PEFT artifacts (adapter_model.safetensors, ~60 MB). Merge onto the base model:

from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel

base = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large-v3")
model = PeftModel.from_pretrained(base, "Maxdorger29/whisper-large-v3-hungarian-lora", subfolder="adapter")
merged = model.merge_and_unload()

processor = WhisperProcessor.from_pretrained("openai/whisper-large-v3")
merged.save_pretrained("./whisper-large-v3-hu-merged")
processor.save_pretrained("./whisper-large-v3-hu-merged")

MLX conversion (Apple Silicon)

After merging (above), convert the merged HF checkpoint with mlx-examples/whisper convert.py:

python convert.py --torch-name-or-path ./whisper-large-v3-hu-merged --mlx-path ./whisper-hu-mlx --dtype float16

Training Details

Parameter	Value
Base model	openai/whisper-large-v3
Method	PEFT LoRA (r=32, alpha=64, dropout=0.05)
Target modules	q_proj, v_proj
Dataset	Google FLEURS hu_hu (3,502 train+val / 905 test)
Export	merged FP16 → CTranslate2

Repository Contents

File	Size	Description
model.bin	~3.1 GB	CTranslate2 FP16 model weights (merged)
config.json	<3 KB	CTranslate2 configuration
vocabulary.json	~1.1 MB	Tokens and vocabulary
preprocessor_config.json	<1 KB	Feature extractor config (128 mel bins — required for correct loading)
adapter/adapter_model.safetensors	~60 MB	Original PEFT LoRA adapter
adapter/adapter_config.json	<1 KB	PEFT adapter configuration

Links

• Hugging Face (Model Weights): huggingface.co/Maxdorger29/whisper-large-v3-hungarian-lora
• GitHub (Issues & Discussion): github.com/Maxdorger/whisper-hungarian-lora

Companion Model (F5-TTS)

We originally fine-tuned this model as the transcription backend for Maxdorger29/f5-tts-hungarian, providing accurate reference texts (ref_text) for voice cloning.

Support the Project

Training and fine-tuning open-source TTS/STT models requires significant GPU resources. If this Hungarian-optimized Whisper LoRA speeds up your workflow, consider buying me a coffee — it directly supports the compute costs of future epochs! ☕

License

MIT License for the fine-tuning weights and scripts, provided usage does not violate the original terms of the OpenAI Whisper large-v3 model.

---

Fine-tuned for the Hungarian Open-Source AI Community.