library_name: peft
base_model: openai/whisper-large-v3
Model Card for Model ID
As of our knowledge SOTA in swiss german with wer=14.269151618793657 and normalized_wer=12.800897299473698.
Model Details
Trained on RTX 3070 for 30 hours using SwissDial all Dialects with following guide: https://github.com/Vaibhavs10/fast-whisper-finetuning/blob/main/Whisper_w_PEFT.ipynb
Model Description
- Developed by: Flurin17
- Language(s) (NLP): swiss-german
- License: IDK ask openai
- Finetuned from model [optional]: openai/whisper-large-v3
Model Sources [optional]
Uses
from peft import PeftModel, PeftConfig from transformers import WhisperForConditionalGeneration, Seq2SeqTrainer
peft_model_id = "Flurin17/whisper-large-v3-peft-swiss-german" # Use the same model ID as before. peft_config = PeftConfig.from_pretrained(peft_model_id) model = WhisperForConditionalGeneration.from_pretrained( peft_config.base_model_name_or_path, load_in_8bit=True, device_map="auto" ) model = PeftModel.from_pretrained(model, peft_model_id) model.config.use_cache = True
from transformers import WhisperFeatureExtractor
feature_extractor = WhisperFeatureExtractor.from_pretrained(peft_model_id)
from transformers import WhisperTokenizer
tokenizer = WhisperTokenizer.from_pretrained(peft_model_id, language=language, task=task)
from transformers import AutomaticSpeechRecognitionPipeline import torch pipe = AutomaticSpeechRecognitionPipeline(model=model, tokenizer=tokenizer, feature_extractor=feature_extractor)
with torch.cuda.amp.autocast(): result = pipe(r"L:\Coding\random\audio.mp3", generate_kwargs={"language": "german"}) print(result["text"])
Framework versions
- PEFT 0.7.1