Whisper large-v3 model for CTranslate2

This repository contains the conversion of openai/whisper-large-v3 to the CTranslate2 model format.

This model can be used in CTranslate2 or projects based on CTranslate2 such as faster-whisper.

Example

from faster_whisper import WhisperModel

model = WhisperModel("large-v3")

segments, info = model.transcribe("audio.mp3")
for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

Conversion details

The original model was converted with the following command:

ct2-transformers-converter --model openai/whisper-large-v3 --output_dir faster-whisper-large-v3 \
    --copy_files added_tokens.json special_tokens_map.json tokenizer_config.json vocab.json  --quantization float16

Note that the model weights are saved in FP16. This type can be changed when the model is loaded using the compute_type option in CTranslate2.

Note that while "openai/whisper-large-v3" does not come with a "tokenizer.json" file, you can generate it using AutoTokenizer.

from transformers import AutoTokenizer
self.hf_tokenizer = AutoTokenizer.from_pretrained("openai/whisper-large-v3")
self.hf_tokenizer.save_pretrained("whisper-large-v3-test")

How faster-whisper working with Whisper-large-v3

In faster-whisper version 0.10.0, there is no need to perform this handling.

~~Working with Whisper-large-v3 #547 by. UmarRamzan~~

- from faster_whisper import WhisperModel

- model = WhisperModel(model_url)

- if "large-v3" in model_url:
-     model.feature_extractor.mel_filters = model.feature_extractor.get_mel_filters(model.feature_extractor.sampling_rate, model.feature_extractor.n_fft, n_mels=128)

More information

For more information about the original model, see its model card.