mijuanlo/whisper-small-ct2-int8

Whisper small INT8 CTranslate2

Fast-Inference with CTranslate2

Speedup inference while reducing memory using INT8 quantization with CTranslate2.

This is a quantized version of openai/whisper-small converted to CTranslate2 format.

Compatible with faster-whisper and CTranslate2 directly.

pip install faster-whisper

Checkpoint compatible with CTranslate2 >= 3.22.0

• compute_type=int8 for both CPU and GPU

Usage example

from faster_whisper import WhisperModel

model = WhisperModel("/path/to/whisper", device="cuda", compute_type="int8")

segments, info = model.transcribe("audio.mp3", beam_size=5, language="es")
for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

Conversion command

ct2-transformers-converter --model openai/whisper-small --output_dir /path/to/whisper \
    --copy_files tokenizer.json --quantization int8

License

This model is a quantized version of openai/whisper-small, which is released under the MIT license. The same license applies to this conversion.

Model description

Whisper is a general-purpose speech recognition model from OpenAI. It is trained on a large dataset of diverse audio and is also a multi-task model that can perform multilingual speech recognition, speech translation, and language identification.

• Paper: Robust Speech Recognition via Large-Scale Weak Supervision (Radford et al., 2022)
• Original model: openai/whisper-small
• Repository: github.com/openai/whisper

Supported languages

99 languages: Afrikaans, Arabic, Armenian, Azerbaijani, Belarusian, Bosnian, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Galician, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Kazakh, Korean, Latvian, Lithuanian, Macedonian, Malay, Marathi, Maori, Nepali, Norwegian, Persian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tagalog, Tamil, Thai, Turkish, Ukrainian, Urdu, Vietnamese, Welsh, and more.

Metrics

Evaluated using Word Error Rate (WER) on LibriSpeech and other benchmarks. See the original model card for detailed metrics.

Evaluation Data

Whisper was evaluated on LibriSpeech, Common Voice, Fleurs, and other multilingual speech datasets.

Training Data

Trained on 680,000 hours of multilingual and multitask supervised data collected from the web.

Ethical Considerations

Whisper may transcribe speech inaccurately, particularly for accented speech, low-resource languages, or noisy environments. The model should not be used as a sole decision-making tool in sensitive domains. Whisper's training data was sourced from the web and may contain biases.

Caveats and Recommendations

Performance varies by language and domain. For best results, use audio with clear speech and minimal background noise. The model is not intended for speaker identification or verification.

Repository

Hugging Face: mijuanlo/whisper-small-ct2-int8