Package: faster-whisper
Faster Whisper transcription with CTranslate2faster-whisperis a reimplementation of OpenAI's Whisper model usingCTranslate2, which is a fast inference engine for Transformer models.This implementation is up to 4 times faster thanopenai/whisperfor the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU.BenchmarkWhisperFor reference, here's the time and memory usage that are required to transcribe13 minutesof audio using different implementations:openai/whisper@6dea21fdwhisper.cpp@3b010f9faster-whisper@cce6b53eLarge-v2 model on GPUImplementationPrecisionBeam sizeTimeMax. GPU memoryMax. CPU memoryopenai/whisperfp1654m30s11325MB9439MBfaster-whisperfp16554s4755MB3244MBfaster-whisperint8559s3091MB3117MBExecuted with CUDA 11.7.1 on a NVIDIA Tesla V100S.Small model on CPUImplementationPrecisionBeam sizeTimeMax. memoryopenai/whisperfp32510m31s3101MBwhisper.cppfp32517m42s1581MBwhisper.cppfp16512m39s873MBfaster-whisperfp3252m44s1675MBfaster-whisperint852m04s995MBExecuted with 8 threads on a Intel(R) Xeon(R) Gold 6226R.Distil-whisperImplementationPrecisionBeam sizeTimeGigaspeech WERdistil-whisper/distil-large-v2fp164-10.36faster-distil-large-v2fp165-10.28distil-whisper/distil-medium.enfp164-11.21faster-distil-medium.enfp165-11.21Executed with CUDA 11.4 on a NVIDIA 3090.testing details (click to expand)Fordistil-whisper/distil-large-v2, the WER is tested with code sample fromlink. forfaster-distil-whisper, the WER is tested with setting:fromfaster_whisperimportWhisperModelmodel_size="distil-large-v2"# model_size = "distil-medium.en"# Run on GPU with FP16model=WhisperModel(model_size,device="cuda",compute_type="float16")segments,info=model.transcribe("audio.mp3",beam_size=5,language="en")RequirementsPython 3.8 or greaterUnlike openai-whisper, FFmpeg doesnotneed to be installed on the system. The audio is decoded with the Python libraryPyAVwhich bundles the FFmpeg libraries in its package.GPUGPU execution requires the following NVIDIA libraries to be installed:cuBLAS for CUDA 11cuDNN 8 for CUDA 11There are multiple ways to install these libraries. The recommended way is described in the official NVIDIA documentation, but we also suggest other installation methods below.Other installation methods (click to expand)Use DockerThe libraries are installed in this official NVIDIA Docker image:nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu22.04.Install withpip(Linux only)On Linux these libraries can be installed withpip. Note thatLD_LIBRARY_PATHmust be set before launching Python.pipinstallnvidia-cublas-cu11nvidia-cudnn-cu11exportLD_LIBRARY_PATH=`python3-c'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`Download the libraries from Purfview's repository (Windows & Linux)Purfview'swhisper-standalone-winprovides the required NVIDIA libraries for Windows & Linux in asingle archive. Decompress the archive and place the libraries in a directory included in thePATH.InstallationThe module can be installed fromPyPI:pipinstallfaster-whisperOther installation methods (click to expand)Install the master branchpipinstall--force-reinstall"faster-whisper @ https://github.com/guillaumekln/faster-whisper/archive/refs/heads/master.tar.gz"Install a specific commitpipinstall--force-reinstall"faster-whisper @ https://github.com/guillaumekln/faster-whisper/archive/a4f1cc8f11433e454c3934442b5e1a4ed5e865c3.tar.gz"UsageFaster-whisperfromfaster_whisperimportWhisperModelmodel_size="large-v3"# Run on GPU with FP16model=WhisperModel(model_size,device="cuda",compute_type="float16")# or run on GPU with INT8# model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")# or run on CPU with INT8# model = WhisperModel(model_size, device="cpu", compute_type="int8")segments,info=model.transcribe("audio.mp3",beam_size=5)print("Detected language '%s' with probability%f"%(info.language,info.language_probability))forsegmentinsegments:print("[%.2fs ->%.2fs]%s"%(segment.start,segment.end,segment.text))Warning:segmentsis ageneratorso the transcription only starts when you iterate over it. The transcription can be run to completion by gathering the segments in a list or aforloop:segments,_=model.transcribe("audio.mp3")segments=list(segments)# The transcription will actually run here.Faster-distil-whisperFor usage offaster-ditil-whisper, please refer to:https://github.com/guillaumekln/faster-whisper/issues/533model_size="distil-large-v2"# model_size = "distil-medium.en"model=WhisperModel(model_size,device="cuda",compute_type="float16")segments,info=model.transcribe("audio.mp3",beam_size=5,language="en",max_new_tokens=128,condition_on_previous_text=False)NOTE: Empirically,condition_on_previous_text=Truewill degrade the performance offaster-distil-whisperfor long audio. Degradation on the first chunk was observed withinitial_prompttoo.Word-level timestampssegments,_=model.transcribe("audio.mp3",word_timestamps=True)forsegmentinsegments:forwordinsegment.words:print("[%.2fs ->%.2fs]%s"%(word.start,word.end,word.word))VAD filterThe library integrates theSilero VADmodel to filter out parts of the audio without speech:segments,_=model.transcribe("audio.mp3",vad_filter=True)The default behavior is conservative and only removes silence longer than 2 seconds. See the available VAD parameters and default values in thesource code. They can be customized with the dictionary argumentvad_parameters:segments,_=model.transcribe("audio.mp3",vad_filter=True,vad_parameters=dict(min_silence_duration_ms=500),)LoggingThe library logging level can be configured like this:importlogginglogging.basicConfig()logging.getLogger("faster_whisper").setLevel(logging.DEBUG)Going furtherSee more model and transcription options in theWhisperModelclass implementation.Community integrationsHere is a non exhaustive list of open-source projects using faster-whisper. Feel free to add your project to the list!WhisperXis an award-winning Python library that offers speaker diarization and accurate word-level timestamps using wav2vec2 alignmentwhisper-ctranslate2is a command line client based on faster-whisper and compatible with the original client from openai/whisper.whisper-diarizeis a speaker diarization tool that is based on faster-whisper and NVIDIA NeMo.whisper-standalone-winStandalone CLI executables of faster-whisper for Windows, Linux & macOS.asr-sd-pipelineprovides a scalable, modular, end to end multi-speaker speech to text solution implemented using AzureML pipelines.Open-Lyricsis a Python library that transcribes voice files using faster-whisper, and translates/polishes the resulting text into.lrcfiles in the desired language using OpenAI-GPT.wscribeis a flexible transcript generation tool supporting faster-whisper, it can export word level transcript and the exported transcript then can be edited withwscribe-editoraTrainis a graphical user interface implementation of faster-whisper developed at the BANDAS-Center at the University of Graz for transcription and diarization in Windows (Windows Store App) and Linux.Whisper-Streamingimplements real-time mode for offline Whisper-like speech-to-text models with faster-whisper as the most recommended back-end. It implements a streaming policy with self-adaptive latency based on the actual source complexity, and demonstrates the state of the art.WhisperLiveis a nearly-live implementation of OpenAI's Whisper which uses faster-whisper as the backend to transcribe audio in real-time.Faster-Whisper-Transcriberis a simple but reliable voice transcriber that provides a user-friendly interface.Model conversionWhen loading a model from its size such asWhisperModel("large-v3"), the corresponding CTranslate2 model is automatically downloaded from theHugging Face Hub.We also provide a script to convert any Whisper models compatible with the Transformers library. They could be the original OpenAI models or user fine-tuned models.For example the command below converts theoriginal "large-v3" Whisper modeland saves the weights in FP16:pipinstalltransformers[torch]>=4.23

ct2-transformers-converter--modelopenai/whisper-large-v3--output_dirwhisper-large-v3-ct2
--copy_filestokenizer.jsonpreprocessor_config.json--quantizationfloat16The option--modelaccepts a model name on the Hub or a path to a model directory.If the option--copy_files tokenizer.jsonis not used, the tokenizer configuration is automatically downloaded when the model is loaded later.Models can also be converted from the code. See theconversion API.Load a converted modelDirectly load the model from a local directory:model=faster_whisper.WhisperModel("whisper-large-v3-ct2")Upload your model to the Hugging Face Huband load it from its name:model=faster_whisper.WhisperModel("username/whisper-large-v3-ct2")Comparing performance against other implementationsIf you are comparing the performance against other Whisper implementations, you should make sure to run the comparison with similar settings. In particular:Verify that the same transcription options are used, especially the same beam size. For example in openai/whisper,model.transcribeuses a default beam size of 1 but here we use a default beam size of 5.When running on CPU, make sure to set the same number of threads. Many frameworks will read the environment variableOMP_NUM_THREADS, which can be set when running your script:OMP_NUM_THREADS=4python3my_script.py