Package: whisper-ctranslate2
IntroductionWhisper command line client compatible with originalOpenAI clientbased on CTranslate2.It usesCTranslate2andFaster-whisperWhisper implementation that is up to 4 times faster than openai/whisper for the same accuracy while using less memory.Goals of the project:Provide an easy way to use the CTranslate2 Whisper implementationEase the migration for people using OpenAI Whisper CLIInstallationTo install the latest stable version, just type:pip install -U whisper-ctranslate2Alternatively, if you are interested in the latest development (non-stable) version from this repository, just type:pip install git+https://github.com/Softcatala/whisper-ctranslate2CPU and GPU supportGPU and CPU support are provided byCTranslate2.It has compatibility with x86-64 and AArch64/ARM64 CPU and integrates multiple backends that are optimized for these platforms: Intel MKL, oneDNN, OpenBLAS, Ruy, and Apple Accelerate.GPU execution requires the NVIDIA libraries cuBLAS 11.x and cuDNN 8.x to be installed on the system. Please refer to theCTranslate2 documentationBy default the best hardware available is selected for inference. You can use the options--deviceand--device_indexto control manually the selection.UsageSame command line as OpenAI Whisper.To transcribe:whisper-ctranslate2 inaguracio2011.mp3 --model mediumTo translate:whisper-ctranslate2 inaguracio2011.mp3 --model medium --task translateWhisper translate task translates the transcription from the source language to English (the only target language supported).Additionally using:whisper-ctranslate2 --helpAll the supported options with their help are shown.CTranslate2 specific optionsOn top of the OpenAI Whisper command line options, there are some specific options provided by CTranslate2 or whiper-ctranslate2.Quantization--compute_typeoption which acceptsdefault,auto,int8,int8_float16,int16,float16,float32values indicates the type ofquantizationto use. On CPUint8will give the best performance:whisper-ctranslate2 myfile.mp3 --compute_type int8Loading the model from a directory--model_directoryoption allows to specify the directory from which you want to load a CTranslate2 Whisper model. For example, if you want to load your own quantifiedWhisper modelversion or using your ownWhisper fine-tunnedversion. The model must be in CTranslate2 format.Using Voice Activity Detection (VAD) filter--vad_filteroption enables the voice activity detection (VAD) to filter out parts of the audio without speech. This step uses theSilero VAD model:whisper-ctranslate2 myfile.mp3 --vad_filter TrueThe VAD filter accepts multiple additional options to determine the filter behavior:--vad_threshold VALUE (float)Probabilities above this value are considered as speech.--vad_min_speech_duration_ms (int)Final speech chunks shorter min_speech_duration_ms are thrown out.--vad_max_speech_duration_s VALUE (int)Maximum duration of speech chunks in seconds. Longer will be split at the timestamp of the last silence.Print colors--print_colors Trueoptions prints the transcribed text using an experimental color coding strategy based onwhisper.cppto highlight words with high or low confidence:whisper-ctranslate2 myfile.mp3 --print_colors TrueLive transcribe from your microphone--live_transcribe Trueoption activates the live transcription mode from your microphone:whisper-ctranslate2 --live_transcribe True --language enhttps://user-images.githubusercontent.com/309265/231533784-e58c4b92-e9fb-4256-b4cd-12f1864131d9.movDiarization (speaker identification)There is experimental diarization support usingpyannote.audioto identify speakers. At the moment, the support is a segment level.To enable diarization you need to follow these steps:Installpyannote.audiowithpip install pyannote.audioAcceptpyannote/segmentation-3.0user conditionsAcceptpyannote/speaker-diarization-3.1user conditionsCreate access token athf.co/settings/tokens.And then execute passing the HuggingFace API token as parameter to enable diarization:whisper-ctranslate2 --hf_token YOUR_HF_TOKENand then the name of the speaker is added in the output files (e.g. JSON, VTT and STR files):[SPEAKER_00]: There is a lot of people in this roomThe option--speaker_name SPEAKER_NAMEallows to use your own string to identify the speaker.Need help?Check ourfrequently asked questionsfor common questions.ContactJordi Masjmas@softcatala.org