| # Options | |
| To transcribe or translate an audio file, you can either copy an URL from a website (all [websites](https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md) | |
| supported by YT-DLP will work, including YouTube). Otherwise, upload an audio file (choose "All Files (*.*)" | |
| in the file selector to select any file type, including video files) or use the microphone. | |
| For longer audio files (>10 minutes), it is recommended that you select Silero VAD (Voice Activity Detector) in the VAD option. | |
| ## Model | |
| Select the model that Whisper will use to transcribe the audio: | |
| | Size | Parameters | English-only model | Multilingual model | Required VRAM | Relative speed | | |
| |--------|------------|--------------------|--------------------|---------------|----------------| | |
| | tiny | 39 M | tiny.en | tiny | ~1 GB | ~32x | | |
| | base | 74 M | base.en | base | ~1 GB | ~16x | | |
| | small | 244 M | small.en | small | ~2 GB | ~6x | | |
| | medium | 769 M | medium.en | medium | ~5 GB | ~2x | | |
| | large | 1550 M | N/A | large | ~10 GB | 1x | | |
| ## Language | |
| Select the language, or leave it empty for Whisper to automatically detect it. | |
| Note that if the selected language and the language in the audio differs, Whisper may start to translate the audio to the selected | |
| language. For instance, if the audio is in English but you select Japaneese, the model may translate the audio to Japanese. | |
| ## Inputs | |
| The options "URL (YouTube, etc.)", "Upload Audio" or "Micriphone Input" allows you to send an audio input to the model. | |
| Note that the UI will only process the first valid input - i.e. if you enter both an URL and upload an audio, it will only process | |
| the URL. | |
| ## Task | |
| Select the task - either "transcribe" to transcribe the audio to text, or "translate" to translate it to English. | |
| ## Vad | |
| Using a VAD will improve the timing accuracy of each transcribed line, as well as prevent Whisper getting into an infinite | |
| loop detecting the same sentence over and over again. The downside is that this may be at a cost to text accuracy, especially | |
| with regards to unique words or names that appear in the audio. You can compensate for this by increasing the prompt window. | |
| Note that English is very well handled by Whisper, and it's less susceptible to issues surrounding bad timings and infinite loops. | |
| So you may only need to use a VAD for other languages, such as Japanese, or when the audio is very long. | |
| * none | |
| * Run whisper on the entire audio input | |
| * silero-vad | |
| * Use Silero VAD to detect sections that contain speech, and run Whisper on independently on each section. Whisper is also run | |
| on the gaps between each speech section, by either expanding the section up to the max merge size, or running Whisper independently | |
| on the non-speech section. | |
| * silero-vad-expand-into-gaps | |
| * Use Silero VAD to detect sections that contain speech, and run Whisper on independently on each section. Each spech section will be expanded | |
| such that they cover any adjacent non-speech sections. For instance, if an audio file of one minute contains the speech sections | |
| 00:00 - 00:10 (A) and 00:30 - 00:40 (B), the first section (A) will be expanded to 00:00 - 00:30, and (B) will be expanded to 00:30 - 00:60. | |
| * silero-vad-skip-gaps | |
| * As above, but sections that doesn't contain speech according to Silero will be skipped. This will be slightly faster, but | |
| may cause dialogue to be skipped. | |
| * periodic-vad | |
| * Create sections of speech every 'VAD - Max Merge Size' seconds. This is very fast and simple, but will potentially break | |
| a sentence or word in two. | |
| ## VAD - Merge Window | |
| If set, any adjacent speech sections that are at most this number of seconds apart will be automatically merged. | |
| ## VAD - Max Merge Size (s) | |
| Disables merging of adjacent speech sections if they are this number of seconds long. | |
| ## VAD - Padding (s) | |
| The number of seconds (floating point) to add to the beginning and end of each speech section. Setting this to a number | |
| larger than zero ensures that Whisper is more likely to correctly transcribe a sentence in the beginning of | |
| a speech section. However, this also increases the probability of Whisper assigning the wrong timestamp | |
| to each transcribed line. The default value is 1 second. | |
| ## VAD - Prompt Window (s) | |
| The text of a detected line will be included as a prompt to the next speech section, if the speech section starts at most this | |
| number of seconds after the line has finished. For instance, if a line ends at 10:00, and the next speech section starts at | |
| 10:04, the line's text will be included if the prompt window is 4 seconds or more (10:04 - 10:00 = 4 seconds). | |
| Note that detected lines in gaps between speech sections will not be included in the prompt | |
| (if silero-vad or silero-vad-expand-into-gaps) is used. |