Transcribing interview data from video to text with Python | by Alan Davies | Towards Data Science

Occasionally I have the need to interview participants for various research projects. Often grant funding for such projects will cover transcription costs. Human transcribers remain the gold standard for this sort of work and usually do an excellent job. You could of course transcribe your own interviews but this can be a very time consuming and laborious task. Some qualitative researchers also advocate transcribing your own interviews as a way of becoming more familiar with the data.
There also exist various software for this task, including free trials and software with various pricing models. Creating a script to do this yourself using Python is also relatively easy. Faced with having to transcribe several fairly lengthy interviews (> 1 hour each) carried out online, I decided to create a simple program with a graphical interface to manage this process. This short blog describes and implements the basic concept as a single script and then shows how we can add a Graphical User Interface (GUI) and use threads to improve performance.
This assumes the reader has some experience with Python and is familiar with concepts like Object Orientated Programming (OOP). The example uses Python 3.x and PyQt5.
The basic working example
For this we need 3 modules, speech_recognition, wave and moviepy. These can be installed in the standard way using pip.
pip install Wavepip install moviepypip install SpeechRecognition
They can then be imported into the Python file along with the other required modules:
import wave, math, contextlibimport speech_recognition as srfrom moviepy.editor import AudioFileClip
For now, we will hard code the mp4 video file to import and the audio file we will create when converting the video to audio:
transcribed_audio_file_name = "transcribed_speech.wav"zoom_video_file_name = "zoom_0.mp4"
Next, we can use the AudioFileClip class from the moviepy.editor to convert the video to audio. This will convert the video to audio, specifically a wav file.
audioclip = AudioFileClip(zoom_video_file_name)audioclip.write_audiofile(transcribed_audio_file_name)
The next step is to convert this audio file into text. The issue with this is that there is a limit of around 10 MB for single requests sent to the API. This means we need to transcribe the audio in stages. To do this it would be helpful to first to know the duration of the audio file so that we can divide it up into manageable chunks.
The contextlib.closing works with the with statement to inform the context manager when the code block is entered and exited. Here we open the wav file using the wave library and obtain the number of frames and the framerate with which we can calculate the duration of the file.
with contextlib.closing(wave.open(transcribed_audio_file_name,'r')) as f:    frames = f.getnframes()    rate = f.getframerate()    duration = frames / float(rate)
This can now be divided into some suitable number for transcribing the file in chunks. In this example, we used 60 seconds.
total_duration = math.ceil(duration / 60)
Next, we can create an instance of the speech recognizer
r = sr.Recognizer()
Finally, we can convert the audio to text. The record function can use a duration (in seconds). Subsequent record lines in the with statement will continue where the last left off. Here we use an offset to capture the next 60-second chunk and write the transcribed text to a text file. To recognize the speech we use recognize_google. There are alternatives such as recognize_bing(), recognize_wit() and recognize_google_cloud(). We can write the transcribed text to a text file. We add an extra space so the subsequent sentences have spaces between them.
for i in range(0, total_duration):    with sr.AudioFile(transcribed_audio_file_name) as source:        audio = r.record(source, offset=i*60, duration=60)    f = open("transcription.txt", "a")    f.write(r.recognize_google(audio))    f.write(" ")f.close()
The full script can be seen below:
import wave, math, contextlibimport speech_recognition as srfrom moviepy.editor import AudioFileCliptranscribed_audio_file_name = "transcribed_speech.wav"zoom_video_file_name = "zoom_0.mp4"audioclip = AudioFileClip(zoom_video_file_name)audioclip.write_audiofile(transcribed_audio_file_name)with contextlib.closing(wave.open(transcribed_audio_file_name,'r')) as f:    frames = f.getnframes()    rate = f.getframerate()    duration = frames / float(rate)total_duration = math.ceil(duration / 60)r = sr.Recognizer()for i in range(0, total_duration):    with sr.AudioFile(transcribed_audio_file_name) as source:        audio = r.record(source, offset=i*60, duration=60)    f = open("transcription.txt", "a")    f.write(r.recognize_google(audio))    f.write(" ")f.close()
Adding a Graphical User Interface (GUI)
The basic script works but it would be more useful if we could add a GUI. This can help with locating files and adding features like a progress bar to monitor progress for large transcriptions. We can also display the transcribed text in a text box on the screen rather than using the console. I mostly use web frameworks like Flask for building front ends and add GUI elements using HTML, CSS so this was the first time I had attempted to make a non-web-based GUI in Python. After doing a bit of research on which tools to use, I came across PyQt. Qt is a cross-platform framework (written in C++) that provides an open-source toolkit for creating GUIs.
The latest version of this is PyQt5, which provides a set of useful libraries for generating GUIs and can be installed with pip like so:
pip install PyQt5
Another useful tool associated with Qt is the QT Designer tool for Windows or Mac (which can be downloaded from: https://build-system.fman.io/qt-designer-download). This allows one to drag and drop common user interface (UI) elements onto a window to generate a front end visually. The image below shows the simple front end generated for the transcription application.
Here we have a simple menu with “File” which contains “Open mp4 video recording” and “new”. We also have an “About” option which contains an “About vid2text”. There is a label on the main window to display the video file that was selected by the user. The user can type in a file name in the box below for the transcription to be written to. Next we have the transcribe button which is disabled by default until a video file is selected. Under this there is a progress bar to update the user on the current progress of the transcription. This is followed by a text box to display the text of the transcription. Finally there is a label at the bottom of the screen to show various messages to the users.
Once the front end has been constructed visually using the designer, we can save the file as a *.ui file. Using the command line (e.g. the terminal or Anaconda prompt) we can run the following command to generate Python code from the ui file.
pyuic –x vidui.ui –o vid2speechapp.py
When we run the following code, it produces Python code for the GUI we designed with the QT Designer as shown below. The –o writes the output to a file and –x generates additional code to test the class.
from PyQt5 import QtCore, QtGui, QtWidgetsclass Ui_MainWindow(object):    def setupUi(self, MainWindow):        MainWindow.setObjectName("MainWindow")        MainWindow.resize(653, 836)        self.centralwidget = QtWidgets.QWidget(MainWindow)        self.centralwidget.setObjectName("centralwidget")        self.label = QtWidgets.QLabel(self.centralwidget)        self.label.setGeometry(QtCore.QRect(50, 20, 161, 41))        font = QtGui.QFont()        font.setPointSize(14)        self.label.setFont(font)        self.label.setObjectName("label")        self.selected_video_label = QtWidgets.QLabel(self.centralwidget)        self.selected_video_label.setGeometry(QtCore.QRect(230, 20, 371,41))        font = QtGui.QFont()        font.setPointSize(14)        self.selected_video_label.setFont(font)        self.selected_video_label.setFrameShape(QtWidgets.QFrame.Box)        self.selected_video_label.setText("")       self.selected_video_label.setObjectName("selected_video_label")        self.label_3 = QtWidgets.QLabel(self.centralwidget)        self.label_3.setGeometry(QtCore.QRect(50, 90, 161, 41))        font = QtGui.QFont()        font.setPointSize(14)        self.label_3.setFont(font)        self.label_3.setObjectName("label_3")        self.transcribed_text = QtWidgets.QTextBrowser(self.centralwidget)        self.transcribed_text.setGeometry(QtCore.QRect(230, 320, 381, 431))        self.transcribed_text.setObjectName("transcribed_text")        self.label_5 = QtWidgets.QLabel(self.centralwidget)        self.label_5.setGeometry(QtCore.QRect(230, 280, 161, 41))        font = QtGui.QFont()        font.setPointSize(14)        self.label_5.setFont(font)        self.label_5.setObjectName("label_5")        self.transcribe_button = QtWidgets.QPushButton(self.centralwidget)        self.transcribe_button.setEnabled(False)        self.transcribe_button.setGeometry(QtCore.QRect(230, 150, 221, 81))        font = QtGui.QFont()        font.setPointSize(14)        self.transcribe_button.setFont(font)        self.transcribe_button.setObjectName("transcribe_button")        self.progress_bar = QtWidgets.QProgressBar(self.centralwidget)        self.progress_bar.setGeometry(QtCore.QRect(230, 250, 381, 23))        self.progress_bar.setProperty("value", 0)        self.progress_bar.setObjectName("progress_bar")        self.message_label = QtWidgets.QLabel(self.centralwidget)        self.message_label.setGeometry(QtCore.QRect(0, 760, 651, 21))        font = QtGui.QFont()        font.setPointSize(8)        self.message_label.setFont(font)        self.message_label.setFrameShape(QtWidgets.QFrame.Box)        self.message_label.setText("")        self.message_label.setObjectName("message_label")        self.output_file_name = QtWidgets.QPlainTextEdit(self.centralwidget)        self.output_file_name.setGeometry(QtCore.QRect(230, 90, 371, 41))        font = QtGui.QFont()        font.setPointSize(14)        self.output_file_name.setFont(font)        self.output_file_name.setObjectName("output_file_name")MainWindow.setCentralWidget(self.centralwidget)        self.menubar = QtWidgets.QMenuBar(MainWindow)        self.menubar.setGeometry(QtCore.QRect(0, 0, 653, 21))        self.menubar.setObjectName("menubar")        self.menuFile = QtWidgets.QMenu(self.menubar)        self.menuFile.setObjectName("menuFile")        self.menuAbout = QtWidgets.QMenu(self.menubar)        self.menuAbout.setObjectName("menuAbout")        MainWindow.setMenuBar(self.menubar)        self.statusbar = QtWidgets.QStatusBar(MainWindow)        self.statusbar.setObjectName("statusbar")      MainWindow.setStatusBar(self.statusbar)self.actionOpen_mp4_video_recording = QtWidgets.QAction(MainWindow)      self.actionOpen_mp4_video_recording.setObjectName("actionOpen_mp4_video_recording")        self.actionAbout_vid2text = QtWidgets.QAction(MainWindow)       self.actionAbout_vid2text.setObjectName("actionAbout_vid2text")        self.actionNew = QtWidgets.QAction(MainWindow)        self.actionNew.setObjectName("actionNew")        self.menuFile.addAction(self.actionOpen_mp4_video_recording)        self.menuFile.addAction(self.actionNew)        self.menuAbout.addAction(self.actionAbout_vid2text)        self.menubar.addAction(self.menuFile.menuAction())        self.menubar.addAction(self.menuAbout.menuAction())        self.retranslateUi(MainWindow)        QtCore.QMetaObject.connectSlotsByName(MainWindow)    def retranslateUi(self, MainWindow):        _translate = QtCore.QCoreApplication.translate        MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow"))        self.label.setText(_translate("MainWindow", "Selected video file:"))        self.label_3.setText(_translate("MainWindow", "Output file name:"))        self.label_5.setText(_translate("MainWindow", "Transcribed text:"))        self.transcribe_button.setText(_translate("MainWindow", "Transcribe"))        self.output_file_name.setPlaceholderText(_translate("MainWindow", "my_speech_file.txt"))        self.menuFile.setTitle(_translate("MainWindow", "File"))        self.menuAbout.setTitle(_translate("MainWindow", "About"))        self.actionOpen_mp4_video_recording.setText(_translate("MainWindow", "Open mp4 video recording"))        self.actionAbout_vid2text.setText(_translate("MainWindow", "About vid2text"))        self.actionNew.setText(_translate("MainWindow", "New"))if __name__ == "__main__":    import sys    app = QtWidgets.QApplication(sys.argv)    MainWindow = QtWidgets.QMainWindow()    ui = Ui_MainWindow()    ui.setupUi(MainWindow)    MainWindow.show()    sys.exit(app.exec_())
If we run the Python file, it will launch a window that looks just the UI we created. Apart from generating the various UI components there is no other functionality. We need to add this ourselves. To add the functionality from the earlier simple script, we can start with the menu items by connecting a click on the menu to a class method. Here we write the name of the method (i.e. self.open_audio_file) we want to call without the parentheses (round brackets) using triggered.connect().
self.actionOpen_mp4_video_recording.triggered.connect(self.open_audio_file)
It would be better if we could open a file dialog box and let the user choose a file to work with. To implement a file dialog box, we can us QFileDialog which first needs to be imported from PyQt5 QtWidgets.
from PyQt5.QtWidgets import QFileDialog
The corresponding method uses the QFileDialog to launch a file dialog box so the user can select the required video file. The selected file name and path can be stored in the file_name variable. This returns a tuple with the first element containing the file name/path. We can check it ends in mp4 and if so we can then enable the transcribe button. Next, we can store the filename in the self.mp4_file_name variable. We also clear the message label in case it contained a previous error message. Finally, we can then update the label to display the selected file. If the file does not end in mp4, we can display a suitable message to the user asking them to select an mp4 file.
def open_audio_file(self):    """Open the audio (*.mp4) file."""    file_name = QFileDialog.getOpenFileName()    if file_name[0][-3:] == "mp4":        self.transcribe_button.setEnabled(True)        self.mp4_file_name = file_name[0]        self.message_label.setText("")        self.selected_video_label.setText(file_name[0])    else:        self.message_label.setText("Please select an *.mp4 file")
When a user clicks on the menu item we launch a file dialog box for the user to select the appropriate file.
Next, we can create a link to a method to start the transcription. For this we can use clicked.connect and pass in the method we want to call, in this case process_and_transcribe_audio.
self.transcribe_button.clicked.connect(self.process_and_transcribe_audio)
This method disables the transcription button so it can’t be pressed again until the transcription is completed. It then calls two further methods, one to convert the video to audio and the next to transcribe into text.
def process_and_transcribe_audio(self):    """Process the audio into a textual transcription."""    self.transcribe_button.setEnabled(False)    self.convert_mp4_to_wav()    self.transcribe_audio(self.audio_file)
First, we need to import the AudioFileClip class from moviepy.
from moviepy.editor import AudioFileClip
Next, we can add an initialize method to the class. Here we can set some variables to store the mp4 file name, the output file and the audio (wav) file name that we will convert the video file into.
def __init__(self):    """Initialisation function."""    self.mp4_file_name = ""    self.output_file = ""    self.audio_file = "speech.wav"
The convert method works as before and produces the speech.wav audio file for the transcript.
def convert_mp4_to_wav(self):    """Convert the mp4 video file into an audio file."""    audio_clip = AudioFileClip(self.mp4_file_name)    audio_clip.write_audiofile(self.audio_file)
Once converted into audio format, we can now start the transcription. Here we get the filename from the text the user entered into the output file name text box.
We get the text from the textbox using the toPlainText() method. If they don’t enter anything we can use the default name “my_speech_file.txt”. We check this by looking for the length of text entered, which should be greater than 0. The core transcription code works as before. After this we set the progress bar to 100%, re-enable the transcribe button and clear any messages from the message label.
def transcribe_audio(self, audio_file):    """Transcribe the audio file."""    r = sr.Recognizer()    total_duration = self.get_audio_duration(audio_file) / 10    total_duration = math.ceil(total_duration)    if len(self.output_file_name.toPlainText()) > 0:        self.output_file = self.output_file_name.toPlainText()    else:        self.output_file = "my_speech_file.txt"        for i in range(0, total_duration):        with sr.AudioFile(audio_file) as source:            audio = r.record(source, offset=i*10, duration=10)            self.progress_bar.setValue(i)        f = open(self.output_file, "a")        f.write(r.recognize_google(audio))        f.write(" ")    f.close()    self.progress_bar.setValue(100)    self.transcribe_button.setEnabled(True)    self.message_label.setText("")    self.update_text_output()
We also created a method to determine the length of the audio file (self.get_audio_duration) in the same way as before, as well as a method to update the value of the progress bar.
When we run the code, the GUI freezes and says “not responding”. We can’t close the app, or interact with it in any way. The progress bar also won’t update until the task is complete rendering it useless.
This is especially noticeable with larger recordings (an hour plus). Obviously, this doesn’t look very good and provides a terrible user experience. One way to overcome this is to run these tasks as background tasks.
Using threads
To overcome this issue we can use threads. Threads (threads of execution) let us run tasks simultaneously, so rather than locking the GUI until finished, we can instead run this task in the background. Here we use QThreads, these are more or less similar in functionality to Python threads but integrate better with Qt.
There are two processes that tend to take some time in this application. The first is the conversion of video to audio; the second is the transcription itself. We can use threads for both of these to improve the performance.
Let’s start off with a thread for the video to audio conversion. Here we make a new class called convertVideoToAudioThread using QThread. We can pass any variables we need to interact with into the initialization function __init__, in this case the mp4 file name and audio output file name.
class convertVideoToAudioThread(QThread):    """Thread to convert mp4 video file to wav file."""    def __init__(self, mp4_file_name, audio_file):        """Initialization function."""        QThread.__init__(self)        self.mp4_file_name = mp4_file_name        self.audio_file = audio_file
We can then place the main video to audio conversion code in a run method.
def run(self):    """Run video conversion task."""    audio_clip = AudioFileClip(self.mp4_file_name)    audio_clip.write_audiofile(self.audio_file)
It’s worth noting that we don’t call the run method directly when using threads, instead, it is run by using the threads start() method. Finally, we can add a class destructor that uses the wait() method to block the thread. The del method is called when an object is garbage collected.
def __del__(self):    """Destructor."""    self.wait()
We can then update the convert_mp4_to_wav method to use the thread. Here we also add some messages to update the user about what the application is doing. We then create a thread called convert_thread and create an instance of the convertVideoToAudioThread passing in the mp4 and audio filenames. To start the thread we use the start() method which will execute the run method. Finally, we can call another method when the thread has finished using the finished.connect() method. Here we use it to call our finished_converting() method.
def convert_mp4_to_wav(self):    """Convert the mp4 video file into an audio file."""    self.message_label.setText("Converting mp4 to audio (*.wav)...")    self.convert_thread = convertVideoToAudioThread(self.mp4_file_name, self.audio_file)    self.convert_thread.finished.connect(self.finished_converting)    self.convert_thread.start()
The finished converting method displays a message to the user that the conversion is complete and then runs the transcribe_audio() method. We will also create a thread for the main transcription as well.
def finished_converting(self):    """Reset message text when conversion is finished."""    self.message_label.setText("Transcribing file...")    self.transcribe_audio(self.audio_file)
We create a transcriptionThread class to deal with transcribing. This works in much the same way as the initial thread with a few extra features.
class transcriptionThread(QThread):    """Thread to transcribe file from audio to text."""    change_value = pyqtSignal(int)    def __init__(self, total_duration, audio_file, output_file):        """Initialization function."""        QThread.__init__(self)        self.total_duration = total_duration        self.audio_file = audio_file        self.output_file = output_file    def __del__(self):        """Destructor."""        self.wait()    def run(self):        """Run transcription, audio to text."""        r = sr.Recognizer()        for i in range(0, self.total_duration):            try:                with sr.AudioFile(self.audio_file) as source:                    audio = r.record(source, offset=i*10, duration=10)                    f = open(self.output_file, "a")                    f.write(r.recognize_google(audio))                    f.write(" ")                self.change_value.emit(i)            except:                print("Unknown word detected...")                continue            f.close()
We also added a try, except. This is because if the speech recognizer cannot recognize an input, i.e. due to some background noise; it will generate an UnknownValueError which will terminate the program. The change_value = pyqtSignal(int) is used to emit a signal (an integer) while the thread is running which we can use to update the progress bar. In the run method the line:
self.change_value.emit(i)
This emits the value of the for loop counter. This change in value is picked up by the threads change_value method which in turn calls the set_progress_value method:
self.thread.change_value.connect(self.set_progress_value)def set_progress_value(self, val):    """Update progress bar value."""    increment = int(math.floor(100*(float(val)/self.td)))    self.progress_bar.setValue(increment)
We can then modify the transcribe_audio function to use the new thread.
def transcribe_audio(self, audio_file):    """Transcribe the audio file."""    total_duration = self.get_audio_duration(audio_file) / 10    total_duration = math.ceil(total_duration)    self.td = total_duration        if len(self.output_file_name.toPlainText()) > 0:        self.output_file = self.output_file_name.toPlainText()    else:        self.output_file = "my_speech_file.txt"            self.thread = transcriptionThread(total_duration, audio_file, self.output_file)    self.thread.finished.connect(self.finished_transcribing)    self.thread.change_value.connect(self.set_progress_value)    self.thread.start()
The image below shows the final version. The progress bar updates to show the transcription progress. When complete we load the text file contents into the text area, which automatically adds scroll bars if needed.
Finally, we can add functionality to the new and about menu options. New simply clears the text fields and resets the progress bar. The about option makes use of QMessageBox to display a message with the authors details. We can set the text, title and an icon, in this case, an information icon.
def show_about(self):    """Show about message box."""    msg = QMessageBox()    msg.setWindowTitle("About vid2speech")    msg.setText(" Created by Dr. Alan Davies,\n Senior Lecturer,\n      Health Data Science,\n Manchester University, UK")    msg.setIcon(QMessageBox.Information)    msg.exec_()
The output of which looks like this:
The final code can be seen here:
import wave, contextlib, math, timeimport speech_recognition as srfrom moviepy.editor import AudioFileClipfrom PyQt5 import QtCore, QtGui, QtWidgetsfrom PyQt5.QtWidgets import QFileDialog, QMessageBoxfrom PyQt5.QtCore import QThread, pyqtSignalclass Ui_MainWindow(object):    """Main window GUI."""    def __init__(self):        """Initialisation function."""        self.mp4_file_name = ""        self.output_file = ""        self.audio_file = "speech.wav"    def setupUi(self, MainWindow):        """Define visual components and positions."""        # Main window        MainWindow.setObjectName("MainWindow")        MainWindow.resize(653, 836)        self.centralwidget = QtWidgets.QWidget(MainWindow)        self.centralwidget.setObjectName("centralwidget")        self.label = QtWidgets.QLabel(self.centralwidget)        self.label.setGeometry(QtCore.QRect(50, 20, 161, 41))        # Selected video file label        font = QtGui.QFont()        font.setPointSize(14)        self.label.setFont(font)        self.label.setObjectName("label")        self.selected_video_label = QtWidgets.QLabel(self.centralwidget)        self.selected_video_label.setGeometry(QtCore.QRect(230, 20, 371, 41))        font = QtGui.QFont()        font.setPointSize(8)        self.selected_video_label.setFont(font)        self.selected_video_label.setFrameShape(QtWidgets.QFrame.Box)        self.selected_video_label.setText("")        self.selected_video_label.setObjectName("selected_video_label")        self.label_3 = QtWidgets.QLabel(self.centralwidget)        self.label_3.setGeometry(QtCore.QRect(50, 90, 161, 41))        # Transcribed text box        font = QtGui.QFont()        font.setPointSize(14)        self.label_3.setFont(font)        self.label_3.setObjectName("label_3")        self.transcribed_text = QtWidgets.QTextBrowser(self.centralwidget)        self.transcribed_text.setGeometry(QtCore.QRect(230, 320, 381, 431))        self.transcribed_text.setObjectName("transcribed_text")        self.label_5 = QtWidgets.QLabel(self.centralwidget)        self.label_5.setGeometry(QtCore.QRect(230, 280, 161, 41))        font = QtGui.QFont()        font.setPointSize(14)        self.label_5.setFont(font)        self.label_5.setObjectName("label_5")        self.transcribe_button = QtWidgets.QPushButton(self.centralwidget)        self.transcribe_button.setEnabled(False)        self.transcribe_button.setGeometry(QtCore.QRect(230, 150, 221, 81))        # Transcribe button        font = QtGui.QFont()        font.setPointSize(14)        self.transcribe_button.setFont(font)        self.transcribe_button.setObjectName("transcribe_button")     self.transcribe_button.clicked.connect(self.process_and_transcribe_audio)        # progeress bar        self.progress_bar = QtWidgets.QProgressBar(self.centralwidget)        self.progress_bar.setGeometry(QtCore.QRect(230, 250, 381, 23))        self.progress_bar.setProperty("value", 0)        self.progress_bar.setObjectName("progress_bar")        self.message_label = QtWidgets.QLabel(self.centralwidget)        self.message_label.setGeometry(QtCore.QRect(0, 760, 651, 21))        # Message label (for errors and warnings)        font = QtGui.QFont()        font.setPointSize(8)        self.message_label.setFont(font)        self.message_label.setFrameShape(QtWidgets.QFrame.Box)        self.message_label.setText("")        self.message_label.setObjectName("message_label")        self.output_file_name = QtWidgets.QPlainTextEdit(self.centralwidget)        self.output_file_name.setGeometry(QtCore.QRect(230, 90, 371, 41))        # Output file name        font = QtGui.QFont()        font.setPointSize(14)        self.output_file_name.setFont(font)        self.output_file_name.setObjectName("output_file_name")        # Menubar options        MainWindow.setCentralWidget(self.centralwidget)        self.menubar = QtWidgets.QMenuBar(MainWindow)        self.menubar.setGeometry(QtCore.QRect(0, 0, 653, 21))        self.menubar.setObjectName("menubar")        self.menuFile = QtWidgets.QMenu(self.menubar)        self.menuFile.setObjectName("menuFile")        self.menuAbout = QtWidgets.QMenu(self.menubar)        self.menuAbout.setObjectName("menuAbout")        MainWindow.setMenuBar(self.menubar)        self.statusbar = QtWidgets.QStatusBar(MainWindow)        self.statusbar.setObjectName("statusbar")        MainWindow.setStatusBar(self.statusbar)        self.actionOpen_mp4_video_recording = QtWidgets.QAction(MainWindow)        self.actionOpen_mp4_video_recording.setObjectName("actionOpen_mp4_video_recording") self.actionOpen_mp4_video_recording.triggered.connect(self.open_audio_file)        self.actionAbout_vid2text = QtWidgets.QAction(MainWindow)       self.actionAbout_vid2text.setObjectName("actionAbout_vid2text")        self.actionAbout_vid2text.triggered.connect(self.show_about)        self.actionNew = QtWidgets.QAction(MainWindow)        self.actionNew.setObjectName("actionNew")        self.actionNew.triggered.connect(self.new_project)        self.menuFile.addAction(self.actionOpen_mp4_video_recording)        self.menuFile.addAction(self.actionNew)        self.menuAbout.addAction(self.actionAbout_vid2text)        self.menubar.addAction(self.menuFile.menuAction())        self.menubar.addAction(self.menuAbout.menuAction())        self.retranslateUi(MainWindow)        QtCore.QMetaObject.connectSlotsByName(MainWindow)    def retranslateUi(self, MainWindow):        """Translate UI method."""        _translate = QtCore.QCoreApplication.translate        MainWindow.setWindowTitle(_translate("MainWindow", "MainWindow"))        self.label.setText(_translate("MainWindow", "Selected video file:"))        self.label_3.setText(_translate("MainWindow", "Output file name:"))        self.label_5.setText(_translate("MainWindow", "Transcribed text:"))        self.transcribe_button.setText(_translate("MainWindow", "Transcribe"))      self.output_file_name.setPlaceholderText(_translate("MainWindow", "interview1.txt"))        self.menuFile.setTitle(_translate("MainWindow", "File"))        self.menuAbout.setTitle(_translate("MainWindow", "About"))     self.actionOpen_mp4_video_recording.setText(_translate("MainWindow", "Open mp4 video recording"))        self.actionAbout_vid2text.setText(_translate("MainWindow", "About vid2text"))        self.actionNew.setText(_translate("MainWindow", "New"))    def open_audio_file(self):        """Open the audio (*.mp4) file."""        file_name = QFileDialog.getOpenFileName()        if file_name[0][-3:] == "mp4":            self.transcribe_button.setEnabled(True)            self.mp4_file_name = file_name[0]            self.message_label.setText("")            self.selected_video_label.setText(file_name[0])        else:            self.message_label.setText("Please select an *.mp4 file")    def convert_mp4_to_wav(self):        """Convert the mp4 video file into an audio file."""        self.message_label.setText("Converting mp4 to audio (*.wav)...")        self.convert_thread = convertVideoToAudioThread(self.mp4_file_name, self.audio_file)        self.convert_thread.finished.connect(self.finished_converting)        self.convert_thread.start()    def get_audio_duration(self, audio_file):        """Determine the length of the audio file."""        with contextlib.closing(wave.open(audio_file,'r')) as f:            frames = f.getnframes()            rate = f.getframerate()            duration = frames / float(rate)            return duration    def transcribe_audio(self, audio_file):        """Transcribe the audio file."""        total_duration = self.get_audio_duration(audio_file) / 10        total_duration = math.ceil(total_duration)        self.td = total_duration        if len(self.output_file_name.toPlainText()) > 0:            self.output_file = self.output_file_name.toPlainText()        else:            self.output_file = "my_speech_file.txt"        # Use thread to process in the background and avoid freezing the GUI        self.thread = transcriptionThread(total_duration, audio_file, self.output_file)        self.thread.finished.connect(self.finished_transcribing)        self.thread.change_value.connect(self.set_progress_value)        self.thread.start()    def finished_converting(self):        """Reset message text when conversion is finished."""        self.message_label.setText("Transcribing file...")        self.transcribe_audio(self.audio_file)    def finished_transcribing(self):        """This run when transcription finished to tidy up UI."""        self.progress_bar.setValue(100)        self.transcribe_button.setEnabled(True)        self.message_label.setText("")        self.update_text_output()    def set_progress_value(self, val):        """Update progress bar value."""        increment = int(math.floor(100*(float(val)/self.td)))        self.progress_bar.setValue(increment)    def process_and_transcribe_audio(self):        """Process the audio into a textual transcription."""        self.transcribe_button.setEnabled(False)        self.message_label.setText("Converting mp4 to audio (*.wav)...")        self.convert_mp4_to_wav()    def update_text_output(self):        """Update the text box with the transcribed file."""        f = open(self.output_file, "r")        self.transcribed_text.setText(f.read())        f.close()    def new_project(self):        """Clear existing fields of data."""        self.message_label.setText("")        self.transcribed_text.setText("")        self.selected_video_label.setText("")        self.output_file_name.document().setPlainText("")        self.progress_bar.setValue(0)    def show_about(self):        """Show about message box."""        msg = QMessageBox()        msg.setWindowTitle("About vid2speech")        msg.setText(" Created by Dr. Alan Davies,\n Senior Lecturer,\n Health Data Science,\n Manchester University, UK")        msg.setIcon(QMessageBox.Information)        msg.exec_()class convertVideoToAudioThread(QThread):    """Thread to convert mp4 video file to wav file."""    def __init__(self, mp4_file_name, audio_file):        """Initialization function."""        QThread.__init__(self)        self.mp4_file_name = mp4_file_name        self.audio_file = audio_file    def __del__(self):        """Destructor."""        self.wait()    def run(self):        """Run video conversion task."""        audio_clip = AudioFileClip(self.mp4_file_name)        audio_clip.write_audiofile(self.audio_file)class transcriptionThread(QThread):    """Thread to transcribe file from audio to text."""    change_value = pyqtSignal(int)    def __init__(self, total_duration, audio_file, output_file):        """Initialization function."""        QThread.__init__(self)        self.total_duration = total_duration        self.audio_file = audio_file        self.output_file = output_file    def __del__(self):        """Destructor."""        self.wait()    def run(self):        """Run transcription, audio to text."""        r = sr.Recognizer()        for i in range(0, self.total_duration):            try:                with sr.AudioFile(self.audio_file) as source:                    audio = r.record(source, offset=i*10, duration=10)                    f = open(self.output_file, "a")                    f.write(r.recognize_google(audio))                    f.write(" ")                self.change_value.emit(i)            except:                print("Unknown word detected...")                continue            f.close()if __name__ == "__main__":    import sys    app = QtWidgets.QApplication(sys.argv)    MainWindow = QtWidgets.QMainWindow()    ui = Ui_MainWindow()    ui.setupUi(MainWindow)    MainWindow.show()    sys.exit(app.exec_())
This application provides a very basic example with minimal functionality. This could be improved in many ways, such as providing an option to load either an mp4 ort wav file, being able to choose different speech recognizers and displaying additional information such as file size/duration. Simple applications can be improved with graphical user interfaces which are easy to add in Python using tools like PyQt and QT Designer, we can also overcome performance issues using threading and making computationally expensive tasks run in the background so as not to impact the user experience.