Speech Recognition in Real-Time using Python | by Behic Guven | Towards Data Science

In this post, I will show you how to convert your speech to text in real-time using Python. We will write a program that understands what we are saying and translates it into written words. This translation is known as speech recognition. Speech Recognition is a pretty exciting and fun field to get started with Machine Learning and Artificial Intelligence.
In my previous posts, I’ve covered similar topics like converting speech to text using Google’s Speech Recognition API and building a speech emotion recognizer using Python. In these projects, the recognition and the training was done over static audio files. Today, we will be doing something a little more advanced and more extraordinary in this project. We will learn how to convert our speech to text in real-time.
If you are ready, let’s get to work!
Getting Started
Step 1 — Libraries
Step 2 — Live Stream
Step 3 — Async Function
Final Step — Testing the Real-time Recognizer
In today’s tutorial, we will use AssemblyAI’s real-time transcription API. As a software engineer and tech enthusiast, it’s great to see that the speech recognition field is overgrowing. I believe that these kinds of developments are making our lives easier.
For example, one area where real-time transcription can be helpful is online lectures. While listening to the instructor, we can also read the instructor’s speech as a subtitle. This feature will make the learning process easier for people with hearing disabilities. This is just a basic example; I am sure there are many areas where real-time transcription can be implemented.
By the way, the AssemblyAI’s API is free to access. You will get a unique API key after creating an account. We will need that API key to use the services and features.
We are going to use three Python libraries in this project. They can list as follows: PortAudio, PyAudio, and Websockets. Here is a short definition for each of them.
PortAudio is a free, cross-platform, open-source audio input/output library. It is also a prerequisite library to get the subsequent library, PyAudio, to work.
Official documentaion.
PyAudio is a free, open-source library to play and record audio files on various platforms. This library works as an extension of PortAudio.
Official documentation.
Websockets is a dependency built on top of the asyncio, which is Python’s standard asynchronous I/O framework. It provides an elegant coroutine-based API.
Official documentation.
Let’s get to work by installing them.
We are going to run the following lines inside the terminal window. To install PortAudio we have to use a python library manager called brew.
brew install portaudio
After that, it’s time to install the other two libraries. This time we are using another python library manager called pip. Did you know that we can install one or more libraries in just one line?
pip install pyaudio websockets
Our libraries are installed. Time to get the streaming system ready!
Time to open our coding platform and start coding. Our code is static, so I will go with Atom text editor. The whole code will be in one python file. Easy to run.
import pyaudio FRAMES_PER_BUFFER = 3200FORMAT = pyaudio.paInt16CHANNELS = 1RATE = 16000p = pyaudio.PyAudio() stream = p.open(   format=FORMAT,   channels=CHANNELS,   rate=RATE,   input=True,   frames_per_buffer=FRAMES_PER_BUFFER)
In the code above, we are defining the parameters of our stream system. Audio world is really complicated when it comes to details. I recommend searching online about each of the variables to understand them better.
In this step, we will write an async function to send and receive audio data with the API. Async functions are under the topic Coroutines and Tasks.
Coroutines declared with async syntax are the preferred way of writing asyncio applications. Since we want to stream the speech in real-time, this is a great way to display the recognized words into the terminal window with some wait times.
First thing first, let’s import the python libraries.
import websocketsimport asyncioimport base64import jsonfrom configure import auth_key
Now, we will write a function, which I named: send_receive_data(). Many things are going on within this function:
Connecting the program to AssemblyAI’s endpoint url.
The audio data is being transferred to the API to be recognized.
The API returns a json data with the recognized speech.
It is basically a two-way highway, where we send our voice data through the microphone of our laptop/ pc. Then, receive the recognized speech from the API. And everything happens so fast.
Isn’t that cool? And guess what, no rush hour :)
# the AssemblyAI endpoint we're going to hitURL = "wss://api.assemblyai.com/v2/realtime/ws?sample_rate=16000" async def send_receive_data():   print(f'Connecting websocket to url ${URL}')   async with websockets.connect(       URL,       extra_headers=(("Authorization", "our API key"),),       ping_interval=5,       ping_timeout=20   ) as _ws:       await asyncio.sleep(0.1)       print("Receiving SessionBegins ...")       session_begins = await _ws.recv()       print(session_begins)       print("Sending messages ...")       async def send():           while True:               try:                   data = stream.read(FRAMES_PER_BUFFER)                   data = base64.b64encode(data).decode("utf-8")                   json_data = json.dumps({"audio_data":str(data)})                   await _ws.send(json_data)               except websockets.exceptions.ConnectionClosedError as e:                   print(e)                   assert e.code == 4008                   break               except Exception as e:                   assert False, "Not a websocket 4008 error"               await asyncio.sleep(0.01)                     return True             async def receive():           while True:               try:                   result_str = await _ws.recv()                   print(json.loads(result_str)['text'])               except websockets.exceptions.ConnectionClosedError as e:                   print(e)                   assert e.code == 4008                   break               except Exception as e:                   assert False, "Not a websocket 4008 error"             send_result, receive_result = await asyncio.gather(send(), receive())
The send_receive_data() function is all set!
Now, let’s put it in a loop so that our program listens continuously to our voice. Since we don’t want it to run forever, there is a timeout, which is 1 minute. Asyncio will do this for us.
asyncio.run(send_receive_data())
After running main.py in the the terminal, the program starts to listen. And display the recognized speech right at that moment. I read the introduction part of this article, and here are results.
Congrats! In this hands-on tutorial, we learned how to build a program that detects our speech and converts it into text in real-time using Python. I enjoy creating and working on these projects because they are natural applications of machine learning and artificial intelligence in our daily lives. Hoping that you enjoyed reading this guide and learned something new today.
I am Behic Guven, and I love sharing stories on programming, education, and life. Subscribe to my content to stay inspired. Ty,
If you are wondering what kind of articles I write, here are some:
Building a Face Recognizer using Python
Step-by-Step Guide — Building a Prediction Model in Python
Building a Speech Emotion Recognizer using Python