Building a multi-functionality Voice Assistant in 10 minutes | by Sakshi Butala | Towards Data Science

Nowadays people don’t have time to manually search the internet for information or the answers to their questions, rather they expect someone to do it for them, just like a personal assistant who listens to the commands provided and acts accordingly.Thanks to Artificial Intelligence, this personal assistant can now be available to everyone in the form of a voice assistant which is much faster and reliable than a human. An assistant that is even capable of accomplishing difficult tasks like placing an order online, playing music, turning on the lights, etc. just by listening to the users command
In this article, I am going to show you how to build a voice assistant that responds to basic user queries. After reading this article you shall get a basic idea about what web scraping is and how can it be used to build a voice assistant.
Note: You will need to have a basic understanding of the Python language to follow this article.
Voice Assistant
Voice Assistant
2. Web Scraping
3. Implementation
Voice Assistant is a software agent that can perform tasks or services for an individual based on commands or questions. In general, voice assistants react to voice commands and give the user relevant information about his/her queries.
The assistant can understand and react to specific commands given by the user like playing a song on YouTube or knowing about the weather. It will search and/or scrape the web to find the response to the command in order to satisfy the user.
Presently voice assistants are already able to process orders of products, answer questions, perform actions like playing music or start a simple phone call with a friend.
The implemented voice assistant can perform the following tasks:
Provide weather detailsProvide corona updatesProvide latest newsSearch the meaning of a wordTake notesPlay YouTube videosShow location on Google MapsOpen websites on Google Chrome
Provide weather details
Provide corona updates
Provide latest news
Search the meaning of a word
Take notes
Play YouTube videos
Show location on Google Maps
Open websites on Google Chrome
When building a voice assistant, there are 2 important libraries that you should consider. Python’s SpeechRecognition package helps the voice assistant understand the user. It can be implemented as follows:
import speech_recognition as sr#initalises the recognizerr1 = sr.Recognizer()#uses microphone to take the inputwith sr.Microphone() as source:    print('Listening..')    #listens to the user    audio = r1.listen(source)    #recognises the audio and converts it to text    audio = r1.recognize_google(audio)
Python’s pyttsx3 package helps the voice assistant respond to the user by converting the text to audio. It can be implemented as follows:
import pyttsx3#initialises pyttsx3engine = pyttsx3.init('sapi5')#converts the text to audioengine.say('Hello World')engine.runAndWait()
Web Scraping refers to the extraction of data from websites. The data on the websites are unstructured. Web scraping helps collect these unstructured data and store it in a structured form.
Some applications of web scraping include:
Scraping Social Media such as Twitter to collect tweets and comments for performing sentiment analysis.
Scraping E-Commerce Website such as Amazon to extract product information for data analysis and predicting market trends.
Scraping E-Mail Address to collect email IDs and then send bulk emails for marketing and advertisement purpose.
Scraping Google Images to create datasets for training a Machine Learning or Deep Learning model.
Although it can be done manually, Python’s library Beautiful Soup makes it easier and faster to scrape the data.
To extract data using web scraping with python, you need to follow these basic steps:
Find the URL of the website that you want to scrapeExtract the entire code of the websiteInspect the website and find the data you want to extractFilter the code using html tags to get the desired dataStore the data in the required format
Find the URL of the website that you want to scrape
Extract the entire code of the website
Inspect the website and find the data you want to extract
Filter the code using html tags to get the desired data
Store the data in the required format
Let us start by importing the following libraries in you python notebook as shown below:
import requests from bs4 import BeautifulSoup import reimport speech_recognition as sr from datetime import dateimport webbrowserimport pyttsx3
Now let us create our main function which consists of a bunch of if-else statements which tell the assistant how to respond under certain conditions.
engine = pyttsx3.init('sapi5')r1 = sr.Recognizer()with sr.Microphone() as source:    print('Listening..')    engine.say('Listening')    engine.runAndWait()    audio = r1.listen(source)    audio = r1.recognize_google(audio)    if 'weather' in audio:        print('..')        words = audio.split(' ')        print(words[-1])        scrape_weather(words[-1])    elif 'covid' in audio:        print('..')        words = audio.split(' ')        corona_updates(words[-1])    elif 'meaning' in audio:        print('..')        words = audio.split(' ')        print(words[-1])        scrape_meaning(words[-1])            elif 'take notes' in audio:        print('..')        take_notes()        print('Noted!!')            elif 'show notes' in audio:        print('..')        show_notes()        print('Done')            elif 'news' in audio:        print('..')        scrape_news()            elif 'play' in audio:        print('..')        words = audio.split(' ')        print(words[-1])        play_youtube(audio)            elif 'open' in audio:        print('..')        words = audio.split('open')        print(words[-1])        link = str(words[-1])        link = re.sub(' ', '', link)        engine.say('Opening')        engine.say(link)        engine.runAndWait()        link = f'https://{link}.com'        print(link)        webbrowser.open(link)            elif 'where is' in audio:        print('..')        words = audio.split('where is')        print(words[-1])        link = str(words[-1])        link = re.sub(' ', '', link)        engine.say('Locating')        engine.say(link)        engine.runAndWait()        link = f'https://www.google.co.in/maps/place/{link}'        print(link)        webbrowser.open(link)                        else:        print(audio)        print('Sorry, I do not understand that!')        engine.say('Sorry, I do not understand that!')        engine.runAndWait()
Case 1: If the user wants to know about the Weather, he/she can ask the assistant “Hey! What is the weather today in Mumbai?”
Since the word “weather” is present in the audio, the function scrape_weather(words[-1]) will be called with the parameter as “Mumbai”.
Let us take a look at this function.
def scrape_weather(city):    url = 'https://www.google.com/search?q=accuweather+' + city    page = requests.get(url)soup = BeautifulSoup(page.text, 'lxml')links = [a['href']for a in soup.findAll('a')]    link = str(links[16])    link = link.split('=')    link = str(link[1]).split('&')    link = link[0]        page = requests.get(link, headers={'User-Agent': 'Mozilla/5.0'})soup = BeautifulSoup(page.content, 'lxml')         time = soup.find('p', attrs = {'class': 'cur-con-weather-card__subtitle'})    time = re.sub('\n', '', time.text)    time = re.sub('\t', '', time)    time = 'Time: ' + timetemperature = soup.find('div', attrs = {'class':'temp'})    temperature = 'Temperature: ' + temperature.text        realfeel = soup.find('div', attrs = {'class': 'real-feel'})    realfeel = re.sub('\n', '',realfeel.text)    realfeel = re.sub('\t', '',realfeel)    realfeel = 'RealFeel: ' + realfeel[-3:] + 'C'climate = soup.find('span', attrs = {'class':'phrase'})    climate = "Climate: " + climate.text        info = 'For more information visit: ' + link         print('The weather for today is: ')    print(time)    print(temperature)    print(realfeel)    print(climate)    print(info)    engine.say('The weather for today is: ')    engine.say(time)    engine.say(temperature)    engine.say(realfeel)    engine.say(climate)    engine.say('For more information visit accuweather.com' )    engine.runAndWait()
We shall use the website “accuweather.com ” to scrape all the weather related information. The function request.get(url) sends a GET request to the URL whose entire HTML code is extracted using BeautifulSoup(page.text, ‘lxml’)
Once the code is extracted, we shall inspect the code to find the data of interest. For example, the numeric value of temperature present in the following format
<div class = "temp">26°</div>
can be extracted using
soup.find('div', attrs = {'class':'temp'})
Similarly we extract the time, the real feel and the climate, and using engine.say(), we make the assistant respond back to the user.
Case 2: If the user wants current COVID-19 updates, he/she can ask the assistant “Hey! Can you give me the COVID updates of India?” or “Hey! Can you give me the COVID updates of the World?”
Since the word “covid” is present in the audio, the function corona_updates(words[-1]) will be called with the parameter as “India” or “World”
Let us take a look at this function.
def corona_updates(audio):        audio = audiourl = 'https://www.worldometers.info/coronavirus/'    page = requests.get(url)soup = BeautifulSoup(page.content, 'lxml')totalcases = soup.findAll('div', attrs =  {'class': 'maincounter-number'})    total_cases = []    for total in totalcases:        total_cases.append(total.find('span').text)world_total = 'Total Coronavirus Cases: ' + total_cases[0]    world_deaths = 'Total Deaths: ' + total_cases[1]    world_recovered = 'Total Recovered: ' + total_cases[2]        info = 'For more information visit: ' + 'https://www.worldometers.info/coronavirus/#countries'if 'world' in audio:        print('World Updates: ')        print(world_total)        print(world_deaths)        print(world_recovered)        print(info)else:        country = audiourl = 'https://www.worldometers.info/coronavirus/country/' + country.lower() + '/'        page = requests.get(url)soup = BeautifulSoup(page.content, 'lxml')totalcases = soup.findAll('div', attrs =  {'class': 'maincounter-number'})        total_cases = []        for total in totalcases:            total_cases.append(total.find('span').text)total = 'Total Coronavirus Cases: ' + total_cases[0]        deaths = 'Total Deaths: ' + total_cases[1]        recovered = 'Total Recovered: ' + total_cases[2]info = 'For more information visit: ' + urlupdates = country + ' Updates: 'print(updates)        print(total)        print(deaths)        print(recovered)        print(info)                engine.say(updates)        engine.say(total)        engine.say(deaths)        engine.say(recovered)        engine.say('For more information visit: worldometers.info')        engine.runAndWait()
We shall use the website “worldometers.info ” to scrape all the corona related information. The function request.get(url) sends a GET request to the URL whose entire HTML code is extracted using BeautifulSoup(page.text, ‘lxml’)
Once the code is extracted, we shall inspect the code to find the numerical values of the total corona cases, total recovered and total deaths.
These value is present inside a span of a div having class “maincounter-number” as shown below.
<div id="maincounter-wrap" style="margin-top:15px"><h1>Coronavirus Cases:</h1><div class="maincounter-number"><span style="color:#aaa">25,091,068 </span></div></div>
These can be extracted as follows.
totalcases = soup.findAll('div', attrs =  {'class': 'maincounter-number'})    total_cases = []    for total in totalcases:        total_cases.append(total.find('span').text)
We first find all the div elements having class “maincounter-number”. Then we iterate through each div to obtain the span containing the numerical value.
Case 3: If the user wants to know about the News, he/she can ask the assistant “Hey! Can you give me the news updates?”
Since the word “news” is present in the audio, the function scrape_news() will be called.
def scrape_news():    url = 'https://news.google.com/topstories?hl=en-IN&gl=IN&ceid=IN:en '    page = requests.get(url)    soup = BeautifulSoup(page.content, 'html.parser')    news = soup.findAll('h3', attrs = {'class':'ipQwMb ekueJc RD0gLb'})    for n in news:        print(n.text)        print('\n')        engine.say(n.text)    print('For more information visit: ', url)    engine.say('For more information visit google news')    engine.runAndWait()
We shall use “Google News” to scrape the headlines of news. The function request.get(url) sends a GET request to the URL whose entire HTML code is extracted using BeautifulSoup(page.text, ‘lxml’)
Once the code is extracted, we shall inspect the code to find the headlines of the latest news.
These headlines are present inside the href attribute of the h3 tag having class “ipQwMb ekueJc RD0gLb” as shown below.
<h3 class="ipQwMb ekueJc RD0gLb"><a href="./articles/CAIiEA0DEuHOMc9oauy44TAAZmAqFggEKg4IACoGCAoww7k_MMevCDDW4AE?hl=en-IN&amp;gl=IN&amp;ceid=IN%3Aen" class="DY5T1d">Rhea Chakraborty arrest: Kubbra Sait reminds ‘still not a murderer’, Rhea Kapoor says ‘we settled on...</a></h3>
We first find all the h3 elements having class “ipQwMb ekueJc RD0gLb”. Then we iterate through each element to obtain the text (news headline) present inside the href attribute.
Case 4: If the user wants to know the Meaning of any word, he/she can ask the assistant “Hey! What is the meaning of scraping?”
Since the word “meaning” is present in the audio, the function scrape_meaning(words[-1]) will be called with the parameter as “scraping”
Let us take a look at this function.
def scrape_meaning(audio):    word = audio    url = 'https://www.dictionary.com/browse/' + word    page = requests.get(url)    soup = BeautifulSoup(page.content, 'html.parser')    soup    meanings = soup.findAll('div', attrs =  {'class': 'css-1o58fj8 e1hk9ate4'})    meaning = [x.text for x in meanings]    first_meaning = meaning[0]    for x in meaning:        print(x)        print('\n')    engine.say(first_meaning)    engine.runAndWait()
We shall use the website “Dictionary.com” to scrape the meanings. The function request.get(url) sends a GET request to the URL whose entire HTML code is extracted using BeautifulSoup(page.text, ‘lxml’)
Once the code is extracted, we shall inspect the code to find all html tags containing the meaning of the word passed as the parameter.
These values are present inside the div having class “css-1o58fj8 e1hk9ate4” as shown below.
<div value="1" class="css-kg6o37 e1q3nk1v3"><span class="one-click-content css-1p89gle e1q3nk1v4" data-term="that" data-linkid="nn1ov4">the act of a person or thing that <a href="/browse/scrape" class="luna-xref" data-linkid="nn1ov4">scrapes</a>. </span></div>
We first find all the div elements having class “css-1o58fj8 e1hk9ate4”. Then we iterate through each element to obtain the text (meaning of the word)present inside the div.
Case 5: If the user wants the assistant to Take Notes, he/she can ask the assistant “Hey! Can you take notes for me?”
Since the word “take notes” is present in the audio, the function take_notes() will be called.
Let us take a look at this function.
def take_notes():r5 = sr.Recognizer()      with sr.Microphone() as source:        print('What is your "TO DO LIST" for today')        engine.say('What is your "TO DO LIST" for today')        engine.runAndWait()        audio = r5.listen(source)        audio = r5.recognize_google(audio)        print(audio)        today = date.today()        today = str(today)        with open('MyNotes.txt','a') as f:            f.write('\n')            f.write(today)            f.write('\n')            f.write(audio)            f.write('\n')            f.write('......')            f.write('\n')            f.close()         engine.say('Notes Taken')        engine.runAndWait()
We start by initialising the recogniser to ask the user for their ‘To-Do list’. We then listen to the user and recognise the audio using recognize_google. Now we will open a notepad named “MyNotes.txt” and jot down the notes given by the user along with the date.
We will then create another function named show_notes() which will read out the notes/To-Do list for today from the notepad named “MyNotes.txt”.
def show_notes():    with open('MyNotes.txt', 'r') as f:        task = f.read()        task = task.split('......')    engine.say(task[-2])    engine.runAndWait()    
Case 6: If the user wants to Play YouTube Video, he/she can ask the assistant “Hey! Can you play Hypnotic?”
Since the word “play” is present in the audio, the function play_youtube(words[-1]) will be called with “hypnotic” passed as the parameter.
Let us take a look at this function.
def play_youtube(audio):url = 'https://www.google.com/search?q=youtube+' + audio    headers = {    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'    }    engine.say('Playing')    engine.say(audio)    engine.runAndWait()    page = requests.get(url, headers=headers)    soup = BeautifulSoup(page.content, 'html.parser')    link = soup.findAll('div', attrs = {'class':'r'})    link = link[0]    link = link.find('a')    link = str(link)    link = link.split('"')    link = link[1]webbrowser.open(link)
We will use Google Videos to search for the video title, and open the first link to play the YouTube video which is present in the div element having class ‘r’.
Case 7: If the user wants to Search for Location, he/she can ask the assistant “Hey! Where is IIT Bombay?”
Since the word “where is” is present in the audio, the following code will be executed.
(This code is present inside the if-else loop of the main function)
elif 'where is' in audio:        print('..')        words = audio.split('where is')        print(words[-1])        link = str(words[-1])        link = re.sub(' ', '', link)        engine.say('Locating')        engine.say(link)        engine.runAndWait()        link = f'https://www.google.co.in/maps/place/{link}'        print(link)        webbrowser.open(link)
We will join the location provided by the user with the Google Maps link and the use webbrowser.open(link) to open the link locating ‘IIT Bombay’.
Case 8: If the user wants to Open a Website, he/she can ask the assistant “Hey! Can you open Towards Data Science?”
Since the word “open” is present in the audio, the following code will be executed.
(This code is present inside the if-else loop of the main function)
elif 'open' in audio:        print('..')        words = audio.split('open')        print(words[-1])        link = str(words[-1])        link = re.sub(' ', '', link)        engine.say('Opening')        engine.say(link)        engine.runAndWait()        link = f'https://{link}.com'        print(link)        webbrowser.open(link)
We will join the website name provided by the user with the standard format of any URL i.e https://{website name}.com to open the website.
So that is how we create a simple voice assistant. You can modify the code to to add more features like performing basic mathematical calculation, telling jokes, creating a reminder, changing desktop wallpapers, etc.
You can find the whole code in my GitHub repository linked below.