How to Make an AI-Assisted Phone Service with Twilio
Tags: python, programming, tutorial, computerscience
Published: 2022-06-24 02:51:07

This article was originally posted on

``` 
https://www.realpythonproject.com/how-to-make-an-ai-assisted-phone-service-with-twilio/
```

<!-- wp:paragraph -->
<p>If you’ve ever called a business to book an appointment or tried to chat on a website to get your questions answered, there’s a good chance you have already interacted with an AI-assisted phone service. AI-assisted phone services have existed for a long time, and with the <a href="https://www.topbots.com/leading-nlp-language-models-2020/">recent advancements in natural language processing (NLP)</a>, they have become more sophisticated.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>AI-assisted phone services can be used in a number of different ways. For example, chatbots can be used to answer customer questions, while interactive voice responses (IVRs) can be used to take orders or book appointments. Additionally, AI-assisted phone services can be used to gather customer feedback and track customer satisfaction. This feedback can then be used to improve customer experience and ensure that customers are happy with the services they receive.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>In this tutorial, you will build a bot that will be used to save a user’s information in a database. This bot will be built using <a href="https://www.twilio.com/">Twilio</a> and developed in <a href="https://replit.com/">Replit</a>. Replit is a powerful browser-based integrated development environment (IDE) that lets you develop your projects online. You will be using various features provided by Replit, such as their unit testing and their live servers and database. Here is a high-level overview of how the bot will work:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/FIEOW3a.png" alt="Architecture of the AI-assisted phone service"/></figure>
<!-- /wp:image -->

<!-- wp:heading -->
<h2>What Are AI-Assisted Phone Services</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>AI-assisted phone services are powered by voice recognition software and NLP, meaning that they can understand the user’s voice and respond in a way that is natural for humans. This makes the user experience more like talking to a human than using a traditional automated phone service.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>There are two general classes of AI-assisted phone services:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li><strong>Personal assistant phone services:</strong> They can do things like provide information about local businesses, book appointments, and connect people with customer service representatives. They are helpful in cases where the user cannot look up the information themselves, such as when driving.</li><li><strong>Customer service phone services:</strong> They provide customer support for businesses. They can take orders, book appointments, cancel reservations, and transfer people to the right department. They are helpful in cases where these tasks need to be done without human intervention or when it would be more practical to do so.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Here are some of the benefits of AI-assisted phone services:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>They are typically cheaper than hiring multiple people to work in shifts and around the clock.</li><li>They have the ability to simultaneously handle multiple customers.</li><li>They can be programmed to automate tasks, like creating tickets and saving users’ feedback.</li><li>They enable users to experience a more efficient and less frustrating customer service experience.</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>As mentioned earlier, the most common AI-assisted phone services are chatbots and IVRs. Chatbots are computer programs that can mimic human conversation via text. They are commonly used to simulate a human’s conversation in online customer service. They are also used to communicate with customers on behalf of brands and businesses, such as in a phone app that lets people order food or ask for a local business’ operating hours. In <a href="https://developers.facebook.com/products/messenger/">Facebook Messenger</a>, they can be added as a friend and sent messages to make exchanges feel more humanlike. Chatbots can also be used in <a href="https://discord.com/">Discord</a> and <a href="https://slack.com/">Slack</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>IVRs are computer programs that interact with people by voice. They are also commonly used to provide customer service. IVRs can be used to ask people questions about their accounts or to direct them to the right department, to provide information about a product or service, or even to take orders over the phone. A customer can call an IVR and press the appropriate buttons corresponding to the information they have or need, and the IVRs then automatically direct them to the right department or provide the details the customer has requested.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Many companies use these technologies in their online support structures. Although it does not completely eliminate the requirement of an actual person for customer support, it makes the process more efficient.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>Building Your Own AI-Assisted Phone Service</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>This tutorial is divided into two sections. First, you will build your own AI-assisted phone service using Twilio. Second, you will build a Flask API to receive data from the phone service and store it in a database. All the development will be done using <a href="https://replit.com/">Replit</a>. You can <a href="https://replit.com/signup">sign up for free here</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Here are some useful links for easy access:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><a href="https://replit.com/@rahulbanerjee26/TwilioAIPhoneServiceAPI?v=1">API repl</a></li><li><a href="https://replit.com/@rahulbanerjee26/TwilioAIPhoneServiceBot?v=1">Twilio bot repl</a></li><li><a href="https://github.com/rahulbanerjee26/Twilio-Bot">GitHub repo</a></li></ul>
<!-- /wp:list -->

<!-- wp:heading {"level":3} -->
<h3>Working with Twilio</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>You will need a <a href="https://www.twilio.com/try-twilio">Twilio account</a> to follow along. After signing up, you should be redirected to your dashboard.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To start, go to <strong>Account</strong> &gt; <strong>API keys &amp; tokens</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/hkoe9ug.png" alt="Twilio Dashboard"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>You should see a button to create an API key, which you will need in order to consume Twilio’s API.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/z0bZ26I.png" alt="Twilio API"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>After you create your API key, you will need to save a couple of strings: the Account SID and the Auth Token.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Now, create a new Python project on <a href="https://replit.com/">Replit</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/TVNybYP.png" alt="Create a repl"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Normally, you would have to create an .env file in which to store your credentials. However, Replit has an alternative for this. Click on the <strong>lock</strong> icon on your sidebar.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/ANTkUcs.png" alt="Lock icon"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Save your Twilio Account SID (<code>account_sid</code>) and Auth Token (<code>account_key</code>), and then scroll down. You should see an option to insert the code to read them using Python.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center"} -->
<figure class="wp-block-image aligncenter"><img src="https://i.imgur.com/UJr0gX6.png" alt="Use secrets in Python"/></figure>
<!-- /wp:image -->

{% embed https://gist.github.com/rahulbanerjee26/6fddc1283e2aa721fbc6fb7a1d37f343 %}

<!-- wp:paragraph -->
<p>Twilio has a <a href="https://www.twilio.com/docs/libraries/python">Python wrapper</a> for their API, and Replit provides a simple way to install packages. On the sidebar, select the cube-like icon and search for Twilio. Click on the <strong>plus</strong> icon to install the package.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/6ZVWBFl.png" alt="Install Twilio"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>You will have to create an <a href="https://www.twilio.com/docs/autopilot/api/assistant">Assistant resource</a>:</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/51a0dd19b0f87b9cab96b5964213ccce %}


<!-- wp:paragraph -->
<p>You can use the code to create the Assistant inside a try-except block to avoid creating a new resource every time you run the script. The code in the except block is run when an Assistant resource with the same unique name is created. In this case, you can simply fetch the existing Assistant resource. You can run the repl to create your own. By default, Replit will run the file named <code>main.py</code>. However, if you wish to, you can click on <strong>shell</strong> and run your file using the <code>python3</code> command.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/vb0dbR2.png" alt="Run command"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Replit has a really cool feature that can explain what a code snippet does. Simply highlight the code snippet and click on <strong>Explain Code</strong>. Note that you will need to upgrade to the <a href="https://replit.com/pricing">Hacker Plan</a> in order to use this feature.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/gzRYwPI.png" alt="Explain Code"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Replit will analyze your code comment and generate an explanation for you. You can also create a thread to add comments (similar to how you would do in code reviews). This was the explanation generated for the highlighted code snippet:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code class="">1. The code creates a variable called the assistant and sets it to None.

2. The code then tries to create an assistant with the unique name "twilio-bot-repl-assistant".

3. If that fails, then the code fetches the assistant with the unique name "twilio-bot-repl-assistant".</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>As you can see, it is pretty accurate.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Next, you can add a task to the above Assistant. A task is basically an action your Assistant resource can perform. You can read more about it in the <a href="https://www.twilio.com/docs/autopilot/api/task">Twilio Docs</a>.</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/18b7fc10ad0ba6c2b495874b8b3b8c61 %}

<!-- wp:paragraph -->
<p>Like the Assistant resource, you can use a try-except block again to avoid creating multiple tasks. Right now, an empty object is being passed to the actions parameter. This will be updated with the action you want your Assistant resource to perform. You can read more about the actions Twilio supports <a href="https://www.twilio.com/docs/autopilot/actions">in their docs</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Now, we want the Assistant resource to ask the user for their name and phone number. For our use case, we will use the <a href="https://www.twilio.com/docs/autopilot/actions/collect"><code>collect</code> action</a>. It is equivalent to an HTML form element in many ways.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Below is the syntax for a <code>collect</code> action:</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/33e535b6ccb23596efd037fd0d5c3531 %}


<!-- wp:paragraph -->
<p>You can add multiple actions, but for now, we just need a single <code>collect</code> action. You can add multiple questions to a single <code>collect</code> action. Here’s what mine looks like:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/tnFX8mJ.png" alt="<code&gt;collect</code&gt; action"/></figure>
<!-- /wp:image -->

{% embed https://gist.github.com/rahulbanerjee26/a5cb74a5b0e495e168c30c5ed312815f %}

<!-- wp:paragraph -->
<p>I used the <a href="https://www.twilio.com/docs/autopilot/built-in-field-types">built-in types</a> provided by Twilio. Optionally, you can define your types by following <a href="https://www.twilio.com/docs/autopilot/api/field-type">these instructions</a>. In the value for the key <code>uri</code> in <code>redirect</code>, you will notice a variable <code>api_url</code>. If you have a functioning API endpoint, you can create a variable named <code>api_url</code> and set it to your API’s URL. If you do not have a functioning API, you do not need to create a variable and can leave the value for <code>uri</code> as it is. You can follow the subsequent sections to build an API and update the value for <code>uri</code> later.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Now that the task is created, you will need to add sample phrases to trigger this task:</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/af90985d286f714befd36be12e50481e %}


<!-- wp:paragraph -->
<p>You can add multiple samples. Twilio recommends adding ten samples per task, but for this tutorial, one will suffice. Whenever the user says “register,” your Assistant resource will perform the collection action you declared earlier.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Now you can build your model:</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/d44c5d660042e19830a24b90179fe899 %}


<!-- wp:paragraph -->
<p>This process will take some time. Head over to your dashboard and click on <strong>Explore Products</strong> &gt; <strong>Autopilot</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/m3R7vrG.png" alt="Autopilot"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Click on <strong>Your bots</strong> and check the <strong>Last model build</strong> time for your Assistant resource.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/gPHDqns.png" alt="Last model build"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>After the build process is complete, select your model and click on <strong>Simulator</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/fs6AZa6.png" alt="Simulator"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>This will allow you to test your model. Since you do not have an API right now, the Assistant will simply collect the information and will not send it to an API. Turn on the <strong>Debug view</strong> to check which task is being performed by the Assistant. You can also see the values being stored in the variables you created using the <code>collect</code> action earlier.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>On the right-hand menu, you should see an option called <strong>Channels</strong>. This will let you add your bot to a Twilio phone number so the bot can work as a chatbot or an IVR.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Building the API</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>To start, create a new Python project on Replit using one of their existing templates. In the <code>main.py</code> file, you should see text asking you if you would like to choose an example. Click on the examples link and select <strong>Server (Flask)</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/JDofu8u.png" alt="Server (Flask)"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Click on the <strong>Run</strong> button to run the Flask server. Replit will automatically install all the required dependencies.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Normally, you would have to use a service like <a href="https://ngrok.com/">ngrok</a> to expose your Flask server via a public URL, but Replit handles that for you. As long your server is running, it can be publically accessible by anyone via the URL shown on the right:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/mvlh71r.png" alt="URL"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>You can set your repl to <strong>Always On</strong>, which basically means that your Flask server will be running even if your close repl, so in a way, it deploys your Flask server. Note that this feature is only available on the Hacker Plan, so you will have to upgrade if you want to use it.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/Oin1U7x.png" alt="**Always On**"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Now create a new endpoint that will accept both GET requests and POST requests. A GET request will return the users who have previously registered, while the POST request will save the information of the user. The Twilio bot you created in the previous section will make a POST request.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Replit has a built-in database that can be used to store the user information:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/Fky9fme.png" alt="Replit DB"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Some helpful code snippets to perform create, read, update, and delete (CRUD) operations will be shown when you click on the database icon, almost like a cheat sheet. Below is the code snippet to handle GET requests:</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/b2f420394555d4682100567c31acf736 %}

<!-- wp:paragraph -->
<p>This code simply reads all the rows in the database and returns them as an array. Replit also allows you to add unit tests. You can add a unit test to verify that the endpoint handles GET requests as expected. The code in the image below creates a simple unit test that checks the response status code of the GET requests and compares the length of the data in the database to the length of the data returned by the GET request:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/H87kd2E.png" alt="Unit test"/></figure>
<!-- /wp:image -->

{% embed https://gist.github.com/rahulbanerjee26/d0e687f5706414b53607cd3b5a0413ec %}

<!-- wp:paragraph -->
<p>You will have to add a couple of import statements for your unit tests to work. Click on <strong>Setup</strong> and add the following imports:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/IoV5i1h.png" alt="Imports"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Click on <strong>Run tests</strong>, and you should see a success message saying that all your unit test cases passed.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Now add the code to handle the POST requests:</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/b8db5c026e3aafdcee66a0efc403bc25 %}

<!-- wp:paragraph -->
<p>Here, the Twilio bot sends a deeply nested object. You can print the data in <code>request.form</code> to check the format of the data. The above code gets the username and phone number, and stores them in the Replit database. Then it returns a Twilio action object so the bot can respond to the user once their information is saved.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Adding the API URL to Your Twilio Bot</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Before testing the bot, you will have to add your API’s endpoint to the Twilio task you created earlier. This can be done using either the Twilio Console or the Twilio API.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4>Using Twilio Console</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Go to the Twilio Console and navigate to your Twilio bot, as shown in the previous sections. Then go to <strong>Tasks</strong> and select the task you already created.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/4d97IrU.png" alt="Tasks"/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Initially, you had the following in your action:</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/ff6031d814e6f361396062c526d1f076 %}

<!-- wp:paragraph -->
<p>Update the value for <code>uri</code> with your API’s URL. In my case, it looks like this:</p>
<!-- /wp:paragraph -->

{% embed https://gist.github.com/rahulbanerjee26/a54c1b7be5508557b136e018770eb370 %}

<!-- wp:paragraph -->
<p>Save the new task and build the model.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4>Using the Twilio API</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Alternatively, if you’d prefer, you can update the API URL value using code. The task resource you created earlier has an update method that can be used to pass a new action with your API URL. You can read more about it in the <a href="https://www.twilio.com/docs/autopilot/api/task?code-sample=code-update-a-task-to-set-task-actions&amp;code-language=Python&amp;code-sdk-version=7.x">Twilio Docs</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Go to the file where you wrote the code for your Twilio bot. In the section where you create a Task resource, replace <code>assistant.tasks.create()</code> with <code>assistant.tasks.update()</code>. This will ensure that your script will update the action instead of creating a new one. Update the JSON object to include your API URL. Once you are done, run the script again to update your action and build the model.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>After you have added your API’s URL to your action resource—either via the Twilio Console or the Twilio API—your bot should work as shown below:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://i.imgur.com/FEk37dn.png" alt="Final Twilio bot"/></figure>
<!-- /wp:image -->

<!-- wp:heading -->
<h2>Conclusion</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>In this tutorial, you built an AI-assisted phone service using Twilio to collect a user’s name and phone number. You also built an API to save this information inside a database.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>As you’ve likely noticed by now, <a href="https://replit.com/">Replit</a> has a lot of useful features that allow you to code online. Although this article only showed its support for Python, it equally supports other programming languages, such as C++, JavaScript, and TypeScript; and frameworks, such as <a href="https://reactjs.org/">React</a> or <a href="https://vuejs.org/">Vue.js</a>. It comes with a built-in browser to preview your frontend or API. Additionally, it also has a built-in database and support for unit testing and storing environment variables.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Furthermore, Replit’s Explain Code feature is able to generate explanations for code snippets. Replit also has support for <a href="https://git-scm.com/">Git</a>, allowing you to add commits or revert to previous ones as needed. Be sure to <a href="https://replit.com/">check out</a> this powerful online IDE that can be used to make phone services, among many other things.</p>
<!-- /wp:paragraph -->