Advanced textual conversational bots using Dialogflow, Sentiment Analysis, Telephone Gateway and Firestore
Tags: sentimentanalysis, firestore, dialogflow, chatbot
Published: 2021-05-19 20:19:13

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/1200/0*MjH-y80704eeqNwZ" alt=""/><figcaption>Demo displayed at Next 18 in&nbsp;Japan</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":4} -->
<h4 id="e97c">Calling a bot from your phone, is that even possible?</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>And the answer is yes!</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><a href="https://stackoverflow.com/users/22656/jon-skeet" rel="noreferrer noopener" target="_blank">Jon Skeet</a>&nbsp;and I presented a mentor search demo in C# and node.js, respectively.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If you give our conversational bot a call, you would be able to talk to it via&nbsp;<strong>phone</strong>, as it is integrated to Dialogflow Phone Gateway Beta.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>A bit more context about the demo — you are able to search the mentor by name, location or piece of technology you’d need mentorship.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/800/0*Rv8noRXHQ50pn3y1" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":4} -->
<h4 id="f83c">Sentiment Analysis</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Ok, so you are in a call with our bot, and in case you say a negative query, the conversation can be escalated to a real human being who can talk to you real time, leveraging both sides of AI and Machine Learning in one computer-human interaction platform.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Dialogflow Enterprise Edition has a sentiment analysis beta feature, that we use the&nbsp;<strong>sentimentAnalysisResult&nbsp;</strong>field to check the score and the magnitude of the queries.</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><code>score</code>&nbsp;- is a number from -1.0 to 1.0 that shows how positive or negative the statement is, the overall emotional leaning of the text.</li><li><code>magnitude</code>&nbsp;- is a number ranging from 0 to infinity, it represents the weight of the statement.</li></ul>
<!-- /wp:list -->

<!-- wp:heading {"level":4} -->
<h4 id="9711">Making Phone&nbsp;calls</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>How we have done it:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>We have bought a US phone number to call the dialogflow agent.</li><li>The “transfer calls” are forwarded to our own numbers.</li></ul>
<!-- /wp:list -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/2560/0*KcawKbSgXzPpV6Vm" alt=""/><figcaption>Demo at Next 18 in San Francisco</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":4} -->
<h4 id="7aca">Telephone Gateway</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>In terms of telephone gateway, I have been excited to try and test this new feature — it provides a telephone interface to your agent.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>I’ve done a number of tests using the simulator but before this demo, not that many real calls, as of now (January 2019) it is only available for US phone numbers.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>So I bought a US phone number and from that moment onwards I began to see errors that could not be seen in test mode.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>It was a challenge to find a solution to make our demo work for a real-life use case, as the resources and questions answered online are really scarce at the moment.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/2560/0*a0VFI60C6z8GVcck" alt=""/><figcaption>Demo at Cloud Next&nbsp;18</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":4} -->
<h4 id="cc0c">Found a bug in production ????</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>I’ve managed to get the functionality working that I could not find the answer anywhere else.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The error was the following: when creating the workflow of the conversation, in the Dialogflow console both in the default text or the telephony gateway tab we could see the conversation flow working neatly.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>However, when you did make a real phone call it would act very glitchy.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/600/1*sJ_zOd7tTSJv2juXM5_PeA.png" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>It would sometimes ignore the message being said, even if we had correctly specified it on the web browser framework response under “SSML or Text response” and the Telephony tab synthesised responses and commands.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>So instead of using the web browser functionalities or even the workaround of adding a “setFollowupIntent” and adding an event — we can write the JSON in full with the texts expected in the platform and whether it is voice or text.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>As you can see on the code snippet below.</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted"></pre>
<!-- /wp:preformatted -->

<!-- wp:group -->
<div class="wp-block-group"><div class="wp-block-group__inner-container"><!-- wp:group -->
<div class="wp-block-group"><div class="wp-block-group__inner-container"><!-- wp:preformatted -->
<pre class="wp-block-preformatted"><code>response.json({"fulfillmentMessages":[
  {
    "text":{
      "text":[
        "I am sorry you feel this way, let me transfer you to a real person!"
      ]
    }
  },
  {
    "platform":"TELEPHONY",
    "telephonySynthesizeSpeech":{
      "text":"I am sorry you feel this way, let me transfer you to a real person!"
    }
  },
  {
    "platform":"TELEPHONY",
    "telephonyTransferCall":{
    "phoneNumber":"&lt;&gt;"
  }
}],]});</code></pre>
<!-- /wp:preformatted --></div></div>
<!-- /wp:group --></div></div>
<!-- /wp:group -->

<!-- wp:heading {"level":4} -->
<h4 id="dc1d"><strong>Jon’s fulfillment in&nbsp;.NET on&nbsp;github:</strong></h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p><a href="https://github.com/jskeet/DemoCode/tree/039ef85aa7f17c8573eac4ec7e7c2acb80ded949/DialogflowDemo/DialogflowDemo"><strong>jskeet/DemoCode</strong><br><em>Demo code used in talks, blog posts etc. Contribute to jskeet/DemoCode development by creating an account on GitHub.</em>github.com</a></p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4 id="f43e">My fulfilment in node.js on&nbsp;Github:</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p><a href="https://github.com/amdcaruso/dialogflow-women-in-tech-demo"><strong>amdcaruso/dialogflow-women-in-tech-demo</strong><br><em>Dialogflow: Women Mentor Search and Escalation to a Human - amdcaruso/dialogflow-women-in-tech-demo</em>github.com</a></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>More specifically the&nbsp;<strong>index.js&nbsp;</strong>file<a href="https://github.com/amdcaruso/dialogflow-women-in-tech-demo/blob/master/functions/index.js"><strong>amdcaruso/dialogflow-women-in-tech-demo</strong><br><em>Dialogflow: Women Mentor Search and Escalation to a Human - amdcaruso/dialogflow-women-in-tech-demo</em>github.com</a></p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4 id="76ca">The node.js demo available on the&nbsp;browser</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p><a href="https://console.dialogflow.com/api-client/demo/embedded/mentor-tech-demo"><strong>mentor-search-demo</strong><br><em>Women Mentor Search and Escalation to a Human</em>console.dialogflow.com</a></p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3 id="06cc">Setting up your dialogflow agent:</h3>
<!-- /wp:heading -->

<!-- wp:heading {"level":4} -->
<h4 id="6ecb">Intents</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Default Welcome Intent</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Default Fallback Intent</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Mentor Search</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/800/1*F3q0B8vwT_lqjRd172UQWA.png" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/800/1*hLa0Bya2i3HdkoPzFDLoTg.png" alt=""/><figcaption>Training phrases</figcaption></figure>
<!-- /wp:image -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/800/1*-5fhimuWIBEmShMsxuDLFg.png" alt=""/><figcaption>Actions and Parameters</figcaption></figure>
<!-- /wp:image -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/800/1*9lbEX6uKbIAY3a_Ddl2AsQ.png" alt=""/><figcaption>Default Tab</figcaption></figure>
<!-- /wp:image -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/800/1*neiatPPjVKlk0OgOJe_e0Q.png" alt=""/><figcaption>Telephony Tab</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Transfer Call Escalation to Human</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Goodbye</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4 id="9817">Entity</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>@ topic</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted"><code>“VR”,”VR”
“Advertising”,”Advertising”
“angular”,”angular”,”Angular JS”
“GCP”,”GCP”,”Google Cloud”,”Google Cloud Platform”
“Java”,”Java”
“firebase”,”firebase”
“Blockchain”,”Blockchain”
“Data analytics”,”Data analytics”,”data analyst”
“python”,”python”
“public speaking”,”public speaking”
“Actions on Google”,”Actions on Google”,”AoG”,”Google Actions”,”Google Assistant”,”NLP”,”Conversational apps”,”conversation design”
“Business intelligence”,”Business intelligence”
“PHP”,”PHP”
“Play Store Optimization”,”Play Store Optimization”
“SQL”,”SQL”
“C++”,”C++”
“app script”,”app script”
“Android”,”Android”
“Javascript”,”Javascript”
“Agile”,”Agile”,”scrum”
“Kotlin”,”Kotlin”
“data privacy”,”data privacy”
“CSS”,”CSS”,”CSS 3"
“React”,”React”
“AI”,”AI”,”machine learning”,”TensorFlow”,”artificial intelligence”
“AR”,”AR”,”augmented reality”
“Backend”,”Backend”
“developer relations”,”developer relations”,”developer advocate”
“Product manager”,”Product manager”,”product managment”
“security”,”security”
“XML”,”XML”
“design”,”design”,”graphic design”
“objective C”,”objective C”
“Usability”,”Usability”
“UX”,”UX”
“Community Management”,”Community Management”
“Community managment”,”Community managment”
“data mining”,”data mining”
“UI”,”UI”
“Marketing”,”Marketing”
“Systems planning”,”Systems planning”
“design thinking”,”design thinking”
“event managment”,”event managment”
“game design”,”game design”
“HTML5”,”HTML5",”HTML”
“network security”,”network security”
“online marketing”,”online marketing”
“Social media”,”Social media”
“Front end developement”,”Front end developement”
“Students in Bootcamps”,”Students in Bootcamps”</code></pre>
<!-- /wp:preformatted -->

<!-- wp:heading {"level":4} -->
<h4 id="dc33">Firebase Cloud Firestore Beta&nbsp;Database</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>We're using the very famous&nbsp;<a href="https://medium.com/u/97e183b9e5ec" target="_blank" rel="noreferrer noopener">Tracy Lee | ladyleet</a>’s&nbsp;<a href="https://github.com/fempire/women-tech-speakers-organizers" rel="noreferrer noopener" target="_blank">#fempire</a>&nbsp;list, she kindly allowed us to use.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>All the kudos to the software developer and Google Developer Expert to&nbsp;<a href="https://medium.com/u/7120d24a2a09" target="_blank" rel="noreferrer noopener">Jason Berryman</a>&nbsp;for pairing with me late at night to make the conversion from firebase real-time database to Firestore. If you’re interested in getting started with Firestore and Dialogflow you should definitely check&nbsp;<a href="https://medium.com/u/5c7ddf46d313" target="_blank" rel="noreferrer noopener">Mandy Chan</a>’s&nbsp;<a href="https://medium.com/google-developers/do-you-need-to-share-data-between-your-actions-and-your-mobile-and-web-apps-8cf6464d85fd" target="_blank" rel="noreferrer noopener">article</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://cdn-images-1.medium.com/max/800/1*Ble6BpAZIMTYwaJ5UKrkPg.png" alt=""/><figcaption>Screenshot of the database “mentors” containing the mentors&nbsp;records</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>—</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4 id="f9b5">Was this article useful to you? Let me know in the comments!</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Thank you for reading this, I hope it encourages you to create your own agent! I try to be active on&nbsp;<a href="https://medium.com/u/d53dd768d047" target="_blank" rel="noreferrer noopener">Stack Overflow</a>&nbsp;answering questions regarding actions on google and dialogflow, you can find my profile&nbsp;<a href="https://stackoverflow.com/users/5113155/amanda-cavallaro" rel="noreferrer noopener" target="_blank">here</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To me personally, this has been a very exciting journey as I’ve had the opportunity to create something that has potential to be very meaningful in real life, and definitely doing it collaboratively with Jon, who has kindly mentored me in the past months, it makes it worth for me to want to pass it forward so that other people can believe more in themselves while being guided.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>By&nbsp;<a href="https://medium.com/@amdcaruso">Amanda Cavallaro</a>&nbsp;on&nbsp;<a href="https://medium.com/p/1924e5914352"><time datetime="2019-01-28T23:54:03.860Z">January 28, 2019</time></a>.</p>
<!-- /wp:paragraph -->