ICM : Intent and Conversational Mining from Conversation Logs (2022)

Abstract: Building conversation agents requires a large amount of manual effort in creating training data for intents / entities as well as mapping out extensive conversation flows. In this demonstration, we present ICM (Intent and conversation Mining), a tool which can be used to analyze existing conversation logs and help a bot designer analyze customer intents, train a custom intent model as well as map and optimize conversation flows. The tool can be used for first time deployment or subsequent deployments of chatbots.

## introduction in-spite of the proliferation of gui based chatbot development environments and availability of open source and commercial tools with low code or no code environments, building chatbots remains a challenge. many frameworks exist to help a non technical user build chatbots 1 2 , including mechanisms to enter the training data, and drag n drop methods for creating conversation flows. similarly, research works focus on designing chat bots as end-to-end neural systems or using reinforcement learning based methods. however, limited work exists on techniques to automatically obtain and prepare training data by leveraging existing conversatons that can then be used by commercial tools building task-oriented chatbots. in this paper , we discuss a tool that takes a user through a guided step by step process of clustering intents, reviewing the intent labels and conversation states, grouping of conversation flows, analyzing individual conversations and, finally exporting the training data for intents and conversation flow. this information can be used by non technical chatbot 1 https://dialogflow.cloud.google.com 2 https://botmock.com/ developers to create chatbots using any of the available chatbot building tools. the rest of the paper is organized as follows: in section 2 we discuss some of the related work in this space. section 3 high lights the key features in our tool. section 4 discusses the technical details of the tool including the algorithms used. 5 discusses the key aspects of the demo. finally, in section 6 we conclude the paper. ## related work multiple methods have been proposed in using realistic data for developing chatbots. @xcite suggest a modified version of the wizard of the oz approach by collecting transcripts of real conversations between service agents and customers. many bots are being built to augment human service agents, and hence there is a rich set of information available as human (customer) -human (service agent) conversations.the tool graph2bot @xcite analyzes such existing conversations but fails to create a format that can be leveraged by commercial tools. in the absence of human conversation logs, text in the form of emails and service now tickets can also provide insights about the queries that can possibly be handled by the chatbot. @xcite provide a mechanism to bootstrap conversational agents by helping select the necessary training samples. once a chatbot is deployed, there are existing tools that perform various forms of conversation analytics. however, these tools do not provide a direct mechanism to leverage these insights back into the enhancement of the chatbot. ## key steps in analyzing intent and conversations icm enables multiple people supporting all phases of the chatbot development process to identify the key intents, the conversation flows, conversation flow analytics including volumetric and temporal analyses, the user sentiment and emotion as well as evolution of conversations over time. this is done in an offline manner by importing the conversation logs. the conversation logs may be human-human conversations captured at the beginning of the chatbot development life cycle, and / or human-bot based conversations at the run phase of the chatbot life cycle. the key features of the tool are as follows: ## icm : technical details the tool contains a front end for labelling, analyzing and reviewing existing conversations, as well as the backend containing a rich set of clustering algorithm options, conversation summarization, sentiment and emotion detection options. figure 1 shows the high level diagram of the tool. ## conclusion in this paper, we highlight a key gap in the existing technology used to build chatbots -the ability to leverage existing data in the form of human-human or human-bot conversations automatically. we discuss a the tool that enables an end user to analyze this data, derive detailed and varied insights and export it in a form that can be leveraged by existing technology to build chatbots.