Intent Detection and Zero-shot Intent Classification for Chatbots (2023)

Abstract: In this paper we give in detail how seen and unseen intent is detected and classified. User intent detection has a critical role in dialogue systems. While analysing the intents it has been found that intents are diversely expressed and new variety of intents emerge continuously. Here we propose a capsule-based approach that classifies the intent and a zero-shot learning to identify the unseen intent. There are recently proposed methods on zero-shot classification which are implemented differently from ours. We have also developed an annotated corpus of free conversations in Tamil, the language we have used for intent classification and for our chatbot. Our proposed method on intent classification performs well.

## introduction with the emergence of conversational ai, chatbots or dialogue systems are becoming central tools in many applications such as virtual assistants helping the citizens in getting authentic information from governments, academia, health sector and industry and so on. since user interests may change frequently over time, the ai agents may continuously see unknown (new) user intents. it is necessary and essential to understand the intent and accurately identify the intent behind the user utterance. this will help in generating the correct response for the user intent. data created manually by manual annotation cannot catch up with the requirement and algorithms which use annotated data cannot give accurate understanding of the intent by the user. this motivates the problem of detection of intent and its classification and has recently attracted increasing interest from both academia and industry. here we define intent as user's intention in an utterance which is the turn constructional unit (tcu), the fundamental unit, in an utterance. in this paper, we aim to address the issue of detection of intent, seen and unseen intent and its classification. here we tackle this issue using zero-shot intent detection and classification problem with a capsule based @xcite ) model, as tried by @xcite . a capsule encase a vector representation of a group of neurons, and the orientation of the vector encodes properties of an object (like in intent turn composition unit (tcu)), while the length of the vector reflects its probability of existence (intent seen or unseen classification such as request). the capsule model learns a hierarchy of feature detectors via a routing-byagreement mechanism: capsules for detecting low-level features (like in intent turn composition unit (tcu)) send their outputs to high-level capsules (intent seen or unseen classification such as request) only when there is a strong agreement of their predictions to high-level capsules. here we use zero-shot learning approaches for intent classification. in classification we identify the intent which is seen as well as unseen while training. there are several zero-shot learning approaches attempted to address the challenges for classifying intents whose instances are not present during training. the common method adopted to classify is to utilize some external resources @xcite @xcite @xcite like labelled ontologies or manually defined features. there is a dearth of such resources as well as developing such resources are time consuming and expensive. the basic idea is that correctly identifying if the intent of an utterance is known or unknown will ## related work there has been many studies to understand user intent from various domains, ranging from search engine questions @xcite to medical queries @xcite . the approaches used for intent classification include deep learning models such as convolutional neural networks (cnn) @xcite and attentionbased recurrent neural networks (rnn) @xcite . traditional intent classification methods require considerable amount of labelled data for each class to train a discriminative classifier, while zero-shot intent classification @xcite addresses the problem that not all intent categories are seen during the training phase, but new intents could continuously emerge in dialogue systems @xcite @xcite . zeroshot intent classification aims to generalize knowledge and concepts learned from seen intents to recognize unseen intents. early methods include @xcite @xcite . recently, intentcapsnet-zs @xcite extends capsule networks @xcite , utterances from unseen intents in the generalized zero-shot classification scenario, and proposes to solve this issue by transferring the transformation matrices from seen intents to unseen intents. in this paper, we use measuring intent relations to tackle the issue of seen intents to unseen intents. ## our approach here in this work we follow the two architecture approach similar to the one proposed by @xcite . initially we use our model, capsnet, for intent detection and zero-shot learning for classification of unseen intent. here are the outline of the steps that were followed in the implementation of capsnet intent detection: a) dataset preparation: where, ω is a learnable covariance matrix which models the correlation between dimensions of the embedding. note that intentcapsnet @xcite also tries to use eq. ( 1 ) to model the relationship between unseen and seen intents, but it ignores the correlation between dimensions and simply sets ω = σ2i (σ is a scaling hyperparameter), which is actually a scaled squared euclidean distance. ## experiments & results we perform our experiment using the data created by us which is described in the previous section. the data is split into two, training (66 dialogues) and test (20 dialogues). the test partition is formed such that 10 dialogues have topic similarity with the training partition. the remaining 10 dialogues are completely different than the rest (completely unseen). the embedding's needed for the intent detection models, are developed using tamil wikipedia content and copyright free novels digitized content from project madurai (project madurai). these pre-trained word embedding's are used for intent classification using capsnet and unseen internet detection using zero-shot learning. we use three fold cross-validation to choose hyper parameters. the dimension of the prediction vector dp is 10. @xmath0. an additional input dropout layer with a dropout keep rate 0.8 is applied to the intent annotated corpus of ours. in the loss function, the down-weighting coefficient -λ is 0.5, margins m + k and mk are set to 0.9 and 0.1 for all the existing intents. the iteration number iter used in the dynamic routing algorithm is 3. adam optimizer is used to minimize the loss function.