# Development of Dialogue Management System for Banking Services
## 2 Related Work
### Dialogue Management Module  
The first generation of DMS is considered as rule-based [24] and flowchart-based that simulate state transitions during the dialogue that are based on the finite state automata [25; 26]. Both systems require intense domain expert support and cannot be considered flexible with respect to unexpected changes in a dialogue flow. The rapid increase in data has led to the second generation of statistical data based DMS, where the Partially Observable Markov Decision Process (POMDP) is considered as one of the robust representatives [27]. In this type of system, every dialogue stage possesses a state supported by a Bayesian inference, and a particular dialogue policy is selected in order to generate a proper natural language response. The authors of [28] have shown how the continuous component of POMDP can enable the model to incorporate a confidence score of the automated dialogue management and to outperform traditional Markov decision processes. [PERSON] et al. took another approach in their work [29], where they showed how the sequence of certain dialogues are used to estimate Q function to establish the dialogue state and the next action. The third generation of DMS emerged recently after the breakthroughs in deep learning and the drastic increase in conversational data, being more defined as goal-oriented dialogue systems [30; 31; 32; 33]. During this generation, there was a clear shift from generative models, such as Bayesian networks, to deeply discriminative ones, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) [25]. [PERSON] et al. showed how to utilize CNN to generate semantic tagging of each utterance in order to update the dialogue state [34]. [PERSON] et al. discussed how to use CNN with a maximum pooling layer to obtain local features from the provided utterances and then select the most important semantic factor [35]. The authors of [36] propose a neural network framework made of two steps: supervised learning from dialogue data followed by the continuous improvement using the reinforcement learning. [PERSON] and [PERSON] in their work [37] suggest handling a dialogue state by assuming that a particular dialogue is a stack of topics. More recent work introduced by the authors of RavenClaw [38] elevated the idea of the stack and applied it for sub-dialogues. One of the major drawbacks of stack-based architectures is that their structure is strict, and minor deviations from topics within a stack result in a loss of the context and a mix of the dialogue states. RNNs started to become more popular in recent years, and showed promising results within general, open domains in dialogue systems [39; 40]. RNNs in their default settings consume the entire sequence of input elements for encoding purposes, which increases the size of a trained model significantly. Long short-term memory based RNN architectures have proven to perform efficiently when it comes to the natural response generation for goal-oriented dialogue management systems [41; 42; 43]. However, one of the drawbacks of RNNs is that for low-resource and limited corpus, such as the Azerbaijan language, they do not guarantee dialogue state generalization. LSTM type RNNs tend to distort their internal state if, during a dialogue stage, an unexpected sentence is provided (e.g., user starts with credit application and then suddenly switches to debit card order). Recently, the transformer architecture started to substitute RNNs for the language model training [44; 45; 46; 47]. One of the major advantages of transformers is that they achieve independency while making predictions on different dialogue stages based on the self-attention mechanism. This mechanism is used to learn sentence representations by comparing different positions of that sentence and allows preselecting tokens that affect the current state of the encoder [48; 49].