D ialog C onv: A Lightweight Fully Convolutional Network for Multi-view Response Selection (2022)

Abstract: Current end-to-end retrieval-based dialogue systems are mainly based on Recurrent Neural Networks or Transformers with attention mechanisms. Although promising results have been achieved, these models often suffer from slow inference or huge number of parameters. In this paper, we propose a novel lightweight fully convolutional architecture, called DialogConv, for response selection. DialogConv is exclusively built on top of convolution to extract matching features of context and response. Dialogues are modeled in 3D views, where DialogConv performs convolution operations on embedding view, word view and utterance view to capture richer semantic information from multiple contextual views. On the four benchmark datasets, compared with state-of-the-art baselines, DialogConv is on average about 8.5x smaller in size, and 79.39x and 10.64x faster on CPU and GPU devices, respectively. At the same time, DialogConv achieves the competitive effectiveness of response selection.

## introduction an important challenge in building intelligent dialogue systems is the response selection problem, which aims to select an appropriate response from a set of candidates given a dialogue context. such retrieval-based dialogue systems have attracted great attention from academia and industry due to the advantages of informative and fluent responses produced @xcite . the existing retrieval-based dialogue systems can be divided into three patterns according to the way of input handling @xcite : (i) separate pattern @xcite @xcite ; (ii) concatenated pattern @xcite ; (iii) prlm (pretrained language model) pattern @xcite @xcite . separate pattern (i.e., figure 1 (a) ) encodes utterances individually, while concatenated pattern (i.e., figure 1 (b) ) concatenates all utterances into a continuous word sequence. methods based on these two patterns usually have recurrent neural networks (rnns) @xcite and attention mechanism @xcite as the backbone. although promising results have been achieved, these methods are generally slow in training and inference due to their recurrent nature. the prlm pattern (i.e., figure 1 (c )) uses special symbols to connect all utterances into a continuous sequence, similar to concatenated pattern. while prlm pattern has obtained state-of-the-art performance in response selection @xcite @xcite , this method having transformer @xcite as the de facto standard architecture suffer from a large number of parameters and heavy computational cost. very large models not only lead to increased training costs, but also prevent researchers from iterating quickly. at the same time, slow inference hinders development and deployment of dialogue systems in real-world scenarios. furthermore, these three patterns treat dialogue contexts as flat structures @xcite . methods based on such flat structures usually capture the sequential features of text by considering each word as a unit. however, previous work @xcite revealed that given a multi-turn dialogue (e.g., figure 2 we view utterances in each dimension (shown as figure 2 (b) and (c))). as shown in figure 2 (b), the embedding view can represent features of each individual word, the word view can represent the features from the whole conversation and a single utterance, and the utterance view can capture the dependencies between different localities composed of adjacent utterances. existing methods @xcite @xcite ) only extract features based on the flat structures, but cannot simultaneously capture complex features from such stereoscopic views. in this paper, we propose a lightweight fully 1 convolutional network model, called dialogconv, without any rnn and attention module for multiview response selection. different from previous studies @xcite @xcite which model the dialogue in a flat view, dialogconv models the dialogue context and response together in the 3d space of the stereo views, i.e., embedding view, word view, and utterance view (as shown in figure 2 (d) ). in the embedding view, the word-level features will be refined through convolution operations on the plane formed by the word sequence dimension and the utterance dimension. in the word view, the global conversation features will be captured by concatenating all words into a continuous sequence, and the features of each utterance will be refined by performing convolution on each utterance. in the utterance view, the dependency features between different local contexts will be distilled by performing convolution across different utterances. in general, dialogconv can simultaneously extract features with different granularities from the stereo structure. dialogconv is completely based on cnn, which uses much fewer parameters and computing resources. dialogueconv has an average number ## experiments and results the baselines are described in the appendix a.1. ## conclusion in this paper, we propose dialogconv, a multi-view lightweight architecture based exclusively on cnn. dialogconv conducts convolutions on embedding, word, and utterance views to capture matching features. experiment results show that dialogconv has fewer parameters, is faster, and requires less computing resources to achieve competitive results on four benchmark datasets. dialogconv provides a valuable reference for the dialogue system being deployed in real-world scenarios.