How Well Apply Simple MLP to Incomplete Utterance Rewriting? (2023)

Abstract: Incomplete utterance rewriting (IUR) aims to restore the incomplete utterance with sufficient context information for comprehension. This paper introduces a simple yet efficient IUR method. Different from prior studies, we first employ only one-layer M LP architecture to mine latent semantic information between joint utterances for IUR task ( MIUR ). After that, we conduct a joint feature matrix to predict the token type and thus restore the incomplete utterance. The well-designed network and simple architecture make our method significantly superior to existing methods in terms of quality and inference speedOur code is available at https://github.com/IMU-MachineLearningSXD/MIUR .

## introduction multi-turn dialogue modeling is a research area focusing on developing systems that can engage in multiple conversation turns with humans. this type of modeling is often used in the field of humanmachine interaction to improve the ability of artificial intelligence systems to communicate with humans in a natural and intuitive way. one of the challenges of multi-turn dialogue modeling is to accurately understand and respond to the context and meaning of the conversation, as well as to handle incomplete or ambiguous utterances that may be used for brevity or to convey meaning. as shown in what is covid-19 to address this issue, recent works @xcite @xcite proposed the incomplete utterance rewriting (iur) task, which aims to transform an incomplete or context-dependent statement into a self-contained, semantically equivalent one that can be understood without any additional context. as shown in despite previous works achieving promising results, the speed of autoregressive generation remains a limiting factor. to improve the speed, @xcite fuses the sequence labeling and non-autoregressive generation, which predicts missing elements in incomplete utterance and rewritten utterance. in addition, @xcite formulates iur as semantic segmentation task based on u-net @xcite and achieves better performance at a faster speed. however, above mentioned models are still not simple enough. layer norm t t layer norm skip-connections skip-connections linear cos dot mlp 1 mlp 2 joint feature matrix batch norm ... ... solution that our model first employs mlp architecture to simultaneously mine the semantic associations between the context utterances and the incomplete utterance, and capture attention information between them. after mlp architecture, we obtain the joint feature maps and further construct the token-pair edit matrix. finally, the above matrix is edited according to prediction edit type tokens to generate the final rewritten utterance. experiments show that our approach achieves better performance on several datasets across different domains and languages with low resource costs and a much faster inference speed. ## methodology in this section, we elaborate on our proposed approach. as shown in figure 1 , our method mainly consists of two modules: mlp backbone network and joint feature matrix. for a multi-turn dialogue utterances (u 1 , u 2 , ..., u t ), we concatenate all the context utterances to produce an m-length word sequence @xmath0, c 2 , ..., c m ) and employ a special mask @xcite to separate different context utterances. meanwhile, all the incomplete utterances are denoted as an n-length word sequence @xmath1, x 2 , ..., x n ). ## conclusion & future work in this paper, we propose a simple yet effective iur method. we utilize one-layer mlp structure to mine the inter-utterance semantic information from different perspectives. this improves the ability to predict the correct token between incomplete utterance and rewritten utterance. benefiting from the fact that our model effectively employs mlp to iur task, allowing our approach to achieve significant results in terms of performance and inference speed. this study represents the first preliminary exploration of the use of mlp on iur task. in the future, we will investigate on extending our approach to other dialogue areas.