Multi-Granularity Information Interaction Framework for Incomplete Utterance Rewriting
Authors: Haowei Du
Dinghao Zhang
Chen Li
Yang Li
Dongyan Zhao
Venue: EMNLP 2023 Findings (2023)
Keywords: Incomplete Utterance Rewriting
Information Interaction
Multi-Granularity

Abstract: Recent approaches in Incomplete Utterance Rewriting (IUR) fail to capture the source of important words, which is crucial to edit the incomplete utterance, and introduce words from irrelevant utterances. We propose a novel and effective multi-task information interaction framework including context selection, edit matrix construction, and relevance merging to capture
the multi-granularity of semantic information. Benefiting from fetching the relevant utterance and figuring out the important words, 
our approach outperforms existing state-of-the-art models on two benchmark datasets Restoration-200K and CANAND in this field.

Multi-Gra Information Interaction Framework for Incomplete Utterance Rewriting
Haowei Du 1,2,3, Dingyu Zhang"; Chen Li Yang Li
Dongyan Zhao
Wangxuan Institute of Computer Technology, Peking University 2 Institute for Artificial Intelligence, Peking University Ant Group duhaowei@stu. pku. edu. cn, zhangdinghao. zdh@antgroup.cm com zhaodongyan@pku. edu. cn, wenyou. Iw@antgrouppp.o.ooooooooooooo... com, 7022000700aalbbba-- www.iibbbbinncc..o com
Abstract
Turns
Utterance
Parsons studied biology
Recent approaches in Incomplete Utterance Rewriting (IUR) fail to capture the source of important words, which is crucial to edit the incomplete utterance, and introduce words from irrelevant utterances. We propose novel and effective multi-task information interaction framework including context selection, edit ma- trix construction, and relevance merging to cap- ture the multi-g granularitt of semantic informa- tion. Benefiting from fetching the relevant utter- ance and figuring out the important words, approach outperforms existing state-of-the-ane models on two benchmark datasets Restoration- 200K and CANAND in this field.
W1
Amherst college.
Who is one of his professors at Amherst?
U/2
Parsons biology professors at
143
Amherst were Glaser and Henry.
04.1
What are his interest?
Parsons showed from early on, a great interest in the topic of philosophy,
MA5
116
Anything else he was interested to?
Other than philosophy, is there anything else parsons was interested in? Besides biology Glaser and Henry, anything else was he interested to?
1
RUN
1 Introduction
One example from CANARD dataset. W1- us 5 turns of contextual utterances, ug denotes the incomplete utterance, ug denote the golden rewriten utterance, and "RUN" denotes the incorrect result of baseline (Liu al., 2020).
Table 1:
Recently increasing attention has been paid to multi-turn dialogue modeling (Choi et al., 2018; Reddy et al., 2019; Sun et al., 2019) and the major challenge in this field is that speakers tend to use incomplete utterances for brevity, such as referring back to (i.e., co-reference) or omitting (i.e., ellipsis) entities concepts that appear in dialogue history. Su al. (2019) shows that ellipsis and co-reference can exist in more than 70% of dialogue utterances. To handle this, the task of Incomplete Utterance Rewriting (IUR) (Pan et al., 2019; Elgohary et al., 2019) is proposed to rewrite an incomplete utter- ance into an utterance which is semantically equiv- alent but self-contained to be understood without context.
glected. Unaware of which sentences contain the words needed to rewrite the incomplete utterance (important words)(Inou et al., 2022), these models introduce incorrect words from irrelevant sentences into the rewritten utterance.
We take an example in table 1. The incom- plete utterance has the phenomenon of co-reference ("he"" and ellipsis ("anything else"). Because the baseline model RUN (Liu et al., 2020) does not fetch the correct source sentence (u5) and the im- portant words ("philosophy"", it introduces irrele- vant words ("Glaser and Henry") into rewriting the utterance.
To maintain the similarity of semantic struc- ture between the incomplete utterance and rewrit- ten utterance, recent approaches formulate as a word edit task (Liu et al., 2020: Zhang et al., 2022) and predict the edit types by capturing the semantic relations between words. However, the sentence-level semantic relations between contex- tual utterances and incomplete utterance are ne-
To identify the source sentences of important words and incorporate the sentence-level relations among contextual utterances and incomplete utter- ance, we propose our gratiigganullarity information capturing framework for IUR. Firstly, we classify the sentences in contexts into relevant or irrelevant
"These authors contributed equally to this work. *Corresponding Author.

---

utterances and match the rewritten utterance with the relevant contexts. Then we capture the token- level semantic relations to construct the token edit matrix. The predicted relevances among sentences in contexts and incomplete utterance, which en- codes the sentence-level relations, are utilized to mask the edit matrix. The rewritten utterance is de- rived by referring the token matrix. We conduct experiments two benchmark datasets in IUR and outperform the prior state-of-the-ar by about 1.0 score in Restoration-2000 petitive performance in CANARD dataset across different metrics including BLEU, ROUGE and F-score.
Figure I:
Model pipeline. Our model contains 4 parts: context selection, edit matrix construction, relevance merging and intention check.
dataset and derive com-
(PTM) (Devlin et al., 2018), Zhang al. (2022) directly take the self-attention weight matrix the word2word edit matrix. Though these models pro- duce competitive performance, the sentence-level semantic relations are neglected and the models tend to introduce incorrect words from irrelevant contextual utterances.
Our contributions can be summarized as: 1. We are the first incorporate sentence-level semantic relations between the utterances in con- texts and the incomplete utterance to enhance the ability to seize the source sentence and figure out the important words.
2. We propose the multi task information interac- tion framework capture the multi-granularity of semantic information. Our approach outperforms existing methods on the benchmark dataset of this field, becoming the new state-of- of-the-art.
3 Methodology
3.1 Overview
By figure 1, our approach contains four compo- nents: context selection, edit matrix construction, relevance merging and intention check.
2 Related Work
There are two main streams of approaches to tackle the task of IUR: generation-base (Huang et al., 2021; Inoue et al., 2022) and edit-based (Liu et al., 2020; Si et al., 2022). Generation-base models seq2seq problem. Su et al.
3.2 Context Selection
In this part, we capture the semantic relations be- tween contextual utterances and incomplete ut- terance. Following RUN, We utilize BERT to encode the contextual representation of contexts and incomplete utterance. The input to PTM is the sequence of contexts concatenated by incom- plete utterance, and the "[SEP]" token is applied to separate different sentences. The utterance representation is derived by pooling the hidden states of words it contains: [C1, C2 Cn. u Pooling (ooling(BERTw//// WI, wN]), where denotes the representation of i-th utterance in texts, denotes the representation of incomplete utterance and w; denotes the i-th word token in the input sequence. We apply a MLP classifier to predict the relevance between each utterance contexts and incomplete utterance:
solve this task (2019) utilize pointer network respectively pre- dict the prob of tokens in rewritten utterance from contexts and incomplete utterance. Hao et al. (2021) formulate the task as sequence tagging to reduce the search space. Huang et al. (2021) com- bine a source sequence tagger with an LSTM-based decoder to maintain the grammatical correctness. Generation models lack to capture the trait of IUR, where the main semantic structure of rewritten ut- terance is usually similar the original incomplete utterance.
Edit-based models focus on predicting word- level or span-level edit type between contextual utterances and the incomplete utterance. Liu et al. (2020) formulate this task semantic segmenta- tion and propose U-Net (Ronneberger et al., 2015; Oktay et al., 2018) to encode the word-level seman- tie relations. To incorporate the rich information in self-attention weights of pretrained language model
L = (roosEEntropy((r),,, (R,H) r; MLP((c;
(1)
1,2,- (2) ..
where 7; and R₁ denote the predicted relevance and label of i-th utterance in contexts respectively.

---

Model
B1
B2
R1
R2
F1
F2
F3
SARG
92.2
89.6
92.1
86.0
62.4
52.5
46.3
RAST
90.4
89.6
91.2
84.3
RUN
92.3
89.6
92.4
85.1
68.6
56.0
47.7
RAU
92,4
89.6
92.8
86.0
69.9
57.5
49.6
QUEEN
92,4
89.8
92.5
86.3
Ours
93.1
90.4
93.2
86.6
70.8
58.5
50.5
Table 2: Evaluation results on Restoration-200K dataset. "B1", "B2", "RI", "R2" denote BLEU1, BLEU2,
ROUGEI, ROUGE2 respectively. The p-values ROUGE, BLEU and F-score are smaller than 0.001.
Model
B1
B2
R1
RL
incomplete utterance. It can be formulated as:
RAST
53.5
47.6
62.7
61.9
RUN
70.5
61.2
79.1
74.7
Cij MLP(UUW;; wj))
(6)
HCT
68.7
62.3
80.0
79.4
QUEEN
72.4
65.2
82.5
81.8
where Byy E R denotes the predicted probability of 3 edit types between i-th token in contexts and th token in incomplete utterance, s Nc. Nu. U denotes the U-Net architecture, No and Ny denotes the context length and incomplete utterance length.
ChatGPT
61.5
52.5
46.1
40.1
Ours 72.9
63.2
79.2
77.0
Table Evaluation results on CANARD dataset. "BI", "B2", "RI", "RL" denote BLEU1, BLEU2, ROUGEI, ROUGEL respectively. The baseline results are from original papers.
3.4 Relevance Merging
If some utterance in contexts is classified as rele- vant by our Context Selection module, it is more possible for the words in this utterance to be adopted into editing the incomplete utterance. In this part, the relevant confidence of the utterance is equally added to the predicted prob of edit type Insert and Replace for all its constitutive words with the words in incomplete utterance:
Considering the golden label is not provided, we retrieve the utterance that contains the words used to rewrite the utterance the label of relevant utterance. To enhance the compatibility of selected contextual utterance with the rewritten utterance, we sample negative utterances from contexts of other cases in the batch and predict the match score:
Lmat CrossEntrops((m)) (Mi))
(3)
Ledit = CrossEntropy (Eg)) Ci
(7)
m MLP((c: urrl) i CpUUC
(4)
Jnserrt Gjj
Insert Ti
(8)
Mj i Cp
Replace Replace + +1 - (1
(5)
Eig
Cjj
(9)
ie ON
where Ejj denotes the golden edit type between i-th token contexts and j-th token in incomplete utterance and a denotes parameters to tune. The relevance merging process can be seen as utilizing the relevance predicted to "softly mask" the edit matrix. Even if one sentence is not selected relevant contexts, the probabilities of edit types Insert and Replace are strictly zero
where m, and M; denote the predicted and labeled matching score of i-th contextual utterance, Cp and Cy denote the golden contextual utterance that includes important words and the sampled negative utterance.
3.3 Edit Matrix Construction
Following (Liu et al., 2020), predict to- ken2token edit type (Insert, Replace, None) based on word representations from PTM with U-Net and build the word edit matrix. The entry at row and column j in the matrix denotes the edit type between i-th token in contexts and j-th token in
3.5 Intention Check
To maintain the intention consistency between the incomplete utterance and the rewritten utterance, we project the representation of incomplete utter- ance and rewritten utterance into intention space

---

Model
B1
B2
R1
R2
F1
F2
F3
RUN
92.3
89.6
92.4
85.1
68.6
56.0
47.7
92.4
89.5
93.2
86.3
70.2
57.1
48.8
+cs/hm
92.8
90.2
93.3
86.7
68.6
56.4
48.6
+cs/sm
93.0
90.3
93.4
86.8
69.1
56.7
48.7
+cs/sm/ic
93.2
90.5
93.4
86.7
70.0
57.6
49.7
Ours (+cs/sm/ic/em)
93.1
90.4
93.2
86.6
70.8
58.5
50.5
Table 4: Ablation results on Restoration-2000 dataset. "B1". "B2", "R1". "R2" denote BLBU1, BLBU2, ROUGEI, ROUGE2. "es", "em", "sm", "hm", "ic" denote context selection, context marching, relevance soft mask, relevance hard mask, intention checking modules respectively.
and close the distance.
directly extracts the co-reference and omission re- lationship from the self-attention weight matrix the transformer; RAST (Hao et al., 2021) proposes novel gquunce--taginng based model to reduce the search space; QUEEN (Si et al., 2022) designs explicit query template to bring guided semantic structural knowledge; HCT (Jin al., 2022) con- structs a hierarchical context tagger that mitigates the multiple spans issue by predicting slotted rules.
Lint = (M)
(10)
C = Pooling( (G, C Cp))
(11)
where C' denotes the pooling representations of utterances in contexts and KL denotes the KL- divergence function.
3.6 Training and Rewriting
The final training loss function computed by the weighted sum edit loss, selection loss, matching loss and intention loss:
4.2 Results
By table and 3, compared with our backbonc RUN, our model improves about 1.0 ROUGE and BLEU score, and 2.5 F-score in Restoration-2009 dataset, as well as 2.0 ROUGE and BLEU score in CANARD datuset. It demonstrates the efficiency of multi-task framework fetch the source of important words and avoid irrelevant words. Spe- cially, we outperform QUEEN by 0.7 BLEU1, 0.6 BLBU2, 0.7 ROUGEI and 0.3 ROUGE2, and beat RAU by about 1.0 F-score on Restoration-20n0 dataset, becoming the new state-of-the-art More- over, our model shows competitive performance on CANARD dataset, which beats QUEEN by 0.5 BLEU1 score.
Lifinal Lecdit Or Tosel + 022 + Ory Tint
where 0gg, 1,2,3 denote parameters to tune. The rewritten utterance is manipulated based on the predicted edit matrix [ ISSSNo That is, if εij = Insert, we insert the i-th token in contexts before the j-th token in incomplete utterance, which is similar for Replace type.
4 Experiments
4.1 Experimental Setup
Datasets: We do experiments on two benchmark IUR datasets from different languages: Restoration- 200K (Chinese, (Pan et al., 2019)) and CANARD (English.EElgohary et al., 2019)). Metries: Follow- ing Liu et al. (2020), we utilize BLEU (Papineni et al., 2002), ROUGE (Lin, 2004) and F-score (Pan et al., 2019) as evaluation metrics.
4.3 Ablation Study
To explore different modules of our approach, we conduct the ablation study. Compared with the "soft mask" of relevance merging section 3.4, we design an ablation with "hard mask" merging: if a sentence in contexts is classified as irrelevant in section 3.2, the probabilities of Insert and Rewrite between its words and the words in the incomplete utterance set to O. In table 4, context selection, relevance merging and intention checking show progressive improvement across different metries. Compared with hard mask of relevance merging, the soft mask method overall better. The sen- tence classified as irrelevant may still contain im-
Baselines: we compare our approach with recent competitive approaches as follows: RUN (Liu et al., 2020) formulates IUR semantic segmentation task and obtains much faster inference speed than generating from scratch, which can be seen our backbone; SARG (Huang al., 2021) proposes a semi ruto-regressive generator with the high effi- ciency and flexibility; RAU (Zhang et al., 2022)

---

portant words, so merging its relevance in soft way is necessary. Context marching module helps the approach to capture the important words from contexts.
In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages -3055-13633.
Shumpei Inoue, Tsungwei Liu, Nguyen Hong Son, and Minh-Tien Nguyen. 2022. Enhance incomplete utter- ance restoration by joint learning token extraction and text generation. arXiv preprint arXiv:2204.0 03958..
5 Conclusion
In this paper, we argue that capturing the source of important words can diminish introduce ir- relevant words for IUR. We propose novel and effective multi-task framework including context selection, edit matrix construction, and relevance merging to capture the multi- muuti-grranulartty of seman- tic information and fetch the relevant utterance. we do experiments on two benchmark datasets in IUR and show competitive performance.
Lisa Jin, Linfeng Song, Lifeng Jin, Dong Yu, and Daniel Gildea. 2022. Hierarchical context tagging for utter- ance rewriting. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 36, pages 10849-10857.
Cewn Lin. 2004. Rouge: A package for automatic
evaluation of summaries. In Text summarization branckes out, pages 74-81.
Qian Liu, Bei Chen, Jian-Guang Lou, Bin Zhou, and Dongmei Zhang. 2020. Incomplete utterance rewriting as semantic segmentation. arXiv preprint arXiv:2009.
Limitations
We propose a novel and effective multi-task frame- work to capture the multi-granularity of semantic information and fetch the relevant utterance. With the population of large language model (LLM), the multi-task finetuning framework may bring more computation cost. We will explore the combination of our approach with LLM in the future.
Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bern- hard Kainz, et al. 2018. Attention u-net: Learn- ing where to look for the pancreas. arXiv preprint 1rX4v:1804.000 03999..
Zhufeng Pan, Kun Bai, Yan Wang, Lianqiang Zhou, and Xiaojiang Liu. 2019. Improving open-domain dialogue systems via multi-turn incomplete utterance restoration. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1824-1833.
Acknowledgements
This work was supported by the National Key Research and Development Program of China (N0.202IYFC3340304).
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguisties, pages 311-318.
References
Eunsol Choi, He He, Mohit lyyer, Mark Yatskar, Wen- tau Yih, Yejin Choi, Perey Liang, and Luke Zettle- moyer. 2018. Quae: Question answering in context. arXiv preprint wrXiv:1808.07036
Siva Reddy, Danqi Chen, and Christopher D Manning. 2019. Coqa: A conversational question answering challenge. Transactions of the Association for Com- putational Linguistics, 7:249-266.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova, 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint 1rXiv:1810.04805.
Olaf Ronneherger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Conputer-Asistted Intervention-MICCA 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages Springer.
Ahmed Elgohary, Denis Peskov, and Jordan Boyd- Graber. 2019. Can you unpack that? learning to rewrite questions innions----conext.. Can You Unpack That? Learning to Rewrite Questions-in Contextt
Jie Hao. Linfeng Song, Liwei Wang, Kun Xu, Zhaopeng Tu. and Dong Yu. 2021. Rast: Domain-robust dia- logue rewriting as sequence tagging. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4913-4924.
Shuzheng Si, Shuang Zeng, and Baobao Chang. 2022. Mining clues from incomplete utterance: A query- enhanced network for incomplete utterance rewriting. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 4839-4847
Mengzuo Huang, Feng Li, Wuhe Zou, and Weidong Zhang. 2021. Sarg: novel semi autoregressive gen- erator for multi-turn incomplete utterance restoration.

---

Hui Su, Xiaoyu Shen. Rongzhi Zhang, Fei Sun, Pengwei Hu. Cheng Niu, and Jie Zhou. 2019. Improving multi-tum dialogue modelling with utterance rewriter. arXiv preprint 0ttt::9906.00000 07004.
Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie. 2019. Dream: challenge data set and models for dialogue-hased reading compre- hension. Transactions of the Association for Compu- lational Linguistics, 7:217-231.
Yong Zhang, Zhitao Li, Jianzong Wang, Ning Cheng, and Jing Xiao. 2022. Self-attention for incomplete ut- terance rewriting. In ICASSP 2022-2022 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8047-8051. IEEE.
A Example Appendix
This is section the appendix.


Reviews:
[{"review_id": "7GHO7t4Nlq", "reviewer": "Reviewer_DJDn", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "R1: We propose a simple and effective model structure to incorporate sentence-level semantic relations between the utterances in contexts and the incomplete utterance to enhance the ability to seize the source sentence and figure out the important words. This structure does not notably enlarge the model size or latency, and significantly improve the IUR performance compared with the backbone.\n\nR2: We outperform all the baselines by different metrics across BLEU, ROUGE, F-score on Restoration-200K, and baselines other than QUEEN on CANARD. QUEEN constructs the query template by the linguistic feature of the specific dataset and it constrains the effectiveness on general datasets or practical data. And our method can be easily generalized to other datasets without explicit template.\n\nR3: Though bringing a slight reduction (0.1%) on BLEU and ROUGE, context matching improves the F-score by about 1.0%. F-score concentrates more on words from the context (important words) which are argued to be harder to copy [1]. So we keep the context matching module in our method.\n\n[1] Improving open-domain dialogue systems via multi-turn incomplete utterance restoration. Pan et al. EMNLP2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1)The paper has some innovations. The paper proposes a framework including context selection, edit matrix construction, relevance merging and intention check for incomplete utterance rewriting. The training loss combines edit loss, selection loss, matching loss and intention loss.\n(2)The paper constructs comparative experiments and ablation experiments to prove the effectiveness of the proposed method.\n", "reasons_to_reject": "(1)The paper has some innovations but the innovations are not very prominent. The loss function is computed by the weighted sum of edit loss, selection loss, matching loss and intention loss. There are no outstanding innovations in the model structure.\n(2)The evaluation results on the CANARD dataset in Table 3 are not satisfactory.\n(3)From the results of the ablation experiments in Table4, the addition of the context marching module brings about a decrease in the ROUGE and BLEU indicators.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This is a paper in the field of incomplete discourse rewriting. The paper proposes a novel multi-task information interaction framework including context selection, edit matrix construction, relevance merging and intention check to capture the multi-granularity of semantic information. The paper also constructs corresponding comparative experiments and ablation experiments to prove the effectiveness of the proposed method.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pF6tMTckoA", "reviewer": "Reviewer_9o7i", "summary": "", "questions": "1. Does the proposed method  also work for rewriting  dialog state information in incomplete utterances ( the pair of  slot type and slot value)?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "R1: The IUR task aims to rewrite an incomplete utterance into an utterance which is semantically\nequivalent but self-contained to be understood without context. The identification of dialogue state is not explicitly needed.\n\nR2: We conduct an experiment using ChatGPT in-context learning ability (ICL). We randomly select 5 examples and append them to the task instruction as the demonstration, followed by rewriting the incomplete utterance. The performance on CANARD is as follows: B1: 61.5, B2: 52.5, R1: 46.1, R2: 40.1. It shows the ICL performance of LLM is significantly lower than existing IUR methods, and there is still exploration room on fitting the LLM for IUR task. We will add this baseline in revised version.\n\nQ1: We introduce our context selection and matching module into dialogue state tracking with T5 \nsequential generation and improve the JGA score [1] from 51.2 to 52.3 on MultiWOZ 2.2 dataset. It shows incorporating sentence-level semantic relations between the utterances in contexts also helps to seize the important dialogue information.\n\n[1] Dialogue State Tracking with a Language Model using Schema-Driven Prompting. Lee et al. EMNLP2021.\n\n[2] Multiwoz-a largescale multi-domain wizard-of-oz dataset for task-oriented dialogue modeling. Budzianowski et al. EMNLP2018.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper raises a important issue for incomplete utterance rewriting of multi-turn dialogues.\n\n2. This paper propose a novel approach to improve utterance infilling task by considering both utterance-level and word-level  semantic relations.\n\n3.  How the method improve the performance on incomplete utterance infilling of multi-turn dialogues keep basically clear.\n", "reasons_to_reject": "1. The proposed method in the paper fails to consider how to rewrite incomplete utterances which refers to dialog state ( the pair of  slot type and slot value) infilling problem. \n2.  The paper does not carry any experiments on large language models for incomplete utterance infilling performance comparison .", "questions_to_authors": "1. Does the proposed method  also work for rewriting  dialog state information in incomplete utterances ( the pair of  slot type and slot value)?", "paper_topic_and_main_contributions": "This paper introduces a multi-task knowledge interaction architecture to  capture multi-granularity of semantic information (sentences and words) to improve utterance infilling (e.g, ellipsis and co-reference) task.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KHhcizGfYk", "reviewer": "Reviewer_VzV8", "summary": "", "questions": "1. What are the model sizes for different components? How efficient is this system in terms of computational cost/latency?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "R1: We conduct an experiment using ChatGPT in-context learning ability (ICL). We randomly select 5 examples and append them to the task instruction as the demonstration, followed by rewriting the incomplete utterance. The performance on CANARD is as follows: B1: 61.5, B2: 52.5, R1: 46.1, R2: 40.1. It shows the ICL performance of LLM is significantly lower than IUR methods, and there is still exploration room on fitting the LLM for IUR task. We will add this baseline in revised version.\n\nQ1: There are about 110M parameters in text encoder (BERT), 1M parameters in edit matrix construction (U-net), and 20K parameters in context selection, context matching and intention checking. The latency is about 20ms to produce a single sentence with PyTorch on a single NVIDIA V100. The modules we propose will not significantly improve the model size and latency than the backbone.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The chosen methodology demonstrates a clear and relevant motivation, effectively aligning itself with the objectives of the Incomplete Utterance Rewriting (IUR) task. This indicates a thoughtful understanding of the task's requirements and goals. \n2. Its ability to attain SOTA performance proves its efficacy.", "reasons_to_reject": "1. As mentioned in limitation section, given the advances in generative LLMs, an end-to-end LLM may be preferable, assuming it can yield decent performance. Even though LLM is not a focus of the paper, ideally it should be included as a baseline/benchmark for completeness.", "questions_to_authors": "1. What are the model sizes for different components? How efficient is this system in terms of computational cost/latency?", "paper_topic_and_main_contributions": "The authors introduced an innovative and efficient framework for multi-task information interaction, encompassing context selection, edit matrix construction, and relevance merging to capture the multi-granularity of semantic information. The method surpasses current state-of-the-art models on two established benchmark datasets, namely Restoration-200K and CANAND.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a multi-task knowledge interaction architecture to improve performance on the utterance infilling task.

The paper discusses some interesting points regarding incomplete utterance rewriting of multi-turn dialogues
The paper proposes an effective approach and is able to achieve state-of-the-art results.
The included experiments and ablation tests are informative.

The method has limited novelty according to some reviewers.
Some of the results are mixed, while the ablation tests show a decrease when certain components are added.
The paper lacks comparsion to large language models, which may outperform this system, thereby decreasing its relevance to the current research directions.
Decision: Accept-Findings