SlowGuess/NAACL_2021_OCR huggingface dataset:
Dataset As discussed in Sec. 3, we conduct all experiments on the MuDoCo dataset and follow the provided data split<sup>1</sup>. Data from 6 domains are aggregated to form train/dev/test sets with 16k, 1.9k and 1.9k examples respectively. Each example contains the dialogue context, the latest user query, and the corresponding coreference resolution and query rewrite annotations. Statistics for each domain are provided in Table 1. Out of all examples that are not the first turn,  $64.2\%$  of them contain coreference links and  $43.7\%$  of them require query rewrite. This makes the task more challenging, as the model also needs to learn when not to rewrite a query and when to predict no coreference links. Note that not every coreference link requires rewriting, as in the
Domain	Total	Coref.	Rewrite
Calling	10.7k	4.0k (60.5%)	2.2k (33.7%)
Messaging	3.9k	1.7k (69.0%)	1.0k (41.2%)
Music	2.8k	1.4k (77.7%)	1.4k (76.7%)
News	387	156 (66.4%)	189 (76.6%)
Reminders	1.7k	632 (56.4%)	357 (31.9%)
Weather	254	38 (28.6%)	102 (76.6%)
All	19.8k	8.0k (64.2%)	5.3k (43.7%)
Table 1: Total number of examples across six domains in the MuDoCo dataset, number of examples requiring coreference resolution (Coref.) and those that need query rewrite (Rewrite). Percentages are calculated across all follow-up turns (i.e., excluding the first turn).
MuDoCo dataset there are coreference annotations where the mention has the exact same word span as its referent.
Setup The GPT-2 decoder layers and word classification layer in our model are initialized with the pre-trained weights from the GPT-2 small model. We fine-tune the model using Adam (Kingma and Ba, 2014) optimizer with learning rate 5e-05 and batch size 15. The criterion for early stopping is the averaged performance of coreference resolution and query rewrite on the development set. Results are obtained as the average of 5 runs.
5.1 Query Rewrite
Evaluation Metrics The standard BLEU-4 (Papineni et al., 2002) between the generated and the target sentences are reported. In addition, to highlight the quality of the rewritten parts in generated sentences, following the post-processing in Quan et al. (2019), we measure an F1 score calculated by comparing machine-generated words with ground truth words for only the ellipsis / co-reference part of user utterances. We also report the percentage of all referents in ground-truth coreference links that were successfully generated in the query rewrite, denoted as reference match (RM). The RM ratio explicitly reflects the quality of coreference resolution in the generated rewritten query.
Baselines The standard seq-to-seq model with attention (seq2seq) and its pointer network (PN; Vinyals et al. (2015)) and pointer-generator network (PG; See et al. (2017)) variants are implemented as baselines. The concatenation of the dialogue context and the query are fed as input, and the output is the target rewrite. The size of the hidden states is 300 and word vectors are initialized with GloVe embeddings (Pennington et al., 2014).
<https://github.com/facebookresearch/mudoco>
Model	Prec.	Rec.	F1	BLEU	RM
seq2seq model	38.3	29.6	33.4	81.0	54.7
+ pn (Vinyals et al., 2015)	42.4	34.1	37.6	86.0	61.2
+ pg (See et al., 2017)	41.4	39.5	40.4	86.4	63.2
Our QR-only model	58.9	57.1	57.9	89.8	78.7
Our joint model	61.0	59.5	60.2	90.2	82.0
Results Table 2 shows the query rewrite results. The low F1 score and high BLEU score is because of filtering out the non-rewritten repeated tokens in post-processing when calculating F1. This allows us to better evaluate the quality of rewritten parts and to better differentiate between good and bad generation in our task. We find that our joint model substantially outperforms all LSTM-based seq-to-seq models on all metrics. Although the pointer-generator in LSTMs can effectively copy words from the input to its generation, the powerful transformer architecture with pre-trained weights allows better learning of rewriting patterns.
To fairly investigate the impact of coreference modeling on the generation of query rewrite, we train a variant of our model using only the query rewrite objectives (Eqns. (4) and (5)), denoted as QR-only model. We can see that without coreference resolution, the F1 score drops from 60.2 to 57.9 and the reference match drops from 82.0 to 78.7. This illustrates the improved ability of the joint model to rewrite anaphoric expressions, since the model can leverage its coreference resolution predictions to generate more accurate query rewrites. We present a detailed case study with model predictions in Sec. 5.5.
5.2 Coreference Resolution
Evaluation Metrics The MUC,  $\mathbf{B}^3$ , and  $\mathrm{CEAF}_{\phi_4}$  metrics that are widely-used in coreference resolution task are reported. Note that these metrics are calculated based on coreference clusters and we only have ground-truth annotations
for coreference links between mentions and referents. To align the links and clusters, during evaluation we post-process both the ground-truth and the model predictions. All the word spans that are identical to the referent in the dialogue context are combined into a cluster so that a link between a mention and a referent can be transformed into a cluster for the standard coreference resolution evaluation.
Baselines To the best of our knowledge, there is no suitable coreference resolution model that is proposed in the same setup for dialogues<sup>2</sup>. We therefore experiment with the state-of-the-art models of document-based coreference resolution, including the end-to-end model (Lee et al., 2017, 2018) using BERT (Joshi et al., 2019) or SpanBERT (Joshi et al., 2020)<sup>3</sup>. Note that these models can only serve as a reference since they are not specifically designed for dialogue-based tasks. Since they require coreference clusters for training, coreference clusters are built from annotated links as in the post-processing step done for evaluation.
Results As seen in Table 3, SpanBERT obtains better results than BERT, which is consistent with the findings in Joshi et al. (2020). This is mainly because SpanBERT is better at capturing span information, which facilitates tasks such as coreference resolution where reasoning about relationships between spans is required. In comparison, our joint learning model achieves competitive and even slightly better results. This indicates that the design of our model leveraging attention heads inside GPT-2 is effective at predicting coreference links in dialogues. To test if the supervision of query rewrite affects the optimization of coreference resolution in joint learning, we train a model variant
2The baseline in Martin et al. (2020) is not compared for two reasons: 1) their setups in training/evaluation is different than ours in many ways, e.g., they only consider finished dialogues; 2) their source code is not released.
<sup>3</sup>https://github.com/mandarjoshi90/coref
Table 2: Query rewrite results in F1 and BLEU. QR-only model is our model variant trained using only objectives of query rewrite.
MUC	B3	CEAFφ4	Avg. F1
P	R	F1	P	R	F1	P	R	F1
c2f-coref + BERT (Joshi et al., 2019)	72.2	66.7	69.3	74.5	67.9	71.0	77.7	72.6	75.1	71.8
c2f-coref + SpanBERT (Joshi et al., 2020)	71.7	71.4	71.5	73.5	72.5	73.0	77.8	74.9	76.3	73.6
Our coref-only model	78.8	69.4	73.8	79.6	71.3	75.2	80.7	75.1	77.8	75.6
Our joint model	78.3	69.4	73.6	79.5	71.2	75.1	81.1	75.1	78.0	75.6
Table 3: Coreference resolution results. "Our coref-only model" is our model variant trained only using the objectives of coreference resolution.
Prec.	Rec.	F1	BLEU	RM
complete model	61.0	59.5	60.2	90.2	82.0
- coref2qr attention	55.5	59.3	57.3	89.3	80.6
- coref. modeling	58.9	57.1	57.9	89.8	78.7
- binary head	54.6	54.4	54.3	88.9	78.9
Table 4: Ablation study of our joint learning model on query rewrite performance.
Model	Calling	Messaging	Music	All
coref.	elp.	coref.	elp.	coref.	elp.	coref.	elp.
seq2seq+pg	56.0	36.2	63.6	36.2	45.5	38.2	52.0	34.5
QR-only	75.4	51.4	77.6	66.5	59.8	45.5	69.0	49.3
Joint	78.3	52.0	81.3	64.3	63.1	51.1	72.1	50.9
using only the objectives for coreference resolution (Eqns. (2) and (3)), denoted as coref-only model. It is observed that the results of the coref-only model are very close to that of the joint model, showing that the addition of coreference resolution in joint learning is beneficial to query rewrite without sacrificing the performance of the former.
Here, we investigate how the different components in our joint model contribute to the performance of query rewrite. We remove one component at a time and examine the performance of query rewrite. As shown in Table 4, without the designed coref2qr attention layer, the performance degrades with a drop of  $2.9\%$  F1 and  $1.4\%$  RM rate. By further removing the supervision of coreference modeling from our joint learning model, the model is solely optimized towards the objectives of query rewrite and produces worse results compared to the complete model. These results indicate that through joint learning, the model's ability of generating the rewritten query improves, including its ability to rewrite the anaphora with its antecedent, by lever
aging the information from coreference resolution modeling. In addition, the binary head plays an essential role in our model. The accuracy of this binary classifier is  $93.9\%$ . Without the binary head, the performance drop can be up to  $5.9\%$  F1 (60.2 -> 54.3). This shows that with the binary classification, the model is able to focus on rewriting the input query without worrying about whether to rewrite or not.
In this section we analyze query rewrite performance on two different types of rewriting, coreference (coref.) and ellipses (elp.). The F1 score over three main domains and all test sets are reported in Table 5. The seq2seq+pg model is the baseline seq2seq model with pointer-generator; QR-only model is our model variant but trained without coreference modeling. The overall trend shows that 1) when the dialogue contains coreferences, the joint learning model is more capable of rewriting the query by leveraging its coreference predictions; 2) when coreferences are not present but the query still needs rewriting on account of information omission, the joint model can still perform competitively with the QR-only model.
We demonstrate several examples of query rewrites generated by different models to provide more insights into the task and into the benefits of joint learning. The coreference links predicted by the joint learning model are appended after its generated rewrite. Two examples that require coreference resolution in query rewrite are shown in Table 6. In the left dialogue, "the song" in the user query refers back to "Talking to the Moon" mentioned in the first user turn. Both seq2seq+pg and QR-only model fail to generate the correct reference in the rewrite, probably because of the high
Table 5: Query rewrite performance (F1) over three main domains and All test set with respect to two types of rewriting: coreference (coref.) and ellipses (elp.).
Dialogue Context	usr: When was Talking to the Moon by Bruno Mars released?
sys: On April 12, 2011.
usr: Who produced the song?
sys: The Smeezingtons and Bhasker.	usr: I want to send a message.
sys: Who do you want to send a message to?
usr: To Ariana.
sys: Ariana Smith or Ariana Taylor?
User Query	Could you play the song for me?	The second one.
Rewrite Label	Could you play Talking to the Moon for me?	Ariana Taylor.
seq2seq+pg	Could you play the moon for me? ×	Ariana Smith. ×
QR-only	Could you play the song for me? ×	Ariana Smith. ×
Joint	Could you play Talking to the Moon for me? (song -> Talking to the Moon) √	Ariana Taylor. (one -> Ariana Taylor) √
Table 6: Two coreference examples from test set with rewrites generated by three models: 1. seq-to-seq model with pointer-generator (seq2seq+pg); 2. Our QR-only model; 3. Our joint learning model. The rewritten parts are highlighted in bold. The coreference links predicted by 3. are presented as (mention -> antecedent).
Dialogue Context	usr: What's the temperature like in Richmond today? sys: The temperature is going to be a warm 85%, but there is a chance of rain.
User Query	What are the chances of rain today?
Rewrite Label	What are the chances of rain today in Richmond?
seq2seq+pg	What are the chances of rain today? X
QR-only	What are the chances of rain today in Richmond? ✓
Joint	What are the chances of rain today in Richmond? ✓
Table 7: An example with ellipsis from the test set. Rewrites generated by three different models are shown.
complexity of a long dialogue. The joint learning model not only correctly predicts the coreference link pointing from the mention to its referent in the first turn, but also generates a rewrite perfectly consistent with its coreference prediction. A similar trend can be observed in the right example. The first two models cannot identify which "Ariana" to generate, while our model is able to rewrite with the correct one with the aid of the correct coreference resolution. While our model does well on most of the test cases, there are situations where the joint model fails to predict correctly. A representative failure example is provided in Appendix A.2.
Table 7 shows an ellipsis example. The implicit location in the user query can be recovered through rewriting by both GPT-2 based models, while the LSTM-based model tends to keep the query. This indicates that 1) even with the pointer-generator's ability to copy source text, the seq2seq model is not capable enough of handling the difficult information omission rewrite; 2) the joint learning model still performs well on ellipses, while substantially benefiting in coreference cases.
We propose a novel joint learning framework for coreference resolution and query rewrite in dialogues. Modeling coreference resolution not only complements the missing information in query rewrite, but is also beneficial to rewriting anaphoric expressions. Our joint learning model can predict coreference links between the user query and dialogue context, and generate the rewritten query. We show that with the aid of coreference resolution, the performance of query rewrite can be substantially boosted. Furthermore, our model produces competitive results in coreference resolution when compared to state-of-the-art BERT-based systems. We hope that the presented joint learning task with the release of our query rewrite annotations on the
MuDoCo dataset provides a promising research direction in multi-turn dialogue understanding.
One restriction of our model is that by virtue of the model being designed to predict the boundaries of a reference, our model is only able to handle cases involving continuous spans of words. In addition, the influence of query rewrite on coreference resolution is limited due to the nature of the information flow in our current model design. Future work will focus on these perspectives.
The authors would like to thank Hadas Kotek for her help with the data annotation guidelines and the organization of the grading project. The authors would also like to thank Barry-John Theobald, Stephen Pulman, Jason Williams and Murat Akbacak for discussions and feedback, and the anonymous reviewers for their helpful feedback.
Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, and Srinivas Chappidi. 2020. Open-domain question answering goes conversational via question rewriting. arXiv preprint arXiv:2010.04898.
Eric Bengtson and Dan Roth. 2008. Understanding the value of features for coreference resolution. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 294-303.
Daniel M Bikel, Vittorio Castelli, Radu Florian, and Ding-jung Han. 2009. Entity linking and slot filling through statistical processing and inference rules. In TAC.
Anders Björkelund and Jonas Kuhn. 2014. Learning structured perceptrons for coreference resolution with latent antecedents and non-local features. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 47-57.
Kevin Clark and Christopher D Manning. 2016a. Deep reinforcement learning for mention-ranking coreference models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2256-2262.
Kevin Clark and Christopher D Manning. 2016b. Improving coreference resolution by learning entity-level distributed representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 643-653.
Eraldo Fernandes, Cicero dos Santos, and Ruy Luiz Milidiú. 2012. Latent structure perceptron with feature induction for unrestricted coreference resolution. In Joint Conference on EMNLP and CoNLL-Shared Task, pages 41-48.
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64-77.
Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel S Weld. 2019. Bert for coreference resolution: Baselines and analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5807-5812.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Vineet Kumar and Sachindra Joshi. 2016. Non-sentential question resolution using sequence to sequence learning. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2022-2031, Osaka, Japan. The COLING 2016 Organizing Committee.
Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 188-197.
Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 687-692.
Scott Martin, Shivani Poddar, and Kartikeya Upasani. 2020. MuDoCo: Corpus for multidomain coreference resolution and referring expression generation. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 104-111, Marseille, France. European Language Resources Association.
Sebastian Martschat and Michael Strube. 2015. Latent structures for coreference resolution. Transactions of the Association for Computational Linguistics, 3:405-418.
Vincent Ng. 2010. Supervised noun phrase coreference research: The first fifteen years. In Proceedings of the 48th annual meeting of the association for computational linguistics, pages 1396-1411.
Vincent Ng and Claire Cardie. 2002. Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution. In COLING 2002:
The 19th International Conference on Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311-318.
Haoruo Peng, Kai-Wei Chang, and Dan Roth. 2015. A joint framework for coreference resolution and mention head detection. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning, pages 12–21, Beijing, China. Association for Computational Linguistics.
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.
Jun Quan, Deyi Xiong, Bonnie Webber, and Changjian Hu. 2019. Gecor: An end-to-end generative ellipsis and co-reference resolution model for task-oriented dialogue. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4539-4549.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Karthik Raghunathan, Heeyoung Lee, Sudarshan Rangarajan, Nathanael Chambers, Mihai Surdeanu, Dan Jurafsky, and Christopher D Manning. 2010. A multi-pass sieve for coreference resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 492-501.
Pushpendre Rastogi, Arpit Gupta, Tongfei Chen, and Mathias Lambert. 2019. Scaling multi-domain dialogue state tracking via query reformulation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers), pages 97-105.
Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073-1083.
Hui Su, Xiaoyu Shen, Rongzhi Zhang, Fei Sun, Pengwei Hu, Cheng Niu, and Jie Zhou. 2019. Improving multi-turn dialogue modelling with utterance rewriter. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 22-31.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008.
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Advances in neural information processing systems, pages 2692-2700.
Tsung-Hsien Wen, David Vandyke, Nikola Mrkšić, Milica Gasic, Lina M Rojas Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017. A network-based end-to-end trainable task-oriented dialogue system. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 438-449.
Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. CorefQA: Coreference resolution as query-based span prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6953-6963, Online. Association for Computational Linguistics.
Wei Yang, Rui Qiao, Haocheng Qin, Amy Sun, Luchen Tan, Kun Xiong, and Ming Li. 2019. End-to-end neural context reconstruction in Chinese dialogue. In Proceedings of the First Workshop on NLP for Conversational AI, pages 68-76, Florence, Italy. Association for Computational Linguistics.
The average run time for training our joint learning model is 6 hours using GTX 1080 Ti. Based on the GPT-2 architecture, our model has 148M parameters. For the attention heads used for predicting the referent in Equation 1, hyper-parameter boundaries for  $L'$  and  $J'$  are:  $1 \leq L' \leq 12$  and  $1 \leq J' \leq 12$ . The best performance is obtained when only using the last two decode layers  $(L' = 2)$  with 3 attention heads used in each layer  $(J' = 3)$ . Hyper-parameters are tuned based on the averaged performance of query rewrite and coreference resolution on the development set.
A.2 Sample Model-Generated Failure Cases
We find that our joint model makes mistakes when the coreference signal is ambiguous and there is complex dialogue context (e.g., having multiple person names in an utterance). In a representative example (Table 8), even though the joint model predicts one of the coreference links correctly (one  $->$  call) and generates the corresponding rewritten span (call from Sana and Erica), it fails to infer that the pronoun her refers to Deirdre, and simply ignores the corresponding rewrite. This is likely because there are many female names that the pronoun her can refer to in this utterance, and these types of complex cases are too infrequent in the training corpus for the model to learn well.
A.3 Query Rewrite Annotation Guideline