SlowGuess/ACL_2020_OCR huggingface dataset:
In the second example, the multi-task explanation has lower ROUGE scores than the extractive one. We observe that the gold justification contains some sentences that are not relevant to the fact check, and the extractive summary is fooled to select explanation sentences that are close to the gold summary. As a result, the explanation does not provide enough information about the chosen veracity label. The multi-task model, on the other hand, selects sentences that are also contributing to the prediction of the veracity labels. Thus, its explanation turns out to be more beneficial for the final fact check even though it has a lower ROUGE score compared to the gold justification.
5 Manual Evaluation
As the ROUGE score only accounts for word-level similarity between gold and predicted justifications, we also conduct a manual evaluation of the quality of the produced veracity explanations.
Explanation Quality. We first provide a manual evaluation of the properties of three different types of explanations - gold justification, veracity explanation generated by the Explain-MT, and the ones generated by Explain-Extractive. We ask three annotators to rank these explanations with the ranks 1, 2, 3, (first, second, and third place) according to four different criteria:
1. Coverage. The explanation contains important, salient information and does not miss any important points that contribute to the fact check.
7357
Label: Mostly-True Claim: Says opponent U.S. Rep. Gwen Moore is Wisconsin's most absent member of Congress, missing nearly 17 percent of the House votes in the second quarter of 2012. Just: Sebring says Moore is the state's "most absent" member of Congress and that she missed 17 percent of the House votes in the second quarter of 2012. Indeed, Moore missed nearly 17 percent of the votes in the second quarter. That is Moore's worst quarter, but it also amounts to Sebring doing some cherry-picking of data. A longer period shows a lower percentage, but even that places Moore at the top of the state's eight-member delegation. Explain-Extr: Then he turns to the ad's main point: "Gwen Moore is Wisconsin's most absent member of Congress, missing nearly 17 percent of the House votes in the second quarter of 2012." That, again, compares to the median of 2.5 percent, according to GovTrack. Overall, the state delegation missed an average of 2.53 percent of the votes this session. But what about the other part of Sebring's claim, that Moore is the "most absent" member of the state delegation? Explain-MT: Then he turns to the ad's main point: "Gwen Moore is Wisconsin's most absent member of Congress, missing nearly 17 percent of the House votes in the second quarter of 2012." If you take out the first two quarters of 2012, Moore missed an average of 3.4 percent of the votes in the other five quarters of this session. In the second quarter, the one cited by Sebring, the missed votes easily put Moore at the top of the state's eight-member House delegation. Looking at the full session, Moore missed 112 votes, or 6.98 percent.
Label: Half-true Claim: Of the more than 1.3 million temporary mortgage modifications, over half have defaulted. Just: In the final full week of the U.S. Senate race, how did Rubio fare on his numbers about the mortgage modification program? Rubio said "over 1.3 million temporary work-outs, over half have now defaulted," referring to a temporary mortgage modification program. Explain-Extr: Over 1.3 million temporary work-outs, over half have now defaulted," he said. There have been 500,000 permanent (modifications). Rubio also said that more than half of those 1.3 million had defaulted." Rubio: "The temporary modifications." Explain-MT: Rubio also said that more than half of those 1.3 million had "defaulted." "Over 1.3 million temporary work-outs, over half have now defaulted," he said. Of those permanent modifications, the majority survived while almost 29,000 were cancelled. The chart states that 699,924 trials had been cancelled - so note that is slightly more than half.
Table 4: Examples of the generated explanation of the extractive (Explain-Extr) and the multi-task model (Explain-MT) compared to the gold justification (Just).
2. Non-redundancy. The summary does not contain any information that is redundant/repeated/not relevant to the claim and the fact check.
3. Non-contradiction. The summary does not contain any pieces of information that are contradictory to the claim and the fact check.
4. Overall. Rank the explanations by their overall quality.
We also allow ties, meaning that two veracity explanations can receive the same rank if they appear the same.
For the annotation task set-up, we randomly select a small set of 40 instances from the test set and collect the three different veracity explanations for each of them. We did not provide the participants with information of the three different explanations and shuffled them randomly to prevent easily creating a position bias for the explanations. The annotators worked separately without discussing any details about the annotation task.
Explanation Informativeness. In the second manual evaluation task, we study how well the veracity explanations manage to address the information need of the user and if they sufficiently describe the veracity label. We, therefore, design the annotation task asking annotators to provide a veracity label for a claim based on a veracity explanation coming from the justification, the Explain-MT, or the Explain-Extractive system. The annotators have to provide a veracity label on two levels - binary classification - true or false, and six-class classification - true, false, half-true, barely-true, mostly-true, pants-on-fire. Each of them has to
provide the label for 80 explanations, and there are two annotators per explanation.
Explanation Quality. Table 5 presents the results from the manual evaluation in the first set-up, described in Section 5, where annotators ranked the explanations according to four different criteria.
We compute Krippendorff's  $\alpha$  inter-annotator agreement (IAA, Hayes and Krippendorff (2007)) as it is suited for ordinal values. The corresponding alpha values are 0.26 for Coverage, 0.18 for Nonredundancy, -0.1 for Non-contradiction, and 0.32 for Overall, where  $0.67 < \alpha < 0.8$  is regarded as significant, but vary a lot for different domains.
We assume that the low IAA can be attributed to the fact that in ranking/comparison tasks for manual evaluation, the agreement between annotators might be affected by small differences in one rank position in one of the annotators as well as by the annotator bias towards ranking explanations as ties. Taking this into account, we choose to present the mean average recall for each of the annotators instead. Still, we find that their preferences are not in a perfect agreement and report only what the majority agrees upon. We also consider that the low IAA reveals that the task might be "already too difficult for humans". This insight proves to be important on its own as existing machine summarisation/question answering studies involving human evaluation do not report IAA scores (Liu and Lapata, 2019), thus, leaving essential details about the nature of the evaluation tasks ambiguous.
We find that the gold explanation is ranked the best for all criteria except for Non-contradiction,
7358
where one of the annotators found that it contained more contradictory information than the automatically generated explanations, but Krippendorff's  $\alpha$  indicates that there is no agreement between the annotations for this criterion.
Out of the two extractive explanation systems, Explain-MT ranks best in Coverage and Overall criteria, with 0.21 and 0.13 corresponding improvements in the ranking position. These results contradict the automatic evaluation in Section 4.3, where the explanation of Explain-MT had lower ROUGE F1 scores. This indicates that an automatic evaluation might be insufficient in estimating the information conveyed by the particular explanation.
On the other hand, Explain-Extr is ranked higher than Explain-MT in terms of Non-redundancy and Non-contradiction, where the last criterion was disagreed upon, and the rank improvement for the first one is only marginal at 0.04.
This implies that a veracity prediction objective is not necessary to produce natural-sounding explanations (Explain-Extr), but that the latter is useful for generating better explanations overall and with higher coverage Explain-MT.
Explanation Informativeness. Table 6 presents the results from the second manual evaluation task, where annotators provided the veracity of a claim based on an explanation from one of the systems. We here show the results for binary labels, as annotators struggled to distinguish between 6 labels. The latter follows the same trends and are shown in Appendix A.3.
The Fleiss'  $\kappa$  IAA for binary prediction is: Just - 0.269, Explain-MT - 0.345, Explain-Extr - 0.399. The highest agreement is achieved for ExplainExtr, which is supported by the highest proportion of agreeing annotations from Table 6. Surprisingly, the gold explanations from Just were most disagreed upon. Apart from that, looking at the agreeing annotations, gold explanations were found most sufficient in providing information about the veracity label and also were found to explain the correct label most of the time. They are followed by the explanations produced by Explain-MT. This supports the findings of the first manual evaluation, where the Explain-MT ranked better in coverage and overall quality than Explain-Extr.
Generating Explanations. Generating textual explanations for model predictions is an understud
Annotators	Just	Explain-Extr	Explain-MT
Coverage
All	1.48	1.89	1.68
1st	1.50	2.08	1.87
2nd	1.74	2.16	1.84
3rd	1.21	1.42	1.34
Non-redundancy
All	1.48	1.75	1.79
1st	1.34	1.84	1.76
2nd	1.71	1.97	2.08
3rd	1.40	1.42	1.53
Non-contradiction
All	1.45	1.40	1.48
1st	1.13	1.45	1.34
2nd	2.18	1.63	1.92
3rd	1.03	1.13	1.18
Overall
All	1.58	2.03	1.90
1st	1.58	2.18	1.95
2nd	1.74	2.13	1.92
3rd	1.42	1.76	1.82
Table 5: Mean Average Ranks (MAR) of the explanations for each of the four evaluation criteria. The explanations come from the gold justification (Just), the generated explanation (Explain-Extr), and the explanation learned jointly (Explain-MT) with the veracity prediction model. The lower MAR indicates a higher ranking, i.e., a better quality of an explanation. For each row, the best results are in bold, and the best results with automatically generated explanations are in blue.
ied problem. The first study was Camburu et al. (2018), who generate explanations for the task of natural language inference. The authors explore three different set-ups: prediction pipelines with explanation followed by prediction, and prediction followed by explanation, and a joint multi-task learning setting. They find that first generating the explanation produces better results for the explanation task, but harms classification accuracy.
We are the first to provide a study on generating veracity explanations. We show that the generated explanations improve veracity prediction performance, and find that jointly optimising the veracity explanation and veracity prediction objectives improves the coverage and the overall quality of the explanations.
Fact Checking Interpretability. Interpreting fact checking systems has been explored in a few studies. Shu et al. (2019) study the interpretability of a system that fact checks full-length news pages by leveraging user comments from social platforms. They propose a co-attention frame
7359
Just	Explain-Extr	Explain-MT
Agree-C	0.403	0.237	0.300
Agree-NS	0.065	0.250	0.188
Agree-NC	0.064	0.113	0.088
Disagree	0.468	0.400	0.425
Table 6: Manual veracity labelling, given a particular explanation from the gold justification (Just), the generated explanation (Explain-Extr), and the explanation learned jointly (Explain-MT) with the veracity prediction model. Percentages of the dis/agreeing annotator predictions are shown, with agreement percentages split into: correct according to the gold label (Agree-C), incorrect (Agree-NC) or insufficient information (Agree-NS). The first column indicates whether higher ( $\searrow$ ) or lower ( $\searrow$ ) values are better. For each row, the best results are in bold, and the best results with automatically generated explanations are in blue.
work, which selects both salient user comments and salient sentences from news articles. Yang et al. (2019) build an interpretable fact-checking system XFake, where shallow student and self-attention, among others, are used to highlight parts of the input. This is done solely based on the statement without considering any supporting facts. In our work, we research models that generate human-readable explanations, and directly optimise the quality of the produced explanations instead of using attention weights as a proxy. We use the LIAR dataset to train such models, which contains fact checked single-sentence claims that already contain professional justifications. As a result, we make an initial step towards automating the generation of professional fact checking justifications.
Veracity Prediction. Several studies have built fact checking systems for the LIAR dataset (Wang, 2017). The model proposed by Karimi et al. (2018) reaches 0.39 accuracy by using metadata, ruling comments, and justifications. Alhindi et al. (2018) also trains a classifier, that, based on the statement and the justification, achieves 0.37 accuracy. To the best of our knowledge, Long et al. (2017) is the only system that, without using justifications, achieves a performance above the baseline of Wang (2017), an accuracy of 0.415—the current state-of-the-art performance on the LIAR dataset. Their model learns a veracity classifier with speaker profiles. While using metadata and external speaker profiles might provide substantial information for fact checking, they also have the potential to introduce biases towards a certain party or a speaker.
In this study, we propose a method to generate
veracity explanations that would explain the reasons behind a certain veracity label independently of the speaker profile. Once trained, such methods could then be applied to other fact checking instances without human-provided explanations or even to perform end-to-end veracity prediction and veracity explanation generation given a claim.
Substantial research on fact checking methods exists for the FEVER dataset (Thorne et al., 2018), which comprises rewritten claims from Wikipedia. Systems typically perform document retrieval, evidence selection, and veracity prediction. Evidence selection is performed using keyword matching (Malon, 2018; Yoneda et al., 2018), supervised learning (Hanselowski et al., 2018; Chakrabarty et al., 2018) or sentence similarity scoring (Ma et al., 2018; Mohtarami et al., 2018; Xu et al., 2018). More recently, the multi-domain dataset MultiFC (Augenstein et al., 2019) has been proposed, which is also distributed with evidence pages. Unlike FEVER, it contains real-world claims, crawled from different fact checking portals.
While FEVER and MultiFC are larger datasets for fact checking than LIAR-PLUS, they do not contain veracity explanations and can thus not easily be used to train joint veracity prediction and explanation generation models, hence we did not use them in this study.
We presented the first study on generating veracity explanations, and we showed that veracity prediction can be combined with veracity explanation generation and that the multi-task set-up improves the performance of the veracity system. A manual evaluation shows that the coverage and the overall quality of the explanation system is also improved in the multi-task set-up.
For future work, an obvious next step is to investigate the possibility of generating veracity explanations from evidence pages crawled from the Web. Furthermore, other approaches of generating veracity explanations should be investigated, especially as they could improve fluency or decrease the redundancy of the generated text.
This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 801199.
7360
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity Checks for Saliency Maps. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pages 9525-9536, USA. Curran Associates Inc.
Tariq Alhindi, Savvas Petridis, and Smaranda Muresan. 2018. Where is your evidence: Improving fact-checking by justification modeling. In Proceedings of the First Workshop on Fact Extraction and Verification (FEVER), pages 85-90, Brussels, Belgium. Association for Computational Linguistics.
Leila Arras, Ahmed Osman, Klaus-Robert Müller, and Wojciech Samek. 2019. Evaluating recurrent neural network explanations. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 113-126, Florence, Italy. Association for Computational Linguistics.
Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, and Jakob Grue Simonsen. 2019. MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4677-4691, Hong Kong, China. Association for Computational Linguistics.
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-SNLI: Natural Language Inference with Natural Language Explanations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 9539-9549. Curran Associates, Inc.
Tuhin Chakrabarty, Tariq Alhindi, and Smaranda Muresan. 2018. Robust document retrieval and individual evidence modeling for fact extraction and verification. In Proceedings of the First Workshop on Fact Extraction and VERIFICATION (FEVER), pages 127-131, Brussels, Belgium. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186.
Priya Goyal, Piotr Dollar, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew
Tulloch, Yangqing Jia, and Kaiming He. 2017. Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour. arXiv preprint arXiv:1706.02677.
Andreas Hanselowski, Hao Zhang, Zile Li, Daniil Sorokin, Benjamin Schiller, Claudia Schulz, and Iryna Gurevych. 2018. UKP-athene: Multi-sentence textual entailment for claim verification. In Proceedings of the First Workshop on Fact Extraction and VERIFICATION (FEVER), pages 103-108, Brussels, Belgium. Association for Computational Linguistics.
Andrew F Hayes and Klaus Krippendorff. 2007. Answering the Call for a Standard Reliability Measure for Coding Data. Communication methods and measures, 1(1):77-89.
Hamid Karimi, Proteek Roy, Sari Saba-Sadiya, and Jiliang Tang. 2018. Multi-source multi-class fake news detection. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1546-1557, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Yang Liu and Mirella Lapata. 2019. Text Summarization with Pretrained Encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3728-3738, Hong Kong, China. Association for Computational Linguistics.
Yunfei Long, Qin Lu, Rong Xiang, Minglei Li, and Chu-Ren Huang. 2017. Fake News Detection Through Multi-Perspective Speaker Profiles. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 252-256.
Ilya Loshchilov and Frank Hutter. 2017. Fixing Weight Decay Regularization in Adam. arXiv preprint arXiv:1711.05101.
Jing Ma, Wei Gao, and Kam-Fai Wong. 2018. *Detect Rumor and Stance Jointly by Neural Multi-task Learning*. In *Companion Proceedings of the The Web Conference* 2018, WWW '18, pages 585-593, Republic and Canton of Geneva, Switzerland. International World Wide Web Conferences Steering Committee.
Christopher Malon. 2018. Team papelo: Transformer networks at FEVER. In Proceedings of the First Workshop on Fact Extraction and VERIFICATION (FEVER), pages 109-113, Brussels, Belgium. Association for Computational Linguistics.
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. 2016. Cross-stitch networks for multi-task learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3994-4003.
Mitra Mohtarami, Ramy Baly, James Glass, Preslav Nakov, Lluís Márquez, and Alessandro Moschitti. 2018. Automatic stance detection using end-to-end memory networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 767-776, New Orleans, Louisiana. Association for Computational Linguistics.
Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI'17, pages 3075-3081. AAAI Press.
Nina Poerner, Hinrich Schütze, and Benjamin Roth. 2018. Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 340-350, Melbourne, Australia. Association for Computational Linguistics.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain yourself! leveraging language models for commonsense reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4932-4942, Florence, Italy. Association for Computational Linguistics.
Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, and Anders Søgaard. 2019. Latent Multi-Task Architecture Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 33:4822-4829.
Victor Sanh, Lysandre Debut, Julien Chaumont, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. In Proceedings of the 5th Workshop on Energy Efficient Machine Learning and Cognitive Computing, NeurIPS'2019.
Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. 2019. dEFEND: Explainable Fake News Detection. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 395-405.
Dominik Stammbach and Guenter Neumann. 2019. Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task. In Proceedings of the Second Workshop on Fact Extraction and VERIFICATION (FEVER), pages 105-109, Hong Kong, China. Association for Computational Linguistics.
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERIFICATION. In NAACL-HLT.