SlowGuess/NAACL_2022_OCR huggingface dataset:
Cana, Dominican Republic. Association for Computational Linguistics.
Jordan J Louviere, Terry N Flynn, and Anthony Alfred John Marley. 2015. Best-worst scaling: Theory, methods and applications. Cambridge University Press.
Jordan J Louviere and George G Woodworth. 1991. Best-worst scaling: A model for the largest difference judgments. University of Alberta: Working Paper.
Rabeeh Karimi Mahabadi, James Henderson, and Sebastian Ruder. 2021. Compacter: Efficient low-rank hypercomplex adapter layers. arXiv preprint arXiv:2106.04647.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906-1919, Online. Association for Computational Linguistics.
Walaa Medhat, Ahmed Hassan, and Hoda Korashy. 2014. Sentiment analysis algorithms and applications: A survey. Ain Shams engineering journal, 5(4):1093-1113.
Amit Moryossef, Yoav Goldberg, and Ido Dagan. 2019. Step-by-step: Separating planning from realization in neural data-to-text generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2267-2277. Association for Computational Linguistics.
Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulíçehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 280-290, Berlin, Germany. Association for Computational Linguistics.
Shashi Narayan, Yao Zhao, Joshua Maynez, Goncalo Simoes, and Ryan McDonald. 2021. Planning with entity chains for abstractive summarization. arXiv preprint arXiv:2104.07606.
Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 188-197, Hong Kong, China. Association for Computational Linguistics.
Bryan Orme. 2009. Maxdiff analysis: Simple counting, individual-level logit, and hb. Sequim, WA: Sawtooth Software.
Nadav Oved and Ran Levy. 2021. Pass: Perturb-and-select summarizer for product reviews. In ACL/IJCNLP.
Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304.
Clifton Poth, Jonas Pfeiffer, Andreas Rückle, and Iryna Gurevych. 2021. What to pre-train on? efficient intermediate task selection. arXiv preprint arXiv:2104.08247.
Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. ArXiv, abs/1910.10683.
Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 379-389, Lisbon, Portugal. Association for Computational Linguistics.
Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073-1083, Vancouver, Canada. Association for Computational Linguistics.
Yoshihiko Suhara, Xiaolan Wang, Stefanos Angelidis, and Wang-Chiew Tan. 2020. OpinionDigest: A simple framework for opinion summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5789-5798.
Wenyi Tay, Aditya Joshi, Xiuzhen Zhang, SarvNaz Karimi, and Stephen Wan. 2019. Red-faced ROUGE: Examining the suitability of ROUGE for opinion summary evaluation. In Proceedings of the The 17th Annual Workshop of the Australasian Language Technology Association, pages 52-60, Sydney, Australia. Australasian Language Technology Association.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008.
Yumo Xu and Mirella Lapata. 2020. Coarse-to-fine query focused multi-document summarization. In Proceedings of the 2020 Conference on empirical methods in natural language processing (EMNLP), pages 3632-3645.
Yumo Xu and Mirella Lapata. 2021. Text summarization with latent queries. arXiv preprint arXiv:2106.00104.
Tiezheng Yu, Zihan Liu, and Pascale Fung. 2021. Adaptsum: Towards low-resource domain adaptation for abstractive summarization. In arXiv preprint arXiv:2103.11332.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of International Conference on Machine Learning (ICML).
Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. 2014. Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, pages 83-92.
10 Appendices
10.1 Human Evaluation Setup
To performed the human evaluation experiments described in Sec. 4.2, we hired workers with  $98\%$  approval rate,  $1000+$  HITS, Location: USA and the maximum score on a qualification test that we had designed. The test asked if the workers were native English speakers, and verified that they correctly understood the instructions of both the best-worst scaling and content support tasks. We paid the workers an approximate amount of $12 per hour.
10.2 Best-Worst Scaling Details
We performed human evaluation based on the Amazon test set using the AMT platform. We assigned 3 workers to each tuple containing summaries from different systems. We showed summaries and asked to select the best and worst one based on the criterion presented below.
1. Fluency: the summary sentences should be grammatically correct, easy to read and understand;
2. Coherence: the summary should be well structured and well organized;
3. Non-redundancy: there should be no unnecessary repetition in the summary.
Table 12: Example summaries produced by different systems for an Amazon product. We highlight in **bold** keywords that were part of the query for ADAQSUM (5%) + L10.
Table 13: Example summaries produced by different systems for an Amazon product. We highlight in bold keywords that were part of the query for ADAQSUM (5%) + L10.