Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, Mustafa Suleyman, and Andrew Zisserman. The Kinetics Human Action Video Dataset, May 2017. URL http://arxiv.org/abs/1705.06950. arXiv:1705.06950 [cs].

Yerbolat Khassanov, Saida Mussakhojayeva, Almas Mirzakhmetov, Alen Adiyev, Mukhamet Nurpeissov, and Huseyin Atakan Varol. A Crowdsourced Open-Source Kazakh Speech Corpus and Initial Speech Recognition Baseline, January 2021. URL http://arxiv.org/abs/2009.10334. arXiv:2009.10334 [cs, eess].

Tushar Khot, Ashish Sabharwal, and Peter Clark. SciTaiL: A Textual Entailment Dataset from Science Question Answering. *Proceedings of the AAAI Conference on Artificial Intelligence*, 32 (1), April 2018. ISSN 2374-3468, 2159-5399. doi: 10/grm22d. URL https://ojs.aaai.org/index.php/AAAI/article/view/12022.

Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. QASC: A Dataset for Question Answering via Sentence Composition, February 2020. URL http://arxiv.org/abs/1910.11473. arXiv:1910.11473 [cs].

Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and Maarten Sap. ProsocialDialog: A Prosocial Backbone for Conversational Agents, October 2022. URL http://arxiv.org/abs/2205.12688. arXiv:2205.12688 [cs].

Seonhoon Kim, Inho Kang, and Nojun Kwak. Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information, November 2018. URL http://arxiv.org/abs/1805.11360. arXiv:1805.11360 [cs].

Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. Prometheus: Inducing Fine-grained Evaluation Capability in Language Models, March 2024. URL http://arxiv.org/abs/2310.08491. arXiv:2310.08491 [cs].

Andreas Kirkedal, Marija Stepanović, and Barbara Plank. FT Speech: Danish Parliament Speech Corpus, October 2020. URL http://arxiv.org/abs/2005.12368. arXiv:2005.12368 [cs, eess].

Oddur Kjartansson, Supheakmungkol Sarin, Knot Pipatsrisawat, Martin Jansche, and Linne Ha. Crowd-Sourced Speech Corpora for Javanese, Sundanese, Sinhala, Nepali, and Bangladeshi Bengali. In *6th Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU 2018)*, pp. 52–55. ISCA, August 2018. doi: 10/gtwwbs. URL https://www.isca-archive.org/sltu_2018/kjartansson18_sltu.html.

Oddur Kjartansson, Alexander Gutkin, Alena Butryna, Isin Demirsahin, and Clara Rivera. Open-Source High Quality Speech Datasets for Basque, Catalan and Galician. In Dorothee Beermann, Laurent Besacier, Sakriani Sakti, and Claudia Soria (eds.), *Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL)*, pp. 21–27, Marseille, France, May 2020. European Language Resources association. ISBN 979-10-95546-35-1. URL https://aclanthology.org/2020.sltu-1.3.

Kevin Klyman. Acceptable use policies for foundation models, 2024. URL https://arxiv.org/abs/2409.09041.

Rostislav Kolobov, Olga Okhapkina, Olga Omelchishina, Andrey Platunov, Roman Bedyakin, Vyacheslav Moshkin, Dmitry Menshikov, and Nikolay Mikhaylovskiy. MediaSpeech: Multilanguage ASR Benchmark and Dataset, March 2021. URL http://arxiv.org/abs/2103.16193. arXiv:2103.16193 [cs, eess].

Mojtaba Komeili, Kurt Shuster, and Jason Weston. Internet-Augmented Dialogue Generation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), *Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 8460–8478, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10/gr75db. URL https://aclanthology.org/2022.acl-long.579.
Quan Kong, Ziming Wu, Ziwei Deng, Martin Klinkigt, Bin Tong, and Tomokazu Murakami. MMAct: A Large-Scale Dataset for Cross Modal Human Action Understanding. In *2019 IEEE/CVF International Conference on Computer Vision (ICCV)*, pp. 8657-8666, October 2019. doi: 10/ghfhxx. URL https://ieeexplore.ieee.org/document/9009579. ISSN: 2380-7504.

Anastassia Kornilova and Vlad Eidelman. BillSum: A Corpus for Automatic Summarization of US Legislation. In *Proceedings of the 2nd Workshop on New Frontiers in Summarization*, pp. 48–56, 2019. doi: 10/gtwwtd. URL http://arxiv.org/abs/1910.00523. arXiv:1910.00523 [cs].

Matěj Korvas, Ondřej Plátek, Ondřej Dušek, Lukáš Žilka, and Filip Jurčíček. Free English and Czech telephone speech corpus shared under the CC-BY-SA 3.0 license. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), *Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14)*, pp. 4423–4428, Reykjavik, Iceland, May 2014. European Language Resources Association (ELRA). URL http://www.lrec-conf.org/proceedings/lrec2014/pdf/535_Paper.pdf.

Danijel Korzinek, Krzysztof Marasek, Lukasz Brocki, and Krzysztof Wolk. Polish Read Speech Corpus for Speech Tools and Services, June 2017. URL http://arxiv.org/abs/1706.00245. arXiv:1706.00245 [cs].

Suhas Kotha, Jacob Mitchell Springer, and Aditi Raghunathan. Understanding catastrophic forgetting in language models via implicit inference. arXiv preprint arXiv:2309.10105, 2023.

Jonáš Kratochvil, Peter Polák, and Ondřej Bojar. Large Corpus of Czech Parliament Plenary Hearings. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), *Proceedings of the Twelfth Language Resources and Evaluation Conference*, pp. 6363–6367, Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-95546-34-4. URL https://aclanthology.org/2020.lrec-1.781.

Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, et al. Quality at a glance: An audit of web-crawled multilingual datasets. *Transactions of the Association for Computational Linguistics*, 10:50–72, 2022.

H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre. HMDB: A large video database for human motion recognition. In *2011 International Conference on Computer Vision*, pp. 2556–2563, Barcelona, Spain, November 2011. IEEE. ISBN 978-1-4577-1102-2. doi: 10/fxpf8k. URL http://ieeexplore.ieee.org/document/6126543/.

Hilde Kuehne, Ali Arslan, and Thomas Serre. The Language of Actions: Recovering the Syntax and Semantics of Goal-Directed Human Activities. In *2014 IEEE Conference on Computer Vision and Pattern Recognition*, pp. 780–787, Columbus, OH, USA, June 2014. IEEE. ISBN 978-1-4799-5118-5. doi: 10/gqdc3v. URL https://ieeexplore.ieee.org/document/6909500/.

Anoop Kunchukuttan, Divyanshu Kakwani, Satish Golla, Avik Bhattacharyya, Mitesh M Khapra, Pratyush Kumar, et al. Ai4bharat-indicnlp corpus: Monolingual corpora and word embeddings for indic languages. arXiv preprint arXiv:2005.00085, 2020.

Alexey Kurakin, Natalia Ponomareva, Umar Syed, Liam MacDermed, and Andreas Terzis. Harnessing large-language models to generate private synthetic text. June 2023.

Fabrício Kury, Alex Butler, Chi Yuan, Li-heng Fu, Yingcheng Sun, Hao Liu, Ida Sim, Simona Carini, and Chunhua Weng. Chia, a large annotated corpus of clinical trial eligibility criteria. *Scientific Data*, 7(1):281, August 2020. ISSN 2052-4463. doi: 10/gr4ftn. URL https://www.nature.com/articles/s41597-020-00620-0. Publisher: Nature Publishing Group.
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. OpenAssistant Conversations – Democratizing Large Language Model Alignment, October 2023. URL http://arxiv.org/abs/2304.07327. arXiv:2304.07327 [cs].

Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. RACE: Large-scale ReAding Comprehension Dataset From Examinations, December 2017. URL http://arxiv.org/abs/1704.04683. arXiv:1704.04683 [cs].

Brenden M. Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks, June 2018. URL http://arxiv.org/abs/1711.00350. arXiv:1711.00350 [cs].

Lander, T. CSLU: 22 Languages Corpus, November 2005. URL https://catalog.ldc.upenn.edu/LDC2005S26.

Lander, T. CSLU: Foreign Accented English Release 1.2, May 2007. URL https://catalog.ldc.upenn.edu/LDC2007S08. Artwork Size: 1468006 KB Pages: 1468006 KB.

Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), *Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)*, pp. 1311–1316, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10/gqrfvv. URL https://aclanthology.org/D19-1131.

Hugo Laurençon, Lucile Saulnier, Thomas Wang, Christopher Akiki, Albert Villanova del Moral, Teven Le Scao, Leandro Von Werra, Chenghao Mou, Eduardo González Ponferrada, Huu Nguyen, Jörg Frohberg, Mario Šaško, Quentin Lhoest, Angelina McMillan-Major, Gerard Dupont, Stella Biderman, Anna Rogers, Loubna Ben allal, Francesco De Toni, Giada Pistilli, Olivier Nguyen, Somaich Nikpoor, Maraim Masoud, Pierre Colombo, Javier de la Rosa, Paulo Villegas, Tristan Thrush, Shayne Longpre, Sebastian Nagel, Leon Weber, Manuel Muñoz, Jian Zhu, Daniel Van Strien, Zaid Alyafeai, Khalid Almubarak, Minh Chien Vu, Itziar Gonzalez-Dios, Aitor Soroa, Kyle Lo, Manan Dey, Pedro Ortiz Suarez, Aaron Gokaslan, Shamik Bose, David Adelani, Long Phan, Hieu Tran, Ian Yu, Suhas Pai, Jenny Chim, Violette Lepercq, Suzana Ilic, Margaret Mitchell, Sasha Alexandra Luccioni, and Yacine Jernite. The bigscience roots corpus: A 1.6tb composite multilingual dataset. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), *Advances in Neural Information Processing Systems*, volume 35, pp. 31809–31826. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ce9e92e3de2372a4b93353eb7f3dc0bd-Paper-Datasets_and_Benchmarks.pdf.

Remi Lebret, David Grangier, and Michael Auli. Neural Text Generation from Structured Data with Application to the Biography Domain, September 2016. URL http://arxiv.org/abs/1603.07771. arXiv:1603.07771 [cs].

Ariel N. Lee, Cole J. Hunter, and Nataniel Ruiz. Platypus: Quick, cheap, and powerful refinement of llms. *NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following*, 2023a.

Katherine Lee, A Feder Cooper, and James Grimmelmann. Talkin”bout ai generation: Copyright and the generative-ai supply chain. arXiv preprint arXiv:2309.08133, 2023b.

Colin Leong, Joshua Nemecek, Jacob Mansdorfer, Anna Filighera, Abraham Owodunni, and Daniel Whitenack. Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks, October 2022. URL http://arxiv.org/abs/2210.14712. arXiv:2210.14712 [cs].
Mike Lewis, Denis Yarats, Yann N. Dauphin, Devi Parikh, and Dhruv Batra. Deal or No Deal? End-to-End Learning for Negotiation Dialogues, June 2017. URL http://arxiv.org/abs/1706.05125. arXiv:1706.05125 [cs].

Guanyu Li, Hongzhi Yu, Thomas Fang Zheng, Jinghao Yan, and Shipeng Xu. Free linguistic and speech resources for Tibetan. In *2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)*, pp. 733–736, Kuala Lumpur, December 2017. IEEE. ISBN 978-1-5386-1542-3. doi: 10/gtsqzh. URL http://ieeexplore.ieee.org/document/8282130/.

Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation, October 2023a. URL http://arxiv.org/abs/2305.15011. arXiv:2305.15011 [cs].

Haoran Li, Abhinav Arora, Shuohui Chen, Anchit Gupta, Sonal Gupta, and Yashar Mehdad. MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmark. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (eds.), *Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume*, pp. 2950–2962, Online, April 2021a. Association for Computational Linguistics. doi: 10/gtsqxq. URL https://aclanthology.org/2021.eacl-main.257.

Hongyu Li, Seohyun Kim, and Satish Chandra. Neural Code Search Evaluation Dataset, October 2019a. URL http://arxiv.org/abs/1908.09804. arXiv:1908.09804 [cs].

Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. MVBench: A Comprehensive Multi-modal Video Understanding Benchmark, May 2024. URL http://arxiv.org/abs/2311.17005. arXiv:2311.17005 [cs].

Sheng Li, Zhiqiang Tao, Kang Li, and Yun Fu. Visual to text: Survey of image and video captioning. *IEEE Transactions on Emerging Topics in Computational Intelligence*, 3(4):297–312, 2019b. doi: 10.1109/TETCI.2019.2892755.

Tianjiao Li, Jun Liu, Wei Zhang, Yun Ni, Wenqian Wang, and Zhiheng Li. UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles, August 2021b. URL http://arxiv.org/abs/2104.00946. arXiv:2104.00946 [cs].

Xinjian Li, Shinnosuke Takamichi, Takaaki Saeki, William Chen, Sayaka Shiota, and Shinji Watanabe. Yodas: Youtube-oriented dataset for audio and speech. In *2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)*, pp. 1–8. IEEE, 2023b.

Xinjian Li, Shinnosuke Takamichi, Takaaki Saeki, William Chen, Sayaka Shiota, and Shinji Watanabe. Yodas: Youtube-Oriented Dataset for Audio and Speech. In *2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)*, pp. 1–8, Taipei, Taiwan, December 2023c. IEEE. ISBN 978-8-3503-0689-7. doi: 10/gtsqzc. URL https://ieeexplore.ieee.org/document/10389689/.

Yu Li, Kun Qian, Weiyan Shi, and Zhou Yu. End-to-End Trainable Non-Collaborative Dialog System, November 2019c. URL http://arxiv.org/abs/1911.10742. arXiv:1911.10742 [cs].

Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d'Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. Competition-Level Code Generation with AlphaCode. *Science*, 378(6624):1092–1097, December 2022. ISSN 0036-8075, 1095-9203. doi: 10/grggxf. URL http://arxiv.org/abs/2203.07814. arXiv:2203.07814 [cs].

Yuncheng Li, Yale Song, Liangliang Cao, Joel Tetreault, Larry Goldberg, Alejandro Jaimes, and Jiebo Luo. TGIF: A New Dataset and Benchmark on Animated GIF Description, April 2016. URL http://arxiv.org/abs/1604.02748. arXiv:1604.02748 [cs].
Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge, June 2023d. URL http://arxiv.org/abs/2303.14070. arXiv:2303.14070 [cs].

Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's Verify Step by Step, May 2023. URL http://arxiv.org/abs/2305.20050. arXiv:2305.20050 [cs].

Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren. Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models, September 2020a. URL http://arxiv.org/abs/2005.00683. arXiv:2005.00683 [cs].

Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning, November 2020b. URL http://arxiv.org/abs/1911.03705. arXiv:1911.03705 [cs].

Bill Yuchen Lin, Ziyi Wu, Yichi Yang, Dong-Ho Lee, and Xiang Ren. RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge, July 2021a. URL http://arxiv.org/abs/2101.00376. arXiv:2101.00376 [cs].

Kevin Lin, Oyvind Tafjord, Peter Clark, and Matt Gardner. Reasoning Over Paragraph Effects in Situations, December 2019. URL http://arxiv.org/abs/1908.05852. arXiv:1908.05852 [cs].

Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring How Models Mimic Human Falsehoods, May 2022. URL http://arxiv.org/abs/2109.07958. arXiv:2109.07958 [cs].

Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, Peng Xu, Feijun Jiang, Yuxiang Hu, Chen Shi, and Pascale Fung. BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling, June 2021b. URL http://arxiv.org/abs/2106.02787. arXiv:2106.02787 [cs].

Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems, October 2017. URL http://arxiv.org/abs/1705.04146. arXiv:1705.04146 [cs].

Chunhui Liu, Yueyu Hu, Yanghao Li, Sijie Song, and Jiaying Liu. PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understanding, March 2017. URL http://arxiv.org/abs/1703.07475. arXiv:1703.07475 [cs].

Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023a.

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023b.

Jingjing Liu, Panupong Pasupat, Scott Cyphers, and Jim Glass. Asgard: A portable architecture for multilingual dialogue systems. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8386-8390, Vancouver, BC, Canada, May 2013. IEEE. ISBN 978-1-4799-0356-6. doi: 10/gtsqxn. URL http://ieeexplore.ieee.org/document/6639301/.

Jingzhou Liu, Wenhu Chen, Yu Cheng, Zhe Gan, Licheng Yu, Yiming Yang, and Jingjing Liu. VIOLIN: A Large-Scale Dataset for Video-and-Language Inference, March 2020. URL http://arxiv.org/abs/2003.11618. arXiv:2003.11618 [cs].

Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, and Andrew M Dai. Best practices and lessons learned on synthetic data. April 2024a.
Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning, April 2024b. URL http://arxiv.org/abs/2312.15685. arXiv:2312.15685 [cs].

Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. AgentBench: Evaluating LLMs as Agents, October 2023c. URL http://arxiv.org/abs/2308.03688. arXiv:2308.03688 [cs].

Xingkun Liu, Arash Eshghi, Pawel Swietojanski, and Verena Rieser. Benchmarking Natural Language Understanding Services for building Conversational Agents, March 2019. URL http://arxiv.org/abs/1903.05566. arXiv:1903.05566 [cs].

Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. Datasets for large language models: A comprehensive survey. arXiv preprint arXiv:2402.18041, 2024c.

Zeming Liu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, and Wanxiang Che. DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pp. 4335–4347, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10/gtsqxr. URL https://aclanthology.org/2021.emnlp-main.356.

Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, and Daphne Ippolito. A pretrainer's guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity, 2023.

Shayne Longpre, Stella Biderman, Alon Albalak, Hailey Schoelkopf, Daniel McDuff, Sayash Kapoor, Kevin Klyman, Kyle Lo, Gabriel Ilharco, Nay San, et al. The responsible foundation model development cheatsheet: A review of tools & resources. arXiv preprint arXiv:2406.16746, 2024a.

Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisetla, Xinyi (Alexis) Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, and Sara Hooker. A large-scale audit of dataset licensing and attribution in AI. *Nature Machine Intelligence*, 6(8): 975–987, August 2024b. doi: 10/gt8f5p.

Shayne Longpre, Robert Mahari, Ariel Lee, Campbell Lund, Hamidah Oderinwale, William Brannon, Nayan Saxena, Naana Obeng-Marnu, Tobin South, Cole Hunter, et al. Consent in crisis: The rapid decline of the ai data commons. arXiv preprint arXiv:2407.14933, 2024c.

Shayne Longpre, Robert Mahari, Naana Obeng-Marnu, William Brannon, Tobin South, Katy Gero, Sandy Pentland, and Jad Kabbara. Data authenticity, consent, & provenance for ai are all broken: what will it take to fix them? arXiv preprint arXiv:2404.12691, 2024d.

Annie Louis, Dan Roth, and Filip Radlinski. "I'd rather just go to bed": Understanding Indirect Answers, October 2020. URL http://arxiv.org/abs/2010.03450. arXiv:2010.03450 [cs].

Holy Lovenia, Rahmad Mahendra, Salsabil Maulana Akbar, Lester James V Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jonibek Mansurov, Joseph Marvin Imperial, Onno P Kampman, et al. Seacrowd: A multilingual multimodal data hub and benchmark suite for southeast asian languages. arXiv preprint arXiv:2406.10118, 2024.

Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems, February 2016. URL http://arxiv.org/abs/1506.08909. arXiv:1506.08909 [cs].

Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering, October 2022. URL http://arxiv.org/abs/2209.09513. arXiv:2209.09513 [cs].
Alexandra Sasha Luccioni and Joseph D Viviano. What's in the box? a preliminary analysis of undesirable content in the common crawl corpus. May 2021.

Zhaoyang Lv, Nicholas Charron, Pierre Moulon, Alexander Gamino, Cheng Peng, Chris Sweeney, Edward Miller, Huixuan Tang, Jeff Meissner, Jing Dong, Kiran Somasundaram, Luis Pesqueira, Mark Schwesinger, Omkar Parkhi, Qiao Gu, Renzo De Nardi, Shangyi Cheng, Steve Saarinen, Vijay Baiyya, Yuyang Zou, Richard Newcombe, Jakob Julian Engel, Xiaqing Pan, and Carl Ren. Aria Everyday Activities Dataset, February 2024. URL http://arxiv.org/abs/2402.13349. arXiv:2402.13349 [cs].

Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury, Tanmay Sinha, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan. MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems, October 2023. URL http://arxiv.org/abs/2305.14536. arXiv:2305.14536 [cs].

Kikuo Maekawa. Corpus of spontaneous Japanese: its design and evaluation. In *Proceedings of the ISCA/IEEE Workshop on Spontaneous Speech Processing and Recognition*, pp. paper MMO2, 2003. URL https://www.isca-archive.org/sspr_2003/maekawa03_sspr.html.

Rohan Mahadev and Anindya Chakravarti. Understanding gender and racial disparities in image recognition models. *arXiv preprint arXiv:2107.09211*, 2021.

Robert Mahari and Shayne Longpre. Discit ergo est: Training data provenance and fair use. *Robert Mahari and Shayne Longpre, Discit ergo est: Training Data Provenance And Fair Use, Dynamics of Generative AI* (ed. Thibault Schrepel & Volker Stocker), *Network Law Review*, Winter, 2023.

Robert Mahari, Longpre Shayne, Lisette Donewald, Alan Polozov, Alex 'Sandy' Pentland, and Ari Lipsitz. Comment to US copyright office on data provenance and copyright, 2023.

Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, and Dan Roth. ExpertQA: Expert-Curated Questions and Attributed Answers, April 2024. URL http://arxiv.org/abs/2309.07852. arXiv:2309.07852 [cs].

Mishaim Malik, Muhammad Kamran Malik, Khawar Mehmood, and Imran Makhdoom. Automatic speech recognition: a survey. *Multimedia Tools and Applications*, 80:9411–9457, 2021.

Srikanth Malla, Behzad Dariush, and Chiho Choi. TITAN: Future Forecast Using Action Priors. In *2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 11183–11193, Seattle, WA, USA, June 2020. IEEE. ISBN 978-1-72817-168-5. doi: 10/gg99rg. URL https://ieeexplore.ieee.org/document/9156550/.

Pekka Malo, Ankur Sinha, Pyry Takala, Pekka Korhonen, and Jyrki Wallenius. Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts, July 2013. URL http://arxiv.org/abs/1307.5336. arXiv:1307.5336 [cs, q-fin].

Ikram Mamtimin, Wenqiang Du, and Askar Hamdulla. M2ASR-KIRGHIZ: A Free Kirghiz Speech Database and Accompanied Baselines. *Information*, 14(1):55, January 2023. ISSN 2078-2489. doi: 10/gtsqzm. URL https://www.mdpi.com/2078-2489/14/1/55.

Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. When less is more: Investigating data pruning for pretraining llms at scale, 2023. URL https://arxiv.org/abs/2309.04564.

Marcin Marszalek, Ivan Laptev, and Cordelia Schmid. Actions in context. In *2009 IEEE Conference on Computer Vision and Pattern Recognition*, pp. 2929–2936, Miami, FL, June 2009. IEEE. ISBN 978-1-4244-3992-8. doi: 10/d5bs7p. URL https://ieeexplore.ieee.org/document/5206557/.

Lara J. Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark O. Riedl. Event Representations for Automated Story Generation with Deep Neural Nets. *Proceedings of the AAAI Conference on Artificial Intelligence*, 32(1), April 2018. ISSN 2374-3468, 2159-5399. doi: 10/g6k72p. URL http://arxiv.org/abs/1706.01331. arXiv:1706.01331 [cs].
Scott Martin, Shivani Poddar, and Kartikeya Upasani. MuDoCo: Corpus for Multidomain Coreference Resolution and Referring Expression Generation. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), *Proceedings of the Twelfth Language Resources and Evaluation Conference*, pp. 104–111, Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-95546-34-4. URL https://aclanthology.org/2020.lrec-1.13.

Joanna Materzynska, Guillaume Berger, Ingo Bax, and Roland Memisevic. The Jester Dataset: A Large-Scale Video Dataset of Human Gestures. In *2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)*, pp. 2874–2882, Seoul, Korea (South), October 2019. IEEE. ISBN 978-1-72815-023-9. doi: 10/gh5k47. URL https://ieeexplore.ieee.org/document/9022297/.

Cecily Mauran. What was Sora trained on? Creatives demand answers. https://mashable.com/article/openai-sora-ai-video-generator-training-data, 2024. [Accessed 28-09-2024].

Angelina McMillan-Major, Zaid Alyafeai, Stella Biderman, Kimbo Chen, Francesco De Toni, Gérard Dupont, Hady Elsahar, Chris Emezue, Alham Fikri Aji, Suzana Ilić, et al. Documenting geographically and contextually diverse data sources: The bigscience catalogue of language data and resources. *arXiv preprint arXiv:2201.10066*, 2022a.

Angelina McMillan-Major, Zaid Alyafeai, Stella Biderman, Kimbo Chen, Francesco De Toni, Gérard Dupont, Hady Elsahar, Chris Emezue, Alham Fikri Aji, Suzana Ilić, Nurulaqilla Khamis, Colin Leong, Maraim Masoud, Aitor Soroa, Pedro Ortiz Suarez, Zeerak Talat, Daniel van Strien, and Yacine Jernite. Documenting geographically and contextually diverse data sources: The bigscience catalogue of language data and resources, 2022b. URL https://arxiv.org/abs/2201.10066.

James Meese and Jennifer Hagedorn. Mundane content on social media: Creation, circulation, and the copyright problem. *Social Media+ Society*, 5(2):2056305119839190, 2019.

Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer Sentinel Mixture Models, September 2016. URL http://arxiv.org/abs/1609.07843. arXiv:1609.07843 [cs].

Antoine Miech, Dimitri Zhukov, Jean-Baptiste Alayrac, Makarand Tapaswi, Ivan Laptev, and Josef Sivic. HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips, July 2019. URL http://arxiv.org/abs/1906.03327. arXiv:1906.03327 [cs].

Antoine Miech, Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic, and Andrew Zisserman. RareAct: A video dataset of unusual interactions, August 2020. URL http://arxiv.org/abs/2008.01018. arXiv:2008.01018 [cs].

Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering, September 2018. URL http://arxiv.org/abs/1809.02789. arXiv:1809.02789 [cs].

Sewon Min, Suchin Gururangan, Eric Wallace, Weijia Shi, Hannaneh Hajishirzi, Noah A Smith, and Luke Zettlemoyer. Silo language models: Isolating legal risk in a nonparametric datastore. In *NeurIPS 2023 Workshop on Distribution Shifts: New Frontiers with Foundation Models*, 2023.

Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, and Ashwin Kalyan. Lila: A Unified Benchmark for Mathematical Reasoning, March 2023. URL http://arxiv.org/abs/2210.17517. arXiv:2210.17517 [cs].

Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In *Proceedings of the conference on fairness, accountability, and transparency*, pp. 220–229, 2019.
Arindam Mitra, Hamed Khanpour, Corby Rosset, and Ahmed Awadallah. Orca-Math: Unlocking the potential of SLMs in Grade School Math, February 2024. URL http://arxiv.org/abs/2402.14830. arXiv:2402.14830 [cs].

Daniela Moctezuma, Tania Ramírez-delReal, Guillermo Ruiz, and Othón González-Chávez. Video captioning: a comparative review of where we are and which could be the route, 2022. URL https://arxiv.org/abs/2204.05976.

David Erik Mollberg, Olafur Helgi Jonsson, Sunneva Thorsteinsdottir, Steinthor Steingremsson, Eydis Huld Magnusdottir, and Jon Gudnason. Samrómur: Crowd-sourcing Data Collection for Icelandic Speech Recognition. In Nicoletta Calzolari, Frederic Bechet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), *Proceedings of the Twelfth Language Resources and Evaluation Conference*, pp. 3463–3467, Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-95546-34-4. URL https://aclanthology.org/2020.lrec-1.425.

Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ramakrishnan, Sarah Adel Bargal, Tom Yan, Lisa Brown, Quanfu Fan, Dan Gutfreund, Carl Vondrick, et al. Moments in time dataset: one million videos for event understanding. *IEEE transactions on pattern analysis and machine intelligence*, 42(2):502–508, 2019a.

Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ramakrishnan, Sarah Adel Bargal, Tom Yan, Lisa Brown, Quanfu Fan, Dan Gutfreund, Carl Vondrick, and Aude Oliva. Moments in Time Dataset: one million videos for event understanding, February 2019b. URL http://arxiv.org/abs/1801.03150. arXiv:1801.03150 [cs].

Mathew Monfort, SouYoung Jin, Alexander Liu, David Harwath, Rogerio Feris, James Glass, and Aude Oliva. Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions, May 2021a. URL http://arxiv.org/abs/2105.04489. arXiv:2105.04489 [cs, eess].

Mathew Monfort, Bowen Pan, Kandan Ramakrishnan, Alex Andonian, Barry A. McNamara, Alex Lascelles, Quanfu Fan, Dan Gutfreund, Rogerio Feris, and Aude Oliva. Multi-Moments in Time: Learning and Interpreting Models for Multi-Action Video Understanding, September 2021b. URL http://arxiv.org/abs/1911.00232. arXiv:1911.00232 [cs, eess].

Nuno Moniz and Luís Torgo. Multi-Source Social Feedback of Online News Feeds, January 2018. URL http://arxiv.org/abs/1801.07055. arXiv:1801.07055 [cs].

Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. OpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs. In Anna Korhonen, David Traum, and Lluís Màrquez (eds.), *Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pp. 845–854, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10/ggt4wm. URL https://aclanthology.org/P19-1081.

Frank Morton-Park. Licensed to learn: Mitigating copyright infringement liability of generative ai systems through contracts. *Notre Dame Journal on Emerging Technology*, 5:64, 2023.

Johannes E. M. Mosig, Shikib Mehri, and Thomas Kober. STAR: A Schema-Guided Dialog Dataset for Transfer Learning, October 2020. URL http://arxiv.org/abs/2010.11853. arXiv:2010.11853 [cs].

Nasrin Mostafazadeh, Aditya Kalyanpur, Lori Moon, David Buchanan, Lauren Berkowitz, Or Biran, and Jennifer Chu-Carroll. GLUCOSE: GeneraLized and COntextualized Story Explanations, October 2020. URL http://arxiv.org/abs/2009.07758. arXiv:2009.07758 [cs].

Rajiv Movva, Sidhika Balachandar, Kenny Peng, Gabriel Agostini, Nikhil Garg, and Emma Pierson. Topics, authors, and institutions in large language model research: Trends from 17k arxiv papers. In *Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)*, pp. 1223–1243, 2024.
Hussein Mozannar, Karl El Hajal, Elie Maamary, and Hazem Hajj. Neural Arabic Question Answering, June 2019. URL http://arxiv.org/abs/1906.05394. arXiv:1906.05394 [cs].

Nikola Mrksic, Diarmuid O Seaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. Neural Belief Tracker: Data-Driven Dialogue State Tracking, April 2017. URL http://arxiv.org/abs/1606.03777. arXiv:1606.03777 [cs].

Nikola Mrkšić and Ivan Vulić. Fully Statistical Neural Belief Tracking, May 2018. URL http://arxiv.org/abs/1805.11350. arXiv:1805.11350 [cs].

Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, and Ahmed Ali. QASR: QCRI Aljazeera Speech Resource – A Large Scale Annotated Arabic Speech Corpus, June 2021. URL http://arxiv.org/abs/2106.13000. arXiv:2106.13000 [cs, eess].

Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15991–16111, 2023.

Rajdeep Mukherjee, Abhinav Bohra, Akash Banerjee, Soumya Sharma, Manjunath Hegde, Afreen Shaikh, Shivani Shrivastava, Koustuv Dasgupta, Niloy Ganguly, Saptarshi Ghosh, and Pawan Goyal. ECTSum: A New Benchmark Dataset For Bullet Point Summarization of Long Earnings Call Transcripts. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 10893–10906, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10/gtsqxz. URL https://aclanthology.org/2022.emnlp-main.748.

Will Myers, Tyler Etchart, and Nancy Fulda. Conversational Scaffolding: An Analogy-based Approach to Response Prioritization in Open-domain Dialogs: In Proceedings of the 12th International Conference on Agents and Artificial Intelligence, pp. 69–78, Valletta, Malta, 2020. SCITEPRESS - Science and Technology Publications. ISBN 978-989-758-395-7. doi: 10/gtsq86. URL http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0008939900690078.

Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. VoxCeleb: a large-scale speaker identification dataset, May 2018. URL http://arxiv.org/abs/1706.08612. arXiv:1706.08612 [cs].

Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation, August 2024. URL http://arxiv.org/abs/2407.02371. arXiv:2407.02371 [cs].

Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Mutethia Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victoria Lin, Caiming Xiong, Richard Socher, and Nazneen Fatema Rajani. DART: Open-Domain Structured Data Record to Text Generation. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 432–447, Online, June 2021. Association for Computational Linguistics. doi: 10/gnh49f. URL https://aclanthology.org/2021.naACL-main.37. arXiv:2007.02871 [cs].

Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kryściński, Hailey Schoelkopf, Riley Kong, Xiangru Tang, Mutethia Mutuma, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, Dragomir Radev, and Dragomir Radev. FeTaQA: Free-form Table Question Answering. Transactions of the Association for Computational Linguistics, 10:35–49, 2022. doi: 10/gtsqx3. URL https://aclanthology.org/2022.tacl-1.3. Place: Cambridge, MA Publisher: MIT Press.

Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization, August 2018. URL http://arxiv.org/abs/1808.08745. arXiv:1808.08745 [cs].
Linh The Nguyen, Nguyen Luong Tran, Long Doan, Manh Luong, and Dat Quoc Nguyen. A High-Quality and Large-Scale Dataset for English-Vietnamese Speech Translation, August 2022. URL http://arxiv.org/abs/2208.04243. arXiv:2208.04243 [cs].

Tuan-Phong Nguyen, Simon Razniewski, and Gerhard Weikum. Advanced Semantics for Common-sense Knowledge Extraction. In *Proceedings of the Web Conference 2021*, pp. 2636–2647, April 2021. doi: 10/gnnffn. URL http://arxiv.org/abs/2011.00905. arXiv:2011.00905 [cs].

Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, and Lidong Bing. SeaLLMs – Large Language Models for Southeast Asia, December 2023. URL http://arxiv.org/abs/2312.00738. arXiv:2312.00738 [cs].

Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial NLI: A New Benchmark for Natural Language Understanding, May 2020. URL http://arxiv.org/abs/1910.14599. arXiv:1910.14599 [cs].

Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. The E2E Dataset: New Challenges For End-to-End Generation, July 2017. URL http://arxiv.org/abs/1706.09254. arXiv:1706.09254 [cs].

Tobi Olatunji, Tejumade Afonja, Aditya Yadavalli, Chris Chinenye Emezue, Sahib Singh, Bonaventure F. P. Dossou, Joanne Osuchukwu, Salomey Osei, Atnafu Lambebo Tonja, Naome Etori, and Clinton Mbataku. AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR, September 2023. URL http://arxiv.org/abs/2310.00274. arXiv:2310.00274 [cs].

Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. Social data: Biases, method-ological pitfalls, and ethical boundaries. *Frontiers in big data*, 2:13, 2019.

Andreea-Maria Oncescu, João F. Henriques, Yang Liu, Andrew Zisserman, and Samuel Albanie. QuerYD: A video dataset with high-quality text and audio narrations, February 2021. URL http://arxiv.org/abs/2011.11071. arXiv:2011.11071 [cs].

Patrick K. O’Neill, Vitaly Lavrukhin, Somshubra Majumdar, Vahid Noroozi, Yuekai Zhang, Oleksii Kuchaiev, Jagadeesh Balam, Yuliya Dovzhenko, Keenan Freyberg, Michael D. Shulman, Boris Ginsburg, Shinji Watanabe, and Georg Kucsko. SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition, April 2021. URL http://arxiv.org/abs/2104.02014. arXiv:2104.02014 [cs, eess].

OpenAI. Hello gpt-4o: We're announcing gpt-4o, our new flagship model that can reason across audio, vision, and text in real time., 2024. URL https://openai.com/index/hello-gpt-4o/.

Mariam Orabi, Hozayfa El Rifai, and Ashraf Elnagar. Classical Arabic Poetry: Classification based on Era. In *2020 IEEE/ACS 17th International Conference on Computer Systems and Applications (AICCSA)*, pp. 1–6, Antalya, Turkey, November 2020. IEEE. ISBN 978-1-72818-577-4. doi: 10/g6k6b2. URL https://ieeexplore.ieee.org/document/9316520/.

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. *arXiv preprint arXiv:2203.02155*, 2022. URL https://arxiv.org/abs/2203.02155.

Cory Paik, Stéphane Aroca-Ouellette, Alessandro Roncone, and Katharina Kann. The World of an Octopus: How Reporting Bias Influences a Language Model’s Perception of Color, October 2021. URL http://arxiv.org/abs/2110.08182. arXiv:2110.08182 [cs].

Serguei Pakhomov, Bridget McInnes, Terrence Adam, Ying Liu, Ted Pedersen, and Genevieve B. Melton. Semantic Similarity and Relatedness between Clinical Terms: An Experimental Study. *AMIA Annual Symposium Proceedings*, 2010:572–576, 2010. ISSN 1942-597X. URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3041430/.
Xiaqing Pan, Nicholas Charron, Yongqian Yang, Scott Peters, Thomas Whelan, Chen Kong, Omkar Parkhi, Richard Newcombe, and Carl Yuheng Ren. Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perception, June 2023. URL http://arxiv.org/abs/2306.06362. arXiv:2306.06362 [cs].

Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: An ASR corpus based on public domain audio books. In *2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pp. 5206–5210, South Brisbane, Queensland, Australia, April 2015. IEEE. ISBN 978-1-4673-6997-8. doi: 10/gfv84w. URL http://ieeexplore.ieee.org/document/7178964/.

Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales, June 2005. URL http://arxiv.org/abs/cs/0506075. arXiv:cs/0506075.

Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. ToTTo: A Controlled Table-To-Text Generation Dataset. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)*, pp. 1173–1186, Online, November 2020. Association for Computational Linguistics. doi: 10/gm3nmg. URL https://aclanthology.org/2020.emnlp-main.89.

Jeongkyun Park, Jung-Wook Hwang, Kwanghee Choi, Seung-Hyun Lee, Jun Hwan Ahn, Rae-Hong Park, and Hyung-Min Park. OLKAVS: An Open Large-Scale Korean Audio-Visual Speech Dataset, January 2023. URL http://arxiv.org/abs/2301.06375. arXiv:2301.06375 [cs].

Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. Data, data everywhere: A guide for pretraining dataset construction. *arXiv preprint* 2407.06380, 2024.

Panupong Pasupat and Percy Liang. Compositional Semantic Parsing on Semi-Structured Tables. In Chengqing Zong and Michael Strube (eds.), *Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)*, pp. 1470–1480, Beijing, China, July 2015. Association for Computational Linguistics. doi: 10/gfz98s. URL https://aclanthology.org/P15-1142.

Amandalynne Paullada, Inioluwa Deborah Raji, Emily M Bender, Emily Denton, and Alex Hanna. Data and its (dis) contents: A survey of dataset development and use in machine learning research. *Patterns*, 2(11), 2021.

Nikita Pavlichenko, Ivan Stelmakh, and Dmitry Ustalov. CrowdSpeech and VoxDIY: Benchmark Datasets for Crowdsourced Audio Transcription, October 2021. URL http://arxiv.org/abs/2107.01091. arXiv:2107.01091 [cs, eess].

Ted Pedersen, Serguei V. S. Pakhomov, Siddharth Patwardhan, and Christopher G. Chute. Measures of semantic similarity and relatedness in the biomedical domain. *Journal of Biomedical Informatics*, 40(3):288–299, June 2007. ISSN 1532-0464. doi: 10/fghjwr. URL https://www.sciencedirect.com/science/article/pii/S1532046406000645.

Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The RefinedWeb dataset for falcon LLM: Outperforming curated corpora with web data, and web data only. June 2023.

Yifan Peng, Jinchuan Tian, Brian Yan, Dan Berrebbi, Xuankai Chang, Xinjian Li, Jiatong Shi, Siddhant Arora, William Chen, Roshan Sharma, et al. Reproducing whisper-style training using an open-source toolkit and publicly available data. In *2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)*, pp. 1–8. IEEE, 2023.

F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung. A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation. In *2016 IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), pp. 724–732, June 2016. doi: 10/ggdmmw. URL https://ieeexplore.ieee.org/document/7780454. ISSN: 1063-6919.

Laura Perez-Beltrachini, Yang Liu, and Mirella Lapata. Generating Summaries with Topic Templates and Structured Convolutional Decoders, June 2019. URL http://arxiv.org/abs/1906.04687. arXiv:1906.04687 [cs].

Denis Peskov, Nancy Clarke, Jason Krone, Brigi Fodor, Yi Zhang, Adel Youssef, and Mona Diab. Multi-Domain Goal-Oriented Dialogues (MultiDoGO): Strategies toward Curating and Annotating Large Scale Dialogue Data. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), *Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)*, pp. 4526–4536, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10/gkr9hj. URL https://aclanthology.org/D19-1460.

Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. Language Models as Knowledge Bases?, September 2019. URL http://arxiv.org/abs/1909.01066. arXiv:1909.01066 [cs].

Thang M. Pham, Seunghyun Yoon, Trung Bui, and Anh Nguyen. PiC: A Phrase-in-Context Dataset for Phrase Understanding and Semantic Search, February 2023. URL http://arxiv.org/abs/2207.09068. arXiv:2207.09068 [cs].

Sara Pieri, Sahal Shaji Mullappilly, Fahad Shahbaz Khan, Rao Muhammad Anwer, Salman Khan, Timothy Baldwin, and Hisham Cholakkal. BiMediX: Bilingual Medical Mixture of Experts LLM, February 2024. URL http://arxiv.org/abs/2402.13253. arXiv:2402.13253 [cs].

Michel Plüss, Manuela Hürlimann, Marc Cuny, Alla Stöckli, Nikolaos Kapotis, Julia Hartmann, Malgorzata Anna Ulasik, Christian Scheller, Yanick Schraner, Amit Jain, Jan Deriu, Mark Cieliebak, and Manfred Vogel. SDS-200: A Swiss German Speech to Standard German Text Corpus, May 2022. URL http://arxiv.org/abs/2205.09501. arXiv:2205.09501 [cs].

Bilal Porgali, Vítor Albiero, Jordan Ryda, Cristian Canton Ferrer, and Caner Hazirbas. The casual conversations v2 dataset, 2023. URL https://arxiv.org/abs/2303.04838.

Luiza Pozzobon, Beyza Ermis, Patrick Lewis, and Sara Hooker. Goodtriever: Adaptive toxicity mitigation with retrieval-augmented models, 2023. URL https://arxiv.org/abs/2310.07589.

Rohit Prabhavalkar, Takaaki Hori, Tara N Sainath, Ralf Schlüter, and Shinji Watanabe. End-to-end speech recognition: A survey. *IEEE/ACM Transactions on Audio, Speech, and Language Processing*, 2023.

Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. MLS: A Large-Scale Multilingual Dataset for Speech Research. In *Interspeech 2020*, pp. 2757–2761, October 2020. doi: 10/grk6mp. URL http://arxiv.org/abs/2012.03411. arXiv:2012.03411 [cs, eess].

Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, et al. Scaling speech technology to 1,000+ languages. *Journal of Machine Learning Research*, 25(97):1–52, 2024.

Adithya Pratapa, Rishubh Gupta, and Teruko Mitamura. Multilingual Event Linking to Wikidata, July 2022. URL http://arxiv.org/abs/2204.06535. arXiv:2204.06535 [cs].

Kun Qian, Satwik Kottur, Ahmad Beirami, Shahin Shayandeh, Paul Crook, Alborz Geramifard, Zhou Yu, and Chinnadhurai Sankar. Database Search Results Disambiguation for Task-Oriented Dialog Systems. In *Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pp. 1158–1173, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10/gtsqx2. URL https://aclanthology.org/2022.nacl-main.85.
Jun Quan, Deyi Xiong, Bonnie Webber, and Changjian Hu. GECOR: An End-to-End Generative Ellipsis and Co-reference Resolution Model for Task-Oriented Dialogue. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), *Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)*, pp. 4547-4557, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10/gk3btq. URL https://aclanthology.org/D19-1462.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. *arXiv preprint arXiv:2103.00020*, 2021.

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In *International Conference on Machine Learning*, pp. 28492-28518. PMLR, 2023.

Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. *arXiv preprint arXiv:2305.18290*, 2023.

Nishant Rai, Haofeng Chen, Jingwei Ji, Rishi Desai, Kazuki Kozuka, Shun Ishizaka, Ehsan Adeli, and Juan Carlos Niebles. Home Action Genome: Cooperative Compositional Action Understanding, May 2021. URL http://arxiv.org/abs/2105.05226. arXiv:2105.05226 [cs].

Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. Explain Yourself! Leveraging Language Models for Commonsense Reasoning, June 2019. URL http://arxiv.org/abs/1906.02361. arXiv:1906.02361 [cs].

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. *arXiv preprint arXiv:1606.05250*, 2016a.

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ Questions for Machine Comprehension of Text, October 2016b. URL http://arxiv.org/abs/1606.05250. arXiv:1606.05250 [cs].

Kavitha Raju, Anjaly V, Ryan Lish, and Joel Mathew. Snow Mountain: Dataset of Audio Recordings of The Bible in Low Resource Languages, May 2023. URL http://arxiv.org/abs/2206.01205. arXiv:2206.01205 [cs, eess].

Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv: arXiv:2204.06125, April 2022.

Revanth Rameshkumar and Peter Bailey. Storytelling with Dialogue: A Critical Role Dungeons and Dragons Dataset. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pp. 5121-5134, Online, July 2020. Association for Computational Linguistics. doi: 10/gtsqxp. URL https://aclanthology.org/2020.acl-main.459.

Francis McCann Ramirez, Luka Chkhetiani, Andrew Ehrenberg, Robert McHardy, Rami Botros, Yash Khare, Andrea Vanzo, Taufiquzzaman Peyash, Gabriel Oexle, Michael Liang, et al. Anatomy of industrial scale multilingual asr. *arXiv preprint arXiv:2404.09841*, 2024.

Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tramèr. Red-teaming the stable diffusion safety filter, 2022. URL https://arxiv.org/abs/2210.04610.

Anyi Rao, Linning Xu, Yu Xiong, Guodong Xu, Qingqiu Huang, Bolei Zhou, and Dahua Lin. A Local-to-Global Approach to Multi-modal Movie Scene Segmentation, April 2020. URL http://arxiv.org/abs/2004.02678. arXiv:2004.02678 [cs].

Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y.-Lan Boureau. Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset, August 2019. URL http://arxiv.org/abs/1811.00207. arXiv:1811.00207 [cs].
Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset, January 2020. URL http://arxiv.org/abs/1909.05855. arXiv:1909.05855 [cs].

Siva Reddy, Danqi Chen, and Christopher D. Manning. CoQA: A Conversational Question Answering Challenge, March 2019. URL http://arxiv.org/abs/1808.07042. arXiv:1808.07042 [cs].

Eric P Robinson and Yicheng Zhu. Beyond “i agree”: Users’ understanding of web site terms of service. *Social media+ society*, 6(1):2056305119897321, 2020.

Anna Rogers. Changing the world by changing the data. In *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)*, pp. 2182–2194, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.acl-long.170. URL https://aclanthology.org/2021.acl-long.170.

Omid Rohanian, Mohammadmahdi Nouriborji, and David A. Clifton. Exploring the Effectiveness of Instruction Tuning in Biomedical Language Processing, December 2023. URL http://arxiv.org/abs/2401.00579. arXiv:2401.00579 [cs].

Anna Rohrbach, Marcus Rohrbach, Niket Tandon, and Bernt Schiele. A Dataset for Movie Description, January 2015. URL http://arxiv.org/abs/1501.02530. arXiv:1501.02530 [cs].

Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Pal, Hugo Larochelle, Aaron Courville, and Bernt Schiele. Movie Description, May 2016a. URL http://arxiv.org/abs/1605.03705. arXiv:1605.03705 [cs].

Marcus Rohrbach, Anna Rohrbach, Michaela Regneri, Sikandar Amin, Mykhaylo Andriluka, Manfred Pinkal, and Bernt Schiele. Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Data. *International Journal of Computer Vision*, 119(3):346–373, September 2016b. ISSN 0920-5691, 1573-1405. doi: 10/f8w6kp. URL http://arxiv.org/abs/1502.06648. arXiv:1502.06648 [cs].

Angelika Romanou, Negar Foroutan, Anna Sotnikova, Zeming Chen, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary, Micol Altomare, Mohamed A. Haggag, Snegha A, Alfonso Amayuelas, Azril Hafizi Amirudin, Viraat Aryabumi, Danylo Boiko, Michael Chang, Jenny Chim, Gal Cohen, Aditya Kumar Dalmia, Abraham Diress, Sharad Duwal, Daniil Dzenhaliou, Daniel Fernando Erazo Florez, Fabian Farestam, Joseph Marvin Imperial, Shayekh Bin Islam, Perttu Isotalo, Maral Jabbarishiviari, Börje F. Karlsson, Eldar Khalilov, Christopher Klamm, Fajri Koto, Dominik Krzemiński, Gabriel Adriano de Melo, Syrielle Montariol, Yiyang Nan, Joel Niklaus, Jekaterina Novikova, Johan Samir Obando Ceron, Debjit Paul, Esther Ploeger, Jebish Purbey, Swati Rajwal, Selvan Sunitha Ravi, Sara Rydell, Roshan Santhosh, Drishti Sharma, Marjana Prifti Skenduli, Arshia Soltani Moakhar, Bardia Soltani Moakhar, Ran Tamir, Ayush Kumar Tarun, Azmine Toushik Wasi, Thenuka Ovin Weerasinghe, Serhan Yilmaz, Mike Zhang, Imanol Schlag, Marzieh Fadaee, Sara Hooker, and Antoine Bosselut. Include: Evaluating multilingual language understanding with regional knowledge, 2024. URL https://arxiv.org/abs/2411.19799.

Joseph Roth, Sourish Chaudhuri, Ondrej Klejch, Radhika Marvin, Andrew Gallagher, Liat Kaver, Sharadh Ramaswamy, Arkadiusz Stopczynski, Cordelia Schmid, Zhonghua Xi, and Caroline Pantofaru. AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detection, May 2019. URL http://arxiv.org/abs/1901.01342. arXiv:1901.01342 [cs, eess].

Amélie Royer, Konstantinos Bousmalis, Stephan Gouws, Fred Bertsch, Inbar Mosseri, Forrester Cole, and Kevin Murphy. XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings, July 2018. URL http://arxiv.org/abs/1711.05139. arXiv:1711.05139 [cs].

Askar Rozi, Dong Wang, Zhiyong Zhang, and Thomas Fang Zheng. An open/free database and Benchmark for Uyghur speaker recognition. In *2015 International Conference Oriental COCOSDA*
held jointly with 2015 Conference on Asian Spoken Language Research and Evaluation (O-COCOSDA/CASLRE), pp. 81–85, Shanghai, China, October 2015. IEEE. ISBN 978-1-4673-8279-3. doi: 10/grh5rd. URL http://ieeexplore.ieee.org/document/7357869/.

Alexander M. Rush, Sumit Chopra, and Jason Weston. A Neural Attention Model for Abstractive Sentence Summarization, September 2015. URL http://arxiv.org/abs/1509.00685. arXiv:1509.00685 [cs].

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115(3):211–252, December 2015. ISSN 1573-1405. doi: 10/gcgk7w. URL https://doi.org/10.1007/s11263-015-0816-y.

Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott. Language Modelling with Pixels, April 2023. URL http://arxiv.org/abs/2207.06991. arXiv:2207.06991 [cs].

Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Singh, Tim Rocktäschel, Mike Sheldon, Guillaume Bouchard, and Sebastian Riedel. Interpretation of Natural Language Rules in Conversational Machine Reading, August 2018. URL http://arxiv.org/abs/1809.01494. arXiv:1809.01494 [cs, stat].

Matthew J. Sag. The new legal landscape for text mining and machine learning. In Journal of the Copyright Society of the USA, 2020.

Amrita Saha, Rahul Aralikatte, Mitesh M. Khapra, and Karthik Sankaranarayanan. DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension, October 2018. URL http://arxiv.org/abs/1804.07927. arXiv:1804.07927 [cs].

Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. WinoGrande: An Adversarial Winograd Schema Challenge at Scale, November 2019. URL http://arxiv.org/abs/1907.10641. arXiv:1907.10641 [cs].

Elizabeth Salesky, Matthew Wiesner, Jacob Bremerman, Roldano Cattoni, Matteo Negri, Marco Turchi, Douglas W. Oard, and Matt Post. The Multilingual TEDx Corpus for Speech Recognition and Translation, June 2021. URL http://arxiv.org/abs/2102.01757. arXiv:2102.01757 [cs].

Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. "Everyone wants to do the model work, not the data work": Data cascades in high-stakes AI. In CHI, CHI '21, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450380966. doi: 10.1145/3411764.3445518. URL https://doi.org/10.1145/3411764.3445518.

Ramon Sanabria, Ozan Caglayan, Shruti Palaskar, Desmond Elliott, Loïc Barrault, Lucia Specia, and Florian Metze. How2: A Large-scale Dataset for Multimodal Language Understanding, December 2018. URL http://arxiv.org/abs/1811.00347. arXiv:1811.00347 [cs].

Ramon Sanabria, Nikolay Bogoychev, Nina Markl, Andrea Carmantini, Ondrej Klejch, and Peter Bell. The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR, March 2023. URL http://arxiv.org/abs/2303.18110. arXiv:2303.18110 [cs, eess].

Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. ICLR 2022, 2021. URL https://arxiv.org/abs/2110.08207.

Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen,
Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Tali Bers, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M. Rush. Multitask Prompted Training Enables Zero-Shot Task Generalization, March 2022. URL http://arxiv.org/abs/2110.08207. arXiv:2110.08207 [cs].

Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. SocialIQA: Commonsense Reasoning about Social Interactions, September 2019. URL http://arxiv.org/abs/1904.09728. arXiv:1904.09728 [cs].

Max Savery, Asma Ben Abacha, Soumya Gayen, and Dina Demner-Fushman. Question-Driven Summarization of Answers to Consumer Health Questions, May 2020. URL http://arxiv.org/abs/2005.09067. arXiv:2005.09067 [cs].

Tomohiro Sawada, Daniel Paleka, Alexander Havrilla, Pranav Tadepalli, Paula Vidas, Alexander Kranias, John J. Nay, Kshitij Gupta, and Aran Komatsuzaki. ARB: Advanced Reasoning Benchmark for Large Language Models, July 2023. URL http://arxiv.org/abs/2307.13692. arXiv:2307.13692 [cs].

Madeline C. Schiappa, Yogesh S. Rawat, and Mubarak Shah. Self-supervised learning for videos: A survey. ACM Computing Surveys, 55(13s):1–37, July 2023. ISSN 1557-7341. doi: 10.1145/3577925. URL http://dx.doi.org/10.1145/3577925.

Claudia Schulz, Josh Levy-Kramer, Camille Van Assel, Miklos Kepes, and Nils Hammerla. Biomedical Concept Relatedness – A large EHR-based benchmark, October 2020. URL http://arxiv.org/abs/2010.16218. arXiv:2010.16218 [cs].

Abigail See, Peter J. Liu, and Christopher D. Manning. Get To The Point: Summarization with Pointer-Generator Networks, April 2017. URL http://arxiv.org/abs/1704.04368. arXiv:1704.04368 [cs].

Pararth Shah, Dilek Hakkani-Tür, Gokhan Tür, Abhinav Rastogi, Ankur Bapna, Neha Nayak, and Larry Heck. Building a Conversational Agent Overnight with Dialogue Self-Play, January 2018. URL http://arxiv.org/abs/1801.04871. arXiv:1801.04871 [cs].

Amir Shahroudy, Jun Liu, Tian-Tsong Ng, and Gang Wang. NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis, April 2016. URL http://arxiv.org/abs/1604.02808. arXiv:1604.02808 [cs].

Igor Shalyminov, Sungjin Lee, Arash Eshghi, and Oliver Lemon. Few-Shot Dialogue Generation Without Annotated Data: A Transfer Learning Approach, August 2019. URL http://arxiv.org/abs/1908.05854. arXiv:1908.05854 [cs].

Dandan Shan, Jiaqi Geng, Michelle Shu, and David F. Fouhey. Understanding Human Hands in Contact at Internet Scale, June 2020. URL http://arxiv.org/abs/2006.06669. arXiv:2006.06669 [cs].

Guokan Shang, Wensi Ding, Zekun Zhang, Antoine Tixier, Polykarpos Meladianos, Michalis Vaziri-giannis, and Jean-Pierre Lorré. Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization. In Iryna Gurevych and Yusuke Miyao (eds.), *Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 664–674, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10/gm8jvb. URL https://aclanthology.org/P18-1062.

Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D Sculley. No classification without representation: Assessing geodiversity issues in open data sets for the developing world. *arXiv preprint arXiv:1711.08536*, 2017.

Dian Shao, Yue Zhao, Bo Dai, and Dahua Lin. FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding, April 2020. URL http://arxiv.org/abs/2004.06704. arXiv:2004.06704 [cs].

Aidean Sharghi, Jacob S. Laurel, and Boqing Gong. Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approach, July 2017. URL http://arxiv.org/abs/1707.04960. arXiv:1707.04960 [cs].
Eva Sharma, Chen Li, and Lu Wang. BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization, June 2019. URL http://arxiv.org/abs/1906.03741. arXiv:1906.03741 [cs].

Vivek Sharma, Makarand Tapaswi, and Rainer Stiefelhagen. Deep Multimodal Feature Encoding for Video Ordering, April 2020. URL http://arxiv.org/abs/2004.02205. arXiv:2004.02205 [cs].

Jiatong Shi, Jonathan D. Amith, Xuankai Chang, Siddharth Dalmia, Brian Yan, and Shinji Watanabe. Highland Puebla Nahuatl Speech Translation Corpus for Endangered Language Documentation. In Manuel Mager, Arturo Oncevay, Annette Rios, Ivan Vladimir Meza Ruiz, Alexis Palmer, Graham Neubig, and Katharina Kann (eds.), *Proceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas*, pp. 53–63, Online, June 2021. Association for Computational Linguistics. doi: 10/gtwwcd. URL https://aclanthology.org/2021.americasnlp-1.7.

Ying Shi, Askar Hamdullah, Zhiyuan Tang, Dong Wang, and Thomas Fang Zheng. A free Kazakh speech database and a speech recognition baseline. In *2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APISPA ASC)*, pp. 745–748, Kuala Lumpur, December 2017. IEEE. ISBN 978-1-5386-1542-3. doi: 10/gtsqzf. URL http://ieeexplore.ieee.org/document/8282133/.

E. Shriberg, R. Bates, A. Stolcke, P. Taylor, D. Jurafsky, K. Ries, N. Coccaro, R. Martin, M. Meteer, and C. van Ess-Dykema. Can prosody aid the automatic classification of dialog acts in conversational speech? *Language and Speech*, 41 (Pt 3-4):443–492, 1998. ISSN 0023-8309. doi: 10.1177/002383099804100410.

Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALFWorld: Aligning Text and Embodied Environments for Interactive Learning, March 2021. URL http://arxiv.org/abs/2010.03768. arXiv:2010.03768 [cs].

Gunnar A. Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. Hollywood in homes: Crowdsourcing data collection for activity understanding, 2016a. URL https://arxiv.org/abs/1604.01753.

Gunnar A. Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding, July 2016b. URL http://arxiv.org/abs/1604.01753. arXiv:1604.01753 [cs].

Gunnar A. Sigurdsson, Abhinav Gupta, Cordelia Schmid, Ali Farhadi, and Karteek Alahari. Actor and Observer: Joint Modeling of First and Third-Person Videos, April 2018. URL http://arxiv.org/abs/1804.09627. arXiv:1804.09627 [cs].

Damien Sileo and Marie-Francine Moens. Probing neural language models for understanding of words of estimative probability, June 2023. URL http://arxiv.org/abs/2211.03358. arXiv:2211.03358 [cs].

Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-A-Video: Text-to-Video Generation without Text-Video Data. arXiv: arXiv:2209.14792, September 2022. URL http://arxiv.org/abs/2209.14792.

Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Het-tiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergün, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Minh Chien, Sebastian Ruder, Surya Guthikonda, Emad A. Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, and Sara Hooker. Aya dataset: An open-access collection for multilingual instruction tuning, 2024a.

Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Het-tiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh
Fadaei, Irem Ergün, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Minh Chien, Sebastian Ruder, Surya Guthikonda, Emad A. Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, and Sara Hooker. Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning, February 2024b. URL http://arxiv.org/abs/2402.06619. arXiv:2402.06619 [cs].

Sam Skolnik. Openai sued over using youtube videos without creators' consent. *Bloomberg Law*, August 2024. URL https://news.bloomberglaw.com/litigation/openai-sued-over-using-youtube-videos-without-creators-consent.

Lucas Smaira, João Carreira, Eric Noland, Ellen Clancy, Amy Wu, and Andrew Zisserman. A Short Note on the Kinetics-700-2020 Human Action Dataset, October 2020. URL http://arxiv.org/abs/2010.10864. arXiv:2010.10864 [cs].

Imdat Solak. The M-AILABS Speech Dataset – caito, January 2024. URL https://www.caito.de/2019/01/03/the-m-ailabs-speech-dataset/.

Per Erik Solberg and Pablo Ortiz. The Norwegian Parliamentary Speech Corpus. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, and Stelios Piperidis (eds.), *Proceedings of the Thirteenth Language Resources and Evaluation Conference*, pp. 1003–1008, Marseille, France, June 2022. European Language Resources Association. URL https://aclanthology.org/2022.lrec-1.106.

Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, et al. Dolma: an open corpus of three trillion tokens for language model pretraining research. *arXiv preprint arXiv:2402.00159*, 2024.

Mattia Soldan, Alejandro Pardo, Juan León Alcázar, Fabian Caba Heilbron, Chen Zhao, Silvio Giancola, and Bernard Ghanem. MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions, March 2022. URL http://arxiv.org/abs/2112.00431. arXiv:2112.00431 [cs].

Amir Soleimani, Christof Monz, and Marcel Worring. NLQuAD: A Non-Factoid Long Question Answering Data Set. In *Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume*, pp. 1245–1255, Online, 2021. Association for Computational Linguistics. doi: 10/g6k6th. URL https://aclanthology.org/2021.eacl-main.106.

Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild, December 2012. URL http://arxiv.org/abs/1212.0402. arXiv:1212.0402 [cs].

Sebastian Stein and Stephen J. McKenna. Combining embedded accelerometers with computer vision for recognizing food preparation activities. In *Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing*, UbiComp ’13, pp. 729–738, New York, NY, USA, September 2013. Association for Computing Machinery. ISBN 978-1-4503-1770-2. doi: 10/gtwv9t. URL https://doi.org/10.1145/2493432.2493482.

A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van Ess-Dykema, and M. Meteer. Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech. *Computational Linguistics*, 26(3):339–373, September 2000. ISSN 0891-2017, 1530-9312. doi: 10/dqmv4j. URL http://arxiv.org/abs/cs/0006023. arXiv:cs/0006023.

Nishant Subramani, Sasha Luccioni, Jesse Dodge, and Margaret Mitchell. Detecting personal information in training corpora: an analysis. In *Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023)*, Stroudsburg, PA, USA, 2023. Association for Computational Linguistics.

Haoran Sun, Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin, and Ruohui Huang. Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models, April 2024. URL http://arxiv.org/abs/2404.02823. arXiv:2404.02823 [cs].
Jennifer J. Sun, Ting Liu, Alan S. Cowen, Florian Schroff, Hartwig Adam, and Gautam Prasad. EEV: A Large-Scale Dataset for Studying Evoked Expressions from Video, February 2021. URL http://arxiv.org/abs/2001.05488. arXiv:2001.05488 [cs].

Oyvind Tafjord, Peter Clark, Matt Gardner, Wen-tau Yih, and Ashish Sabharwal. QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships, November 2018. URL http://arxiv.org/abs/1811.08048. arXiv:1811.08048 [cs].

Oyvind Tafjord, Matt Gardner, Kevin Lin, and Peter Clark. QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions, September 2019. URL http://arxiv.org/abs/1909.03553. arXiv:1909.03553 [cs].

Shinnosuke Takamichi, Ludwig Kürzinger, Takaaki Saeki, Sayaka Shiota, and Shinji Watanabe. JTube-Speech: corpus of Japanese speech collected from YouTube for speech recognition and speaker verification, December 2021. URL http://arxiv.org/abs/2112.09323. arXiv:2112.09323 [cs, eess].

Alon Talmor and Jonathan Berant. The Web as a Knowledge-Base for Answering Complex Questions. In Marilyn Walker, Heng Ji, and Amanda Stent (eds.), *Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)*, pp. 641–651, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10/gkz2k6. URL https://aclanthology.org/N18-1059.

Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge, March 2019. URL http://arxiv.org/abs/1811.00937. arXiv:1811.00937 [cs].

Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. MultiModalQA: Complex Question Answering over Text, Tables and Images, April 2021. URL http://arxiv.org/abs/2104.06039. arXiv:2104.06039 [cs].

Niket Tandon, Bhavana Dalvi Mishra, Keisuke Sakaguchi, Antoine Bosselut, and Peter Clark. WIQA: A dataset for "What if..." reasoning over procedural text, September 2019. URL http://arxiv.org/abs/1909.04739. arXiv:1909.04739 [cs].

Raphael Tang, Rodrigo Nogueira, Edwin Zhang, Nikhil Gupta, Phuong Cam, Kyunghyun Cho, and Jimmy Lin. Rapidly Bootstrapping a Question Answering Dataset for COVID-19, April 2020. URL http://arxiv.org/abs/2004.11339. arXiv:2004.11339 [cs].

Yansong Tang, Dajun Ding, Yongming Rao, Yu Zheng, Danyang Zhang, Lili Zhao, Jiwen Lu, and Jie Zhou. COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis, March 2019. URL http://arxiv.org/abs/1903.02874. arXiv:1903.02874 [cs].

Zhiyuan Tang, Dong Wang, Yanguang Xu, Jianwei Sun, Xiaoning Lei, Shuaijiang Zhao, Cheng Wen, Xingjun Tan, Chuandong Xie, Shuran Zhou, Rui Yan, Chenjia Lv, Yang Han, Wei Zou, and Xiangang Li. KeSpeech: An Open Source Speech Dataset of Mandarin and Its Eight Subdialects. In *Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks*, volume 1, December 2021. URL https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/0336dcbab05b9d5ad24f4333c7658a0e-Abstract-round2.html.

Makarand Tapaswi, Martin Bauml, and Rainer Stiefelhagen. StoryGraphs: Visualizing Character Interactions as a Timeline. In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, pp. 827–834, 2014. URL https://openaccess.thecvf.com/content_cvpr_2014/html/Tapaswi_StoryGraphs_Visualizingeming_2014_CVPR_paper.html.

Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. MovieQA: Understanding Stories in Movies through Question-Answering, September 2016. URL http://arxiv.org/abs/1512.02902. arXiv:1512.02902 [cs].
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.

Avijit Thawani, Jay Pujara, and Filip Ilievski. Numeracy enhances the Literacy of Language Models. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pp. 6960–6967, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10/g6k6w5. URL https://aclanthology.org/2021.emnlp-main.557.

James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. FEVER: a large-scale dataset for Fact Extraction and VERification, December 2018. URL http://arxiv.org/abs/1803.05355. arXiv:1803.05355 [cs].

Shubham Toshniwal, Ivan Moshkov, Sean Narethiran, Daria Gitman, Fei Jia, and Igor Gitman. OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset, February 2024. URL http://arxiv.org/abs/2402.10176. arXiv:2402.10176 [cs].

Radim Tyleček and Radim Šára. Spatial Pattern Templates for Recognition of Objects with Regular Structure. In Joachim Weickert, Matthias Hein, and Bernt Schiele (eds.), *Pattern Recognition*, pp. 364–374, Berlin, Heidelberg, 2013. Springer. ISBN 978-3-642-40602-7. doi: 10/ggwb5g.

Herbert Ullrich, Jan Drchal, Martin Rýpar, Hana Vincourová, and Václav Moravec. CsFEVER and CTKFacts: Acquiring Czech data for fact verification. *Language Resources and Evaluation*, 57 (4):1571–1605, December 2023. ISSN 1574-020X, 1574-0218. doi: 10/g6k95h. URL http://arxiv.org/abs/2201.11115. arXiv:2201.11115 [cs].

Asahi Ushio, Fernando Alva-Manchego, and Jose Camacho-Collados. Generative Language Models for Paragraph-Level Question Generation, January 2023. URL http://arxiv.org/abs/2210.03992. arXiv:2210.03992 [cs].

Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, et al. Aya model: An instruction finetuned open-access multilingual language model. arXiv preprint arXiv:2402.07827, 2024.

Dave Uthus, Garrett Tanzer, and Manfred Georg. Youtube-asl: A large-scale, open-domain american sign language-english parallel corpus. In A. Oh, T. Naumann, A. Glober-son, K. Saenko, M. Hardt, and S. Levine (eds.), *Advances in Neural Information Processing Systems*, volume 36, pp. 29029–29047. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/5c61452daca5f0c260e683b317d13a3f-Paper-Datasets_and_Benchmarks.pdf.

Paul Vicol, Makarand Tapaswi, Lluis Castrejon, and Sanja Fidler. MovieGraphs: Towards Understanding Human-Centric Situations from Videos, April 2018. URL http://arxiv.org/abs/1712.06761. arXiv:1712.06761 [cs].

Anja Virkkunen, Aku Rouhe, Nhan Phan, and Mikko Kurimo. Finnish Parliament ASR corpus - Analysis, benchmarks and statistics, March 2022. URL http://arxiv.org/abs/2203.14876. arXiv:2203.14876 [cs, eess].

Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding, February 2019. URL http://arxiv.org/abs/1804.07461. arXiv:1804.07461 [cs].

Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models, January 2022a. URL http://arxiv.org/abs/2111.02840. arXiv:2111.02840 [cs].

Changhan Wang, Anne Wu, and Juan Pino. CoVoST 2 and Massively Multilingual Speech-to-Text Translation, October 2020a. URL http://arxiv.org/abs/2007.10310. arXiv:2007.10310 [cs, eess].
Changhan Wang, Morgane Rivière, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation, July 2021. URL http://arxiv.org/abs/2101.00390. arXiv:2101.00390 [cs, eess].

Dong Wang and Xuewei Zhang. THCHS-30: A Free Chinese Speech Corpus, December 2015. URL http://arxiv.org/abs/1512.01882. arXiv:1512.01882 [cs].

Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Doug Burdick, Darrin Eide, Kathryn Funk, Yannis Katsis, Rodney Kinney, Yunyao Li, Ziyang Liu, William Merrill, Paul Mooney, Dewey Murdick, Devvret Rishi, Jerry Sheehan, Zhihong Shen, Brandon Stilson, Alex Wade, Kuansan Wang, Nancy Xin Ru Wang, Chris Wilhelm, Boya Xie, Douglas Raymond, Daniel S. Weld, Oren Etzioni, and Sebastian Kohlmeier. CORD-19: The COVID-19 Open Research Dataset, July 2020b. URL http://arxiv.org/abs/2004.10706. arXiv:2004.10706 [cs].

Wenhan Wang, Ge Li, Bo Ma, Xin Xia, and Zhi Jin. Detecting Code Clones with Graph Neural Network and Flow-Augmented Abstract Syntax Tree, February 2020c. URL http://arxiv.org/abs/2002.08653. arXiv:2002.08653 [cs].

Wenhao Wang and Yi Yang. Vidprom: A million-scale real prompt-gallery dataset for text-to-video diffusion models. arXiv preprint arXiv:2403.06098, 2024a.

Wenhao Wang and Yi Yang. VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models, September 2024b. URL http://arxiv.org/abs/2403.06098. arXiv:2403.06098 [cs].

Xiaoxuan Wang, Ziniu Hu, Pan Lu, Yanqiao Zhu, Jieyu Zhang, Satyen Subramaniam, Arjun R. Loomba, Shichang Zhang, Yizhou Sun, and Wei Wang. SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models, February 2024. URL http://arxiv.org/abs/2307.10635. arXiv:2307.10635 [cs].

Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, and William Yang Wang. VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research, June 2020d. URL http://arxiv.org/abs/1904.03493. arXiv:1904.03493 [cs].

Yan Wang, Yixuan Sun, Yiwen Huang, Zhongying Liu, Shuyong Gao, Wei Zhang, Weifeng Ge, and Wenqiang Zhang. FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videos, March 2022b. URL http://arxiv.org/abs/2203.09463. arXiv:2203.09463 [cs].

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-Instruct: Aligning Language Models with Self-Generated Instructions, May 2023a. URL http://arxiv.org/abs/2212.10560. arXiv:2212.10560 [cs].

Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Polak Scowcroft, Neel Kant, Aidan Swope, and Oleksii Kuchaiev. HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM, November 2023b. URL http://arxiv.org/abs/2311.09528. arXiv:2311.09528 [cs].

Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. BLiMP: The Benchmark of Linguistic Minimal Pairs for English, February 2023. URL http://arxiv.org/abs/1912.00582. arXiv:1912.00582 [cs].

Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2021.

Wei Wei, Quoc Le, Andrew Dai, and Jia Li. AirDialogue: An Environment for Goal-Oriented Dialogue Research. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3844-3854, Brussels, Belgium, October 2018. Association for Computational Linguistics. doi: 10/gf6gq2. URL https://aclanthology.org/D18-1419.
Philippe Weinzaepfel and Grégory Rogez. Mimetics: Towards Understanding Human Actions Out of Context, February 2021. URL http://arxiv.org/abs/1912.07249. arXiv:1912.07249 [cs].

Johannes Welbl, Nelson F. Liu, and Matt Gardner. Crowdsourcing Multiple Choice Science Questions, July 2017. URL http://arxiv.org/abs/1707.06209. arXiv:1707.06209 [cs, stat].

Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. Constructing Datasets for Multi-hop Reading Comprehension Across Documents, June 2018. URL http://arxiv.org/abs/1710.06481. arXiv:1710.06481 [cs].

Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. Challenges in detoxifying language models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 2447-2469, 2021.

Orion Weller, Nicholas Lourie, Matt Gardner, and Matthew E. Peters. Learning from Task Descriptions, November 2020. URL http://arxiv.org/abs/2011.08115. arXiv:2011.08115 [cs].

Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M. Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks, December 2015. URL http://arxiv.org/abs/1502.05698. arXiv:1502.05698 [cs, stat].

Adina Williams, Tristan Thrush, and Douwe Kiela. ANLIzing the Adversarial Natural Language Inference Dataset, October 2020. URL http://arxiv.org/abs/2010.12729. arXiv:2010.12729 [cs].

E. B. Wilson. Untitled review. *The American Economic Review*, 4(2):442-444, 1914. ISSN 00028282. URL http://www.jstor.org/stable/1804762.

Wongun Choi, Khuram Shahid, and Silvio Savarese. What are they doing?: Collective activity classification using spatio-temporal relationship among people. In *2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops*, pp. 1282-1289, Kyoto, Japan, September 2009. IEEE. ISBN 978-1-4244-4442-7. doi: 10/fv4tmg. URL http://ieeexplore.ieee.org/document/5457461/.

Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. PMC-LLaMA: Towards Building Open-source Language Models for Medicine, August 2023. URL http://arxiv.org/abs/2304.14454. arXiv:2304.14454 [cs].

Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: A Benchmark for Molecular Machine Learning, October 2018. URL http://arxiv.org/abs/1703.00564. arXiv:1703.00564 [physics, stat].

Wangmeng Xiang, Chao Li, Ke Li, Biao Wang, Xian-Sheng Hua, and Lei Zhang. CDAD: A Common Daily Action Dataset with Collected Hard Negative Samples. In *2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)*, pp. 3920-3929, New Orleans, LA, USA, June 2022. IEEE. ISBN 978-1-66548-739-9. doi: 10/gtsqzp. URL https://ieeexplore.ieee.org/document/9857185/.

Wenhan Xiong, Jiawei Wu, Hong Wang, Vivek Kulkarni, Mo Yu, Shiyu Chang, Xiaoxiao Guo, and William Yang Wang. TWEETQA: A Social Media Focused Question Answering Dataset, July 2019a. URL http://arxiv.org/abs/1907.06292. arXiv:1907.06292 [cs].

Yu Xiong, Qingqiu Huang, Lingfeng Guo, Hang Zhou, Bolei Zhou, and Dahua Lin. A Graph-Based Framework to Bridge Movies and Synopses, October 2019b. URL http://arxiv.org/abs/1910.11009. arXiv:1910.11009 [cs].
Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein. Detoxifying language models risks marginalizing minority voices. In *Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pp. 2390–2397, 2021.

Fangyuan Xu, Kyle Lo, Luca Soldaini, Bailey Kuehl, Eunsol Choi, and David Wadden. KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questions, March 2024a. URL http://arxiv.org/abs/2403.03866. arXiv:2403.03866 [cs].

Jun Xu, Tao Mei, Ting Yao, and Yong Rui. MSR-VTT: A Large Video Description Dataset for Bridging Video and Language. In *2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 5288–5296, June 2016. doi: 10/ggv9gj. URL https://ieeexplore.ieee.org/document/7780940. ISSN: 1063-6919.

Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing, June 2024b. URL http://arxiv.org/abs/2406.08464. arXiv:2406.08464 [cs].

Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, and Baining Guo. Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions, July 2022. URL http://arxiv.org/abs/2111.10337. arXiv:2111.10337 [cs].

Yale Song, Jordi Vallmitjana, Amanda Stent, and Alejandro Jaimes. TVSum: Summarizing web videos using titles. In *2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 5179–5187, Boston, MA, USA, June 2015. IEEE. ISBN 978-1-4673-6964-0. doi: 10/gfsj74. URL http://ieeexplore.ieee.org/document/7299154/.

Xinyu Yang, Weixin Liang, and James Zou. Navigating dataset documentations in ai: A large-scale analysis of dataset cards on hugging face, 2024. URL https://arxiv.org/abs/2401.13822.

Zehui Yang, Yifan Chen, Lei Luo, Runyan Yang, Lingxuan Ye, Gaofeng Cheng, Ji Xu, Yaohui Jin, Qingqing Zhang, Pengyuan Zhang, Lei Xie, and Yonghong Yan. Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Dataset, March 2022. URL http://arxiv.org/abs/2203.16844. arXiv:2203.16844 [cs, eess].

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering, September 2018. URL http://arxiv.org/abs/1809.09600. arXiv:1809.09600 [cs].

Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents, February 2023. URL http://arxiv.org/abs/2207.01206. arXiv:2207.01206 [cs].

Seonghyeon Ye, Yongrae Jo, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, and Minjoon Seo. SelFee: Iterative Self-Revising LLM Empowered by Self-Feedback Generation, May 2023. URL https://kaistai.github.io/SelFee/.

Serena Yeung, Olga Russakovsky, Ning Jin, Mykhaylo Andriluka, Greg Mori, and Li Fei-Fei. Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos, June 2017. URL http://arxiv.org/abs/1507.05738. arXiv:1507.05738 [cs].

Wen-tau Yih, Matthew Richardson, Chris Meek, Ming-Wei Chang, and Jina Suh. The Value of Semantic Parse Labeling for Knowledge Base Question Answering. In Katrin Erk and Noah A. Smith (eds.), *Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)*, pp. 201–206, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10/gkz3hq. URL https://aclanthology.org/P16-2033.

Yue Yin, Daijiro Mori, and Seiji Fujimoto. ReazonSpeech: A Free and Massive Corpus for Japanese ASR, 2023. URL https://research.reazon.jp/_static/reazonspeech_nlp2023.pdf.
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhengguo Li, Adrian Weller, and Weiyang Liu. MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models, October 2023. URL http://arxiv.org/abs/2309.12284. arXiv:2309.12284 [cs].

Tao Yu, Rui Zhang, He Yang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter S. Lasecki, and Dragomir Radev. CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases, September 2019a. URL http://arxiv.org/abs/1909.05378. arXiv:1909.05378 [cs].

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task, February 2019b. URL http://arxiv.org/abs/1809.08887. arXiv:1809.08887 [cs].

Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, and Dragomir Radev. SParC: Cross-Domain Semantic Parsing in Context. In Anna Korhonen, David Traum, and Lluís Màrquez (eds.), *Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pp. 4511–4523, Florence, Italy, July 2019c. Association for Computational Linguistics. doi: 10/gj4fwd. URL https://aclanthology.org/P19-1443.

Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning, August 2020. URL http://arxiv.org/abs/2002.04326. arXiv:2002.04326 [cs].

Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning, October 2023. URL http://arxiv.org/abs/2309.05653. arXiv:2309.05653 [cs].

Xiaoxue Zang, Abhinav Rastogi, Srinivas Sunkara, Raghav Gupta, Jianguo Zhang, and Jindong Chen. MultiWOZ 2.2: A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines, July 2020. URL http://arxiv.org/abs/2007.12720. arXiv:2007.12720 [cs].

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a Machine Really Finish Your Sentence?, May 2019. URL http://arxiv.org/abs/1905.07830. arXiv:1905.07830 [cs].

Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. AgentTuning: Enabling Generalized Agent Abilities for LLMs, October 2023. URL http://arxiv.org/ abs/2310.12823. arXiv:2310.12823 [cs].

Kuo-Hao Zeng, Tseng-Hung Chen, Juan Carlos Niebles, and Min Sun. Title Generation for User Generated Videos, September 2016. URL http://arxiv.org/abs/1608.07068. arXiv:1608.07068 [cs].

Binbin Zhang, Hang Lv, Pengcheng Guo, Qijie Shao, Chao Yang, Lei Xie, Xin Xu, Hui Bu, Xiaoyu Chen, Chenchen Zeng, Di Wu, and Zhendong Peng. WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition, February 2022a. URL http://arxiv.org/abs/ 2110.03370. arXiv:2110.03370 [cs].

Dawen Zhang, Boming Xia, Yue Liu, Xiwei Xu, Thong Hoang, Zhenchang Xing, Mark Staples, Qinghua Lu, and Liming Zhu. Tag your fish in the broken net: A responsible web framework for protecting online privacy and copyright. October 2023a.

Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, and Jie Fu. Chinese Open Instruction Generalist: A Preliminary Release, April 2023b. URL http://arxiv.org/abs/2304.07987. arXiv:2304.07987 [cs].
Jianguo Zhang, Kazuma Hashimoto, Yao Wan, Zhiwei Liu, Ye Liu, Caiming Xiong, and Philip S. Yu. Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection, April 2022b. URL http://arxiv.org/abs/2106.04564. arXiv:2106.04564 [cs].

Jiaxing Zhang, Ruyi Gan, Junjie Wang, Yuxiang Zhang, Lin Zhang, Ping Yang, Xinyu Gao, Ziwei Wu, Xiaoqun Dong, Junqing He, Jianheng Zhuo, Qi Yang, Yongfeng Huang, Xiayu Li, Yanghan Wu, Junyu Lu, Xinyu Zhu, Weifeng Chen, Ting Han, Kunhao Pan, Rui Wang, Hao Wang, Xiaojun Wu, Zhongshen Zeng, and Chongpei Chen. Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence, March 2023c. URL http://arxiv.org/abs/2209.02970. arXiv:2209.02970 [cs].

Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level Convolutional Networks for Text Classification, April 2016. URL http://arxiv.org/abs/1509.01626. arXiv:1509.01626 [cs].

Xing Zhang, Zuxuan Wu, Zejia Weng, Huazhu Fu, Jingjing Chen, Yu-Gang Jiang, and Larry Davis. VideoLT: Large-scale Long-tailed Video Recognition, August 2021. URL http://arxiv.org/abs/2105.02668. arXiv:2105.02668 [cs].

Xinlu Zhang, Chenxin Tian, Xianjun Yang, Lichang Chen, Zekun Li, and Linda Ruth Petzold. AlpaCare:Instruction-tuned Large Language Models for Medical Application, April 2024. URL http://arxiv.org/abs/2310.14558. arXiv:2310.14558 [cs].

Yu Zhang, Daniel S. Park, Wei Han, James Qin, Anmol Gulati, Joel Shor, Aren Jansen, Yuanzhong Xu, Yanping Huang, Shibo Wang, Zongwei Zhou, Bo Li, Min Ma, William Chan, Jiahui Yu, Yongqiang Wang, Liangliang Cao, Khe Chai Sim, Bhuvana Ramabhadran, Tara N. Sainath, Francoise Beaufays, Zhifeng Chen, Quoc V. Le, Chung-Cheng Chiu, Ruoming Pang, and Yonghui Wu. Bigssl: Exploring the frontier of large-scale semi-supervised learning for automatic speech recognition. *IEEE Journal of Selected Topics in Signal Processing*, 16(6):1519–1532, October 2022c. ISSN 1941-0484. doi: 10.1109/jstsp.2022.3182537. URL http://dx.doi.org/10.1109/JSTSP.2022.3182537.

Yuan Zhang, Jason Baldridge, and Luheng He. PAWS: Paraphrase Adversaries from Word Scrambling, April 2019. URL http://arxiv.org/abs/1904.01130. arXiv:1904.01130 [cs].

Hang Zhao, Antonio Torralba, Lorenzo Torresani, and Zhicheng Yan. HACS: Human Action Clips and Segments Dataset for Recognition and Temporal Localization, September 2019. URL http://arxiv.org/abs/1712.09374. arXiv:1712.09374 [cs].

Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. WildChat: 1M ChatGPT Interaction Logs in the Wild. In *Proceedings of the Twelfth International Conference on Learning Representations*, October 2023. URL https://openreview.net/forum?id=B18u7ZR1bM.

Liang Zheng, Zhi Bie, Yifan Sun, Jingdong Wang, Chi Su, Shengjin Wang, and Qi Tian. MARS: A Video Benchmark for Large-Scale Person Re-Identification. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), *Computer Vision – ECCV 2016*, pp. 868–884, Cham, 2016. Springer International Publishing. ISBN 978-3-319-46466-4. doi: 10/gtwv9w.

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, December 2023. URL http://arxiv.org/abs/2306.05685. arXiv:2306.05685 [cs].

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset, March 2024a. URL http://arxiv.org/abs/2309.11998. arXiv:2309.11998 [cs].

Lu Zheng, Tongtong Zhou, Rongqi Jiang, and Yueping Peng. Survey of video object detection algorithms based on deep learning. In *Proceedings of the 2021 4th International Conference*
on Algorithms, Computing and Artificial Intelligence, ACAI '21, New York, NY, USA, 2022.
Association for Computing Machinery. ISBN 9781450385053. doi: 10.1145/3508546.3508622.
URL https://doi.org/10.1145/3508546.3508622.

Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, March 2024b. URL https://github.com/hpcaitech/Open-Sora.

Tiankai Zhi, Ying Shi, Wenqiang Du, Guanyu Li, and Dong Wang. M2ASR-MONGO: A Free Mongolian Speech Database and Accompanied Baselines. In 2021 24th Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA), pp. 140–145, Singapore, Singapore, November 2021. IEEE. ISBN 978-1-66540-870-7. doi: 10/gtsqzg. URL https://ieeexplore.ieee.org/document/9660401/.

Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, and Dragomir Radev. QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization, April 2021. URL http://arxiv.org/abs/2104.05938. arXiv:2104.05938 [cs].

Victor Zhong, Caiming Xiong, and Richard Socher. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning, November 2017. URL http://arxiv.org/abs/1709.00103. arXiv:1709.00103 [cs].

Luowei Zhou, Chenliang Xu, and Jason J. Corso. Towards Automatic Learning of Procedures from Web Instructional Videos, March 2017. URL https://arxiv.org/abs/1703.09788v3. arXiv:1703.09788 [cs].

Shuyan Zhou, Uri Alon, Frank F. Xu, Zhiruo Wang, Zhengbao Jiang, and Graham Neubig. DocPrompting: Generating Code by Retrieving the Docs, February 2023a. URL http://arxiv.org/abs/2207.05987. arXiv:2207.05987 [cs].

Xuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson, Jena D. Hwang, Swabha Swayamdipta, and Maarten Sap. COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements, June 2023b. URL http://arxiv.org/abs/2306.01985. arXiv:2306.01985 [cs].

Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks, September 2019. URL http://arxiv.org/abs/1909.03496. arXiv:1909.03496 [cs, stat].

Chenguang Zhu, Yang Liu, Jie Mei, and Michael Zeng. MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization, March 2021. URL http://arxiv.org/abs/2103.06410. arXiv:2103.06410 [cs].

Cunjuan Zhu, Qi Jia, Wei Chen, Yanming Guo, and Yu Liu. Deep learning for video-text retrieval: a review, 2023. URL https://arxiv.org/abs/2302.12552.

Ming Zhu, Aneesh Jain, Karthik Suresh, Roshan Ravindran, Sindhu Tipirneni, and Chandan K. Reddy. XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence, June 2022. URL http://arxiv.org/abs/2206.08474. arXiv:2206.08474 [cs].

Yi Zhu, Xinyu Li, Chunhui Liu, Mohammadreza Zolfaghari, Yuanjun Xiong, Chongruo Wu, Zhi Zhang, Joseph Tighe, R. Manmatha, and Mu Li. A comprehensive study of deep video action recognition, 2020. URL https://arxiv.org/abs/2012.06567.

Dimitri Zhukov, Jean-Baptiste Alayrac, Ramazan Gokberk Cinbis, David Fouhey, Ivan Laptev, and Josef Sivic. Cross-task weakly supervised learning from instructional videos, April 2019. URL http://arxiv.org/abs/1903.08225. arXiv:1903.08225 [cs].