SlowGuess/NAACL_2022_OCR huggingface dataset:
The 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
reveal | brainspace
ISBN 978-1-955917-71-1
Welcome to the 2022 meeting of the North American Association for Computational Linguistics! Due to the COVID-19 pandemic, NAACL-2021 was held virtually, and NAACL-2022 is the first major NLP conference that is run as an hybrid conference in North America. It is my pleasure to welcome many of you who are joining us in Seattle, as well as those who chose to participate in the conference virtually.
COVID safety is important to us and we will do whatever we can to help you enjoy the in-person conference despite the difficulties we all experience coming back to normality. At the same time, thanks to the virtual conference platform put together by Underline, we hope that our virtual attendees will experience the conference almost as if they are in Seattle and enjoy the conference.
NAACL-2022 decided, along with ACL-2022, to experiment with a new reviewing process, based on "rolling review" (ARR). While we believe that, eventually, this process will converge to an efficient review process that would benefit our community, pioneering such a process is not without difficulties. This would not have been possible without the incredible effort, devotion, thoughtfulness, patience, and many work hours put by our program chairs, Marine Carpuat, Marie-Catherine de Marneffé, and Ivan Vladimir Meza Ruiz, and the help from the ACL-2022 program chairs, Smaranda Muresan, Preslav Nakov, Aline Villavicencio. This process necessitated developing a new software package to support our publication, an effort that was done in collaboration with ACL-2022, and I am thankful to Ryan Cotterell who led this effort.
Among other innovations we installed in NAACL-2022 is a reproducibility track, where we attempted to incentivize authors to release models, code, and other information necessary to reproduce the main results and findings of their papers. We hope that this effort, led by Niranjan Balasubramanian, Jesse Dodge, Annie Louis, Daniel Deutsch and Yash Kumar Lal, will be followed in future conferences. Other initiatives include incorporating a "Responsible NLP Research" checklist into the submission process, a new special theme on "Human-Centered Natural Language Processing", and many innovative activities led by our very active and thoughtful Diversity and Inclusion Committee, led by Diana Galván, Snigdha Chaturvedi and Yonatan Bisk, with Pranav A and Luciana Benotti as advisors.
Organizing a conference as large as NAACL, especially under the constraints of the times we live in, requires the support of a large number of volunteers who care deeply about our community and are willing to spend a lot of time and effort in this long process. It is an honor to coordinate such a team. I would like to thank the members of the organizing committee for their dedication, creativity, and hard work.
First, it is hard to imagine the amount of thought, care, and time, our program chairs Marine Carpuat, Marie-Catherine de Marneffé, and Ivan Vladimir Meza Ruiz put into all aspects of organizing this conference – resulting in an exciting and high quality scientific program.
Many other volunteers have worked hard to make this conference a success and it would not be possible to name all of them here. I will only list the chairs of the main committees whose dedication, creativity, hard work and lively communication contributed to making NAACL-2022 a successful event:
- The diversity & inclusion committee chaired by Diana Galván, Snigdha Chaturvedi and Yonatan Bisk, with advisors Pranay A and Luciana Benotti.
- The industry track chairs, Rashmi Gangadharaiiah, Anastassia Loukina and Bonan Min, and advisors Owen Rambow and Yunyao Li.
- The tutorial chairs, Cecilia Alm, Miguel Ballesteros and Yulia Tsvetkov.
- The demonstration chairs, Hannaneh Hajishirzi, Qiang Ning and Avi Sil.
- The workshop chairs, Dan Goldwasser, Yunyao Li and Ashish Sabharwal.
- The student research workshop chairs, Daphne Ippolito, Liunian Harold Li, Maria Leonor Pacheco and advisors, Danqi Chen and Nianwen Xue.
- The publication chairs, Ryan Cotterell, Danilo Croce and Jordan Zhang.
- The reproducibility chairs, Niranjan Balasubramanian, Jesse Dodge, Annie Louis, Daniel Deutsch and Yash Kumar Lal.
- The sponsorship chair, Byron Wallace.
- The volunteer chair, Daniel Khashabi.
- The publicity chairs, Nanyun (Violet) Peng, Emily Sheng, Sameer Singh.
- The virtual infrastructure chairs, Deepak Ramachandran, Martin Villalba, and Rishita Anubhai.
- The website chairs, Ice Pasupat and Vered Shwartz.
Many thanks to Chris Callison-Burch, the ACL Sponsorship Director, for helping the NAACL-2022 Sponsorship chair, Byron Wallace, managing the relations between the sponsors and NAACL-2022.
I am also very grateful to the chairs of previous years' conferences, who were always ready to help and share their experience, and to the members of the ACL and NAACL Executive Committees for their support, feedback and advice.
As usual, special thanks go to Priscilla Rasmussen and to Jennifer Rachford who has stepped into the role of the ACL business manager just in time to help us with NAACL-22. They have been our local organizers and have dealt with all aspects of organizing and managing the conference, from room assignment, to food, to COVID tests.
Finally, I would like to thank all authors, invited speakers and panelists, area chairs and reviewers, the volunteers organizing and chairing sessions, and all attendees, in-person and virtual, for making this a scientifically exciting and socially engaging conference.
Welcome and hope you all enjoy the conference!
University of Pennsylvania and AWS AI Labs
NAACL-2022 General Chair
June 2022
Welcome to the 2022 Annual Conference of the North American Association for Computational Linguistics! For the first time, NAACL-HLT 2022 is a hybrid conference. After two years of exclusively virtual conferences due to the COVID-19 pandemic, we are pleased that attendees and presenters will be able to join us in person in Seattle and from all over the world online for this year's edition.
Review Process NAACL 2022 invited the submission of long and short papers featuring substantial, original, and unpublished research in all aspects of Computational Linguistics and Natural Language Processing (NLP). Our paper review process was organized in a hierarchical structure similar to recent years. We recruited 62 senior area chairs (SACs) for 26 areas, following the areas defined for NAACL 2022. There were two paths for submitting papers: special theme papers were directly submitted to the NAACL OpenReview site, and other main conference papers were reviewed through a new ACL-wide centralized reviewing process. In coordination with the ACL 2022 organizers, we experimented with the ACL Rolling Review (ARR) introduced as part of an initiative to improve efficiency and turnaround of reviewing for ACL conferences. Within this system, reviewing and acceptance of papers to publication venues was done in a two-step process: (1) centralized rolling review via ARR, where submissions receive reviews and meta-reviews from ARR reviewers and action editors; (2) commitment to a publication venue (e.g., NAACL 2022), so that Senior Area Chairs and Program Chairs make acceptance decisions for a submission using the ARR reviews and meta-reviews. During the first phase of the review process, we served as guest Editors in Chief for the ACL Rolling Review and worked to ensure that all papers submitted received at least three review and one meta-review, while balancing the reviewing load for reviewers and action editors. NAACL SACs acted as guest senior area chairs in the ARR system, by helping monitor review progress and supporting the 408 action editors and 3379 reviewers in their work. While the new reviewing mechanism was not as smooth as one could have hoped for, all papers submitted to ARR received at least three reviews and a meta-review, so that authors could decide to commit it to NAACL 2022 if they wanted to. The ACL Executive Committee, based on feedback from the community, will decide whether the advantages of a centralized rolling review system outweigh the disadvantages, taking into account the fast growth of our research field. Once papers were committed to the NAACL OpenReview site, SACs were in charge of making acceptance recommendation per area, taking into account the submission itself, (meta-)reviews, as well as comments to SACs provided by the authors and ethics reviews when applicable.
In coordination with Jesse Dodge, Anna Rogers, Margot Mieskes, Amanda Stent, and the ACL Ethics Committee, we incorporated a "Responsible NLP Research" checklist into the submission process, designed to encourage best research practices in our field, from an ethics and reproducibility perspective. The ARR Responsible NLP Research checklist is largely based on the NeurIPS 2021 paper checklist, the reproducible data checklist from Rogers, Baldwin, Leins's paper "Just What do You Think You're Doing, Dave? A Checklist for Responsible Data Use in NLP", and the NLP Reproducibility checklist introduced by Dodge, Gururangan, Card, Schwartz and Smith in "Show Your Work: Improved Reporting of Experimental Results". Authors were asked to follow the ACL code of ethics and to fill the checklist to ensure that best practices are put in place. Reviewers were asked to consult the checklist when deciding whether the paper requires ethics review. Based on input from reviewers and action editors, SACs flagged papers that required an in-depth ethics review, which was handled by a committee of 11 ethics reviewers. The ethics chairs provided guidance and office hours to help SACs decide when ethics review was required. The ethics reviews were integrated in the final acceptance recommendation by SACs and decisions by PCs.
Special Theme We highlighted "Human-Centered Natural Language Processing" as the special theme for the conference. As NLP applications increasingly mediate people's lives, it is crucial to understand how the design decisions made throughout the NLP research and development lifecycle impact people,
whether there are users, developers, data providers or other stakeholders. For NAACL 2022, we invited submissions that address research questions that meaningfully incorporate stakeholders in the design, development, and evaluation of NLP resources, models and systems. We particularly encouraged submissions that bring together perspectives and methods from NLP and Human-Computer Interaction. Given their interdisciplinary nature, theme papers were reviewed through a dedicated process by reviewers with expertise in NLP and in Human-Computer Interaction. We received 52 submissions to the special theme, of which 14 have been accepted to appear at the conference.
Submission Statistics The ACL Rolling Review received 196 submissions in December and 1897 in January, which were the two submission deadlines between the ACL and NAACL commitment deadlines. Of these 2103 submissions,  $56\%$  (1073) were committed to NAACL 2022 for the senior program committee to make an acceptance decision. We accepted a total of 442 papers (358 long papers and 84 short papers), representing  $21.96\%$  of papers submitted to ARR in December and January and to the NAACL special theme, and  $41.19\%$  of papers committed to NAACL (including the special theme papers). As a reference point, NAACL-HLT 2021 received 1797 submissions and accepted 477 papers, including 350 long and 127 short, for an overall acceptance rate of  $26\%$ . From the accepted papers, and based on the nominations from SACs, the best paper committee selected best papers, as well as a small number of outstanding papers with the goal of recognizing diverse types of contributions (including contributions to the special theme on human-centered NLP; innovation in model design, training or evaluation; resource or dataset contribution).
Additionally, 209 submissions (183 long and 26 short) were accepted for publications in the "Findings of ACL: NAACL 2022" (or Findings for short), an online companion publication for papers that are not accepted for publication in the main conference, but nonetheless have been assessed by the program committee as solid work with sufficient substance. A total of 5 accepted Findings papers were withdrawn. Findings paper were given the option to be presented as posters during the main conference: 183 took this opportunity and will be presented either in person or virtually.
NAACL 2022 will also feature 15 papers that were published at Transactions of the Association for Computational Linguistics (TACL) and 3 papers from the journal of Computational Linguistics (CL).
Program Format The conference program was designed to allow for presentation and attendance in person in Seattle and virtually from all over the world. Oral sessions will consist of presentations done either in person or virtually. The Q&A session for each paper will alternate between in-person and online questions, with a volunteer helping monitor the online questions. All oral sessions will be live-streamed and recorded. All main conference posters will be presented with a 5-minute video pitch available online and with a virtual Q&A session, where papers will be grouped by topic to foster discussion. In addition, authors who attend the conference in Seattle will present their poster in person during traditional poster sessions. Finally, asynchronous interaction between authors and attendees will be made possible before, during and after the conference on the Underline platform. We also chose to start the conference early in the morning to overlap with normal waking hours in distant time zones.
The program includes several plenary sessions, which we hope will provide thought-provoking perspectives that will enrich discussions during the conference and beyond. In addition to a session for best paper awards, we are delighted to have keynote talks by Batya Friedman (University of Washington) and Manuel Montes-y-Gomez (National Institute of Astrophysics, Optics and Electronics of Mexico). Dan Roth (University of Pennsylvania and Amazon) will moderate a discussion on the role of linguistics and symbolic representations in NLP, with panelists Chitta Baral (Arizona State University), Emily Bender (University of Washington), Dilek Hakkani-tur (Amazon), and Christopher D. Manning (Stanford University). The industry track, demonstrations track and the student research workshop will have dedicated sessions during the main conference to round up the program, including a plenary panel on careers in NLP organized by the industry track chairs.
Gratitude NAACL would not have been possible without the hard work of many volunteers. We are very grateful to all who contributed to make the conference possible, especially given the ongoing challenges raised by the COVID-19 pandemic.
We would like to start by thanking all the authors who submitted their work to the ACL Rolling Review and NAACL 2022. We could only accept a small fraction of submissions but hope that most papers received valuable feedback.
Next, we would like to thank all members of the Program Committee – who are too numerous to be all named here, but are listed elsewhere in the proceedings:
- The senior area chairs, who were incredibly responsive throughout the reviewing process and patiently helped improve the new reviewing infrastructure.
- ARR action editors and reviewers. Special thanks to those who stepped in at the last minute to serve as emergency reviewers. This was tremendously appreciated!
- The special theme area chair, Jeff Bigham, and all reviewers, with a special note of appreciation for those who contributed their time and expertise even though they do not usually publish in NLP conferences.
- The ethics Chairs, Kai-Wei Chang, Dirk Hovy and Diyi Yang, for designing a process to encourage consistent evaluation of ethical considerations during the review process, and their timely input to ensure the integration of ethics review in acceptance recommendations and decisions.
- The ethics reviewers: Yonatan Bisk, Kevin Bretonnel Cohen, Francien Dechesne, Jack Hessel, Jin-Dong Kim, Anne Lauscher, Dave Lewis, Margot Mieskes, Xanda Schofield, Lyle Ungar, Jingbo Xia.
- The outstanding reviewers and action editors who were nominated by the senior area chairs for writing reviews that were particularly helpful in the decision making process. They are recognized by name later in this volume.
Experimenting with a new reviewing system on the large scale required by NAACL would not have been possible without the following people:
- Amanda Stent and Goran Glavaš, as ARR Editors-in-Chiefs, for their tireless work in support of the ARR December and January cycles.
- Graham Neubig, Dhruv Naik and Nils Dycke, as ARR Tech Team for these two cycles.
Celeste Martinez Gomez, Melisa Bok, and Nadia L'Bahy, as OpenReview Tech Team.
- Elijah Rippeth for his help coordinating the special theme submissions.
The following committees helped shape the conference in countless ways:
- The best paper committee: Thamar Solorio (Chair), Isabelle Augenstein, Gemma Bel Enguix, Alona Fyshe, Shafiq Joty and Emily Prud'hommeaux who enthusiastically read and recommended papers for awards in a short time frame.
- The diversity & inclusion committee chaired by Diana Galván, Snigdha Chaturvedi and Yonatan Bisk, with advisors Pranay A and Luciana Benotti.
- The industry track chairs, Rashmi Gangadharaiiah, Anastassia Loukina and Bonan Min, and advisors Owen Rambow and Yunyao Li.
- The tutorial chairs, Cecilia Alm, Miguel Ballesteros and Yulia Tsvetkov.
- The demonstration chairs, Hannaneh Hajishirzi, Qiang Ning and Avi Sil.
- The workshop chairs, Dan Goldwasser, Yunyao Li and Ashish Sabharwal.
- The student research workshop chairs, Daphne Ippolito, Liunian Harold Li, Maria Leonor Pacheco and advisors, Danqi Chen and Nianwen Xue.
- The publication chairs, Ryan Cotterell, Danilo Croce and Jordan Zhang.
- The reproducibility chairs, Niranjan Balasubramanian, Jesse Dodge, Annie Louis, Daniel Deutsch and Yash Kumar Lal.
- The sponsorship chair, Byron Wallace.
- The volunteer chair, Daniel Khashabi.
- The publicity chairs, Nanyun (Violet) Peng, Emily Sheng, Sameer Singh.
- The virtual infrastructure chairs, Deepak Ramachandran, Martin Villalba, and Rishita Anubhai.
- The website chairs, Ice Pasupat and Vered Shwartz for their exceptional reactivity and thorough checks of the conference schedule.
Finally, we would not have been able to organize this conference without the guidance, advice and cooperation of the following people:
- Damira Mrsic, Jernej Masnec, and Sol Rosenberg from Underline, who have been very prompt at answering all our questions and very helpful in setting up the virtual platform.
- Priscilla Rasmussen and Jenn Rachford who make all the logistics of the conference possible.
- Smaranda Muresan, Preslav Nakov, Aline Villavicencio, the Program co-Chairs of ACL 2022 who shared with us their materials and recent experience, and provided moral support.
- Anna Rumshisky, Thamar Solorio and Luke Zettlemoyer, as previous Program co-Chairs of NAA-CL, who answered many questions and provided invaluable guidance.
- TACL Editorial Assistant Cindy Robinson, and CL Editor-in-Chief Hwee Tou Ng for coordinating TACL and CL presentations with us.
- And last but not least, our General Chair, Dan Roth, whose guidance and support throughout the process were truly invaluable: his quiet strength, big picture thinking, and respect for all the parties involved were a soothing balm and an inspiration.
We hope you will enjoy the NAACL 2022 conference!
Marie-Catherine de Marneffe, Marine Carpuat and Ivan Vladimir Meza Ruiz  
NAACL 2022 Program Committee Co-Chairs  
June 2022
Marine Carpuat, University of Maryland  
Marie-Catherine de Marneffe, The Ohio State University  
Ivan Vladimir Meza Ruiz, National Autonomous University of Mexico
Priscilla Rasmussen, ACL  
Jennifer Rachford, ACL
Rashmi Gangadharaiah, Amazon  
Anastassia Loukina, Grammarly  
Bonan Min, Raytheon BBN Technologies
Owen Rambow, Stony Brook University Yunyao Li, IBM Research
Cecilia Alm, Rochester Institute of Technology  
Miguel Ballesteros, Amazon  
Yulia Tsvetkov, University of Washington
Hannaneh Hajishirzi, University of Washington  
Qiang Ning, Amazon  
Avi Sil, IBM Research
Dan Goldwasser, Purdue University  
Yunyao Li, IBM Research  
Ashish Sabharwal, Allen Institute for AI
Academic Inclusion Chairs
Aakanksha Naik, National University of Córdoba
Khyati Chandu, National University of Córdoba
Childcare Chairs
Fatemehsadat Mireshghallah, Univeristy of California, San Diego
Abhilasha Ravichander, Carnegie Mellon University
Saadia Gabriel, Carnegie Mellon University
Accessibility Chairs
Arjun Subramonian, University of California, Los Angeles  
Divyansh Kaushik, Carnegie Mellon University
Yuntian Deng, Harvard University
Shirley Hayati, Georgia Institute of Technology
Financial Accessibility Chairs
Nedjma Ousidhoum, University of Cambridge
Kailirroi Georgila, University of Southern California
Micha Elsner, The Ohio State University
Reut Tsarfaty, Bar-Ilan University
Pitler, Barbara Plank, Edoardo Ponti, Simone Ponzetto, Kashyap Popat, Maja Popovic, Soujanya Poria, Vinodkumar Prabhakaran, Daniel Preotiac-Pietro, Emily Prud'hommeaux
Marcos Zampieri, Marcely Zanon Boito, Fabio Massimo Zanzotto, Amir Zeldes, Luke Zettle-moyer, Justine Zhang, Weinan Zhang, Xiangliang Zhang, Xingxing Zhang, Yi Zhang, Yue Zhang, Zhe Zhang, Michael Zock
X1X
Luis Fernando D'Haro, Jennifer D'Souza, Hervé Déjean, Giovanni Da San Martino, Raj Dabre, Deborah Dahl, Hongliang Dai, Wenliang Dai, Xiang Dai, Xinyu Dai, Yinpei Dai, Zhuyun Dai, Siddharth Dalmia, Sandipan Dandapat, Marina Danilevsky, Verna Dankers, Anubrata Das, Rajarshi Das, Sarthak Dash, Vidas Daudaravicius, Sam Davidson, Brian Davis, Ernest Davis, Heidar Davoudi, Steve DeNeefe, Alok Debnath, Marco Del Tredici, Louise Deleger, Agustín Delgado, David Demeter, Xiang Deng, Yang Deng, Yuntian Deng, Zhenyun Deng, Zhongfen Deng, Tejaswini Deoskar, Jan Deriu, Franck Dernoncourt, Ameet Deshpande, Roberto Dessi, Tim Dettmers, Sunipa Dev, Jwala Dhamala, Maria Pia Di Buono, Shizhe Diao, Gaël Dias, Emily Dinan, Chenchen Ding, Haibo Ding, Kaize Ding, Liang Ding, Ning Ding, Shuoyang Ding, Long Doan, Charles Dognin, Miguel Domingo, Lucia Donatelli, Domenic Donato, Haoyu Dong, Qian Dong, Yue Dong, Bonaventure F. P. Dossou, Longxu Dou, Zi-Yi Dou, Doug Downey, Mark Dras, Markus Dreyer, Rotem Dror, Aleksandr Drozd, Andrew Drozdov, Jingfei Du, Jinhua Du, Li Du, Mengnan Du, Pan Du, Wanyu Du, Xinya Du, Yupei Du, Xiangyu Duan, Kumar Dubey, Pablo Duboue, Shiran Dudy, Philipp Duffer, Jonathan Dunn, Benjamin Durme, Ritam Dutt, Gaël de Chancelar, Gustavo de Rosa, Flor Miriam del Arco