Understanding Ethics in NLP Authoring and Reviewing (2023)

Abstract: With NLP research now quickly being transferred into real-world applications, it is important to be aware of and think through the consequences of our scientific investigation. Such ethical considerations are important in both authoring and reviewing. This tutorial will equip participants with basic guidelines for thinking deeply about ethical issues and review common considerations that recur in NLP research. The methodology is interactive and participatory, including case studies and working in groups. Importantly, the participants will be co-building the tutorial outcomes and will be working to create further tutorial materials to share as public outcomes.

## motivation and structure in late 2021, the association for computational linguistics' executive committee appointed an ethics committee to investigate long-term ethical issues of the community's research and legislate any policy and workflow changes to the authoring, reviewing and other processes. the committee surveyed the constituency's opinions, wants and needs, finding that the majority of respondents felt that clear guidelines on acceptable practices regarding authoring and reviewing were needed. specifically, in response to the question "what do you think are the most urgent tasks for the global *cl ethics committee?", 50% of respondents highlighted the need for more resources and discussion forums to raise awareness in the community about ethical issues in research and to clarify ethical review policies, 36% specifically mentioned the importance of creating dedicated training materials for authors and reviewers, and 26% encouraged more outreach initiatives to facilitate discussion about ethical research in the community. this tutorial proposal thus follows from the mandate from the survey, such that more interactive opportunities exist to best communicate and train our membership on ethical guidelines and research practices. the tutorial also draws on related, successful past tutorials on nlp reviewing and socially responsible nlp (≈100 participants) @xcite , where some of the proposed tutorial instructors have been involved. we propose a hybrid tutorial to best allow equitable access to the topic of this tutorial, especially to familiarize new community members and those who cannot afford access to attend physically. we plan to have dedicated presenters that can coordinate activities for the expected online participants. we may plan to use specific e-resources that can help facilitate virtual group discussions (e.g., padlet, polleverywhere, google docs, slack). we intend to make the tutorial presentation materials publicly available, in alignment with the stated goals of the tutorials. as an example, annotated presentation slides (with presenter notes) will be made available, such that tutorial participants can bring exercises of different lengths into classroom settings for research groups as well as undergraduate and graduate classes. we will organize a separate website via a github repository foot_0 (to be owned by the acl) to centralize our tutorial resources for long-term and public access. however, due to the sensitive and formative nature of the small-group discussions, we will not record the small-group discussions so that participants can speak freely and off-the-record. the plenary, lecture-styled sessions (sessions 1 and 7) may be recorded live, or pre-recorded offline. this proposal tutorial aligns with the theme track "reality check"of acl 2023. most of the challenges addressed by the theme track, including out-of-domain generalization, adversarial attacks, spurious patterns (both linguistic and social), insensitivity to basic linguistic perturbations such as segment topic led by 1. introduction and foundations for ethics presenters 2. case studies: problematic ethical research -first reading participants 3. structured interaction / dialogue presenters, participants 4. case studies -second reading (rotation) participants 5. group presentations group leads 6. summary and common issues presenters 7. discussing and troubleshooting ethics and further resources presenters ## tutorial content type: 1/2 day, introductory expected attendees: 100 audience: authors and reviewers, interested parties desired location: preferably acl (toronto, canada) prerequisites: introductory background in natural language processing and deep learning, including a basic familiarity of commonly-used approaches to text classification and generation, and standard nlp tasks. fluent command of english. ethical consideration overarch our duties as researchers and scientists. as members of our community, and representatives of our works to both the general public and practitioners, we need to consider the ramifications of our work. the need for a better understanding of ethics is reflected in both authoring and reviewing, key functions of our community's peer review process. unintended and harmful ethical lapses and consequences can be largely avoided through contin-uing communication. rather than assume that research is purely an intellectual pursuit, our tutorial invites participants to consider ethics as an integral component of the holistic framework of impactful research work. ethics requires healthy debate and deep thought, and for these reasons, our structure incorporates a socratic exercise, where participants spend a large part of the session discussing a concrete case of problematic research. a community of inquiry foot_1 approach will be taken such that participants engage in role-playing and discussing about ethical issues through reading 1-2 problematic hypothetical research abstracts from a curated set ( § 2.1). using socratic-style questioning, presenters guide the participants to engender discussion and realise ethical issues in the works. importantly, the participants will be co-building the tutorial outcomes and will be working to create further tutorial materials to share as public outcomes of the exercises. for many issues in ethics, the evolving discussion creates more value than the actual conclusions. this is why we propose such a dialectic approach. to encapsulate the exercise, the presenters will first introduce the key ways that ethics impacts authoring and reviewing (segment 1), summarise the group discussions' key points (segment 6) and conclude with pointers to references and other training materials (segment 7), including best practices for authoring ethical consideration sections @xcite and reviewing. due to the necessary interactivity of the session, we plan to limit the registrations for the tutorial to 100. this is to cater to having approximately a 25:1 ratio for presenters to participants. a larger volume than this jeopardizes the necessary interactive nature of the tutorial, which requires input from all participants. ## diversity considerations the instructors of this tutorial are affiliated in different geographic regions. luciana benotti is in latin america, kären fort in europe, min-yen kan in asia and yulia tsvetkov in north america. three of them identify with the female gender and one with the male gender. all of them are part of the acl ethics committee. we will promote this tutorial to all the acl members but in particular to affinity groups such as masakane, lat-inx, north africans, disabled in ai, indigenous in ai, khipu and similar groups with the help of equicl. equicl is the only big interest group in the acl, its scope is equity and diversity and its current officers are marine carpuat (chair), aline villavicencio (secretary), zeerak waseem (communication with workshops and affinity groups). we think it is crucial to reach a diverse audience for this tutorial. ## ethical considerations we are well aware that we do not compose a perfectly diverse committee and commit to pay close attention to ensure all participants' points of views are faithfully acknowledged. we decided to use synthetic case studies in the form of abstracts, rather than real and complete articles, in order to preserve the anonymity of the authors, to refrain from personal criticism, and to allow the participants to focus more on the discussion than on the reading. we will create a variety of abstracts, with different forms, exemplifying different ethical issues, however, they will not cover all the possible ethical issues in the domain. finally, the synthetic case studies will be clearly identified as such.