Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processing
Authors: Lucie-Aimée Kaffee
Arnav Arora
Zeerak Talat
Isabelle Augenstein
Venue: EMNLP 2023 Findings (2023)
Keywords: dual use
ai ethics
checklist
harms
survey
TL;DR: Dual use is ill-defined in NLP; based on a survey of NLP researchers and practitioners we propose a checklist for mitigating dual use

Abstract: Dual use, the intentional, harmful reuse of technology and scientific artefacts, is an ill-defined problem within the context of Natural Language Processing (NLP). As large language models (LLMs) have advanced in their capabilities and become more accessible, the risk of their intentional misuse becomes more prevalent. To prevent such intentional malicious use, it is necessary for NLP researchers and practitioners to understand and mitigate the risks of their research. Hence, we present an NLP-specific definition of dual use informed by researchers and practitioners in the field. Further, we propose a checklist focusing on dual-use in NLP, that can be integrated into existing conference ethics-frameworks. The definition and checklist are created based on a survey of NLP researchers and practitioners.

As usage of NLP artefacts (e.g., code, data, and models) increases in research and commercial ef- forts, it becomes more important to examine their rightful use and potential misuse. The Associa- tion for Computing Machinery code of ethics? which has been adopted by ACL3, states that "[c]omputing professionals should consider whether the results of their efforts (...) will be used in so- cially responsible ways". New regulation in the European Union also requires producers of LLMs to outline foreseeable misuses and mitigation tech- niques (EU, 2023). However, without guidance on how to assess the impacts society, researchers and practitioners (henceforth professionals) are ill- equipped to consider social responsibility in de- ciding on tasks work on or managing resulting Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processing
Lucie-Aime Kaffee', Arnav Arora', Zeerak Talat', Isabelle Augenstein HHsso Plattner Institute, Germany, Unniversity of Copenhagen, Denmark
MMohaeed Bin Zayed University of Artificial Intelligence, United Arab Emirates lucie-aimee kaffee@hpi.ccoo aarêdi ku. zēzeerak. org, augensteinēdi wwt.eddiikkcco ku.dk
Abstract
research artefacts. This is reflected in contemporary ethical review processes which emphasise the im- pacts of research on individual subjects, rather than the wider social impacts of conducted research.
Dual use, the intentional, harmful reuse of tech- nology and scientific artefacts, an ill-defined problem within the context of Natural Lan- guage Processing (NLP). As large language models (LLMs) have advanced in their capabil- ities and become more accessible, the risk of their intentional misuse becomes more preva- lent. To prevent such intentional malicious use, it is necessary for NLP researchers and prac- titioners to understand and mitigate the risks of their research. Hence, we present NLP- specific definition of dual use informed by re- searchers and practitioners in the field. Further, we propose checklist focusing dual-use in NLP, that can be integrated into existing con- ference frameorrr......ss. The definition and checklist created based survey of NLP researchers and practitioners.'
While very few research projects have malicious motivations, some are reused to harm any but partic- ularly marginalised groups of society. This presents a crucial gap in the ethics of artificial intelligence and NLP on malicious reuse, or dual use, which has been particularly absent in literature.
Concerns of dual use of Artificial Intelligence (AI) have been discussed by prior work (e.g.- Shankar and Zare, 2022: Kania, 2018: Schmid et al., 2022; Urbina et al., 2022; Ratner, 2021; Gamage et al., 2021). However, NLP technolo- gies are rarely included in such considerations. As LLMs are being incorporated into wide range of consumer- products, the dual use considera- tions become increasingly critical for research and deployment. For instance, online mental health services company has used ChatGPT to respond people secking advice and aid, raising concerns of informed consent, and impact on vulnerable users (Biron, 2023). To address such misuse of LLMs, it is crucial to support researchers in mak- ing decisions to limit the dual use of their work.
1 Introduction
We propose a checklist that can be implemented by researchers to guide them the decision mak- ing. If integrated into a paper, the checklist further serves to elucidate the considerations made and approaches taken to mitigate dual use to other pro- fessionals. We keep the checklist concise, such that readily be integrated into existing checklists for ethics used by conferences within NLP. The checklist developed based on mixed-methods survey. In the survey NLP professionals, we find that the majority of our respondents do not spend a significant amount of time considering dual use
We make the survey and checklist available https: //gi segittub.ccom cwtuu..com/ooooo ctmm///penu//ual.... "Association for Computing Machinery Code Ethics Associition for Computational Linguisties Code of Ethics

---

concerns arising from their work and that institu- tional support often is inadequate.
the field. Our definition is founded in classifica- tion of harms (Weidinger et al., 2021).
In summary, our paper contributes: (a) a defini- tion of dual use for NLP artefacts; (b) survey of NLP professionals documenting their concerns of misuse of NLP artefacts; and (c) checklist and outlines of methods for mitigating risks of dual use.
2.1 Defining Dual Use in NLP
We conduct a survey of NLP professionals (see Section 3.2) and found that participants were in- terested in a precise definition that communicates clearly to professionals who are unfamiliar with dual use. We therefore formulate NLP specific definition of dual use that is informed by our survey results and prior work. Our definition centres the intent of work being created, and the intent with which it is reused. While our definition is informed by our insights into the NLP rescarch community, may be applicable other fields. We advise that researchers from other fields carefully evaluate whether their field would require any changes to the definition to capture field-specific challenges.
2 Dual Use in NLP
NLP artefacts, such LLMs, can have severe im- pacts society, even when they not developed with malicious intent. Addressing the issue of ma- licious use, the European Union defines dual use as: "goods, software and technology that can be used for both civilian and military applications" However, similarly to many other definitions of dual use, this definition only transfers to subset of NLP projects. From the field of neuroscience, Mahfoud et al. (2018) suggest that the current mili- tary use and export control lens for dual use is not adequate for scientists and policy-makers antic- ipate harms and recommend investigating the po- litical, security, intelligence, and military domains of application. Drawing from dual use research in life sciences, Koplin (2023) highlight the need for considerations in Al in light of NLP models as sophisticated text generators.
We understand dual use as the ma- licious reuse of technical and research artefacts that were developed without harmful intent. Ma- licious reuse signifies applications that are used to harm any, and particularly marginalised groups in society, where harm describes the perceived negative impacts or consequences for members by those groups. That is, dual use describes the in- tentional and malicious re-use of for a harmful (secondary) purpose besides its primary applica- tion area. To define harm in the context of dual use, differentiate between sanction and vio- lence, where our definition of harm focuses on violence rather than sanction. Briefly, sanction can be described simply as violence that is perceived as justified by entities which hold power (e.g., penal- ties by a government for breaking regulations on technological use), whereas violence can simply be described as harms without leave (e.g.. break- ing regulation which causes harm) (Foucault, 2012; Bentham, 1996). The primary mode through which these differ by how power is distributed.
Definition
We argue that the field of NLP is at crossroad on one hand, artefacts have become more accessi- ble. On the other, the increased access to resources limits the control over their use. While availability and open access can facilitate reproducible science, there is a need to develop practices that prevent the malicious use of artefacts. Discussing the social impacts of NLP, Hovy and Spruit (2016) describe dual use as "unintended consequences of research" However, this definition leaves malicious reuse out of scope. Resnik (2009) argue that dual use defi- nitions should neither be too narrow or too broad in scope, as that results in limiting what can be considered or make mitigating risks of dual use unmanageable. Forge (2010) further posit that the context of dual use is critical in determining its implications and context-specific threats and risks should be specified.
Exclusions We exclude from our definition the unintended secondary harms from research arte- facts. For instance, our definition does not address the gendered disparities produced in machine trans- lation, as the reproduction of patriarchal values is a secondary harm that arises from the social context and reflected as model and dataset biases (Van- massenhove et al., 2018: Hovy al., 2020). Simi- larly, we exclude from our definition tasks where the primary uses are malicious or harmful, e.g. de- veloping research artefacts to predict recidivism
Therefore, in our work, we present a NLP spe- cific definition of dual use, and an associated check- list that is based on the insights from our survey of
htt httts:s//polccy.teee trade. ec. ecreur. eu/help-ae exps terttersstnnee and-anndimma aed-ipoorters. exporting- dual qual-use-in muse-iieen itmsseen

---

rates, as the primary use-case is harmful (Leins al., 2020).
POO ardiise
(B
Rossdor (5)
3 Survey of NLP Professionals
62.5%
Ausupeni
Asocate
Profesor HI
Professrr
In order to develop a checklist, we survey members of the NLP community on their stances towards misuse of research artefacts through anonymous online form. Michael al. (2022) conduct a sur- vey of NLP professionals" perspectives on general topics, e.g. potential decline in industry interest and promising research directions, dual use no- tably absent from consideration. Indeed, only one question alludes to the issue: "Is it unethical to build easily-misusable systems?" to which 59% re- spond in the affirmative, indicating that profession- als are responsible for minimising misuse within their practice. We similarly conduct an anonymous survey of NLP professionals, seeking detailed in- sights into their views on dual use concerns. Our survey sought to elicit participants understanding of dual use, their perception of harms arising from it, and how to mitigate such harms.
Academia
Industry
1044
Both
Muse
Odher (
(I)
Lecturer (1)
Other
(I)
Research
Reaarch Ergreer (8)
Figure 1: Survey respondents' occupation.
ML for NLP
Information Escraction
Ceneatiion
interpretabllyy liy
Miving
Applowatinn
Tooging
Sementics
Comput allional Sou Scanca
Reseunces and Evaluction
MT and Mukilingualty
Semtment Analysi
Lostal
Disloque
Summerizaton
Our adapted survey questions were shaped by feedback on preliminary version shared re- searchers within the same group, at different lev- els semiority and discussions with colleagues in academia and industry. The survey was open to the NLP community, widely, from April un- til the end of June 2022 and advertised on Twit- ter, professional mailing lists and relevant Reddit communities." The survey conducted the LimeSurvey platform, and approved by the Uni- versity of Copenhagen ethics board under num- ber 504-0311322-5000.. The survey contained a total of 23 questions, which consist a mixture of multiple-choice questions, free-form text answers, and Likert scales (see Appendix B for the survey setup). We inductively code all free-form text an- swers (see Table 2 for the codebook).. For our anal- ysis, we discard all partial responses, resulting in n 48 complete responses out of 256 participants. We discarded partial responses as they were mostly limited to the participant demographics questions (the first part of the survey).
Prenotogyy Marphology
edbilty
Ungulsttc
Pragmatics
Language
Figure 2: Respondents' area of work.
PhD students comprising the largest group of par- ticipants. Of participants from industry, most work as Research Scientists. Participants are primar- ily based in Europe and North America with only three participants based in Asia, and one in Africa. This limits the validity of our results to Europe and North America (see Section 6 for further discus- sion).
A Area of work Participants are asked to select their areas of work and the task they work on, using the ACL 2020 classification of research areas.? Par- ticipants can select multiple areas and were asked to add at least one task per area. Their responses of work cover a wide range of topics demonstrat- ing high degree of diversity of areas of work and expertise (see Figure 2 for the distribution of sponses). This is further reflected by participants working on 180 unique tasks. Using participants' input, we identify potential harms for each task Section 3.3.
3.1 Demographies
The majority of participants work in academia (62.5%) across career stages (see Figure 1), with
'Reddit communities: rniproo. rMachineLearmingg Ieenooaggeeccnnoogg
"htps//wwww.immssrrry.rrg
"https://ac1200 http:///aaaarrrraaaa////pp/sspps

---

3.2 Definition of Dual Use
consideration of dual use by NLP professionals and serve as basis for the checklist (see Section 5).
As developing a NLP specific definition of dual use was one of the primary goals of conducting our sur- vey, we presented participants with the following definition of dual use and a free-form text field to respond:
Dual Use in the Research Workflow In efforts to understand the stages in which dual is consid- ered, the participants select stages of their research project in which they consider the risks of mis- use (for an overview of responses, see Figure 4). Although 58.3% of respondents selected that they did not find the proposed stages fitting to when they considered misuse, they also selected stages that most closely fit the stage in which they con- sider misuse. This indicates that further specificity is required in identifying which stages such con- siderations occur. For the respondents who select one or more stages for considering dual use, the discussions predominantly happen in collaborative parts of the research process, i.e. ideation and writ- ing phases. Less consideration is given after papers have been submitted and in the post publication pro- cesses. While it is important to consider dual use in the early phases, i.e. when selecting tasks to work on, potential harms can often be more clearly iden- tified once scientific artefacts have been produced. Our checklist (see Section 5) therefore seeks to aid professionals in considering dual use in the later stages of research.
"Dual use describes any task that can be intentionally used for a harmful (sec- ondary) purpose besides its main appli- cation area."
Whilc slight majority of participants (56%) agrec with the definition, their responses revealed that majority found the definition to be too vague. For instance, participants requested clearer definitions of "task" and "harmful" such that the definition would be clear to people unfamiliar with dual use. Further, participants emphasised the need to con- sider the power relations afforded by the defini- tion. Given the participant responses, we adapt our definition of dual use to the one presented in Section 2.1.
3.3 Perception of harm
Given that NLP professionals shape and implement NLP technology, the survey explores participants' perception of harms the field of NLP (see Fig- ure 3). When we asked about the risks of dual use, a majority of participants found NLP overall vulnerable, see Figure 3a. Indeed, we find that a majority of participants (77%) perceive dual use as somewhat very important to their research, how- ever only a minority (46%) of participants reported think about often or always. This demonstrates that while professionals hold a general concern, is not frequently considered in NLP projects.
Harms by NLP tasks We define the areas of work in the survey using the areas of work de- scribed in the ACL 2020 call for papers. 8 We ask participants to identify the areas and tasks that they work to obtain expert views on dual use in these fields. For each task they are asked to provide (1) harmful use cases and (2) how vulnerable they are to misuse. We then codify the harms described, as- signing one code per described harm (see Figure 5). Intentional manipulation, e.g. dis/misinformation and attempted polarisation of users is frequently listed as a potential harm. The use language mod- els for such tasks has previously been discussed Automated Inffuence Operations in detail by Gold- stein al. (2023). Further frequently identified harms include oppressing (marginalised) groups of society, e.g. using research artefacts to produce discriminatory outcomes, and surveillance by gov- ernment or corporate entities. Other concerns cen- tre around reusing NLP technologies for criminal purposes (e.g.. fraud), ethics washing, plagiarism, censorship, and military applications. Participants were also asked score how vulner-
The survey asks participants how they approach dual use considerations in their research. The an- swers cover broad range of concerns. For in- stance, large share of participants considered how scientific artefacts may be reused: (P. 11]: "How could this tool be used for harmful purposes?". Participants also raised questions that indicate im- pacts on what they work on, e.g., "Should I build this?". Participants were further concerned about the application area of their work:
[P. 28]: "How could a corporation or government use this to surveil people?"
The wide range of questions and concerns raised by participants provided insight into the nuanced
# ACI 2020 Call for Papers.

---

Net vutumbre of use?
Neether ner
Wileentle
Nover
Harely
Sonsrmmc
Othan
Awaay
How you of polential your
Imporomn
foer unimporae Somneiot
feether imperteet
Important the potantial your research
Ne ingr
Supported
Somewhhtt 14 epportted
Very expported
4 How supported by your institution do you feel with regards preventing
(upported
faany support considerations your research?
12%
Trariio-zrateral tradenaa
Figure 3: Likert scales of responses on perceptions of harm and institutional support for preventing misuse.
nome of these siages
vulnenbility
Mftrr cation
Applicatinn
oppecssion. monipulation
When presering the camera reads vera
Ethics and
ettics washing. surveillacee.
athical ststementtccheciiee durne sutmicsion en
plagarsm, opperssion
When cateiiting paper
Psycholiiquizes
4.0
cyber bullying, oppeessios crime cyber bullynng manipultioo, oppression, etiics washing
While
What rurningg expunime to
Genzration
3.8
When exchanging keee colleagures
When related work
Dislogue Systems
3.8
suvvillence, crime survillancs.
Ween sop ying for gro
MT Muhilingulity
miliiary application, munipulation. oppression
15
ahiss usshing. eurveillmor.
Figure 4: Participant responses which stage of project they consider misuse.
Resources and Evallattinn
3.1
manipulation
Inierpesabiliy
3.0
ahisa washing, monipulation suveillance, censocstip
Information Extraction IR and Text Mining
1.8
2.7
suveillince
Table I: Average score for vulnerability across ACL
mo
ippression
areas (with least three answers) the participants work on and their associated harms.
surveillacee
10
whisk wishing
cyber bullying
cation
cemorrship
Nora
slagiariam Nagiariam
30.7%
2
10
Figure 5: Distribution of codes for harms identified by participants of the tasks in NLP they work on.
Fundeg
102%
Dataat Unaalion
able each task is to misuse on Likert scale from to 5, where 5 is very vulnerable (see Table 1 for aggregate scores and harms associated with each area). For each area, we find wide range in harms for associated tasks, thus indicating the importance of considering dual use within specific use-cases.
Conmercation
Figure Distribution measures participant take
to limit misuse of the research artefacts they create.
For instance, they highlight factors that they believe are indicative of tasks that could be less prone dual use, e.g., "niche tasks which (hope- fully) can't 't be generalised such that they become harmful". However, the particular methods for se- lecting a task which not prone to dual use, or one that carries risks of it, are not described, indi- cating need for a standardised framework to help guide professionals in selecting tasks. Participants also measure other methods for addressing dual use concerns, c.g., including ethics statements and outlining limitations of the research in the paper. While the ethical impacts statements have only re- cently become mandatory at ACL conferences, they
3.4 Prevention of Misuse
The next part of the survey sought uncover how participants currently mitigate misuse of their re- search artefacts which we inductively code and ag- gregate (see Figure for the distribution of coded answers). The largest share of participants (31.8%) stated that they do not take any steps to prevent the misuse of their work. This stands in contrast participants assessment of the risks of misuse of NLP and the tasks they work on, respectively (see Section 3.3). Participants who do seek to mit- igate misuse primarily (22.7%) do so through se- lecting tasks they believe are not prone to harmful

---

support researchers in clearly communicating their concerns and intentions when publishing research. Participants also highlight other guidelines, such as data statements (Bender and Friedman, 2018) ) as helpful in outlining the limitations and risks of their research. Finally, limiting access to scientific artefacts is also used to mitigate misuse. Although this strategy can help mitigate misuse, it can have adverse effects access for legitimate purposes.
and dual use applications. They present frame- work to assess forbidden knowledge in machine learning research. They further propose grades of availability for the applications resulting in a research project, i.e., in which way it could be re- stricted. The authors point out that it is infeasible to halt the development of machine learning in di- rections that could lead to negative outcomes, machine learning "is in most cases a general pur- pose or dual use technology. meaning that has general capabilities, which are applicable to count- less varying purposes" (Kempner et al., 2011).
Institutional Support The final questions focus on institutional support for participants when it comes to questions of societal harm. When asked about support from institutions, only 37.5% of par- ticipants feel positive about the support they re- ceive (see Figure 3). That is, the majority of par- ticipants do not feel adequately supported by their institution in terms of preventing the misuse of their research. Selecting between Ethics boards, Courses, Training Material, and None most partici- pants selected that they had ethical review boards available (see Figure 3). While 61% % of participants have previously used institutional support, 14% of participants stated that they had no access to insti- tutional support for dual-use concerns. Such lack of support needs to be addressed from all institu- tional partners, i.e., publication venues, universi- ties, and companies engaged in the production of research artefacts. Until there are structured efforts to prevent the misuse of research, however, the re- sponsibility lies with the professionals themselves. Therefore, we propose checklist in Section 5 to provide a starting point for researchers to address some of the concerns of dual use.
Contrarily, Marchant and Pope (2009) point out the problems with forbidding science, including: the unenforceability of forbidding direction of research internationally, especially as international agreements are rare; the legislative imprecision in a rapidly evolving field; the unpredictability of which scientific advances may have harmful applications; the outdated nature of laws regulating science; and the potential misuse of such laws for parochial in- terests. The authors emphasise the challenges of identifying dual use of research artefact ahead of time, the main research outcome of artefact might be beneficial and therefore poses the ques- tion of the likelihood of destructive or malicious applications." 9 They argue that instcad of forbid- ding one research direction altogether, regulation should be put in place to forbid the misuse of re- search. Another option would be to pursue the research direction but limit the publication of the scientific results or the models themselves, how- ever the regulations should come from inside the communities, i.e., self-regulation. Contributing to that discussion, Brundage et al. (2018); Solaiman (2023) discuss strategies for controlled release of AI models as a mitigation method and the associ- ated trade-offs. Going step further, Henderson et al. (2023) argue for more technical approaches to supplement structural strategies to mitigate risks. They demonstrate an approach for embedding task blocking within a language model.
4 Prevention of dual use
In prior work, there are a set of approaches that address the problem of mitigating the misuse of research. In the following, we present four ap- proaches mentioned in the context of machine learning at large; forbidden knowledge, ethics re- view boards, education of researchers, and guide- lines/checklists.
4.2 Ethies Review Boards
4.1 Forhidden Knowledge
In the EU, the Nuremberg Code set the precedent for the importance of consent in medical research and later the Declaration of Helsinki set the stan- dard for ethical medical research in the EU. Any project funded by the EU undergoes an ethics re-
As knowledge holds power, the term forbidden knowledge describes scientific fields that are too dangerous distribute (Johnson, 1996). In the context of machine learning, Kempner et al. (2011) 1) propose to develop and establish an ethical frame- work to address this field's forbidden knowledge
"Similar to our work, they differentiate dual use and morally objectionable research directions.

---

could be repurposed for harmful or negative re- sults, and designing systems so that they do not inadvertently cause harm.". We argue that educat- ing researchers about the potential harms of their research should be a core part of their training. view. 10 In the US and other countries, Institutional Review Boards (IRB) are required by law and are appointed to make decisions on ethics with regard to research on human subjects (Grady, 2015).
Bernstein et al. (2021) propose the creation of an Ethics and Society Review board (ESR), which in- tervenes at the stage of the submission of research proposals. They argue that currently established ethics boards are concerned with harm to human subjects rather than impact on socicty at large. One of the topics that they focus on is dual use, which they define as "Any risks or concerns that arise due to the technology being co-opted for nefarious purposes by motivated actors (e.g., an author- itarian government employed mass surveillance methods)" (Bernstein al., 2021). They test their proposed ESR board and find that the resulting proposals are influenced by the ESR board's focus on ethics. Within machine learning, IRB or ERB reviews are typically done for projects involving human annotators or subjects but the wider societal impacts of research are rarely reviewed. Recently, pre-registration has been proposed as method improve robustness and reliability of machine learning research (van Miltenburg et al., 2021; Sø- gaard al., 2023; Bertinetto al., 2020; Albanie et al., 2021). While improving scientific rigour, pre-registration could also serve as an opportunity to evaluate the ethical and dual use implications of projects at the proposal stage.
However, as a prevention method, since it is lim- ited to people receiving formal research training and does not reach all practitioners creating NLP tools, it should be used in conjunction with other methods for prevention of dual use.
4.4 Checklists and Guidelines
A proposal presented in previous studies is the im- plementation of checklists and guidelines for searchers to navigate the complex issue of ethics in their research area (Madaio et al., 2020b; Rogers et al., 2021). A wide range of checklists have been created in recent years to cover the topic of ethics and fairness in AI, e.g., ARR's Responsible NLP Research check 11, NeurIPS' paper checklist and Microsofts' AI Faireess Checklist 13 However, existing checklists for Al Ethics at large, and for NLP in specific, do not yet cover the topic of dual use. Guidelines such as data statements (Bender and Friedman, 2018) and model cards (Mitchell et al., 2019) encourage researchers to report on their scientific artefacts more thoroughly, document their intended use, and reflect on the impact those can have. Mohammad (2022) present an ethics sheet for AI, which considers the impact and ethics of AT projects. Although dual use is briefly mentioned, it does not cover the different aspects of dual use that are required in order to suffice a starting point for discussion of researchers in the field. In contrast, the checklist proposed in Section is de- signed to be concise and specifically focused on dual use, rather than having long-form text answers as in the ethics sheet.
4.3 Education
Another path to advocating for a more careful inter- action with dual use is the possibility of educating researchers about its danger. Minchata et al. (2013) argue for the education of life scientists w.r.t. dual use, starting as part of their university education. This shifts the responsibility towards universities provide these courses as well as on the re- searchers to have been provided with the courses. While integrating the topic of dual use in the cur- riculum is an important issue, it requires large, coordinated effort, that is yet to materialise.
Checklists method in Al ethics have been re- cently criticised due to their representation of mod- ularity of software and the shifting of responsibility away from the creator of the software (Widder and Nafus, 2023). While we attempt to integrate that criticism into the creation of the checklist itself, checklists can only ever be starting point for a conversation around AI ethics large and misuse
With regards NLP, there have been proposals integrate ethics into the NLP curriculum, in- cluding dual use (Bender et al., 2020; Strube and Pan, 2022). In their proposal, Bender et al. (2020) suggest education on dual use by including "Learn- ing how to anticipate how a developed technology
"nttoss//eadrolllnnrrveeeee. 12 Paperinffommtoon//pprrrrcclklss
responsibbleWLPPrssecch/
"https///eevrrppscc///ooorerreee
10 "https://ecc.cmm europa. httttp///ee.urrppp.e//eseeerrrre htt.a//eeeeeeeee//rrriiipaattttt woreccss-for-rsseeeeee htr.riiccssffo..rreeerreeeesss
project/ai- project/ai--airmss--cceekisss

---

of research artefacts in specific. Checklists provide a productive first step help researchers, who are unfamiliar with an area, to approach and consider it. Therefore, checklists are a productive starting point and should not be the end-goal.
Dual use of scientific artefacts Despite potential unfamiliarity with the concept of dual use among survey participants, we found that they could iden- tify range of potential harms deriving from misuse of their work (see Section 3.3). The objective of the first question in the checklist (C1) is to clicit the full range of concerns held by researcher and ensure they are fully expressed in their paper. The following questions draw from the potential harms survey participants mentioned in Section 3.3 as dis- played Figure We selected from these harms the ones that can be applied to a wide range, leav- ing out ethics washing (as is specific to a sub- set of areas in NLP), crime (as it is too wide of a question and can be covered by the other topics), cyber bullying (as it could be repetitive with oppres- sion), and plagiarism (as it is again covering only specific field, potentially repretative). This leaves questions about whether scientific artefacts result- ing from the research can be used for surveillance (C2), military applications (C3), harm or oppres- sion of (marginalised)) groups in society (C4), or to manipulate people by spreading misinformation or polarising users (C5).
5 Checklist for Dual Use
Based on the results of the survey and previously established guidelines and checklists, we propose the following checklist for the issue of dual use in NLP. This checklist should be integrated into existing checklists and guidelines to extend them towards this topic and give researchers guideline for the issue of malicious reuse of their work. The design of our checklist is inspired by Madaio et al. (2020a), who co-design checklist for fairness in AI with the input of practitioners. They argue that checklists should formalise existing informal pro- cesses. Additionally, our design takes into account the insights of Rosen (2010), who maintain that checklists are essential in complex and specialised fields ensure that the minimum necessary steps are taken. Based on their experience in medicine, Rosen (2010) found that simple checklists are most effective in reducing human error.
Prevention of dual use A large share of survey participants stated that they did nothing to prevent misuse of their work (see Section 3.4). Therefore, for the second part of the checklist, we focus on ways mitigate harm through dual use. The first question in this section (C6) is aimed to cover all possible ways that participants try prevent harm through dual use. This could be by using institu- tional support in form of data distribution frame- works or by using licensing options as provided and described by external entities. With the following two questions, we focus institutional support. We found that while majority of researchers have and access institutional support (see Section 3.4), large share of participants are yet use their institutional support. In Section 4, we identify a number of possibilities to mitigate the malicious reuse of research that can only be implemented on an institutional level, e.g., ethics review boards and education. Towards the goal of better education, we phrase the checklist question C7 about ethics training. Bernstein et al. (2021) propose the exten- sion of ethics review boards to Ethics and Society Review board, indicating the importance of institu- tional support for questions of societal harm. The checklist question C8 covers these concerns, ask- ing whether the scientific artefacts produced were
The dual use of scientific artefacts part of the checklist can be used at the ideation stage of re- search project as well as in the stages before the publication of the paper. We believe that integrat- ing these questions in existing checklists, such as ACL's Responsible NLP Research Checklist will draw attention to the topic of dual use and can start a much-needed discussion in the field of NLP on such issues.
5.1 Checklist Creation
After collecting the feedback of participants on their perspective of dual use in the survey (Sec- tion 3), we grouped and sorted the feedback to be able to extract questions for the checklist that tackles the issues mentioned by participants. We grouped the checklist along the two most important angles for the prevention of misuse as discussed in Section 3.4: the dual use of scientific artefacts and prevention of dual use. The dual use of scien- lific artefacts part aims to give researchers space reflect on their work and the potential for mis- use. The prevention of dual use part aims to make preventive measures explicit and open a discussion about institutional support for the topic of dual use.

---

reviewed for dual use.
Despite the participants' concerns about the poten- tial for misuse of their work in NLP, there appears to a lack of institutional support for addressing these concerns fully. is imperative for structured and comprehensive initiatives to be implemented to address this issue. Utilising the participants in- sights, we have developed a dual use checklist for NLP professionals. This checklist, which can be integrated with existing ethics checklists, addresses both the potential dual use of scientific artefacts and measures to prevent such dual use. In con- clusion, the dual use NLP artefacts is a critical concern that has received limited attention the field. Our paper strives to raise awareness and ini- tiate a conversation among NLP professionals to address this important and overlooked issue.
5.2 Checklist
Dual use of scientific artefacts
C1
Did you explicitly outline the intended use of scientific artefacts you create?
C2 Can any scientific artefacts you create be used for surveillance by companies or governmen- tal institutions?
C3 Can any scientific artefacts you create be used for military application?
C4 Can any scientific artefacts you create be used to harm or oppress any and particularly marginalised groups of society?
C5 Can any scientific artefacts you create be used to intentionally manipulate, such spread disinformation polarise people?
Prevention of dual use
Limitations
C6 Did you access your institution's 's other available resources to ensure limiting the mis- use of your research?
Every study relying on surveys and mixed- methods approach will have a set of limitations. the case of our study, we find that the participants of the survey underlie a bias based on the location and network of the leading researchers of this project. While the survey was distributed both through the researchers' Twitter accounts well as Reddit, which could potentially reach more diverse com- munity, most answers are from researchers based in North America and Europe (see Section 3.1). This constrains our findings the issue of dual use as seen through a Western lens, as concerns of dual use may be different across different geogra- phies and social configurations (Revill et al., 2012). It is therefore important to bear in mind that our findings and proposed methods are not necessarily applicable universally and studies that examine the perspective on dual use in other geographies are important for the field of NLP be able to develop culturally competent methods for addressing dual use. Further, given the topic and name of the sur- vey, it is likely that participants who are interested in the topic of misuse of research are more likely to answer the full survey. However, we do find some researchers who clearly opposing the un- derlying premises of the study. Quoting one of the comments at the end of the survey, one participant wrote: "Please don't ruin science." pointing out their disagreement with the overall topic of ethics in NLP.
C7 Have you been provided by your institution with ethics training that covered potential mis- use of your research?
C8 Were the scientific artefacts you created re- viewed for dual use and approved by your institution's 's ethics board?
6 Conclusion
In this paper, we discussed the topic of dual use, the intentional, harmful misuse of research artefacts, in the context of NLP research. Dual use has been overlooked in the field of NLP, a gap this paper aims to close. In order to gauge the current level of consideration for dual use in the field of NLP, we conducted a survey among NLP professionals. Our survey results revealed that majority of the partic- ipants expressed concerns regarding the potential misuse of their work. They identified several poten- tial dangers associated with their work. including the oppression (marginalised) groups in society, surveillance, and manipulation. However, they re- ported taking limited steps toward preventing such misuse. Based on the participants' inputs on a pro- vided definition of dual use, propose definition of dual use that is more appropriate for researchers in the field of NLP. We discuss existing proposals address dual use machine learning and other research fields. Many of these approaches need in- stitutional structures to be realised on a large scale.

---

Acknowledgements
John Forge. 2010. A note on the definition of "dual use". Science and Engineering Ethics, 16(1):111-118.
This research was partially funded by a DFF Sapere Aude research leader grant under grant agreement No 0171-00034B, as well as supported by the Pio- neer Centre for AI, DNRF grant number PI.
Michel Foucault. 2012. Discipline and punish: The birth of the prison. Vintage.
Dilrukshi Gamage, Kazutoshi Sasahara, and Jiayu Chen. 2021. The emergence deepfakes and its soci- etal implications: A systematic review. In Proceed- ings of the 2021 Truth and Trust Online Conference (TTO 2021), Virtual, October 7-8, 2021, pages 28-39. Hacks Hackers.
References
Samuel Albanie, João F. Henriques, Luca Bertinetto, Alex Hernández-Garae Hazel Doughty, and Güll
Varol, editors. 2021. NeurlPS 2021 Workshop on
Josh A. Goldstein, Girish Sastry, Micah Musser, Re- nee DiResta, Matthew Gentzel, and Katerina Sedova. 2023. Generative language models and automated influence operations: Emerging threats and potential mitigations. CoRR, 02//2301.02446
Pre-registration in Machine Learning, volume 181 of Proceedings of Machine Learning Research. PMLR.
Emily M. Bender and Batya Friedman. 2018, Data statements for natural language processing: Toward mitigating system bias and enabling better science. Trans. Assoc. Comput. Linguistics, 6:587-604.
Christine Grady, 2015. Institutional review boards: Pur- pose and challenges. Chest, (4855):1488-1555
Emily M. Bender, Dirk Hovy, and Alexandra Schofield. 2020. Integrating ethics into the NLP curriculum. In Proceedings of the 58th Annual Meeting the A Association for Computational Linguistics: Tutorial Abstracts, ACL 2020, Online, July 5, 2020, pages 6-9. Association for Computational Linguistics.
Peter Henderson, Eric Mitchell, Christopher Man- ning, Dan Jurafsky, and Chelsea Finn. 2023. Self- destructing models: Increasing the costs of harmful dual uses of foundation models. In Proceedings of the 2023 AAAJ/ACM Conference on AI, Ethics, and Society, AIES '23, page 287-296, New York, NY, USA. Association for Computing Machinery.
Jeremy Bentham. 1996. The collected works of Jeremy Bentham: An introduction to the principles of morals and legislation. Clarendon Press.
Dirk Hovy, Federico Bianchi, and Tommaso Fornaciari. 2020. "you sound just like your father" commer- cial machine translation systems include stylistic bi- ases. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1686 1686-1690, 1690. Online. Association for Computational Linguistics.
Michael S. Bernstein, Margaret Levi, David Magnus, Betsy Rajala, Debra Satz, and Charla Waeiss. 2021. ESR: ethics and society review of artificial intelli- gence research. CoRR, a0s/2106. 15211.
Luca Bertinetto, João F. Henriques, Samuel Albanie, Michela Paganini, and Gül Varol, editors. 2020. NeurlPS 2020 Workshop on Pre-registration in Ma- chine Learning, volume 148 Proceedings of Ma- chine Learning Research. PMLR.
Dirk Hovy and Shannon I.. Spruit. 2016. The social im- pact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Popers. The Association for Computer Linguistics.
Bethany Biron. 2023. Online mental health company uses ChatGPT to help respond users in experiment - raising ethical concerns around healthcare and Al technology. Business Insider.
Deborah G Johnson. 1996. Forbidden knowledge and science as professional activity. The Monist, 79(2):197-217.
Miles Brundage. Shahar Avin, Jack Clark, Helen Toner. Peter Eckersley, Ben Garfinkel. Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, Hyrum Anderson, Heather Roff, Gregory C. Allen, Jacob Steinhardt, Carrick Flynn, Seán Ó hEigcartaigh, Si- mon Beard, Haydn Belfield, Sebastian Farquhar, Clare Lyle, Rebecca Crootof, Owain Evans, Michael Page, Joanna Bryson, Roman Yampolskiy, and Dario Amodei. 2018. The malicious use of artificial in- telligence: Forecasting, prevention, and mitigation. CoRR, abs/1802.0722
Elsa Kania. 2018. Technological entanglement: Co- operation, competition and the dual-use dilemma in artificial intelligence.
Joanna Kempner, Jon F. Merz, and Charles L. Bosk. 201 Forbidden knowledge: Public controversy and the production of nonknowledge. Sociological Forum, 26(3):475-500,
Julian J. Koplin. 2023.
Dual-use implications of AI text generation. Ethics and Information Technology. 25(2):32.
EU. 2023. Artificial intelligence act: Proposal for a reg- ulation of the european parliament and of the coun- cil laying down harmonised rules on artificial intel- ligence act and amending certain union legislative acts.
Kobi Leins, Jey Han Lau, and Timothy Baldwin. 2020. Give me convenience and give her death: Who should decide what uses of NLP are appropriate, and on what basis? In Proceedings of the 58th Annual Meeting of

---

the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020. pages 2908-2913. Association for Computational Linguistics.
James Revill, M Daniela Candia Carnevali, Åke Fors- berg, Anna Holmström, Johannes Rath, Zabta Khan Shinwari, and Giulio M Mancini. 2012. Lessons learned from implementing education on dual-use in austria, italy, pakistan and sweden. Medicine, Con- flict and Survival, 28( (0(())33122)))) 1-:31
Michael A. Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna M. Wallach. 2020a. Co- designing checklists understand organizational challenges and opportunities around fairness in AL. In CHI '20: CHI Conference on Human Factors in Computing Systems, Honolala, HI, USA, April 25-30. 2020, pages 1-14. ACM.
Anna Rogers, Timothy Baldwin, and Kobi Leins. 2021. "just what do you think you're doing, dave?" A check- list for responsible data use in NLP. In Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Domini- can Republic, 16-20 November, 2021, pages 4821- 4833. Association for Computational Linguistics.
Michael A. Madaio, Luke Stark, Jennifer Wort- man Vaughan, and Hanna Wallach. 2020b. Co- designing checklists to understand organizational challenges and opportunities around faimess in ai. In Proceedings of the 2020 CHI Conference on Hu- man Factors in Computing Systems, CHI '20, page 1-14, New York, NY, USA. Association for Comput- ing Machinery.
Dennis Rosen. 2010. The checklist manifesto: How to get things right. JAMA, 303(7))670-6773
Stefka Schmid, Thea Riebe, and Christian Reuter. 2022. Dual-use and trustworthy? A mixed methods analysis of Al diffusion between civilian and defense r&d. Sci. Eng. Ethics, 28(2):12.
Tara Mahfoud, Christine Aircardi, Saheli Datta, and Nikolas Rose. 2018. The limits of dual use. Issues in Science and Technology, 34(4):73-78.
Sadasivan Shankar and Richard N. Zare. 2022. The per- ils of machine learning in designing new chemicals and materials. Nat. Mach. Intell., 4(4):314-313
Gary E. Marchant and Lynda L. Pope. 2009. The problems with forbidding science. Sci. Eng. Ethics, 1555)3375-394.
Anders Søgaard, Daniel Hershcovich, and Miryam de Lhoneux. 2023. two-sided discussion of prereg- istration of NLP research. In Proceddings of the 17th Conference of the European Chapter of the Associ- ation for Computational Linguistics, pages 83-93. Dubrovnik, Croatia. Association for Computational Linguisties.
Julian Michael, Ari Holtzman, Alicia Parrish, Aaron Mueller, Alex Wang. Angelica Chen, Divyam Madaan, Nikita Nangia, Richard Yuanzhe Pang, Ja- son Phang, and Samuel R. Bowman, 2022. What do NLP researchers believe? results of the NLP commu- nity metasurvey. CoRR, 128822
Irene Solaiman. 2023. The gradient of generative ai release: Methods and considerations. In Proceedings of the 2023 ACM Conference on Fairness, Account- ability, and Transparency, FAceT '23, page 111-122 New York, NY, USA. Association for Computing Machinery.
Masamichi Minehata, Judi Sture, Nariyoshi Shinomiya, Simon Whitby, and Malcolm Dando, 2013. Promot- ing education of dual-use issues for life scientists: A comprehensive approach. Journal of Disaster Re- search, 8:674-685.
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the Conference on Fairness, Account- ability, and Transparency, FAT* 2019, Atlanta, GA, USA, Junuary 29-31, 2019, pages 220-229. ACM.
Michael Strube and Shimei Pan. 2022. Ethics in nlp: Bias and dual use. UMBC Faculty Collection.
Fabio Urbina, Filippa Lentzos, Cédric Invernizzi. and Sean Ekins. 2022. Dual use of artificial-intelligerr powered drug discovery. Nat. Mach. Intell., 4(3):199- 191.
Saif M. Mohammad. 2022. Ethics sheets for AI tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume Long Papers), ACL 2022, Dublin, Treland, May 22-27, 2022, pages 8368-8379, Association for Com- putational Linguisties.
Emiel van Miltenburg, Chris van der Lee, and Emiel Krahmer. 2021. Preregistering NLP research. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 613 623, Online. Association for Computa- tional Linguistics.
Claudia Ratner. 2021. When "sweetie" is not so sweet: Artificial intelligence and its implications for child pornography. Family Court Review, 59(2):386-401.
Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting gender right in neural machine translation. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Process- ing, pages 3003-3008, Brussels. Belgium. Associa- tion for Computational Linguistics.
David B. Resnik. 2009. What is "dual use" research? response to miller and selgelid. Sei. Eng. Ethics, 15())33.

---

Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William S. Isaac, Sean Legassick, Geoffrey Irving, and lason Gabriel. 2021. Ethical and social risks of harm from language models. CoRR, abs/21 12.04359.
A
Codebooks
For each the free-form answer fields of the sur- vey, we coded the answers of the participants, for details see Section 3. Below we provide the code- books for the relevant questions reported in this paper.
A.1 Harms by Task
David Gray Widder and Dawn Nafus. 2023. Dislocated accountabilities in the "Al supply chain": Modularity and developers' notions of responsibility. Big Data & Society, 10( 12:205395//231117667
Participants were asked to list the potential harm of each of the tasks they stated they worked on. The codebook provided Table 2. The distribution of the codes among the survey respondents can found in Figure 5. [h]
A.2 Measures to limit misuse
Participants were asked to list measures they take to limit the potential misuse of their work. The codebook is provided in Table 3. The distribution of the codes among the survey respondents be found in Figure 6.
B Survey
We provide the full survey as conducted on LimeSurvey below.

---

Dual Use NLP
queetinn sumnyy
Survey disclaimer
ratural banguage procexang (NU', nessech conduciad by Lucie-lemie (1) Arrou Arewi (Ts, Zeersc Thase EX), bodsels Aupernzaen 111 all llaaed with the 113 Univeriity Capentagen and Digiaal Democrades inaitude with the gas of eiceestencinn how researchers approscn the dunger of ouse research.
survey simeed nossardarr caru and precitaomes
This survey has boon apprtwed by the othics comnen tho the universit Capontagon under number 604 d31822 5000
your
sarny
would you to survay adiecniieity and much sowable.
surney' anonymited
change
Background
Do you work in industry or academia? (If other, please specify in the text box)
Choose one of the following answers chonse only the
Industry
Academiia
Industry best describes your position?
Only onswer this question the following condiions are met
(C322054 NADK UN exioekkmannnnee www.kknnnnnkkkkkkkkkkoo.o.o. www.rr/wwwwwwwweeeeeeoooo www.saawwwws777//....... www.aae/aaaaaeeeeeeee 'A002 C02034 NAOK
Un Sy..wrtonkemmmanneeee www.mmannnnnnwwwweeeeeee. www.ur/wwettrrrtrteeeeee teatrag) (600001)
Cheose
you
please
speaty your
accompanyffng
Phease chones only the fellevingy
Research Sclentis
Resperch ch Enginser
Inductrol postdoc
Management
Other

---

In Academia, what best describes your position?
Only ansvrr this question the following conditons are mec
(1902054 NADK (qp artonekmiin cortrre-mminteeeeeeeeee wwwp:///wwwwwrreeeiiei..oo weigrr//eg///eeeeeeeeee (i0g tn 'ADO3 GU2034 NAOK
werkonkkemnnnndeeeeeeeeee yarterr yesterr regatoag) (400091)
only followings
Niveter
PhO condadate
Posodce
Lecturer
Assssian Professor
Professor
O Other
deploy NLP applications into production?
Where in the world do you work?
only falloxing
Asia
Austrill aesttnliaiiiin Groeros
Allarctioa
Europe
North America
Background fields of research

---

We undentund hd hows
Blew pmide
Please select beiow which Seds you hinen wnrood in thess aus the ACI madde, inagiaa whach yourhove pirkeeeyy weol hows orlill butiil papeos nt) Plaass foos work
Por sush bess, plaas provds tst baita tuat you von e that sod. asperated by commas.
qpeatii resarrh recemuurly pribler firmuiation nasvpll tuike Receentios could exeliig pencation sic
mas on, justrppeat specily the
Comment only witen chtacee aroovee
Please chosse all that auply and provide commentt
Cogn Modelling ug and inudhooiiguiidss
of Socaai Staenoo and Gultural
Analytics
D and Intaractive Systemm
Diccoume and
Generation
Information Extrction
Information Reoloval
Interpeeniilly and Anstyes Medelts
NLP
Language Grounding Visio,, Robolics and Beyond
Linquaae Theories, Coqrilive Modaling, Paych Tschoiinusstes
Lewning for
Machine Traratation and Muttingusityy
NLP Appleattonn
Phonology, Merphology, Segmentetion
Answering
Resources and Evolustion
Semandic:: Lovical
inferonos,
Sentiment Analysis, Styfitil Maving
Sysiko: Tecaing, Crunking and Pensing
Dual use definition and tasks

---

you think it is important have definition use can applied NLP?
only the followings
Do you timms that our def iion of day use (obovo)is sporoph die for NUPY
Please chosse only one of the following
Yee
Ni
sou pww.saadddipiisuaaaaaaaa Sokoeing
SQ009commentt
INEERIAN588776XXX11011666110
[NNSERTANS:8777806XXX55050
SQ02 ccomment
INSERTANS::
iNSERTANS:98778880
Conne:
which
could
modele Interdionaly deployed conted

---

Tank Vulremabiity:: Oe the ices, dellne now vultaradle the bak miszss the form dual tug for the uae case you deverbedi.
Please shoose the spprop lase resporie for each hom:
Neither vulnerable nor Invulnerable
Somewhat vulnerable
Not vulnerable
vulnerable
Not vulnerable
Taak
Taak 5
Tank
Taak 7
Task
Taak
Tuak
Taak
Task
Teak
Task
Tank
Taak
Task
Dual Use definition and considerations

---

Overall, as field, how vulnerable is NLP dual use?
Nelther Involnerable
Very vuiinerble
vulnerable
Not vulnerable
Wikdh, Vary. of soict antoloot dront sould prosicm cookyy tap poosle in cociety to dual
Check apply
Mithodds
Data
Model
Anadyala
Irlarfooe
When during cumea of reserrh project atat, conaderr polembe mouee tedhnolgiis knowledge you create
Check apply
Plesss chorow all thet apply:
creation phuse
apelving
rolated
When exchanging ideas with coll eagues
Whee surning sopartments
willing
submiting paper
othical sartemansicchelfaa submissiion
When roading rovlows
When preparing the cornera roady version
After publication
Other
How often you think of the question of potential misuse your research?
Please chosse resporise for each Bant:
Always
Otten
Sometimes
Ranely
Never
Which questions do you ask yourself regarding the potential misuse of your research?

---

How important question of potential misuse of your research to you?
Netther unimportant
Very
Important
What measures do you limit the potential misuse your research (if any)?
your
How supported by your institution do you feel with regards preventing the misuse your research?
Masse chocow
for each
Natthe unsupported
supported
Very supported
supported
supported
your
that appty:
Coursea
Training matert!
Nore
Other
Have you utilised said support for any of your projects?
Please chosse only of the falloing
Yea

---

Do you hane any comment for the rescarcten??
Please will your answer here:
01-06-300r--- 12101
Submik your survay.
Thank you for completing this survey.

---

code
description
includes(sub- codes)
examples
surveillance
automation of surveillance by any entity
surveillance by cor- porations; surveil-
large scale data col- lection and process-
lance by govern- ment
ing of users
manipulation
manipulation of users of technology, trying to change their beliefs world-view
disinformation, po- disinformation gen- larisation
eration
technologies to op- of groups in press, marginalise disadvantage any group in a society
oppression
language standardi- sation; propagation of racist believes or classification
generation of hate speech; classifica- tion based on di- alects in language
society
crime
reuse of technolo- gies for general criminal purpose
personalised phish- ing
ethics washing minimise the un- derlying problems of system
reuse technologies
explainability
fairwash an algo- rithm exhibiting bias
cyber bullying
target individuals with harmful, oppressive or disad-
using technologies to bully kids schools
vantageous content
military applica- tion censorship
reuse of technolo-
target systems
gies by the military
automation of cen- sorship
censorship by gov- ernment
finding content to censor on a large scale
plagiarism
automatic creation summarisation of content for plagia-
automatically rephrasing existing academic work to republish
rism
Table 2: Codebook for the harms identified by participants for each of the tasks they work on. Question: Please add for each task the following: Dual Use Use Cases: the text field, state which use cases of the task could problematic (e.g., when models intentionally deployed out of context cause harm).

---

code
description
includes (sub-codes)
examples
Task Selection
considerations regarding the tasks the participants work on
selecting tasks or research problems to work on that are less prone to mis- use: decline jobs or re- search projects where par- ticipants are not comfort- able how their models used
Communication
communicating the risks related through the work: discussing po- tential risks with the community
ethics state- ments
writing the paper to clearly communicate risks; using ethics statements
Limited Access
limiting access to scientific arte- facts
regulation work; licens-
limit access data and code
ing
Dataset
Cre-
careful considerations when the dataset created with regards to how it will be reused
anonymisation of datasets; dataset state- ments
ation
keep processing of data lo- cal
Funding
considerations regarding the sources of funding for research
selection and rejecting projects based on the fund- ing agency
None
no measures taken to limit mali- cious reuse
comment "None"
Table Codebook for how participants select tasks work on. Question: What measures do you take to limit the potential misuse of your research (if any)?.


Reviews:
[{"review_id": "k4lu8ekHXS", "reviewer": "Reviewer_mJ7P", "summary": "", "questions": "A. Can you clarify which type of inductive coding you perform (e.g. is this thematic analysis?) and share any resulting codebook?\n\nB. Can you comment on the choice of a checklist in light of recent critique of checklist-based approaches to ethics? For instance, the critiques of Widder & Nafus [1]:  \"asking developers to anticipate every conceivable outcome by diligently following elaborate checklists as if they occupied a view from nowhere (what Gansky and McDonald (2022) call “metadata maximalism”) does not portend a meeting of partial knowledges.\"\n\n[1] Widder, D. G., & Nafus, D. (2023). Dislocated accountabilities in the  “AI supply chain”: Modularity and developers’ notions of responsibility. Big Data & Society, 10(1). https://doi.org/10.1177/20539517231177620\n\nC. The number of full responses is a small fraction of the total respondents (48 of 256). What areas did partial responses leave unfinished, and would it be possible to use some of the data from these partial responses?\n\nD. What do you see as the key contribution of your definition of dual use, in comparison to existing definitions?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments and their suggestions for related work, we want to take this moment to clarify some of our positions. \n\n**W1.** We believe that a NLP specific definition of dual use is necessary due to how language and identity are co-constructive. This requires specific considerations around language. Pre-existing definitions for dual use are often too narrow or too broad for the purposes of NLP (e.g., regulation). We argue that the general definitions are not adequate to cover the potential misuses, we, therefore, propose a definition tailored with NLP artefacts in mind.  We appreciate the reference to Leins et al., however, their discussion does not address in detail the issue of dual use, but consider it from a higher and more abstract position. We seek to address this with our paper. We further contribute to the understanding of dual use by practitioners through the survey as well as provide an actionable checklist, which enables researchers to reflect on the issue. \nWe will be more clear about this in the camera ready.\n\n**W2.** To the best of our knowledge, there isn’t work on mitigating dual use in NLP within those fields. We believe that this further provides evidence for the need to address the concern of dual use. However, the goal of our work was to 1) identify how practitioners view issues of dual use, 2) how they are supported within their institutions, and 3) what might be productive paths forward for considering dual use. We do agree that STS and other social scientific fields have many relevant considerations to NLP, however, it is out of scope for our work, as we are seeking first to establish how the issue is viewed internally.\n\n**W3.** To understand *why* a particular phenomenon exists, we have to first establish that it does, which is what we aim to do with the survey. The survey highlights the stance of the field and builds a foundation for further investigation of the questions posed by dual use in NLP. We believe that future work should address these questions with qualitative and mixed-methods studies to get an in-depth understanding, e.g., of the lack of institutional support, which depends on the context of the individual.\n\n**W4.** We thank the reviewer for the appreciation of our mixed-methods approach. As stated earlier, we wanted to build a foundation for the research in the field of dual-use and NLP and believe that a general overview, including surveys, will build this foundation upon which future qualitative research can build. We would like to suggest that these qualitative studies should be done with the context of the participants in mind, and can then deliver more fine-grained understandings of their needs in their specific context, whereas we wanted to give an introduction to the topic at large.\n\n###  Questions For The Authors \n**A.** We did indeed use thematic analysis for the coding of the responses. We have added the codebooks at the bottom of the response as a table and will add them to the appendix of the manuscript.\n\n**B.** One of the primary reasons we chose to use a checklist was to integrate our work in existing frameworks around ethics in NLP and not introduce a new format to researchers. We believe that based on discussion within the community, as a result of this paper, this could be extended to other formats in the future, similar to an ethics statement. And while we agree with the critique, checklists provide a productive first step to help researchers, who are unfamiliar with an area, to think about it. We therefore believe that checklists are a productive starting point and should not be the end-goal. We will add these points to the camera ready to encourage future work to develop new formats.\n\n**C.** The partial responses were mostly limited to the participant demographics questions (the first part of the survey), we therefore decided not to use the answers in our analysis of the survey. We will clarify this in the manuscript.\n\n**D.** The main contribution of our definition of dual use is that it is developed specifically with NLP artefacts in mind. Existing definitions of dual use were developed for civilian applications used by the military [1]. In contrast, ours is shaped for the exact scope of the research output of NLP researchers. \n\n[1] Selgelid, Michael J. \"Governance of dual-use research: an ethical dilemma.\" Bulletin of the World Health Organization 87 (2009): 720-723.\n\n### Codebook for Harms Identified (Figure 5)\n\n|code            |description                                                                                    |includes (sub-codes)      |examples                                                                                                                                                                         |\n|----------------|-----------------------------------------------------------------------------------------------|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n|surveillance    |automation of surveillance by any entity                                                       |surveillance by corporations; surveillance by government|large scale data collection and processing of users                                                                                                                              |\n|manipulation    |manipulation of users of technology, trying to change their believes or world-view             |disinformation, polarisation|disinformation generation                                                                                                                                                        |\n|opression of groups in society|technologies to oppress, marginalise or disadvantage any group in a society                    |language standardisation; propagation of racist believes or classification|generation of hate speech; classification based on dialects in a language                                                                                                        |\n|crime           |reuse of technologies for general criminal purpose                                             |                          |personalised phishing                                                                                                                                                            |\n|ethics washing  |reuse technologies to minimise the underlying problems of a system                             |                          |explainability to fairwash an algorithm exhibiting bias                                                                                                                          |\n|cyber bullying  |target individuals with harmful, oppressive or disadvantageous content                         |                          |using technologies to bully kids in schools                                                                                                                                      |\n|military application|reuse of technologies by the military                                                          |                          |target systems                                                                                                                                                                   |\n|censorship      |automation of censorship                                                                       |censorship by government  |finding content to censor on a large scale                                                                                                                                       |\n|plagiarism      |automatic creation or summarisation of content for plagirarism                                 |                          |automatically rephrasing existing academic work to republish                                                                                                                     |\n\n\n\n### Codebook for Task Selection (Figure 6)\n\n|code            |description                                                                                    |includes (sub-codes)      |examples                                                                                                                                                                         |\n|----------------|-----------------------------------------------------------------------------------------------|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n|Task Selection  |considerations regarding the tasks the participants work on                                    |                          |selecting tasks or research problems to work on that are less prone to misuse; decline jobs or research projects where participants are not comfortable how their models are used|\n|Communication   |communicating the risks related through the work; discussing potential risks with the community|ethics statements         |writing the paper to clearly communicate risks; using ethics statements                                                                                                          |\n|Limited Access  |limiting access to scientific artefacts                                                        |regulation work; licensing|limit access to data and code                                                                                                                                                    |\n|Dataset Creation|careful considerations when the dataset is created with regards to how it will be reused       |dataset statements        |anonymisation of datasets; keep processing of data local                                                                                                                         |\n|Funding         |considerations regarding the sources of funding for research                                   |                          |selection and rejecting projects based on the funding agency                                                                                                                     |\n|None            |no measures taken to limit malicious reuse                                                     |                          |comment \"None\"                                                                                                                                                                   |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "S1. Dual use is an important ethical consideration, and the authors discuss it in depth, including discussing the benefits and disadvantages of several definitions, which could be useful for later work in the area.\n\nS2. The authors discuss the literature around several intervention options for dual use, which may not be well-known to NLP practitioners. \n\nS3. The authors highlight the limited discussion of dual use among NLP practitioners (as reflected in their survey), raising a point for concern.", "reasons_to_reject": "W1. The proposal of the definition is presented as a key contribution. However, the need for an NLP-specific definition of dual use is not well-motivated. While the authors argue clearly that the specific definitions quoted in section 2 are (respectively) too narrow and too broad, it is not clear that discussion of \"dual use\" in NLP is limited by either of these definitions. In fact, several recent papers engage with \"dual use\" in ways that seem to parallel the authors' definition, and the ARR Responsible NLP checklist seems to include several of the identified harms explicitly as examples of \"malicious [...] uses\":\n>Examples of risks include potential malicious or unintended harmful effects and uses (e.g., disinformation, generating fake profiles, surveillance), environmental impact (e.g., training huge models), fairness considerations (e.g., deployment of technologies that could further disadvantage or exclude historically disadvantaged groups), privacy considerations (e.g., a paper on model/data stealing), and security considerations (e.g., adversarial attacks). See discussion in Leins et. al. (2020) as examples.\n\n While \"making the `obvious' explicit\" can be a valuable contribution of ethics research, it is not clear to me that this clarifies the discussion around dual use or provides a new perspective. \n\nW2. The work could engage more deeply with the STS and ethics literature outside of NLP. Particularly, more discussion of work on how people in technical fields apply or interpret ethics research would be helpful contextualization-- e.g., how do people generally take concepts like \"dual use\" into NLP ethics work? Does this demonstrate the need for an emic definition of the term, as you suggest? \n\nW3. Overall, the work is interesting descriptively, but could provide more detailed analysis. Particularly, participants indicate that they do not use institutional support and do not explicitly aim to prevent dual use, but without more questions/analysis on *why* participants feel this way, it is not clear that a checklist solution would have any benefits.\n\nW4. Relatedly, the complicated ethical questions around dual use are challenging to address in a survey; long-form interviews may be more appropriate for studying this issue. However, I do appreciate that the authors used some free-response questions in the survey.\n\n**Edit after rebuttal**: the authors have addressed my concern about the contribution (W1), and thus I have raised my excitement score (2->3).", "questions_to_authors": "A. Can you clarify which type of inductive coding you perform (e.g. is this thematic analysis?) and share any resulting codebook?\n\nB. Can you comment on the choice of a checklist in light of recent critique of checklist-based approaches to ethics? For instance, the critiques of Widder & Nafus [1]:  \"asking developers to anticipate every conceivable outcome by diligently following elaborate checklists as if they occupied a view from nowhere (what Gansky and McDonald (2022) call “metadata maximalism”) does not portend a meeting of partial knowledges.\"\n\n[1] Widder, D. G., & Nafus, D. (2023). Dislocated accountabilities in the  “AI supply chain”: Modularity and developers’ notions of responsibility. Big Data & Society, 10(1). https://doi.org/10.1177/20539517231177620\n\nC. The number of full responses is a small fraction of the total respondents (48 of 256). What areas did partial responses leave unfinished, and would it be possible to use some of the data from these partial responses?\n\nD. What do you see as the key contribution of your definition of dual use, in comparison to existing definitions?", "paper_topic_and_main_contributions": "The paper studies dual use in an NLP context. The authors propose a definition of dual use that emphasizes the reuse of technology with intent to harm. They conduct a survey of NLP practitioners that asks about their proposed definition and prompts practitioners to speculate about potential dual use in their research area. They propose an ethics checklist focused on dual use.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 354: broken reference\n\nBecause of the large images (screenshots) in the appendix, the PDF file size is quite large."}, "extra_text": {"Missing_References": "work that discusses dual use in NLP/related fields:\n* Koplin, J.J. Dual-use implications of AI text generation. Ethics Inf Technol 25, 32 (2023). https://doi.org/10.1007/s10676-023-09703-z\n* K. Nikolskaia and V. Naumov, \"Ethical and Legal Principles of Publishing Open Source Dual-Purpose Machine Learning Algorithms,\" 2020 International Conference Quality Management, Transport and Information Security, Information Technologies (IT&QM&IS), Yaroslavl, Russia, 2020, pp. 56-58, doi: 10.1109/ITQMIS51053.2020.9322897.\n* Peter Henderson, Eric Mitchell, Christopher Manning, Dan Jurafsky and Chelsea Finn. \"Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models.\" AIES 2023.\n\nwork that discusses ethics interventions/thinking about dual use in NLP/related fields:\n* Widder, D. G., & Nafus, D. (2023). Dislocated accountabilities in the “AI supply chain”: Modularity and developers’ notions of responsibility. Big Data & Society, 10(1). https://doi.org/10.1177/20539517231177620\n* MAHFOUD, T., AICARDI, C., DATTA, S., & ROSE, N. (2018). The Limits of Dual Use. Issues in Science and Technology, 34(4), 73–78. https://www.jstor.org/stable/26597992"}}, {"review_id": "DTT0oAMnVA", "reviewer": "Reviewer_AsLm", "summary": "", "questions": "A. I believe the added exclusion of \"unintended secondary harms\" is rather vague. In most cases, I think it would be these harms that are identified by people with malicious intents and then further refined or channeled into dual use. Would that not mean that the difference between dual use and \"unintended secondary harms\" lies only in the intentionality?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their effort and insights through the review. We will integrate the feedback in the manuscript.\n\n**A.** To address the reviewer’s point that “*researchers often fail to see past their own intention and can easily overlook creative ways in which the technology can be misused.*” We believe that that is exactly why documentation of the researchers’ intention and standardised protocol in the peer review process is needed. The checklist documents the author’s intended use case and helps them think about ways in which their work could be misused and inculcate mechanisms for protection against them. This process does not ensure that all harms will be considered, but we think it plays an important role in putting systematic deterrents to known malicious use cases and in the identification of similar ones in the future.\n\n**B.** We believe that the space for a long paper was necessary to give the different components of the manuscript (the definition, survey, the checklist, and existing interventions for dual use) their due attention in analysing such a nuanced topic, each of which play an important role in getting a holistic overview of the topic. Further, we wanted to give space to the description of the survey to let the reader follow on how the proposed checklist was created.\n\n### Questions For The Authors\n\n**A.** Intentionality is precisely the distinction we make between dual use and \"unintended secondary harms\". We believe intentionality is a core component when it comes to understanding the nature of harms of NLP artefacts. Thus, the devised mitigation methods should explicitly factor in the intentions of a potential malicious actor. Currently, work on ethics in NLP primarily focuses on unintended secondary harms [1,2,3,4]. While these are very important topics to address, it also shows the lack of discussion with regards to dual use. \n\n[1] Weidinger, Laura, et al. \"Taxonomy of risks posed by language models.\" Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 2022.\n\n[2] Dev, Sunipa, et al. \"On Measures of Biases and Harms in NLP.\" Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022. 2022.\n\n[3] Hardmeier, Christian, et al. \"Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP).\" Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP). 2022.\n\n[4] Blodgett, Su Lin, et al. \"Stereotyping Norwegian salmon: An inventory of pitfalls in fairness benchmark datasets.\" Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A. The topic of the paper is highly relevant as it discusses how to prevent unethical (dual) use of NLP technology. This is an issue that is not sufficiently researched.\n\nB. The paper concretely proposes a checklist that can be an important step towards building solid ethical guidelines.\n\nC. The work is well-written and the provided argumentation is clear.\n\nD. Great title", "reasons_to_reject": "A. The definition of dual use is bound to remain vague in some sense, as it relies on the intention of the creator and the user. Additionally, researchers often fail to see past their own intention and can easily overlook creative ways in which the technology can be misused.\n\nB. The paper is rather long. I believe the core message of the paper could also have been written down in a short paper.", "questions_to_authors": "A. I believe the added exclusion of \"unintended secondary harms\" is rather vague. In most cases, I think it would be these harms that are identified by people with malicious intents and then further refined or channeled into dual use. Would that not mean that the difference between dual use and \"unintended secondary harms\" lies only in the intentionality?", "paper_topic_and_main_contributions": "In this paper, the authors discuss to what extent the NLP community is currently combating dual use, the unintended misuse of technology. Based on their latest survey, they argue that the members of the NLP an CL communities are not sufficiently aware of the dangers of dual use and discuss how this issue can be addressed in the future. The primary contribution of this paper is a checklist that can be a useful tool or guide for researchers in the community. Additionally, the topic of this paper is rather novel and therefore, this paper can spark interesting and relevant discussions about the future of NLP at conferences. Finally, the authors also present some insights into the general opinion of the research community (regarding ethics and dual use).", "excitement": 4, "reproducibility": null, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Missing reference on line 354", "Typos_Grammar_Style_and_Presentation_Improvements": "112: “a survey of NLP professional” → professionals\n227 “survey survey” duplicate word\n256 “we which” → missing word"}, "extra_text": {}}, {"review_id": "UsbUqlrlrf", "reviewer": "Reviewer_chMo", "summary": "", "questions": "See previous section on this.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Regarding the **first example**, we do not discuss whether the researchers should have the “right” to produce harmful artefacts, but that they should consider the consequences of their artefacts on society. They can, consequently, decide that the possible harms are reasonable compared to the positive impact of their applications. We ask the reviewer to clarify what their definition of legitimate interest is, since that would entail whether the harm falls under our definition of dual use. Our work provides an operating definition of a kind of harm that we argue should be given more attention and initiate a discussion of researchers along this topic. \n\n**Second example**: Indeed we do not make that distinction. However, in the context of our discussion, we believe that distinction is irrelevant since the intention of the researcher and the primary domain are documented and released with the artefact. Our work focuses on establishing systematic guardrails and providing the researchers with tools to predict and prevent a malicious actor's use of said artefact.\n \nRegarding the **third example**, the issue of dual use is well-defined and described as problematic in the related literature at large [1,2,3]. Our paper is concerned with adapting the topic of dual use to the domain of NLP, not providing a discussion on whether malicious reuse is problematic from an ethical or moral perspective.\n\n[1] Urbina, Fabio, et al. \"A teachable moment for dual-use.\" Nature machine intelligence 4.7 (2022): 607-607.\n\n[2] Forge, John. \"A note on the definition of “dual use”.\" Science and Engineering Ethics 16 (2010): 111-118.\n\n[3] Miller, Seumas, and Michael J. Selgelid. \"Ethical and philosophical consideration of the dual-use dilemma in the biological sciences.\" Science and engineering ethics 13 (2007): 523-580.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper makes an original contribution in that the problems associated with dual use of NLP techniques (called 'artefacts' in the paper) have not been properly explored in the relevant literature. This provides a reason to accept the paper.", "reasons_to_reject": "The paper gives several reasons to reject it. First, it is conceptually rather sloppy. Let me give a few examples to back this assessment. Throughout, authors use the concept of 'harm' in an unproblematised way, defined roughly as 'adverse consequences on someone' (the standard definition here would be 'setback of people's interests'). But this neutral concept of harm notoriously does not give reasons, in itself, to others to stop harming people. Only individuals' LEGITIMATE interests count as 'harm' in this second, reason-giving, way. The difference is huge and reveals a big flaw in the paper. So the mere fact that I shall do something that 'harms' you in the first way does not mean that I do not have a right to do it. I only lack the right if your interest is LEGITIMATE. By not attending to this distinction, the authors blur different things that should be kept separate and misidentify both 'malicious intent' and 'harm'. Second example: the authors do not distinguish between a primary domain of application of an NLP artefact and INTENTIONS that an artefact apply to a primary domain. These, again, are not the same. Third example: the authors do not even strive to identify a moral principle that would make their guidelines and discussion plausible. If, however, someone asks 'why exactly is dual use a problem to begin with?', the authors provide no answer. They take it for granted that it does. But that in itself rests (probalbly) on consequentialist grounds (i.e., that agents should avoid actions that lead to suboptimal - under some criterion of value - consequences). The problem is that this principle is very controversial. Some discussion would help, but the authors do not even seem aware that there is an issue here.\n\nADDENDUM AFTER REBUTTALS: (a) A legitimate interest is a morally relevant interest, whereas a non-legitimate interest is a morally irrelevant interest. Let me give you some prosaic examples to drive the point home. My interest in gaining profits from my company is a legitimate interest in the sense that I have private property in my company, so that of you interfere with it against my will I have a moral claim against you that you stop, but NOT a legitimate interest in the sense that you can open your own company across the road and 'harm' me through competition. Clearly there are adverse consequences for me in the second scanario, but, in a morally relevant sense, you do not harm me. In the same way, I might be sexually attracted to X and s/he might turn me down. Turning me down can have all sorts of adverse consequences, but is morally irrelevant. In quite the same way, if I do not have a right (say) that information about me in the public domain be used (say) to terminate my employment, precisely because (say) there is no 'privacy' in the publlic domain, then adverse consequences do not amount to harm (in the relevant moral sense). .Since harm in the first sense (adverse consequences) is morally neutral, your argument overshoots. To put it provocatively: there is no moral problem with dual use of NLP-artifacts that merely harms people in the sense of non-morally relevant setback of people's interests.\n(b) The difference is important because someone might create without harmful intent an NLP-artifact whose primary domain harms people. Does that the use of that artifact to harm people count as dual use?\n(c) Sure, but I have tried to say that it's only 'problematic' in the morally-relevant sense, not the morally-neutral one. Again: just because something bad (i.e. something that sets back my interests) happens to me does not mean that someone 'harmed' me in the morally relevant sense. If that's the case, then taking a stance on why dual-use is wrong (presumably because people's legitimate interests can be set back by using NLP artifacts not designed and/or primarily used to set those interests back0 could help identify a normatively proper checklist.", "questions_to_authors": "See previous section on this.", "paper_topic_and_main_contributions": "The paper's topic is about dual use and its risks in NLP. A specific definition of dual use is provided and the paper ends with a dual use checklist. Along the way, empirical results as to the attitudes of members of the NLP community vis-a-vis dual use are provided.", "excitement": 3, "reproducibility": null, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "I do not know af any references that should be included, but that is not the main issue.", "Typos_Grammar_Style_and_Presentation_Improvements": "There are lots of typos throughout, and the paper would benefit from extended proofing."}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors formulate an NLP specific definition for the dual use and propose a checklist to be reflected on in ethical considerations of NLP tools. This checklist can be used in addition to a datasheet or model card or other related artefacts to make a complete assessment. The work identifies a major gap in our current discourse of harmful impact of language technologies, and lays foundational work towards defining and checking for the same.
The survey of the NLP practitioners is particularly interesting as a contribution and helps reflect on the state of models being released. The checklist however seems a little under detailed. Some more thought on what one can do with the answers recorded, as well as the agency and knowledge of an NLP researcher to reliably answer them are also points that could be reflected upon in the paper some more, since the checklist is also counted by the authors as a core contribution. 
All reviewers in their reviews and comments post rebuttal concur that the paper makes foundational progress towards consideration of dual use in NLP, and with the minor adjustments recommended (such as expansions of terms such as 'secondary unintended harms', image and text typos, mistyped citations in line 354) will consist of significant insights for the community.
Decision: Accept-Findings