‘Just What do You Think You’re Doing, Dave?’ A Checklist for Responsible Data Use in NLP (2021)

Abstract: A key part of the NLP ethics movement is responsible use of data, but exactly what that means or how it can be best achieved remain unclear. This position paper discusses the core legal and ethical principles for collection and sharing of textual data, and the tensions between them. We propose a potential checklist for responsible data (re-)use that could both standardise the peer review of conference submissions, as well as enable a more in-depth view of published research across the community. Our proposal aims to contribute to the development of a consistent standard for data (re-)use, embraced across NLP conferences.

## introduction in nlp, as in other research areas that are heavily data-driven, it is impossible to overstate the significance of the creation, collection, storage, and use of data @xcite . machine learning models are fundamentally a way of storing (and ideally generalising over) training data in the form of latent representations. the vast majority of nlp tasks involve supervised training of some sort. ideally, the training data would come from the real-world, be of high quality, diverse, available in large quantities, obtained with revocable consent for specific use cases, and in compliance with licensing and other legal obligations, as well as broader ethical considerations. unfortunately, many of these considerations are often overlooked, to the potential detriment of both quality and integrity of the resulting datasets and models. a key issue is that the legal frameworks vary from jurisdiction to jurisdiction, as do the workplace cultures and norms around research institutions.we may not even have established definitions for commonly used terms like "fairness" @xcite and "bias" @xcite . the result is differing expectations and standards, which makes it harder for fellow researchers to understand the terms of (re-)use of new resources and models, and also complicates peer review. experts in biomedical data research have concluded that "truly informed consent ... requires (1) comprehensive disclosure of informational risks to participants, (2) independent governance entities, and (3) data sharing policies that offer guidance for physicians and researchers" @xcite . the ai community has already started to work on the latter @xcite @xcite . unfortunately, the codes of conduct and promulgation of ethics by industry, government, and academia remain highly contextual, and have "substantive divergence in relation to how these principles are interpreted, why they are deemed important, what issue, domain or actors they pertain to, and how they should be implemented" @xcite . further, a more cynical, albeit well-founded view, is that much of the flurry of activity around codes of conduct and ethics has been to avoid regulation, and is heavily driven by industry @xcite . in nlp, the association for computational linguistics (acl) has adopted the acm code of ethics @xcite , conducted a series of workshops @xcite , and, most recently, implemented ethics reviews and submission guidelines. 1 but we are far from a set of rules that would account for all the diverse types of nlp data and applications. initial discussions after the first round of ethics review in naacl 2021 highlighted many disagreements. we submit that the community needs to work towards a common standard for what constitutes responsible use of data, what review mechanisms should be in place to ensure it, and what should happen when that standard is not upheld. ideally, the discussion would advance collectively and with representation across different sub-populations of the field, as well as other stakeholders. hopefully this process would result in more comprehensive disclosure, independent governance structures, and clearer policies that reflect a common purpose by the nlp community. to this end, we analyse the core legal and ethical principles of data collection, and distill bestpractice recommendations into a (non-exhaustive) checklist for potential adoption/adaptation by nlp conferences. above all, we hope that this paper will advance discussion towards a single standard of responsible data (re-)use in nlp. ## key principles of data collection we do not yet have unified standards for responsible data use, but there are several guiding principles upon which we widely agree. this section briefly discusses such principles and the interactions between them. ## tension between data collection principles while all of the above principles are important, there is a tug-of-war between them, as well as tensions with other factors in the research environment. the reproducibility principle aims to maximise data sharing in the interest of open science, while the others all limit it from ethical and legal perspectives. the transparency principle means that the data processors have to disclose what they are doing with the data, but that complicates the protection of trade secrets. the privacy principle is sometimes fundamentally at odds with public interest (crime, harassment, and accountability for public figures and organisations). reproducibility and preservation of records of public interest may be in conflict with data subject privacy, the right to be forgotten, client confidentiality, and legitimate business objectives of industry research. tos of individual platforms may further crosscut these principles: e.g. twitter's requirement that only tweet ids may be distributed maximises user control over their data, but it makes anonymisation impossible, and also sacrifices reproducibility (due to data attrition). add to that the differences in national legislation, cultural norms, and priorities of individual researchers, and it is evident that there are no clear solutions that simply follow from all the above principles. the result is that different research communities have come up with different norms and combinations of these principles, depending on their agenda, the power differential of data controllers and data subjects, and simply the goodwill of researchers willing to volunteer time to engage in unheralded administrative and advocacy work to change the norms. as an example, consider the tension between privacy and open science in medical research @xcite . fundamentally, patient data has to be kept private, and most clinical studies do not publish it. this, however, prevents joint analysis of data from independent clinical trials, which could yield critical insights and literally save lives of other patients. one solution is adaptive clinical trial platforms @xcite , which enable cross-institutional coordination and data sharing in ongoing trials. but currently such initiatives rely largely on volunteer service work by researchers @xcite . the result is patchy: for example, there is a platform for consolidating pancreatic cancer research across institutions, foot_4 but not for lung cancer. 10 nlp is not an exception: as an interdisciplinary field, it has a mixture of players with different priorities and incentive structures, who argue for different combinations of the above principles, in part predicted by the domain they work in (e.g. clinical nlp with a strong focus on privacy and transparency vs. machine learning for nlp with a strong focus on reproducibility). more recently, as part of the rush to develop and distribute pretrained language models, there has arguably been an over-focus on data volume and too little focus on any of the data principles, including the use of corpora of dubious legality and quality such as bookcorpus @xcite . ## responsible data use checklist the conflicting principles of data collection, coupled with conflicting priorities of different researchers, make for a torn, confused community. the movement towards standardising ethical and legal expectations has already started: neurips 2020 pioneered obligatory broader impact statements inspired by @xcite , and naacl, acl-ijcnlp and emnlp 2021 adopted a similar ethics policy, with a separate committee ethically reviewing papers flagged by regular reviewers. in parallel with all that, the *acl conferences now use an adapted version of the reproducibility checklist @xcite . in this paper, we are hoping to refine the conversation regarding responsible data use, collection and distribution, as one thread of a richer dialog about nlp ethics. to this end, we propose to re-structure the host of issues that the paper authors are asked to consider into three areas: our proposal draws on the gdpr framework, naacl ethics guidelines, and work on documenting the populations represented in nlp resources @xcite . while by no means a complete solution, we believe that it is a useful starting point. the main section of the checklist focuses on the safe use of data, with many questions overlapping with those expected in institutional ethics review (see section 5). it also has a section dedicated to safe reuse, which aims to nudge the resource authors towards specifying limitations and safe use cases. at the same time, the model developers are nudged towards specifying whether their use is consistent with any such limitations. a key provision of safe resuse is ensuring consistent terms of data sharing. in many cases researchers are allowed access to data that would not be granted to commercial entities, under the provision that this is done solely for research purposes, or for 'social good'. for example, based on article 89 of gdpr and their local legislation, eu-based researchers may be exempted from observing some data subject rights "for scientific or historical research purposes or statistical purposes" and research performed "in public interest". if researchers gain access to data under such provisions, and then publish the data (or models derived from that data) under licenses allowing commercial use, this creates a loophole. for papers using a previously-published resource: the authors explain their choice of data, given the available resources and their known limitations (e.g. representativeness issues, biases, annotation artifacts) and any data protection issues (e.g. inclusion of sensitive health data). see section . the authors discuss whether their use of a previously-published resource is compatible with its original purpose and license, and any known limitations (e.g. if the target user group is represented in the sample). see section . for papers contributing a new resource: the authors have the legal basis for processing the data and, if it is made public, for distributing it. (check one) the data is used with consent of its creators or copyright holders; 1.3. if the data is used without consent, the paper makes the case to justify its legal basis (e.g. research performed in the public interest under gdpr). see section . ## challenges the above checklist is by no means exhaustive in terms of legal and ethical considerations in nlp projects, and does not solve all the problems. in particular, the following topics require a lot more legal thinking in the community. web crawled data nlp models are often pretrained on large volumes of web crawled text: from 6b words of news data in word2vec @xcite to 42b words of common crawl data in glove @xcite , and now 300b tokens in gpt3 @xcite , and counting @xcite . these studies, like any other, have to have a legal basis for data collection, storage, and processing. gdpr recognises "public interest" as a legal basis for processing of personal data (in this case, data attributable to individuals through public records of authorship). but most current largescale models are trained by industry labs and made available for commercial use. the only possible basis for such training is the business' "legitimate interest", but then it needs to outweigh the interests of the data subjects, who may suffer significant harms @xcite furthermore, those involved in data storage and processing still have to comply with the privacy principles, including the protection of personal data. there is ongoing work in this direction in the big science workshop. 15 all the above only concerns privacy legislation, which copyright law is orthogonal to. copyright would be directly violated by the act of copying the data for training, and also potentially by the fact of storing the protected text in trained models. with static word embeddings, it could be argued that the model itself does not "store" any text, but the current generation of language models clearly do reconstruct excerpts of training data. since this is a blackbox process, it is hard to tell how much "creativity" there is in any particular sample, but the worst-case scenario is that such models could make their users potentially liable for plagiarism. legacy data many datasets that are currently in use have been released with little consideration for copyright, speaker demographics, or proper handling of personal information @xcite . since such resources have been published in prestigious venues, some nlp researchers might feel that there is precedent to use them and build new resources in a similar way. the ongoing use of such data raises many questions. it is not realistic to prohibit its use overnight, but, if a responsible data-use standard is established, it should give rise to a new generation of resources that do not have such problems. in the short term, such a standard should also stimulate efforts to address documentation debt (bandy and vincent, 2021; dodge et al., 15 bigscience.huggingface.co 2021), as well as analysis and cleaning of the older resources (similarly to the problem of removal of pornographic images that the computer vision community is tackling 16 ). note that our checklist includes a section for previouslypublished resources, which should nudge authors towards more carefully documented data (since it would be easier to motivate). ## conclusion as nlp technologies become more mainstream and commercialised, it is increasingly important that the nlp community leads best practice in legal and ethical standards. this paper is a step in that direction. we propose a tentative checklist for responsible data (re-)use, intended to serve as a complement to the existing reproducibility checklist. we hope that the community will continue working towards a common standard for data collection and sharing, complemented with transparent review mechanisms to ensure that the standard is upheld.