SlowGuess/NAACL_2021_OCR huggingface dataset:
5.1 Researchers Cannot Exempt Themselves from an IRB Application
The Final Rule includes provisions for IRB exemptions, and we expect the vast majority of crowdsourced NLP tasks to fall into that category. However, it is crucial to understand that granting a research project an IRB exemption is not the prerogative of the researcher; it is only IRB that hands out exemptions following an initial review:
[T]he determination of exempt status (and the type of review that applies) rests with the IRB or with an administration official named by your institution. The determination does not rest with the investigator. Therefore, all projects must be submitted to the IRB for initial review. (American Association for Public Opinion Research, 2020)
5.2 Worker IDs Constitute IPI
Researchers often obtain and store worker IDs — unique and persistent identifiers assigned to each worker by the crowdsourcing platform — even when workers are “anonymous”. MTurk, for example, assigns each worker a fixed 14-digit string, which is provided to the researcher with completed tasks. A worker ID is part of the worker’s account, and is therefore linked to their personal details, including full name, email address, and bank account number. As a consequence, the worker ID constitutes IPI (identifiable private information). If the worker ID is obtained by the researcher, the research mandates an initial IRB review.
To avoid obtaining this IPI, researchers can create and store pseudonymized worker IDs, provided that these IDs cannot be mapped back to the original worker IDs.
5.3 Obtaining Anonymous Data Does Not Automatically Absolve from IRB Review
Even if IPI is not obtained, and only anonymous and non-identifiable data is collected, we have
Some universities employ the Precautionary Principle and require all crowdsourced-enabled research to go through an IRB application.
shown in Section 4.2 that crowdsourced NLP tasks involve interaction between researcher and participant through which the data about the worker may be collected, and thus often require an initial review by an IRB.
5.4 Payment to Crowdworkers Does Not Exempt Researchers from IRB Review
Remuneration of human subjects does not change their status to independent contractors beyond the scope of research ethics. In fact, compensation of human subjects for the time and inconvenience involved in participating is a standard practice "especially for research that poses little or no direct benefit for the subject" and at the same time "should not constitute undue inducement to participate", as the University of Virginia, Human Research Protection Program (2020) points out.
5.5 Non-Published Research Also Requires IRB Review
Some researchers believe that research that will not be published is not subject to an IRB review. For example, Carnegie-Mellon University, Language Technologies Institute (2020) teaches students that "Paid labeling does not require IRB approval... [b]ut sometimes you want to discuss results in papers, so consider IRB approval".<sup>9</sup>
The definition of research in the Final Rule is not contingent on subsequent publication of the results. Given the uncertainties of the peer-review process, whether research eventually finds its way into a publication can often be ascertained only ex post. An important exception not requiring IRB approval is student work as part of course assignments. However, subsequent use of research data collected originally as part of a student assignment is not mentioned in the Final Rule. Consequently, universities handle this situation differently. For example, University of Michigan allows retroactive IRB approval:
Class assignments may become subject to this policy... if the faculty member or the students change their plans... application to the IRB for permission to use the data is required. (University of Michigan, Research Ethics and Compliance, 2021)
while Winthrop University does not:
IRB approval cannot be granted retroactively, and data may need to be recollected for the project. (University of Winthrop, The Office of Grants and Sponsored Research Development, 2021)
6 Risks and Harms for Crowdworkers
Previous work on the ethics of crowdsourcing focused on labor conditions, such as fair pay, and on privacy issues (Fort et al., 2011; Gray and Suri, 2019). However, even when payment is adequate and the privacy of the workers is preserved, there are additional ethical considerations that are often not taken into account, and might put the worker at risk. We propose using the three ethical principles outlined by the Belmont Report — Respect for Persons, Beneficence, and Justice — to guide the action of researchers. We outline some of the specific risks and harms that might befall NLP task crowdworkers in light of these principles. While the list is not comprehensive, it can serve as a starting point to be used by researchers when planning their crowdsourced task, as well as by reviewers examining the ethical implications of a manuscript or research proposal.
6.1 Inducing Psychological Harms
NLP researchers are increasingly cognizant that texts can potentially harm readers, as evident by trigger warnings they add to their own papers (e.g., Sap et al., 2020; Nangia et al., 2020; Sharma et al., 2020; Han and Tsvetkov, 2020). Moreover, researchers have long known that annotation work may be psychologically harmful. The Linguistic Data Consortium (LDC), for example, arranged stress-relieving activities for annotators of broadcast news data, following reports of “negative psychological impact” such as intense irritation, overwhelmed feelings, and task-related nightmares (Strassel et al., 2000). Although literature on the emotional toll on crowdworkers is still scant (Huws, 2015), there is growing literature on the psychological cost of work done by commercial content moderators (Steiger et al., 2021). Crowdworkers deserve similar consideration: while NLP tasks can be as benign as the POS tagging of a children's poem, they may also involve exposure to disturbing textual or visual content.
In 45 CFR 46.110 the Final Rule allows expedited IRB review for research posing no more than
9Moreover, whether the labeling is paid or unpaid is irrelevant; see Section 5.4.
minimal risk to the human subjects involved. According to 45 CFR 46.102(j) minimal risk "means that the probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine [...] psychological examinations or tests." Exposing crowdworkers to sensitive content may exceed the threshold of minimal risk if the data that requires labeling or evaluation might be psychologically harmful.
The amount of harm (if any) depends on the sensitivity of the worker to the specific type of content. The risk is generally higher in data labeling or text evaluation tasks, where workers might be repeatedly asked to categorize offensive tweets, transcribe violent texts, evaluate hateful text that may expose them to emotional stimuli, or, depending on the content and worker, traumatize them. In some cases, sexual material can be highly offending or shocking and cause an emotional disturbance. Although less likely, hazards can also occur when workers are asked to produce text, since workers are elicited to produce texts based on given input. For example, when users are asked to compose a story based on images, certain images might trigger a harmful response in the worker.
6.2 Exposing Sensitive Information of Workers
A crowdworker might inadvertently or subconsciously expose sensitive information about themselves to the researcher. This is more pronounced in text production, where the responses produced by such tasks reveal as much about the individual workers as they do about the produced text. However, workers also reveal information about themselves when evaluating or labeling text, especially when subjective labeling is in place. Moreover, even seemingly trivial data — for example, the elapsed time taken by the worker to label, evaluate, or produce text — may contain valuable information about the worker (and this information is automatically captured by MTurk). Table 3 shows the risk level for the different task categories.
Moreover, researchers can obtain sensitive information about workers because the crowdsourcing platforms allow screening of workers using built-in qualification attributes including age, financial situation, physical fitness, gender, employment status, purchasing habits, political affiliation, handedness,
Task Category	Risk of Exposure of Workers' Sensitive Information
Objective Labeling	Low
Subjective Labeling	Medium
Evaluation	Medium
Production	High
Table 3: Potential risk level by task category.
marital status, and education level (Amazon Mechanical Turk, 2020).
Researchers may also obtain other types of sensitive information by creating their own, arbitrary qualification tests as a quality control measure (Daniel et al., 2018).
In summary, the information obtained by researchers may reveal as much about the individual crowdworkers as they do about the data being labeled, evaluated, or produced.
6.3 Unwittingly Including Vulnerable Populations
Research on crowdsourcing platforms such as MTurk is inherently prone to the inclusion of vulnerable populations. In 45 CFR 46.111(b) the Final Rule (2018) non-exhaustively lists "children, prisoners, individuals with impaired decision-making capacity, or economically or educationally disadvantaged persons" as vulnerable groups. The Final Rule requires that additional safeguards are implemented to protect these vulnerable populations and makes IRB approval contingent on these safeguards.
A great proportion of MTurk crowdworkers are located in developing countries, such as India or Bangladesh, which makes MTurk an attractive proposition to those offering a task (Gray and Suri, 2019), but increases the risk of including economically-disadvantaged persons. Furthermore, it is difficult to ensure that MTurk crowdworkers are above the age of majority, or fall into any other of the defined vulnerable populations (Mason and Suri, 2011).
Moreover, given the power imbalances between researchers in industrialized countries and crowdworkers in developing countries, ethical consideration should occur regardless of whether the jurisdiction in which the crowdworkers are located even has a legal framework of research ethics in place, or whether such a local framework meets the standard of the Belmont Report or Final Rule.
6.4 Breaching Anonymity and Privacy
There is a perception among researchers that crowdworkers are anonymous and thus the issue of privacy is not a concern. This is not the case. Lease et al. (2013), for example, discuss a vulnerability that can expose the identity of an Amazon Mechanical Turk worker using their worker ID — a string of 14 letters and digits — because the same worker ID is used also as the identifier of the crowdworker's account on other Amazon assets and properties. As a result, a Google search for the worker ID can lead to personal information such as product reviews written by the crowdworker on Amazon.com, which in turn can disclose the worker's identity. Researchers might be unaware of these issues when they make worker IDs publicly available in papers or in datasets. For example, Gao et al. (2015) rank their crowdworkers using MTurk worker IDs in one of the figures.
Moreover, breaches of privacy can also occur unintentionally. For example, workers on MTurk are provided with an option to contact the researcher. In this case, their email address will be sent to the researcher, who is inadvertently exposed to further identifiable private information (IPI). We maintain that the anonymity of crowdworkers cannot be automatically assumed or guaranteed, as this is not a premise of the crowdsourcing platform.
6.5 Triggering Addictive Behaviour
Graber and Graber (2013) identify another source for harmful effects, which ties in with the risk of psychological harm, and is specific to gamified crowdsourced tasks: a possibility of addiction caused by dopamine release following a reward given during the gamified task. Gamification techniques can be added to data labeling, evaluation, and production. Indeed, some NLP work is using gamification, mostly for data collection (e.g., Kumaran et al., 2014; Ogawa et al., 2020; Ohman et al., 2018).
Moreover, the crowdsourcing platform may add elements of gamification over which the researcher has no control. For example, MTurk recently introduced a "Daily Goals Dashboard", where the worker can set game-like "HITs Goal" and "Reward Goal", as shown in Figure 1.
Figure 1: The "Daily Goals Dashboard" on MTurk
7 Ways Forward
The use of crowdworkers is growing within the NLP community, but the ethical framework set in place to guarantee their ethical treatment (whether de jure or de facto) did not anticipate the emergence of crowdsourcing platforms. In most crowdsourced NLP tasks, researchers do not intend to gather information about the worker. However, the crowdsourcing platform often autonomously collects such information. As a result, it is often difficult to determine whether crowdworkers constitute human subjects — which hinges on whether the researcher collects information about the worker. However, a determination that all crowdworkers are human subjects — and thus mandate an IRB approval for government-supported institutions — might create a "chilling effect" and disadvantage university researchers compared to industry-affiliated researchers. The effect is exacerbated in institutions where the ethics committee is heavily bureaucratic and does not offer a streamlined, expedited exemption process for low-to-no risk studies.
Whether their crowdsourced task requires IRB application or not, we recommend that the ethics-aware researcher should carefully examine their study in light of the three principles set up by the Belmont Report: Respect for persons, Beneficence, and Justice. And while this mandates fair pay, it is important to note that this is just one of the implications. There are other ethical considerations that are often overlooked — in particular risk assessment of causing psychological harm and exposure of sensitive information. Thus, we recommend increasing awareness of the potential ethical implications of crowdsourced NLP tasks. As NLP researchers are now encouraged to add an "ethical considerations" section to their papers (NAACL, 2020), they should also be encouraged to carefully weigh potential benefits against risks related to the crowdsourced task.
We also propose increasing awareness by disseminating relevant knowledge and information. An educational ethics resource created using a community effort could serve as a beneficial first step. Such a resource can include guidelines, checklists, and case studies that are specific to the ethical challenges of crowdsourced tasks in the context of NLP research. We believe that the creation of such a resource can serve as a springboard for a necessary nuanced conversation regarding the ethical use of crowdworkers in the NLP community.
We thank the anonymous reviewers for their valuable comments and suggestions which helped improve the paper. This research was partially supported by the Ministry of Science and Technology in Taiwan under grants MOST 108-2221-E-001-012-MY3 and MOST 109-2221-E-001-015- [sic].
Amazon Mechanical Turk. 2020. Amazon Mechanical Turk Pricing. https://web.archive.org/web/20201126061244/https://requester.mturk.com/pricing. Accessed: 2020-11-26.
American Association for Public Opinion Research. 2020. IRB FAQs for Survey Researchers. https://www.aapor.org/Standards-Ethics/Institutional-Review-Boards/IRB-FAQs-for-Survey-Researchers.aspx. Accessed: 2020-11-11.
Emily M. Bender, Dirk Hovy, and Alexandra Schofield. 2020. Integrating ethics into the NLP curriculum. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 6-9, Online. Association for Computational Linguistics.
Adrian Benton, Glen Coppersmith, and Mark Dredze. 2017. Ethical research protocols for social media health research. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 94-102.
Allan M. Brandt. 1978. Racism and research: The case of the Tuskegee syphilis study. The Hastings Center Report, 8(6):21-29.
Arthur L. Caplan. 1992. When evil intrudes. The Hastings Center Report, 22(6):29-32.
Alexander M. Capron. 2008. Legal and regulatory standards of informed consent in research. In Ezekiel J. Emanuel, Christine Grady, Robert A. Crouch, Reidar K. Lie, Franklin G. Miller, and David Wendler, editors, The Oxford Textbook of Clinical Research Ethics, pages 613-632. Oxford University Press, New York, NY.
Carnegie-Mellon University, Language Technologies Institute. 2020. Human subjects research determination worksheet. https://web.archive.org/web/2020111132614/http://demo.clab.cs.cmu.edu/11737fa20/slides/multilingual-21-annotation.pdf. Accessed: 2020-11-11.
Committee on Publication Ethics. 2020. https:// publicationethics.org. Accessed: 2020-11-22.
Council for International Organizations of Medical Sciences. 2002. International Ethical Guidelines for Biomedical Research Involving Human Subjects. https://cioms.ch/wp-content/uploads/2016/08/International_Ethical_Guidelines_for_Biomedical_Research_Involving_Human_Subjects.pdf. Accessed: 2021-04-06.
Florian Daniel, Pavel Kucherbaev, Cinzia Cappiello, Boualem Benatallah, and Mohammad Allahbakhsh. 2018. Quality control in crowdsourcing: A survey of quality attributes, assessment techniques, and assurance actions. ACM Comput. Surv., 51(1).
Tao Ding and Shimei Pan. 2016. Personalized emphasis framing for persuasive message generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1432-1441, Austin, Texas. Association for Computational Linguistics.
Final Rule. 2018. Protection of Human Subjects, Part 46. In US Department of Health and Human Services, Code of Federal Regulations (CFR) Title 45.
Karen Fort, Gilles Adda, and K Bretonnel Cohen. 2011. Amazon mechanical turk: Gold mine or coal mine? Computational Linguistics, 37(2):413-420.
Mingkun Gao, Wei Xu, and Chris Callison-Burch. 2015. Cost optimization in crowdsourcing translation: Low cost translations made even cheaper. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 705-713, Denver, Colorado. Association for Computational Linguistics.
Richard Gillespie. 1989. Research on human subjects: An historical overview. Bioethics News, 8(2):s4-s15.
Igor Gontcharov. 2018. Qualitative ethics in a positivist frame: The Canadian experience 1998-2014. In The SAGE Handbook of Qualitative Research Ethics, pages 231-246. SAGE Publications Ltd.
Mark A Graber and Abraham Graber. 2013. Internet-based crowdsourcing and research ethics: the case for IRB review. Journal of medical ethics, 39(2):115-118.
Mary L. Gray and Siddharth Suri. 2019. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Houghton Mifflin Harcourt, Boston, MA.
Xiaochuang Han and Yulia Tsvetkov. 2020. Fortifying toxic speech detectors against veiled toxicity. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7732-7739, Online. Association for Computational Linguistics.
Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591-598, Berlin, Germany. Association for Computational Linguistics.
Ursula Huws. 2015. Online labour exchanges or 'crowdsourcing': Implications for occupational safety and health. European Agency for Safety and Health at Work (EU-OSHA).
Mark Israel. 2015. Research Ethics and Integrity for Social Scientists: Beyond Regulatory Compliance, 2nd edition. SAGE Publications, Thousand Oaks, CA.
Andrew Conway Ivy. 1948. The history and ethics of the use of human subjects in medical experiments. Science, 108(2792):1-5.
Molly Jackman and Lauri Kanerva. 2015. Evolving the IRB: Building robust review for industry research. Wash. & Lee L. Rev. Online, 72:445.
Albert R. Jonsen. 1998. The ethics of research with human subjects: A short history. In Albert R. Jonsen, Robert M. Veatch, and LeRoy Walters, editors, Source Book in Bioethics: A Documentary History, pages 3-10. Georgetown University Press, Washington, DC.
Albert R. Jonsen. 2005. On the origins and future of the Belmont report. In James F. Childress, Eric M. Meslin, and Harold T. Shapiro, editors, Belmont Revisited: Ethical Principles for Research with Human Subjects, pages 3-11. Georgetown University Press, Washington, DC.
Robert L. Klitzman. 2015. The Ethics Police?: The Struggle to Make Human Research Safe. Oxford University Press, Oxford.
A Kumaran, Melissa Densmore, and Shaishav Kumar. 2014. Online gaming for crowd-sourcing phrase-equivalents. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1238-1247.
Matthew Lease, Jessica Hullman, Jeffrey Bigham, Michael Bernstein, Juho Kim, Walter Lasecki, Saeideh Bakhshi, Tanushree Mitra, and Robert Miller. 2013. Mechanical Turk is not anonymous. Available at SSRN 2228728.
Susan E. Lederer. 1995. Subjected to Science: Human Experimentation in America Before the Second World War. Johns Hopkins University Press, Baltimore, MD.
Jochen L. Leidner and Vassilis Plachouras. 2017. Ethical by design: Ethics best practices for natural language processing. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 30-40, Valencia, Spain. Association for Computational Linguistics.