The ethical ambiguity of AI data enrichment:  Measuring gaps in research ethics norms and practices
Authors: Will Hawkins
Brent Mittelstadt
Venue: Submitted to ICLR 2023 (2023)
Keywords: ethics
disclosures
crowdsourcing
data enrichment

Abstract: The technical progression of artificial intelligence (AI) research has been built on breakthroughs in fields such as computer science, statistics, and mathematics. However, in the past decade AI researchers have increasingly looked to the social sciences, turning to human interactions to solve the challenges of model development. Paying crowdsourcing workers to generate or curate data, or ‘data enrichment’, has become indispensable for many areas of AI research, from natural language processing to inverse reinforcement learning. Other fields that routinely interact with crowdsourcing workers, such as Psychology, have developed common governance requirements and norms to ensure research is undertaken ethically. This study explores how, and to what extent, comparable research ethics requirements and norms have developed for AI research and data enrichment. We focus on the approach taken by two leading AI conferences: ICLR and NeurIPS. In a longitudinal study of accepted papers, and a comparison with Springer journal articles and Psychology papers, this work finds that ICLR and NeurIPS have established protocols for human data collection which are inconsistently followed by authors. Whilst Psychology papers engaging with crowdsourcing workers frequently disclose ethics reviews, payment data, demographic data and other information, such disclosures are far less common in leading AI conferences despite similar guidance. The work concludes with hypotheses to explain these gaps in research ethics practices and considerations for its implications.

Under review as a conference paper at ICLR 2023
THE ETHICAL AMBIGUITY OF AI DATA ENRICHMENT: MEASURING GAPS IN RESEARCH ETHICS NORMS AND PRACTICES
Anonymous authors Paper under double-bling review
ABSTRACT
The technical progression of artificial intelligence (AI) research has been built on breakthroughs in fields such computer science, statistics, and mathemat- ics. However, the past decade Al researchers have increasingly looked to the social sciences, turning to human interactions to solve the challenges of model development. Paying crowdsourcing workers to generate or curate data, or 'data enrichment', has become indispensable for many areas of AI research, from nat- ural language processing to inverse reinforcement learning. Other fields that rou- tinely interact with crowdsourcing workers, such as Psychology, have developed common governance requirements and norms to ensure research is undertaken ethically. This study explores how, and to what extent, comparable research ethics requirements and norms have developed for AI research and data enrichment. We focus on the approach taken by two leading conferences: ICLR and NeurIPS, and journal publisher Springer. In longitudinal study of accepted papers, and via. a comparison with Psychology and CHI papers, this work finds that leading Al venues have begun establish protocols for human data collection, but these are are inconsistently followed by authors. Whilst Psychology papers engaging with crowdsoureing workers frequently disclose ethics reviews, payment data, demo- graphic data and other information, similar disclosures are far less common leading AI venues despite similar guidance. The work concludes with hypothe- ses to explain these gaps in research ethics practices and considerations for its implications.
INTRODUCTION
When the creators of the seminal image recognition benchmark, ImageNet, pronounced that the use of Amazon's Mechanical Turk (MTurk) monumental impact crowdsourcing platforms were set to have on Al research (Li) 2019). In the decade that has followed, crowdsourced workers, 'crowdworkers' have been central contributor to machine learning research, enabling low-cost human data collection at scale.
"godsend" for their research, they foreshadowed the
Ethics questions posed by research involving human participants are traditionally overseen by gover- nance groups, such as Institutional Review Boards (IRBs) in the United States (US). Whilst medical fields and social sciences have long history of IRB engagement, the relatively recent rise of crowd- sourcing tasks in AI research means guidelines and norms have been developed in recent years to consider research ethics. The proliferation of guidelines and publication policies have risen along- side critiques of Al crowdsourced work focused on issues such as payment and worker maltreatment.
In response, this study seeks understand how AI research involving crowdworkers engages with research ethics. It does this via an assessment of the expectations put forward by publication venues on researchers, and by analysing how these expectations translate into practices. To make this de- termination the policies and practices of major Al conferences, ICLR and NeurIPS, along with AI research submitted to Springer journals, are reviewed. This is compared with other benchmarks to understand whether Al research at these venues follows norms within more established disciplines. The results show that Al research at these venues involving crowdworkers lacks robust research ethics norms, with venue policies not translated into practice. Whilst ICLR, NeurlPS and Springer

---

Under review as a conference paper at ICLR 2023
provide research ethics guidance, the interpretation of these appears inconsistent, and fails to meet the same standards of disclosure as seen in other fields engaging with crowdworkers.
2 RELATED WORK
Oversight in research involving human subjects is no recent phenomenon, with the Nuremberg Code of 1948 formalising the idea that humans involved in research required protection (Sass) 1983
Shuster 1997p. Research ethics in the United States arose during the 1960s, prompted by various
scandals biomedical rescarch, and followed by scandals in social science studies (Beecher 1966
Stark 2016 Emanuel 2008 Heller Milgram 1963] Zimbardo 1972). These cases led to regulation
standardising Institutional Review Board (IRB) oversight of research involving human subjects, requirement that exists to this day, with similar processes existing in over 80 countries globally (Grady) 2015). IRBs only oversee research involving living 'human subjects', as defined by the
Code of Federal Regulations (Office for Human Research Protections 2017).
2.1 CROWDSOURCING AND AI
In the twenty-firs. first century the scope of research ethics has been extended by the rise of internet research (Taylor 2000). The ability to recruit, engage with, and study human subjects online has led to the rise of crowdsourcing platforms, such as MTuur, becoming a key tool across a variety of academic disciplines (Howe 2006)). Launched in 2005, MTurk was early pioncer of the crowdsourcing model (Cobanoglu et al., 202T). MTurk has remained popular due to its low cost, ease of access, and large user base (Williamson) 2016). The platform has been of particular to the Al field, with Amazon marketing the platform as "artificial artificial intelligence" (Schwartz, 2019; Stephens, 2022).
Whilst early AI crowdwork often involved labelling tasks, such as Fei-Fei Li's seminal ImageNet work, the use of crowdworkers has diversified (Deng al. 2009) Vaughan 2018). Shmueli et al. offer three categories of data collection seen in NLP research papers: (1) labelling, (2) evaluation, and (3) production (Shmueli et al., 202I). For the purposes of this work these categories can be generalised across AI research.
Labelling includes the processing of existing data by a crowdworker and then the selection com- position of a label or labels for that data. Labelling can be objective, for example crowdworkers may be asked to label objects in images (e.g. dogs or cats), or subjective, with one study asking MTurk workers to label their predicted political leanings of images (Thomas & Kovashka 2019). Eval- uation involves an assessment of outputs or data according to predefined criteria, such as fluency. This could be asking humans to provide feedback model-generated language or produce 'mean opinion score' by assessing the outputs of various models (Clark et al. [221]] Défossez et al.) 2018 Stiennon et al. (020). Production studies ask workers produce their own data, rather than label or evaluate existing data. For example, studies might explicitly ask crowdworkers to write questions for question-anaaane answer dataset (Talmor et al., (2018).
These categories can be broadly encapsulated by the Partnership on Al's (PAI) definition of 'data enrichment" work, defined as data curation tasks which require human judgement and intelligence (Partnership on Al (022). However, this does not include research studying the behaviour of crowd- workers themselves (Vaughan 2018). For example, researcher might assess how individuals re- spond to interaction with algorithms deployed in an educational setting, or assess human perception of artificial systems (Fahid al. |02T, Latikka et al., 2021 Koster al., 2022). Behavioural stud- ies are different to data enrichment tasks as they treat crowdworkers as the subject of research, rather than as a worker providing input to a model which is itself the subject research.
2.2 ETHICS IMPLICATIONS OF CROWDWORK
In parallel to the rise of crowdsourcing in AI research, crities have questioned the ethics of these practices in lieu of employment law protections for workers (Aloisi, 2016). Concerns centre on issues of payment, maltreatment, power asymmetry, and demographics.
Crowdsourcing platforms are often utilised due to their low costs, and consequently many critiques of crowdwork relate payment (Scholz, 2016). MTuur allows requesters place tasks online
2

---

Under review as a conference paper at ICLR 2023
for little as $0.01 per task, with mean payment rates estimated to be around S3 per hour (Hara
jet al. 2018 Pew Toxtli et al. 202Ip. Considering around 75% of MTurk workers are based in the US (with 16% based in India), this is far below federal minimum wage levels (Difallah et al. 2018). This has taken focus on Al developers with the issue being reported in various media outlets
Naylor 2021 Slater
202T).
The rise of underpaid uncontracted work has ushered in an "era of digital sweatshops", raising con- cerns regarding worker wellbeing ZZittrain 2009). An individual might be tasked with identifying harmful content, such as pornographic, violent, or offensive images, text, or video, in order to train algorithms, or might evaluate a model's moderation performance (Dang et al.) 2018: Edstedt al.) 2021 Hettiachchi & Goncalves] (219)). Such subjection to harmful content has been shown to have severe psychological impacts on workers (Steiger et al. 2021 Benjelloun & Otheman, 2020).
In many cases, Al research involving crowdworkers will not include such harmful content, but that does not mean ethics concerns beyond payment are absent. Platforms have been criticised because of inadequate feedback mechanisms and intransparent instructions leading to inability to meaningfully consent to studies (Schlagwein et al. 2019; Zimmer & Kinder-Kurlanda, 2017). Some studies have been found to deceive crowdworkers, whilst researchers have the ability to reject workers outputs
for unclear reasons, leading to vastly imbalanced power dynamics against workers who are often without the protection of employment contracts (Diaz et al. 2022 Irani 2015).
The lack of consideration for workers has led to the idea that crowdsourced workers are "inter- changeable" (Diaz al. 2022). This is despite work demonstrating demographics having drastic impacts on research outcomes (Salkind) 2010] [eel et al.] [0133] Welbl et al. [202T) When curating datasets with crowdworkers, lack of diversity can lead the "preservation' of bias in future uses
of data (Celi et al. 2022: Wachter et al. 2021 Crawford & Paglen] (021). In combination, these
issues have led crowdworkers being dehumanised. and labelled "Ghost Workers" (Barbosa & Chen Gray & Suri 2019 Mohamed et al. 2020 Prabhu & Birhaane 2020).
2019
2.3
ETHICS DISCLOSURES
How these ethics issues impact research practices has been assessed by examinations of publication practices. Santy et al. have undertaken this in the context of the NLP field, assessing how many papers engage with formal ethics review through IRBs. They find that very few papers (0.8%) cite IRB review (Santy et al. 2021). This is unsurprising considering many NLP papers will not involve any direct interaction with crowdworkers or human participants, and so would not be subject to IRB review. The authors additionally provide comparisons with other fields, such cognitive sciences to demonstrate that NLP research lacks the same level of engagement with formal ethics processes Santy et al. 2021).
Shmueli et al. conduct a similar review of NLP research, focusing on issues beyond payment Shmueli et al. 2021). The authors find that whilst 10% of accepted papers 3 major NLP confer- ences use crowdsourcing techniques, just 17% of these mention payment, and fewer refer to an IRB review (Shmueli al., 202T). The paper notes that it is often unclear whether crowdworkers meet the definition of human subjects because of narrow criteria provided by US regulation (Shmueli al. (202T). This definitional dilemma supported by (Kaushik et al.] (022) who highlight the inconsistencies in how crowdwork is defined under research ethics regulations. These papers point to need for further understanding of how researchers engage with research ethics issues.
METHODOLOGY
To assess how Al research involving crowdworkers engages with research ethics, this paper conducts (1) policy analysis of publication venues and (2) paper analysis to assess how policies and norms translate to practice at major venues.
3.1 POLICY ANALYSIS
The first portion of this study assesses the publication requirements of AI papers leading venues. Google Scholar impact ratings show that conferences are the leading publication venues for AI re- search, and therefore the top two, the International Conference on Learning Representations (ICLR)
3

---

Under review as a conference paper at ICLR 2023
and the Conference on Neural Information Processing (NeurIPS) are selected for analysis (Google Scholar 2022). Journals are also common publication venues, and can act as a comparison point for policy analysis. Springer Nature, leading publisher with universal disclosure policies across academic disciplines and with various journals dedicated to AI, is selected to compare policy docu- mentation (Springer 2022: Torres-Salinal al. 2013
3.2
PAPER ANALYSIS
The second component of this study bridges the gap between expectation and reality. line with the policy analysis, ICLR, NeurlPS, and Springer Nature papers are assessed to understand how AT research papers adhere to the requirements prescribed by publishers. Papers were included within the study if they:
(a) Utilised data generated by humans recruited via a crowdsourcing platform and (b) Collected data directly for the purposes of the study; and (c) Contributed to artificial intelli- gence research
To determine whether paper met criteria, a full-text search performed. ICLR papers cessible via OpenReviow, a peer-review portal, whilst NeurIPS papers were accessed through the NeurIPS site. Accepted papers were then analysed via Python seript using the PyPDF package. This code identified papers which may use crowdsourcing by searching for terms such as "Me- chanical Turk", "annotator" and "rater" (see Appendix A.3). Each paper identified was manually reviewed to determine whether the criteria for study was met. For Springer papers full text-search was available without additional coding via the Springer online portal. As Springer includes hun- dreds of journals, only papers within the Artificial Intelligence field referencing the use of MTurk, the most-used platform in the conference data analysis, were included to manage data collection. Psychology papers from Springer journals and papers from the Conference on Human Factors in Computing Systems (CHI) were also considered for one year as a benchmark of current best prac- tices in (1) a well-established field with strong research ethics practices and (2) a conference where AI research frequently crosses into human participant research. Psychology has a long history of engagement with human participants and has similarly utilised crowdsourcing platforms in recent years, whilst CHI is arguably the leading conference sitting at the intersection of computer science and human behavioural studies (Buhrmester et al. 2018 Stewart al. 2017).
For each paper analysed data collected included whether IRB (or equivalent) review was disclosed, whether payment terms were outlined, what type of task undertaken, and whether any type of demographic data of workers was noted in the paper. For a full list of data points collected, see Appendix A.4.
VENUE POLICY ANALYSIS
4.1
ICLR
ICLR provided no requirements of authors related to research ethics until 2021, when code of ethics was introduced. This code outlines principles such as "avoid harm", and "respect privacy" which have direct implications to research involving crowdsoureing.
The conference explicitly notes the need for the disclosure of ethics review when considering the principle of upholding scientific excellence. The code states: "Where human subjects involved in the research process (e.g., in direct experiments, or as annotators), the need for ethical approvals from an appropriate ethical review board should be assessed and reported" (ICLR) 2022). This is the only reference to human subjects within the code, but suggests that papers should report research ethics reviews. or disclose when research ethics reviews were deemed exempt by a review body. The code does not require papers to report on other issues such as consent, payment, or demographics. Reviewers are asked to raise potential violations of the ICLR Code of Ethics, and authors are encouraged to discuss ethics questions.
Where uncertainty exists over whether the study was conducted using crowdsourcing platform, papers included within analysis for completeness
4

---

Under review as a conference paper at ICLR 2023
4.2 NEURIPS
NeurIPS, a year before the ICLR Code of Ethics, piloted an ethics review process (Ashurst al, 2022). This process focused on an assessment of the "broader impacts" of research, asking re- searchers include a statement on how the research might lead beneficial harmful outcomes to society. The guidance for these statements was limited, and did not explicitly require disclosures of IRBs, payment rates, or consent protocols.
However, in 2021 NeurIPS replaced the requirement of an ethics statement with checklist (Beygelzimer et al., 202T). When announcing the checklist, the program chairs stated that they aimed to "encourage best practices for responsible machine learning research, taking into consider- ation reproducibility, transparency, research ethics, and socictal impact" (Beygelzimer et al. (002I). The checklist includes specific requests for disclosing information about crowdsourced workers or human subjects, asking the following:
"(a) Did you include the full text of instructions given participants and screenshots, if applicable? (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?" (NeurIPS) 2021)
This ciscklist shows actionable approach to engaging with research ethics issues such payment and ethics review, showing a clear interest in the impacts of research on crowdworkers.
4.3 SPRINGER
Whilst codes of ethics and requirements pertaining research ethics are a relatively new phe- nomenon for AI conferences, this is not the case for journals released by established publishing houses. Publishers can have editorial policies that apply to all journals submitted, and this the case for Springer, which publishes number of AI journals.
Springer has a dedicated editorial policy to studies involving human subjects, stating that papers should: "include statement that confirms that the study was approved (or granted exemption) by the appropriate institutional and/or national research ethics committee (including the name of the ethics committee) and certify that the study performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards" (Springer 2022).
The policy goes on to state that any exemptions should be detailed within manuscripts, including the reasons for exemption. The publisher does not provide guidance disclosure standards research requirements beyond research ethics approvals, but this requirement applies to any journal submitted Springer, meaning all papers which have engaged with an IRB should disclose this.
These policies demonstrate some deviation between the requirements of publishers across AI venues, whilst broader analysis of publication policies across venues including ICML, AAAI, and CHI can be found in Appendix A.8, demonstrating that these policies are in-line with practices of other venues.
5 PAPER ANALYSIS
Whilst policy can inform how AI researchers are expected to engage with research ethics consider- ations, paper analysis enables an assessment of this engagement in practice.
5.1 CONFERENCE ANALYSIS
Table show the number of papers which meet this study's inclusion criteria for crowdsourcing from NeurIPS and ICLR. For both conferences the proportion of papers meeting this criteria is low, at between 2-3% for NeurIPS and 2-6% for ICLR each year. This is a small but significant number of papers, particularly considering many more papers use previously collected datasets which may have derived from crowdsourcing, but were considered out of scope for this study.
5

---

Under review as a conference paper at ICLR 2023
Analysis also found that papers overwhelmingly utilised the MTurk platform compared with others. Over half of all papers using crowdsourcing at NeurIPS, and 65% at ICLR, state that MTurk was used for data collection, largely remaining consistent over the four years of data collection. Over 40% of papers at NeurlPS, and a third at ICLR, did not disclose a data collection platform, whilst the only other platform to be mentioned in more than one paper was Prolific, with four references. Full platform details can be found in the Appendix A.6.
Table 1: Proportion of accepted papers meeting crowdsourcing criteria between 2018-2021
Year
NeurIPS Papers
NeurIPS Crowdsourcing
ICLR Papers
ICLR Crowdsourcing
2022
N/A no data
N/A
1,068
24 (2%)
2021
2,331
48 (2%)
874
40 (5%)
2020
1,896
34 (2%)
696
23 (3%)
2019
1,426
37 (3%)
500
30 (6%)
2018
1,009
24 (2%)
334
11 (3%)
Total
6,662
143 (2%)
3,472
128 (4%)
5.2 CONFERENCES: ETHICS DISCLOSURES
Table shows analysis of research ethics disclosures for NeurIPS. In 2018 and 2019 few papers disclosed research ethics considerations. However, in 2020 18% of papers discussed IRB review, and 12% disclosed payments. This increase may have resulted from the introduction of broader impact statements, explicitly asking authors to consider the societal implications of their research (Ashurst al., 2022). In 2021 this system changed, and a checklist was introduced explicitly requesting authors to disclose payment terms and disclose IRB reviews (Beygelzimer el al.] 202T). Whilst this had a huge impact on payment disclosures, this did not significantly increase other disclosures, indicating some, but limited, success of the checklist.
Table 2: NeurIPS: Crowdsourcing papers' research ethics disclosures between 2018-2021
Year Crowdsoureing
IRB
Payment
Consent
Demographics
2021
48
9 (19%)
26 (54%)
8 (17%)
5 (10%)
2020
34
6 (18%)
4 (12%)
3 (9%)
4 (12%)
2019
37
0 (0%)
4 (11%)
(3%)
4 (11%)
2018
24
0 (0%)
1 (4%)
(0%)
0 (0%)
Total
143
15 (10%)
35 (24%)
12 (8%)
13 (9%)
In contrast, ICLR's Code of Ethics, whilst also requesting IRB disclosures, makes fewer additional disclosure requirements for papers, and this can be seen from the results in Table 3. Across 2018 and 2019 none the 41 papers which involved crowdsourcing data collection referenced any of the categories analysed. In 2020 this changed, with some IRB and payment disclosures. Following the introduction of the Code of Ethics disclosures have become slightly more common, but remain very infrequent.
5.3 JOURNAL COMPARISON
The data from NeurlPS and ICLR can be compared to papers and articles submitted to Springer journals to understand whether this is a unique issue to conferences. Table 4 outlines disclosures within AI papers which utilise MTurk for data collection between 2018 and 2021. Over the four years there has been steady increase in research ethics disclosures suggesting a trend towards AI researchers taking greater interest in ethics considerations when using crowdworkers.

---

Under review as a conference paper at ICLR 2023
Table 3: ICLR:
Crowdsourcing papers' research ethics disclosures between 2018-2021
Year
Crowdsoureing
IRB
Payment
Consent
Demographics
2022
24
3 (13%)
5 (21%)
(8%)
(4%)
2021
40
1 (3%)
5 (13%)
2 (5%)
3 (8%)
2020
23
2 (9%)
(13%)
0 (0%)
0 (0%)
2019
30
0 (0%)
0 (0%)
0 (0%)
0 (0%)
2018
11
0 (0%)
0 (0%)
0 (0%)
0 (0%)
Total
128
6 (5%)
(10%)
4 (3%)
4 (3%)
Table 4: Springer Journals: MTurk papers' research ethics disclosures between 2018-2021
Year
Crowdsoureing
IRB
Payment
Consent
Demographics
2021
76
16 (21%)
22 (29%)
14 (18%)
26 (34%)
2020
47
6 (13%)
19 (40%)
6 (13%)
16 (34%)
2019
66
(6%)
8 (12%)
(8%)
16 (24%)
2018
86
4 (5%)
12 (14%)
(5%)
11 (13%)
Total
275
(11%)
62 (22%)
30 (11%)
(25%)
5.4 PSYCHOLOGY AND CHI COMPARISON
We can compare this data to benchmark collected from an academic field with a history of research ethics considerations, Psychology, and Computer Science venue with history of human data collection, CHI. Table 5 outlines the results of data collection for Psychology papers within Springer journals and CHI papers which reference the use of MTurk. The results show that Psychology papers disclose research ethics considerations most frequently, whilst CHI papers are more likely to include these compared with other Al venues, particularly when considering payment and demographic information. This could indicate either substantive difference in the nature of the data collection between venues, or cultural divide.
Table 5: Benchmark: Springer Psychology and CHI MTTurk papers' research ethics disclosures
Venue
Year
Papers
IRB
Payment
Consent
Demographics
ICLR
2018-22
128
6 (5%)
13 (10%)
4 (3%)
4 (3%)
NeurIPS
2018-21
143
15 (10%)
35 (24%)
12 (8%)
13 (9%)
Springer AI
2018-21
275
30 (11%)
61 (22%)
29 (11%)
69 (25%)
CHI
2022
66
28 (42%)
49 (74%)
27 (41%)
41 (62%)
Springer Psych
2021
268
193 (72%)
173 (65%)
194 (72%)
240 (90%)
5.5 TASK TYPE ANALYSIS
To explore this gap, the type of task undertaken within the research can be examined (see Table 6) to understand if disclosures differ depending on the type of engagement with crowdworkers (with task types defined in Section 2.1 95% of Psychology papers analysed classified as involving a "hehaviour" task, while around one in ten Al papers at Al conferences and a third of Al journal papers involve behaviour tasks. At AI conferences. evaluation tasks most prominent, comprising 70% of all papers, whilst in journals this figure is only 39%. This may indicate that behaviour tasks more likely to engage with research ethics issues.
Note categories not mutually exclusive, with some studies employing crowdworkers for multiple task types.

---

Under review as a conference paper at ICLR 2023
Table 6: Task type comparison venues between 2018-2021
Venue
Behaviour
Evaluation
Labelling
Production
AI: NeurIPS
15% (22/143)
70% (100/143)
6% (8/143)
11% (16/143)
AI: ICLR
9% (9/104)
70% (73/104)
9% (9/104)
16% (17/104)
AI: Springer
35% (95/275)
39% (108/275)
17% (48/275)
9% (24/275)
CHI (2022 only)
74% (49/66)
20% (13/66)
12% (8/66)
3% (2/66)
Psychology (2021 only)
95% (254/268)
2% (6/268)
2% (5/268)
1% (3/268)
Table 7 explores this hypothesis, demonstrating that a gap appears to remain between the Al papers analysed and the Psychology field. However, NeurlPS disclosures except when considering payment and demographic data. The gap is most stark when considering ICLR behavioural papers, which rarely report on research ethics issues.
similar those at CHI,
Table 7: Disclosures of papers utilising behavioural tasks across venues between 2018-200 2018-2021
Venue
Behaviour
IRB
Payment
Consent
Demographics
AI: NeurIPS AI: ICLR
15% (22/143)
45% (10/22)
50% (11/22)
45% (10/22)
45% (10/22)
9% (9/104)
0% (0/9)
11% (1/9)
22% (2/9)
22% (2/9)
AT: Springer
35% (95/275)
24% (23/95)
48% (46/95)
23% (22/95)
64% (61/95)
CHI (2022 only)
74% (49/66)
47% (23/49)
80% (39/49)
45% (22/49)
73% (36/49)
Psychology (2021 only)
95% (254/268)
73% (186/254)
67% (170/254)
74% (187/254)
94% (239/254)
5.6 INSTITUTION TYPE ANALYSIS
Another explanation for this gap might be the types of institutions submitting papers, where private companies, who may be less familiar with research ethics process, could be less likely to engage with research ethics considerations. This hypothesis is explored in Appendix A.7, which demonstrates that this cannot be easily concluded, with disclosures across institutions in the papers analysed falling short of those seen at Psychology and CHI.
6
KEY FINDINGS
6.1 LEADING RESEARCH VENUES DO NOT ALIGN WITH TRADITIONAL RESEARCH ETHICS DISCLOSURE STANDARDS
Research ethics disclosures appear to be less common at leading Al research venues compared with Psychology and CHI. One might argue that this is due the nature of the tasks being different, with Psychology research concerning the behaviour of participants. However, the gap persists in AI research involving behavioural tasks, whilst ethics issues are not exclusive to behavioural stud- ies. This difference may result from how the definition of 'human subject' is interpreted, with AI researchers unclear on when studies require engagement with research ethics review processes
Shmueli et al. 2021 Kaushik et al. 2022).
This gap may also exist because the AI field lacks the same history engagement with human subjects, with recent crowdsourcing possibilities provoking greater interest in direct human engage- ment. Alternatively, this may result from a lack of research ethics education in Computer Science departments, meaning there is less focus these concerns.
8

---

Under review as a conference paper at ICLR 2023
6.2 JOURNALS AND CONFERENCES HAVE POWER TO INFLUENCE ENGAGEMENT WITH RESEARCH ETHICS
The research ethics discrepancies noted are not equal across publication venues, with Springer and NeurIPS papers more frequently citing IRB reviews compared with ICLR papers. For Springer, this may be due to the varied nature of journals, many of which are related to Al's impact on society (e.g. "AI Society" and "Artificial Intelligence in Education"). For NeurIPS, drastic increase in reporting of IRB engagement and payment terms may be explained by changes in conference policy, with the 2021 introduction of the checklist. The lack of equivalent impact of the ICLR Code of Ethies may be due to the code providing less stringent stipulations.
6.3 LEADING AI RESEARCH BREAKS WITH SCIENTIFIC TRADITION BY SCARCELY CONSIDERING DEMOGRAPIIC IMPACTS
One type of disclosure which stands out across Al papers relates demographic data. Demographic data is frequently reported in social science studies (including the assessed Psychology papers), from a scientific integrity and reproducibility perspective, and because of research ethics considerations [Connelly] 2013: 'NNatrre) ["2022"] Robinson et al., 2017). However, these disclosures were not seen in many of the Al papers analysed. Whilst demographics might not impact some studies (e.g. those involving objective labelling tasks), many AI research tasks involve some subjectivity, and the distinction between objective and subjective labelling not always clean or self-evident. De- mographics are frequently demonstrated to be an important influence on datasets in AI, and can have multiplicative effects as datasets are reused, locking biases which can cause representational harms to those not adequately considered within dataset (Paullada et al., 2020). This can be hard to identify and mitigate when datasets are re-used without demographic data, and means addressing these issues up front is critical.
The relative lack of disclosure may exist because demographic data is not collected for legal reasons or avoid data which might be considered identifiable. However, this type of data is available for collection via crowdsourcing platforms, so this is a design choice from researchers, rather than an imposition. Studies may also argue that they do not require demographic diversity; for example if engaging in objective labelling tasks. However, labelling tasks account for 12% of papers analysed, and many cases data enrichment tasks involve some subjectivity. Subjectivity is not the only reason to include demographic data; different demographic groups may be impacted differently by tasks, or data collected could be re-used by other actors in domains where demographic differences are impactful. Demographic reporting is an important aspect of best practices in research, and there is little excuse for the AI field to depart from this norm.
7 CONCLUSION AND FUTURE WORK
This paper shows how AI researchers at leading venues engage with research ethics questions when employing crowdsourced workers, and illustrates the norms developing in the field. The work shows that research ethics disclosures are infrequent at leading research venues, whilst publication policies are emerging but inconsistently followed.
The gap demonstrated in this work between policies and practices of Al venues has been conducted with relatively small pool of papers (owing to the content of published papers, not methodological limitations) and venues. Future work could extend this study to other venues, and should prompt further exploration of how AI data enrichment research should engage with research ethics norms which exist in other fields. While our findings are limited to the reviewed venues, they are nonethe- less significant given the leading position of ICLR and NeurIPS and the role these venues play in setting research culture and norms across machine learning and Al research. This may result from ambiguity set by the regulatory requirements, which were designed for different fields and different types of work, or from a lack of experience in the AI field with this type of work. Future work could also further explore the motivations for the lack of demographic reporting, with this gap appearing consistent across the Al papers analysed, in stark contrast to research norms. With these directions in mind, this work hopes to encourage the field to move towards agreed ethics norms, fit for Al research and crowdwork, and consistently applied across publications.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Antonia Aloisi. Commoditized Workers. Case Study Research on Labour Law Issues Arising from Set of Oon-Demand/Gig Economy' Platforms. Comparative Labor Law Policy Journal, 37(3): 1354-1360, 2016. doi:
Carolyn Ashurst, Markus Anderljung, Carina Prunkl, Jan Leike, Yarin Gal, Toby Shevlane, and Allan Dafoe. Code of ethics, 2022. URL Itttps://meeiiiie com/ @GovAI/ 4
Nată M. Barbosa and Monchu Chen. Rehumanized crowdsourcing: A labeling framework ad- dressing bias and ethics in machine learning. In Proceedings of the 2019 CHI Conference Human Factors in Computing Systems, CHI '19, pp. 1-12, New York, NY, USA, 2019. Associ- ation for Computing Machinery. ISBN 9781450359702. doi: 10.1145/2290005333007777 URL
ht.tss://oii htt:::://661..rr//0..111.. 01t.////....1/32200000000
3300773
Henry Beecher. Ethics and Clinical Research. New England Journal of Medicine, 274(24):
1354-1360, 1966. doi: http: 1056/FJM111660061777777777
Joeran Beel, Stefan Langer, Andreas Nürnberger, and Marcel Genzmehr. The impact of demograph- ics (age and gender) and other user-characteristics evaluating recommender systems. volume doi: (0.1007/77---6422-400
8092, pp. 396-3966499 400, 09 2013. ISBN 777-3--642-40000-6
Roukaya Benjelloun and Yassine Otheman. Psychological distress in social media content moder- ator: A case report. Archives of Psychiatry and Mental Health, 2020. doi: do1:10.2932/juuraal. apmh. 1001024.
Alina Beygelzimer, Yann Dauphin, Percy Liang, and Jennifer Wortman Vaughan. Introducing the neurips 2021 paper checklist, 2021. URL
medi um. com/
introducing -ereduuiing-te--neee the- ps- 2021 ssa
Michael D. Buhrmester, Sanaz Talaifar, and Samuel D. Gosling. An evaluation of amazon's me- chanical turk, its rapid rise, and its effective use. Perspectives on Psychological Science, 13 (2):149-154 154, 2018. doi: 10. 70.1777774566111177667 174569161770651 PMID: 29928846.
URL htt.:htpps////o /tttps:///ol.org110...11
wt..//...01.... 11777
Leo Celi, Jacqueline Cellini, Marie Charpignon, Edward Dee, Franck Dernoncourt, Rene Eber, Julian Schirmer, Julia Situ, Joseph Alexander Paguio, Joel Park, Judy Wawira, Seth Yao, and William Mitchell. Sources of bias in artificial intelligence that perpetuate healthcare disparities-a global review. PLoS Medicine, e0000022, 03 2022. doi: 10.1371/journal 00000000002.
0000022.
Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. All that's 'human' is not gold: Evaluating human evaluation of generated text. In Pro- ceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference pp. 7282-7296, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/ v1/2021 .acl-long..565. URL https: http://acaaatttto /ttpsttss/////cccaaaooooooooo fttt:s://cclaathoooo...ogg////// wwt.gyy.orgg/200 http://2222..a111ooooo acl-
Natural Language Processing (Volume 1: Long Papers),
Cihan Cobanoglu, Muhittin Cavusoglu, and Gozde Turktarhan. A beginner's guide and best practices for using crowdsourcing platforms for survey research: The case of Amazon Me- chanical Turk (MTurk). Journal of Global Business Insights, 6(1):92-97, 2021. doi: https:
Lynne Connelly. Demographic data in research studies. Medsurg nursing official journal of the Academy of Medical-Surgical Nurses, 22:269-70, 10 2013.
Kate Crawford and Trevor Paglen. Excavating ai: the politics images machine learning training sets. AI SOCIETY, 06 2021. doi: 00.10077/000146--01100016666
Brandon Dang, Martin J. Riedl, and Matthew Lease. Toward safer crowdsourced content modera- 10999
tion. CoRR, abs/180s 10999, 2018.
10

---

Under review as a conference paper at ICLR 2023
Alexandre Défossez, Neil Zeghidour, Nicolas Usunier, Léon Bottou, and Francis R. Bach. SING:
symbol-to-imstrument neural generator. CoRR, abs/1810.09785, 2018. URL http:://rxix
org/abs/1810. www.a////........ 09785
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: large-scale hierarchical image database. 2009. doi: 10. 10R.1//9...000.520000
"Mark Díaz, lan Kivlichan, Rachel Rosen, Dylan Baker, Razvan Amironesei, Vinodkumar Prab- hakaran, and Emily Denton". ACM title CrowdWorkSSeets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation, booktitle = 2022 ACM Con- ference Fairness, Accountability, and Transparency, jum 2022. doi: 1..1445/53114463344666 URL 10
35346477
Djellel Difallah, Elena Filatova, and Panos Ipeirotis. Demographics and dynamics of mechanical turk workers. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 18, pp. 135-143, New York, NY. USA. 2018. Association for Computing Machinery. ISBN 9781 450355880... doi: 10.1145/115963231599661.. URL https::/doi.
org/10. 011.111144//311522.33333
Johan Edstedt, Johan Karlsson, Francisca Benavente, Anette Novak, Amanda Berg, and Michael Felsberg, Is this harmful? learning predict harmfulness ratings from video. CoRR, abs/2106.08323, 2021.
08323
Ezekiel J. Emanuel. The Oxford Textbook of Clinical Research Ethics. Oxford University Press, Oxford, 2008.
Fahmid Morshed Fahid, Jonathan P. Rowe, Randall D. Spain, Benjamin S. Goldberg, Robert Pokorny, and James Lester. Adaptively Scaffolding Cognitive Engagement with Batch Con- strained Deep Q-Networks. Artificial Intelligence in Education, 337(20):113-12,, 2021. doi:
https://doi.org/10.
Google Scholar. Artificial intelligence top publications. 2022.
URL
https:
eng ang_arttiffciaalineellggg
Christine Grady. Institutional Review Boards, Chest, 148(5): -15-5 2015. doi: https://doi.og/!
10.1378/chest11500766
Mary L. Gray and Siddharth Suri. Ghost work how to stop Silicon Valley from building new global underclass. Houghton Mifllin Harcourt, Boston, 2019.
Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jeffrey P. Bigham. A data-driven analysis of workers' earnings on amazon mechanical turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI '18, pp. 1-14, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 4781450356206. doi: 10.
1145/3173574.3174025 URL BRRLaatps:://ee wwwpwo............oo.o.o..../... 1et...11111111711 3174023
Jean
Heller.
Syphilis
victims
in u.s. study went untreated for 40
years.
URL
https: //ww:://wwwww... nytimes com wtt.I1922/772////rrrr////
html
Danula Hettiachchi and Jorge Goncalves. Towards effective crowd-powered online content mod- eration. In Pr Proceedings of the 31st Australian Conference on Human- Caapmaa--Coomuur---eeeeeaaae
OZCHI' 19, pp. 342-346. New York, NY, USA, 2019. Association for Computing Machin- ery. ISBN 9781450376969,
doi: 01144//33694573336442 URL https:: http//oi..rg//...
1145 /3533333347777 1145/336955.3369977
Jeff Howe. The Rise of Crowdsoureing? 2006/06/crowds/
Wired, 2006. URL https:// http:::://wwww.ieee.
ICLR. Code of ethics, 2022. /ttp////tt////err..o/////l////aa
11

---

Under review as a conference paper at ICLR 2023
Lilly Irani. The cultural work of microwork. New Media & Sociery, 17(5):720 739, 2015. doi: URL
Divyansh Kaushik, Zachary C. Lipton, and Alex John London. Resolving the human subjects sta- of machine learning's crowdworkers, 2022. URL https::/arxiv.. org/ absp//bs/2220
04039
Raphael Koster, Jan Balaguer, Andrea Tacchetti, Ari Weinstein, Tina Zhu, Oliver Hauser, Duncan Williams, Lucy Campbell-Gillinghan,, Phoebe Thacker, Matthew M. Botvinick, and Christopher Summerfield. Human-centered mechanism design with democratic AL. CoRR, abs/2201.1144 2022. URL https::/axx.....//////////////// 2201://1abb2221114111
Rita Latikka, Nina Savela, Aki Koivula, and Atte Oksanen. Perceived robot attitudes of other peo- ple and perceived robot use self-efficacy as determinants of attitudes toward robots. Human-
Computer Interaction. Interaction Techniques and Novel Applications: Thematic Area, HCI 2021, Held as Purt of the 23rd HCI International Conference, HCII 2021, Virtual Event, July 24-29, 2021, Proceedings, Part II, pp. 262-274, Berlin, Heidelberg, 2021. Sppinger-VVelag. ISBN 978- doi: 10. (0007997--3300077466666 978 3- 030 78465-8822 2_ 20
3-030-78464-5.
Fei-Fei Li. Where did imagenct come from?, 2019. URL httpp::/www.yooteettt com/
watch?v 27 7naKluqqF8
Stanley Milgram. Behavioral Study of obedience. The Journal of Abnormal and Social Psychology, 67(4):371-375 1963. doi: https://doi.org/ hrt: http://00.rrr/111100033/00000000 13770/000000
Shakir Mohamed, Therese Png, and William Isaac. Decolonial AI: decolonial theory as sociotechnical foresight in artificial intelligence. CoRR, 0bss2200.044063 2020. URL https: //arx/iw.rrg/bb///0007 04068
"Nature". "nature journals raise the bar on sex and gender reporting in research". ""0'"""96'),, "2022".
Aliide Naylor. Underpaid Workers Are Being Forced Train Biased AI Mechani-
wwtpc//..o/////aaiieeelee.e/ cal Turk. Vice, 2021. URL http::/sswww....ccccccc///////ts
monerpaiidworkeen are httpap/eeeoooooot..taaaaaaaaaaaa
NeurIPS. Neurips 2021 paper checklist guidelines, 2021. URL htttp:://neurpps....
Office for Human Research Protections. 2018 requirements (2018 common rule), 2017.
URL httpa////s.///aaaa/////////////r
45- http::/ewee..eaaacccooooo..reeaa
Partnership on AL Responsible sourcing of data enrichment services, 2021. URL
Amandalynne Paullada, Inioluwa Deborah Raji, Emily Bender, Emily Denton, and Alex Hanna. Data and its (dis)contents: A survey of dataset development and use in machine learning research. CoRR, abs/2012.05345, 2020. URL
/ttp://wwww..r//////22.2.22222 2532.03445
abs/
Pew.
Research in the crowdsourcing age, a case
study.
URL
https:/ www. www.:eeweeearcc..oo//eteeeeeeeee 20ti://0nee//110/7111111
research in the wwwwwoourciigg.....ooooo.o age a case study/
Vinay Uday Prabhu and Abeba Birhane. Large image datasets: A pyrrhi win for computer vision?
CoRR, abs/2006. 16923, 2020. URL htt.:httt:://aatt wwwp:xwwwwwoorooooooooooooooeeee haargg/ab//// abs/2bbs.555 16923
June K. Robinson. Amy J. McMichael, and Claudia Hernandez. Transparent Reporting of Demo-
graphic Characteristics of Study Participants. JAMA Dermatology, 153(3):2632664, 2017. doi: https://do..org/0. h00..00000mmmmmmmmmm..000000
Neil J. Salkind, Encyclopedia of Research Design. SAGE Publications, Californa, 2010.
12

---

Under review as a conference paper at ICLR 2023
Sebastin Santy, Anku Rani, and Monojit Choudhury. Use of formal ethical reviews in NLP literature: Historical trends and current practices. CoRR, abs/2106.01 105, 2021. URL https://arxxx
horg/bbs/an 2106. 01105
Hans-Martin Sass. Reichsrundschreiben 1931: Pre-Nuremberg German Regulations Concerning New Therapy and Human Experimentation. The Journal of Medicine and Philosophy: A Forum for Bioethics and Philosophy of Medicine, 8(2):99-111, 1983. doi: 1oi: 109.0000999.........
Daniel Schlagwein, Dubravka
and Benjamin Hanckel. Ethical norms and is- sues in crowdsourcing practices: A habermasian analysis. Information Systems Journal, 29(4): 811-837, 2019. doi: http:://doi.orr//0.1111111 12227. URL pnttps:///nt womllinelibrrrr librryy.r.... wi wttp.//////o//aao.o111111..11111
12227
Trebor Scholz. Uberworked and Underpaid: How Workers Are Disrupting the Digital Economy. Polity Press, Cambridge, 2016.
Boaz Shmueli, Jan Fell, Soumya Ray, and Lun-Wei Wei Ku. Beyond fair pay: Ethical implications of NLP crowdsourcing. CoRR, abs/2104. 10097, 2021. URL
21 1404. 10097
Evelyn Shuster. Fifty Years Later: The Significance of the Nuremberg Code. New England Journal of Medicine, 337(20):1436-144)) 1997. doi: http://dooorrr0.....00////111111
Anita
Slater.
How
Artificial
Intelligence
Depends
on
Low-Paid Work-
Tribune, 2021.
URL https://triiunnemaa.cco
how now--artine www.iiiiiialaiiiiaaeiaaa oorpalaeee--e-pepeee----o--o----
Springer Editorial policies. 2022.
URL
https://ww..pringer..coom http::auau-aaaaaa----aaaaaaaaaaa
editorial editorial-policeess wow.alooooiieieeeeeere---------e
177697761
Luke Stark. The unintended ethics of Henry K Beecher. The Lancet, 337(10003)22377423552
doi: https://do..org/10.. http:://do..og1100010///01100000 0011///011006766166000000
Miriah Steiger, Timir Bharucha, Sukrit Venkatagiri, Martin J. Riedl, and Matthew Lease. The psychological well-being of content moderators: The emotional labor of commercial moderation and avenues for improving support. In Proceedings of the 202/ CHI Conference on Human Factors in Computing Systems, CHI 211, New York, NY, USA, 2021. Association for Computing URL https://doi.
Machinery. ISBN 9781450380966 doi: 10. 10.1145//411744444559999
0tte/0..114/34111150 3445092]
Neil Stewart, Jesse Chandler, and Gabriele Paolacci. Crowdsourcing samples in cognitive science. Trends in Cognitive Sciences, 21, 08 2017. doi: ttt.:///ccc.22....00
Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. Learning summarize from human feedback. CoRR, abs/2009.01325, 2020. URL
01325
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. CoRR, abs/1811.00037,, 2018. URL
http::/aaxiv.. httrgg/bb//8111 .09337
Humphrey Taylor. Does Internet Research Work?. International Journal of Market Research, 42(1): 1-11, 2000. doi: http:://k.ooog///1111///00000000
Christopher Thomas and Adriana Kovashka. Predicting the politics of an image using webly super- vised data. CoRR, abs/1911.000 2019. UtL org/ abs 1911 0014
Daniel Torres Salina,, Nicolás Robinson-Garcis Juan Miguel Campanario, and Emilio Delgado López-Cózar. Coverage. field specialization and impact of scientific publishers indexed in the 'book citation index'. CoRR, abs/1312..2991, 2013. htthhtt::/// //ttp:///xxiv...//// http://ogggabb/.b....
2791
13

---

Under review as a conference paper at ICLR 2023
Carlos Toxtli, Siddharth Suri, and Saiph Savage. Quantifying the invisible labor in crowd work.
CoRR, abs/2110.00169, 2021. URL http:t::/axxxi..ootom http://ttxxvoo..//.//.ooo http:////Eaav.oo0/////221......0 wttp://///bb//....1.1.... 2110..01669
Jennifer Wortman Vaughan. Making better use of the crowd: How crowdsourcing can advance
machine learning research. Journal of Machine Learning Research, 18(193):1-46, 2018. URL
"Sandra Wachter, Brent Daniel Mittelstadt, and Chris Russell". Bias preservation in machine learn- ing: The legality of fairness metrics under eu non-discrimination law. SSRN Electronic Journal, 2021.
"Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang". Challenges in detoxifying language models. CoRR, abs/2109.07445, 2021. URL ptttps:/aaxxv..or
0aas/2109.0744
Vanessa Williamson, On the Ethics of Crowdsourced Research. PS: Political Science Politics, 49 (1):77-81, 2016. doi:
Philip Zimbardo. The stanford prison experiment: A simulation study of the psychology of impris- onment conducted august 1971 at stanford university, 1972.
Michael Zimmer and Katharina Kinder-Kurlanda Internet Research Ethics for the Social Age. Peter Lang Publishing, New York, 2017.
Jonathan Zittrain. The Internet Creates New Kind of Sweatshop. Newsweek, 2009. URL https:
7 7/www. www.://wwwwwecccccceoo /ttp://wwwweeeee/c//////oooanonn
hrapa//ceeecttttatteeeeeeeeeeenn 20in$20Kenya.
A APPENDIX
A.1 CODE
The code used in order access and search NeurIPS and ICLR papers for this project is available on Github: [link removed for anonymity].
A.2 RESEARCH ETHICS STATEMENT
This project reviewed and approved by the institutions in accordance with the procedures laid down by the institution for ethical approval of all research involving human participants, reference number [redacted for anonymity].
A.3 SEARCH TERMS
The below terms were used within free-text search to identify papers which involved crowdsourced workers at the NeurIPS and ICLR conferences:
'mechanical turk', 'mturk', 'prolific', 'crowd', 'rater', 'annotator', 'participant', 'amt', 'labeller', 'labeler', 'figure eight'.
Figure Eight and Prolific were included to identify whether platforms other than MTurk were com- mon in Al research, and different practices. Only 4 papers citing Prolific were identified, and zero citing Figure Eight, limiting this exploration.
The below terms used within free-text search the Springer platform to identify Springer AI and Psychology papers which involved the use of Mechanical Turk workers:
Mechanical turk', 'mturk"
14

---

Under review as a conference paper at ICLR 2023
A.4 DATA COLLECTION CRITERIA
The below categories of data were collected for each paper examined in this study, with sub-bullets outlining the justification for data collection.
1. Is IRB or equivalent process mentioned (including disclosure of exemptions)? (Yes/No) Disclosure of IRB review norm for human subjects research, per the Declaration of Helsinki, with IRBs ensuring the welfare of subjects in research.
2. Are payment terms for workers disclosed? (Yes/No)
Payment key issue for crowdworkers within and beyond research, raising ethical and legal concerns (Felstiner, 2).
3. Was worker consent discussed in the paper? (Yes/No)
Participant informed consent is key facet of research ethics, as the Declaration of Helsinki and Belmont Report (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1978; World Medical Association, 2009).
4. What type of data collection hww.://aaaaaattilaaaaalllr/nrooo
did
this
work
involve?
(Be-
Provides data on the types of tasks which AI research engages in determine whether ethics disclosures differ between task types. Task definitions align with those de- scribed in Section (Shmueli et al., 2021; Vaughan, 2018). These categories are not mutually exclusive, with some studies engaged in multiple types of tasks.
5. Were worker demographics disclosed in the paper? (Yes/No)
The demographics of workers ethics reviews considering issues of representation in participant samples. This may be of particular importance in Al research, with various examples of biased data leading to unequal outcomes in AI systems (Mehrabi et al., 2022; Paullada et al., 2021).
impact the outcomes of research, with research
6. What location was the institution of the lead author based in?
Provides insight whether disclosures differ across geography, following the methodology outlined in Santy et al. (Santy et al., 2021).
7. In what type of institution(s) were the authors of the paper based? (Univer- sity/Induustr/Sttae//oonn)
Provide insight whether disclosures differ between private, academic, and state institutions, following methodology outlined in Santy et al. (Santy et al., 2021). State institutions include state-run research labs (e.g. military research bodies).
8. If disclosed, which crowdsourcing platform was used to collect data?
Identifies which platforms are most prominently used, and may identify variance in practices between platforms.
A.5 GEOGRAPHIC DISTRIBUTION OF PAPERS ACROSS VENUES
The table below demonstrates the total number of papers assessed in this work across geographies and venues. As shown, the first authors from over half of the papers identified using crowdsoure- ing across the venues derived from the US with 13 percent from the European Union, and percent from China.
This provides context to the results that follow and shows a heavy US-bias to these venues, and the study whole.
A.6 PLATFORM USE
Tables below show the breakdown of platforms used across ICLR and NeurlPS papers meeting study criteria.
The table demonstrates the overwhelming reliance on MTurk for crowdsourced data, and frequency of papers choosing not to disclose platforms used.
15

---

Under review as a conference paper at ICLR 2023
Table 8: Geographic distribution of Al crowdsourcing papers across publication venue between 2018 and 2021
Geography
NeurIPS
ICLR
Springer AI
Total
United States
(60%)
(66%)
131 (48%)
286 (55%)
European Union
11 (8%)
4 (4%)
52 (19%)
67 (13%)
China
17 (12%)
11 (11%)
21 (8%)
49 (9%)
United Kingdom
6 (4%)
3 (3%)
(4%)
19 (4%)
Canada
2 (1%)
(5%)
11 (4%)
18 (3%)
South Korea
6 (4%)
(6%)
(1%)
16 (3%)
Switzerland
4 (3%)
(0%)
9 (3%)
13 (2%)
Rest of World
11 (8%)
6 (6%)
37 (13%)
54 (10%)
Total
143 (100%)
104 (100%)
275 (100%)
522 (100%)
Table 9: NeurlPS: Crowdsourcing papers' platform use between 2018-2021
Year
MTurk
Other
Unknown
2021
23 (48%)
4 (8%)
21 (44%)
2020
14 (41%)
(0%)
20 (59%)
2019
25 (68%)
(3%)
11 (30%)
2018
15 (63%)
0 (0%)
(38%)
Total
77 (54%)
5 (3%)
61 (43%)
A.7 INSTITUTION ANALYSIS
Table 11 analyses disclosures from ICLR, NeurIPS and Springer Al papers across different institu- tion types. "Joint" indicates that co-authors on a paper represent multiple types of institution.
A.8 VENUE POLICY ANALYSIS
Table 12 table outlines the disclosure requirement venues assessed in this paper, plus other major Al venues, ICML and AAAI. These two venues were added as the next most influential research venues per Google Metrics.
*CHI publication policy is set by the Association for Computing Machinery (ACM), with the conference adhering to this code of conduct which includes IRB requirements. #*ICML publication policy advises authors to follow the NeurIPS Code of Ethics. See: http://lonlccccnnntccc///////ttt
16

---

Under review as a conference paper at ICLR 2023
Table 10: ICLR: Crowdsourcing papers' platform between 2018-2021
Year
MTurk
Other
Unknown
2022
(58%)
(0%)
10 (42%)
2021
26 (65%)
(3%)
13 (33%)
2020
15 (65%)
(0%)
(35%)
2019
18 (60%)
0 (0%)
12 (40%)
2018
9 (82%)
0 (0%)
(18%)
Total
68 (65%)
(1%)
35 (34%)
Table 11: Institution type comparison between 2018-2022 for ICLR, NeurIPS and Springer Al Pa-
pers
Institution
Crowdsoureing
IRB
Payment
Consent
Demographics
University
307 (56%)
42 (14%)
78 (25%)
33 (11%)
62 (20%)
Industry
47 (9%)
1 (2%)
(6%)
(2%)
3 (6%)
State
(1%)
0 (0%)
(%)
(0%)
0 (0%)
Joint
188 (34%)
5 (3%)
24 (13%)
8 (4%)
18 (10%)
Total
546 (100%)
48 (9%)
105 (19%)
42 (8%)
83 (15%)
Table 12: Comparison of venue policy requirements venues considered within this paper, plus
other major Al conferences
Venue
Code of Conduct?
Author Checklist?
IRB required?
Payment Disclosure Required?
NeurIPS
Yes
Yes
Yes
Yes
ICLR
Yes
No
No
No
Springer
Yes
Journal specific
Yes
No
CHI
Yes*
No
Yes
No
ICML
Yes**
No
No
No
AAAI
Yes
No
No
No
17


Reviews:
[{"review_id": "kkaEHUrq8Q", "reviewer": "Reviewer_rXNM", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a study of how ethics considerations related to crowdsourcing are disclosed in the AI community compared to psychology. The study includes ICLR, NeurIPS and Springer journals, which is a reasonable representative sample of AI publications. The primary findings are that AI lags behind psychology in ethics disclosures, but has improved greatly since 2018 particularly at NeurIPS. The paper is unique, important and very clearly written.", "summary_of_the_review": "This paper is important and would likely generate much discussion and debate at the conference. The study could be deeper by considering more conferences, but in its current form it is sufficiently justified and complete to warrant publication.", "strength_and_weaknesses": "Strengths\n\nThe paper tackles a very timely topic with a thorough, well-designed study of the AI community in comparison to psychology. Choosing psychology as a baseline is well founded, as it performs much HSR and has an established practice of rigorous disclosure requirements in its publications.\n\nThe paper reveals important and clear differences between ICLR and NeurIPS, largely stemming from NeurIPS adoption of an explicit checklist of ethics considerations for authors and reviewers. While ICLR has a similar ethics policy as NeurIPS, the introduction of explicit mechanisms to state and review compliance appears to make a huge difference.\n\nSection 6.3 makes an excellent point about the lack of consideration and disclosure of demographic data in AI. This gap has become a source of mistrust of the AI community by the general public in high-profile cases regarding bias in facial recognition systems, for example.\n\nWeaknesses\n\nThe paper has few weaknesses, and these are minor points.\n\nIn 4.1, it would be clearer to state that ICLR does not have an explicit ethics criterion as part of its review process. The reality is that many or most authors and reviewers do not read conference policy documents, nor are they necessarily aware when these are changed. However authors and reviewers will pay much attention to requirements that are encoded into the submission and review processes, such as a checklist that must be filled out, and associated text.\n\nThere is no general discussion of ethics as a review criterion for conference papers, but this is a major recent development that should lead to greater enforcement of ethics policy. It would strengthen the paper to include a table of major AI conferences vs. ethics policies and how they are enforced, even for those that are not included in the detailed study such as CVPR, ICML, AAAI.\n\nThe finding that conference papers “overwhelmingly utilised the MTurk platform compared with others”, sec. 5.1, could be biased by the nature of the paper selection criteria, which searched for MTurk explicitly. What about papers that leveraged labeling service companies, which are increasingly abundant and often comparable in price to MT?\n\nIt would be helpful to add a summary table in sec. 5.4 with comparative numbers between the two conferences and the journal so that the reader does not have to skip back and forth between tables on different pages in order to see their differences.\n\nA major difference between AI and psychology research is the much higher proportion requiring IRB review in the latter, as shown in Table 5. This difference could explain virtually all of the other differences, as IRB review forces researchers to consider many criteria such as potential harm to participants that may not be considered thoroughly otherwise. Data labeling tasks are usually not HSR in the US, and many researchers (correctly) do not seek IRB review for them. Considering only behavioral tasks, table 7, is an insightful way to break down the data as such tasks are more likely to require IRB review. However, I don’t think the data supports the statement that “AI behaviour studies still do not meet the same standards as Psychology experiments” for NeurIPS. IRB review was reported for 45% vs. 73% in psychology, but this could easily reflect a true difference in the nature of the behavioral science being conducted. For ICLR and AI in general the statement is more supportable.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "_jbON-ll-g", "reviewer": "Reviewer_QRKz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper looks at papers in AI submitted at ICLR and NeurIPS for adherence to ethical declaration standards when elements of work has been crowdsourced.This has been compared to the standards met by papers in Psychology and Springer journals.\nThe evidence clearly demonstrates a significant gap in disclosures across all standards including ethics reviews, payment data, demographic data.\n", "summary_of_the_review": "Although the work is good and important, it is unclear to me if this is ahead of time, if the requirements for disclosure for ICLR and NeurIPS were changed in 2021-22.It is hard to interpret this data for ambiguity when there was no requirement,although authors could have volunteered it.", "strength_and_weaknesses": "Strengths:\n- Addresses an important aspect of appropriateness of disclosure and details provided.\n- Clearly demonstrates a gap.\n\nWeaknesses:\n- Although there is a requirement to disclose and they found missing reporting, it is unclear if this is significant. \n- As stated in the paper, while the requirements for disclosure are present now but these requirements seem have been made in 2021-22 for ICLR and NeurIPS.But the data is from 2018-2021. Clearly, voluntary disclosure standards should or could have  be higher than what had been required previously. Hence, not sure if this study is too early. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "L3sndh0DcS", "reviewer": "Reviewer_aCRE", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigated the ethics disclosure in the AI papers that use crowdsourcing as a data collection method published in ICLR, NeurIPS, and Springer journals in the past four years. The main finding of the investigation is that compared with the Psychology paper, ethics disclosures are far less common. Also, the paper has some other interesting findings, such as the journals and conferences have the power to influence engagement with research ethics.", "summary_of_the_review": "This paper investigated the ethics disclosure in the AI papers that use crowdsourcing as a data collection method published in ICLR, NeurIPS, and Springer journals in the past four years. The topic of the paper is interesting. As crowdsourcing platforms have been widely used in AI research, the researchers in this field have not paid enough attention to the ethics in the usage of crowdsourcing. This paper will inspire the thinking of this issue. The paper is clearly presented and easy to follow. However, the paper lacks more deep insights. Some critical issues were not deeply touched and analyzed. Thus, it is below the acceptance bar of this venue. ", "strength_and_weaknesses": "Strength:\n- The topic of the paper is interesting. As crowdsourcing platforms have been widely used in AI research, the researchers in this field have not paid enough attention to the ethics in the usage of crowdsourcing. This paper will inspire the thinking of this issue.\n- The authors chose the Psychology papers as a baseline, which is sound and can disclose the difference between psychology research that involves ethics as a tradition and AI research.\n- The venues where the AI papers were published were chosen soundly. The findings of this research are convincing and interesting.\n\nWeaknesses:\n- The main weakness of the paper is that some findings lack deep insights and thorough discussion. For example, in 2020, the disclosure of the IRB reviews started to appear at AI conferences. Why did the research community begin to pay attention to the IRB reviews? What benefits can the IRB disclosure bring to the crowd workers and the research itself? Another example is in Table 6. For behavior research, the ethical disclosure of AI research is far less than that of psychology but for the other research type, the former is higher than the latter.  For what reason, in behavior research, the ethical disclosure of AI research is rather low?\n- The venues of papers can be extended. As far as we know, there are also many AI papers that utilize crowdsourcing published in AAAI and IJCAI. Also, the papers that utilize crowdsourcing in a more complicated way can be found in WWW, CHI, and HCOMP. Why did the authors not include the paper in these venues?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2UIQpd9HLUp", "reviewer": "Reviewer_8fXg", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper reviews the information reported around human subjects experiments in articles focused on artificial intelligence published at ICLR, at NeurIPS, and in Springer Nature. The authors identity papers that use crowdsourced labor (specifically papers that meet three criteria that they define), and on this set of works they analyze whether IRB is mentioned, payment terms are outlined, workers' consent and demographics are disclosed. They then analyze how reporting varies between \"evaluation\" and \"behavioral\" tasks, and compare it with trends in psychology. They also analyze how trends were affected by policy changes at ICLR and NeurIPS, e.g., the introduction of the checklist. The authors conclude that \"AI research does not meet the standard of psychology research for research ethics disclosures\" and advise that demographic data, when collected, should be disclosed. \n", "summary_of_the_review": "In my review, I have raised several concerns and pointed to the lack of clarity around some details. The paper is not ready for publication at the current stage. The authors may address some of my concerns in the rebuttal.", "strength_and_weaknesses": "The topic is certainly interesting and I agree with the authors that this is something that should be clearly discussed (more) in the machine learning community. However, I have a few concerns. First, I am not totally sure that the topic is appropriate for ICLR (cf the call for papers). While the topic is of interest to the community, this work seems to be more appropriate for CSCW or HCOMP, venues where researchers have deeply thought about this issue. Second, there are several details that are not clear from the paper, described below:\n* How is \"AI research\" defined? There are so many venues (HCOMP/CSCW/CHI etc.) where AI research is done and authors in my experience tend to be careful about IRB reporting. Even other conferences that are noteworthy (CVPR/ACL/FAccT/ICML/AISTATS etc.) are not taken into account here. The authors seem to focus on ICLR and Neurips because of their use of OpenReview, but this is a huge limitation and narrow view of the field. \n* Related to my previous concern, there are subfields within AI/ML where authors often report the demographics of study participants. This is the case for almost all of the papers with Human-AI experiments, which likely fall in the \"behavioral\" tasks (and a clear definition of what is \"behavior\" of study participants should be given), but only some of them are published at the venues considered. The simple partition defined by the authors does not recognize this. \n* There is a very thin and blurry line between what does and does not constitute human subjects research. For example, see the thorough discussion of https://arxiv.org/pdf/2206.04039.pdf. This is not discussed in the paper but is certainly relevant. \n* Sample size in table 1 (and in many other) is very small. Although I think the few trends are large enough that this issue does not undermine the message of the paper, it should be mentioned in the limitations of the analysis. \n* For certain applications, demographics of the participants are important. For some others, they are simply not. For example, in a task where subjects are asked to draw bounding boxes around objects, knowing the demographics of the workers is not fundamental. The authors do not seem to acknowledge this. In addition, (Lum and Isaac, 2016) and (Raghavan et al, 2019) do not mention at all crowdsourcing: They deal with biases in data collection that are totally unrelated to crowdsourcing.\n* In table 8, the analysis should be done conditioning on the venue and on the type of task. The observed patterns could be explained by these factors.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject