Consensus and Subjectivity of Skin Tone Annotation for ML Fairness
Authors: Candice Schumann
Gbolahan Oluwafemi Olanubi
Auriel Wright
Ellis Monk
Courtney Heldreth
Susanna Ricco
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: Fairness
Skin Tone
MST
Annotation
Dataset

Abstract: Understanding different human attributes and how they affect model behavior may become a standard need for all model creation and usage, from traditional computer vision tasks to the newest multimodal generative AI systems. In computer vision specifically, we have relied on datasets augmented with perceived attribute signals (eg, gender presentation, skin tone, and age) and benchmarks enabled by these datasets. Typically labels for these tasks come from human annotators. However, annotating attribute signals, especially skin tone, is a difficult and subjective task. Perceived skin tone is affected by technical factors, like lighting conditions, and social factors that shape an annotator's lived experience.
This paper examines the subjectivity of skin tone annotation through a series of annotation experiments using the Monk Skin Tone (MST) scale, a small pool of professional photographers, and a much larger pool of trained crowdsourced annotators. Along with this study we release the Monk Skin Tone Examples (MST-E) dataset, containing 1515 images and 31 videos spread across the full MST scale. MST-E is designed to help train human annotators to annotate MST effectively.Our study shows that annotators can reliably annotate skin tone in a way that aligns with an expert in the MST scale, even under challenging environmental conditions. We also find evidence that annotators from different geographic regions rely on different mental models of MST categories resulting in annotations that systematically vary across regions. Given this, we advise practitioners to use a diverse set of annotators and a higher replication count for each image when annotating skin tone for fairness research.

Consensus and Subjectivity of Skin Tone Annotation for ML Fairness
Candice Schumann Google United States
Gbolahan O. Olanubi Google United States f femi Gemio@google.co
Auriel Wright Google United States
cschumann@google. com
aurielwrig aurieelrrighh@google.coom
Ellis Monk, Jr. Harvard University*
Courtney Heldreth
Susanna Ricco Google United States ri ricco@google.co com
Google
United States
United States
emonkêfas harvard. edu
cheldrethh@oogge... com com
Abstract
Understanding different human attributes and how they affect model behavior may become a standard need for all model creation and usage, from traditional computer vision tasks the newest multimodal generative Al systems. computer vision specifically, we have relied on datasets augmented with perceived attribute signals (e.g., gender presentation, skin tone, and age) and benchmarks enabled by these datasets. Typically labels for these tasks come from human annotators. However, annotating attribute signals, especially skin tone, is difficult and subjective task. Perceived skin tone is affected by technical factors, like lighting conditions, and social factors that shape an annotator's 's lived experience.
This paper examines the subjectivity skin tone annotation through a series annotation experiments using the Monk Skin Tone (MST) scale (59], small pool professional photographers, and much larger pool of trained crowdsourced annotators. Along with this study we release the Monk Skin Tone Examples (MST- E) dataset, containing 1515 images and 31 videos spread across the full MST scale. MST-E is designed help train human annotators to annotate MST effectively. Our study shows that annotators can reliably annotate skin tone in way that aligns with an expert in the MST scale, even under challenging environmental conditions. We also find evidence that annotators from different geographic regions rely different mental models of MST categories resulting annotations that systematically vary across regions. Given this, we advise practitioners to use diverse of annotators and higher replication count for each image when annotating skin tone for fairness research.
1 Introduction
Machine learning models are ubiquitous in today's society, influencing almost every aspect of our lives. Widespread adoption brings with it heightened scrutiny, with focus responsible model creation from data curation to evaluations As a result, understanding different human attributes and how they affect model behavior may become a standard need for all model creation and usage, from traditional computer vision tasks the newest multimodal generative AI systems 4T]. In computer vision specifically, we foresce skin tone becoming significant part of this conversation. Indeed, the past few years have seen
evident in regulation and policy discussions 24 40 67).
Ellis Monk performed this work while also a Visiting Researcher at Google.
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

MSTI
MST2
MST3
MST4
MST5
MST6
MST7
MST8
MST9
MSTI0
Figure I: Monk Skin Tone (MST) scale (59
significant investment toward computer vision models that work well for everyone, enabled by improved standards for evaluation [8] 71). better data curation and analysis [22] 81], and algorithmic advances 4 64 [6]. To leverage these advances, many practitioners rely image datasets enriched
with third-party fairness annotations, such as perceived age, gender presentation, and skin tone 777 82]. In this paper we focus on the collection of fine-grained perceived skin tone annotations via third party annotators.
Our focus skin tone is inspired by research in the social sciences and machine learning. Research shows that perceived skin tone affects how we view and treat other people. Specifically, it affects interpersonal interactions 44
62
65
job applications [12. interpersonal relationships [17], wages,
educational attainment and occupation, and health outcomes among minority ethnic and racial groups
in the United States [31 52
54
55
60
61
63 74)
Machine learning algorithms often perform poorly on images of people with darker skin tones reinforcing the importance of considering these subgroups during model selection. Skin tone measures have been valuable tool allowing fairness researchers
address top-of-mind challenges [16].
Researchers can devise metrics to measure diversity and representation in dataset collection (70]
image generation [4], and image retrieval systems [TO 58 Finer-grained skin tone scales, such
as MST (see Figure I promise to further improve the accuracy and fairness of machine learning systems.
These subjective judgements can be affected by social factors and lived experience [18 72 For example, social factors such as political affiliation [9. or the inferred race/cthnicity of an individual [30 can affect the perception of skin tone. Noisy estimates can lead to incorrect causal attribution of model bias 42), and wer-indexing on outlier annotations or using annotations that are unduly affected by transient environmental factors can overestimate measures of diversity and representation. We must address these challenges to ensure that downstream analyses using skin tone annotations are trustworthy.
Annotating skin tone is an important but subjective task 34 35 48
This raises the question: What is the best method to systematically annotate skin build datasets and benchmarks for fairness research? In this paper we focus on the following major questions:
a) Who can annotate skin tone and what are the implications for different types of annotators?
b) Does annotator location, a proxy for cultural context and lived experience, matter and have any effect on annotation quality? If so, how can this best be managed?
c) How can practitioners best navigate the complexity of skin tone annotation and are there any best practices to consider?
To address these questions. we release the MST-E dataset, a curated dataset of exemplar images with MST annotations, help train new annotators and study the behavior of annotator pools. Based on the results, we provide suggestions for best practices for designing skin tone annotation tasks.
2 Background and Related Work
In this section we look into related work on subjective annotations, skin tone, and fairness computer vision. For a more in-depth look at work in these areas see Appendix A
Subjective Annotations Soliciting annotations for subjective concepts has been a subject of con- siderable research across machine learning, from natural language processing (NLP) and affective computing to classical computer vision tasks such as attribute recognition. Goyal et al. 32) found differences annotations of toxicity in online conversations when using specialized annotator pools comprised of individuals identifying African American and LGBTQ, respectively. Aroyo et al. 2

---

annotate conversations for safety using annotator pools across two geographic regions. In both papers the authors significantly increased replications compared standard practice, finding differences in annotator behavior between groups where single annotator's s judgements would not necessarily be stable across multiple views of the same content. Díaz et al. (20] argue for careful design of annotation tasks, accounting for the socio-cultural backgrounds of annotators and considering lived experiences as aspects of expertise. Given annotations for a subjective task from a diverse set of annotators, researchers are increasingly adopting model architectures that target individual annotators instead of or addition to an overall consensus estimate [5] 18 37.. Our work adds to the growing literature that considers the importance of better understanding and modeling the individual perspec- tives of annotators for subjective tasks, and extends the general recommendations for practitioners by focusing specifically skin tone annotations for fairesss in computer vision.
Skin Tone Research The Fitzpatrick Skin Type (FST) scale has become gold standard of sub- jective skin tone measurement in dermatology [26]. The six-point scale, initially conceived as four-points, was developed skin to phototherapy. exposure to ultraviolet light. Despite its wide use, there are some notable limitations of FST for skin tone annotation. In particular, the scale was not developed to assess skin tone; rather, skin, hair, eyes, and history of sunburn were used to assess likelihood of skin burning during phototherapy. In addition, to make the scale more inclusive of people with Asian, Indian, and A African heritage, Type and Type VI were later added to the four-point scale [27]. Even with the two
dermatological tool to measure the response of different types of
darker additions, FST has been criticised for being skewed toward lighter skin tones (23 29 8 with researchers calling for finer grain skin tone measurements within dermatology [66] and for fairness in
computer vision 59 That being said, fine-grained measurement is not without its own challenges
and limitations.
Skin tone scales with too many shades, like the 36 36-point Felix von Luschan Skin Color chart [80], may be too granular and introduce additional cognitive load for annotators during an already challenging task, potentially reducing inter-rater agreement. In addition, it may be difficult for annotators to identify differences between skin tone shades at such fine-grained measurement, leading to lower annotator accuracy compared to an expert ground truth assessment.
We have chosen to have annotators use the 10-point MST scale (Figure D. At 10 points, the MST scale was designed for a broader and more diverse range of skin tones compared to the FST without having too many options and overburdening annotators [59]. Although a finer-grained scale adds more complexity and difficulty to achieve consensus at the annotation stage (compared to, for example, annotations for a binarized skin tone annotation task), the MST allows for a more nuanced understanding of system behavior, better targeted disaggregated evaluations and audits, and more capable measures of representation. In addition, over the past year the MST has been adopted by computer vision fairness researchers expand fairness annotations and evaluate models [14] 66). We focus on exploring annotator behavior through the MST scale to better understand its potential as a more inclusive skin tone measurement tool in computer vision.
Annotations for ML Fairness in Computer Vision Perceived signals such gender expression, skin tone, and age are commonly used in fairness evaluations and bias mitigation techniques in computer vision 8 77 82 84] There are four main methods currently used in collecting these anno- tations: third party crowdsourced annotations, annotations from captions, self reported annotations, and algorithmic or model generated annotations.
The most scalable solution for obtaining fairness annotations is through automated annotations with models or objective measures. However, most fairness annotation models require face or person detection or other preprocessing steps, and variable performance of upstream components across skin tone groups lead to missing annotations for underrepresented groups during fairness evaluations [77]. In addition, these models must first be trained on annotated data. Alternatively, prior work has used measurements computed from pixel values of detected skin regions such as ITA [II] or matched skin tone pixels directly to an RGB scale [4]. In both cases these results can vary due to the influence of scene lighting or other imaging artifacts, making them less reliable as annotation
tool for in wild datasets [25
45 48] (see Appendix G
Image captions are often used to provide gender fairness annotations for images 57
85
However, relying on captions for skin tone fairness indicators not a viable option because captions in existing datasets rarely include description of image subject's 's skin tone.
3

---

Hazirbas et al. [3]] released the Casual Conversations Dataset which includes both self reported gender and age. Datasets with self-reported annotations are expensive to collect and tend to be very "clean" and not in-the-wild images or videos. There is also an important distinction between subjective attributes such as skin tone, non-subjective attributes such as gender identity and age, and the perception of these attributes. While self-reported values are extremely useful and important [8)] the choice to use self-reported values vs third-party annotated values depends on the perspective practitioners want to understand: a) how models interact with the identities and non-subjective attributes of users or b) how models perform on different perceived attributes across a wide range of imagery (including generated imagery). We note that both Hazirbas et al. [88 and the expansion in Porgali et al. [69] choose the perspective of third-party annotations for skin tone.
Data collected with annotations from topical experts and crowdsource annotators are common in computer vision. For skin tone, these annotations are often based on FST using dermatologists as annotators 8
38 46 due to their expertise with the FST as a dermatological assessment tool. However, topical expert annotation is not scalable to larger datasets, and has been reserved for relatively smaller annotation exercises. Crowdsourced annotations leveraging non-expert annotators has been a scalable alternative to expert annotation [38 [5]. Critically for this work, studies find that expert and crowdsource annotators tend toward fairly similar annotations on the FST [3 48 50). In
this paper we take a third party annotation approach and use both topical expert and crowdsource
annotators using the MST scale.
3 The Monk Skin Tone (MST) Scale and the MST-E Dataset
The Monk Skin Tone (MST) scale defines 10 representative categories using exemplar color swatches, either as flat patches, or spheres which also incorporate shading into the stimuli. Figure shows these spheres. An annotator's 's task is to translate from this "illustrative reference" " [59] to assess the skin tone of people depicted in images captured under varying environmental conditions.
This paper introduces the open-access accesss MST-E datase consiising of images from 19 subjects spanning the 10 point scale (see Figure 2 The goal of this dataset is to enable and test consistent skin tone annotation across environment capture conditions. It provides a starting point to support practitioners to teach their annotators to annotate skin tone consistently on the fine-grained MST scale and controlled set through which to study annotator behavior. See Appendix C for the MST-E data card.
MST-E models, and subsequent images, were sourced through TONI We selected 30 candidate subjects from an initial pool of hundreds of models, 19 of whom were eventually photographed The 19 subjects span the full MST scale and include individuals of different ethnicities and gender identities to help human annotators decouple the concept of skin tone from race.
All images and videos of a subject were collected within a short timespan of 4 hours to eliminate skin tone variation across images due to seasonal or other temporal effects. Each subject was photographed in various poses, facial expressions (smiling, neutral, sad/frown face, and silly face), and lighting conditions. Some subjects provided a video of themselves talking to the camera, and some subjects provided images of themselves with a facial mask. See Figure 2 for example images of each of the 19 subjects. In total, the MST-E dataset contains 1515 images and 31 videos across all subjects.
The images in the dataset were reviewed and annotated using the MST scale by the authors as well as the MST creator, Dr. Ellis Monk, an expert in social perception and inequality. Images were first annotated by an author and sent to Dr. Monk for final golden annotations. Dr. Monk selected an image for each subject that best represents their skin tone (called the Golden Images set) and approved the corresponding MST annotation. We refer to these annotations as the oracle or intended annotations for the images. We curated a corresponding Challenging Images set which mirrored the Golden Images set with different lighting conditions. See Figure (b)) for an example of "golden" versus "challenge" images and Appendix B for the full Golden Images and Challenging Images sets.
Consent and terms of use Each subject provided consent for their images and videos to be released. TONL has given permission for these images be released and used for research, model evaluations.
httpa://skinton htt////iiintnneggggggg///t/teeee dataset. https://tonl.co/

---

Figure 2: The MST-E dataset contains subjects that span the full MST scale. Each subject has images in variety of lighting conditions and poses. Images by TONL. Copyright TONL.CO 2022
ALL RIGHTS RESERVED. Used with permission.
annotator- training purposes only. The images cannot be used to train machine learning models. The images and annotations are licensed under CC BY
4 The Annotation Experiment
We perform two different studies in this research. The first is qualitative survey with group of skin tone "experts", photographers from two different geographical regions. We describe our design choices concerning the qualifications of these experts in detail in Section The "experts" annotated the Golden and Challenging MST-E sets ($3). The second study expands on the results of the small scale study in two ways: extending to larger pool of trained crowdsourced annotators located in five different geographical regions, and to an MI.-training-sea
dataset ($6). Crowdsourced annotators were asked to annotate skin tone across the full MST-E dataset and subset of the Open Images dataset [49] explore whether findings from the controlled settings generalized to practical settings. Both of these studies were conducted under the guidance of our institution's 's ethics body.
In each study, examine the degree which annotators within the same region agree with each other, whether annotators from different regions agree, and how well the consensus annotations match the intended annotation across different conditions. In the following sections, when calculating the consensus MST annotation for an image we use the median MST annotation from the responses.
Typically, the only available indicator of annotation quality is measure of inter-rater reliability. However, in this instance we have MST annotations provided by the creator of the MST scale. Thus, have the unique opportunity to compare annotations from the annotators to his annotations.
Previous research has taken a similar approach comparing FST annotations between crowdsource annotators and an expert dermatologist, and has found that crowdsource annotations are typically within I-point of expert dermatologist annotations 33). This threshold of 1-point discrepancy has continued to be used when comparing skin tone annotations between crowdsource annotators and experts [34). Here, we consider alignment between the consensus from the photography experts or crowdsoureed annotators and Dr. Monk's intent using two metrics: the percentage of median MST annotations that are within 1-point of discrepancy from the oracle annotations (higher is better), and the average median distance, the expected difference between the median annotation from group of annotators and the oracle computed across all images (ideally less than 1).
5 Exploration with Experts
We used Gerson Lehrman Group (GLG) to recruit and compensate panel of five experts in photog- raphy India and five experts in photography the United States to complete skin tone annotation task. Experts in India were selected because we later recruit crowdsource annotators from this market. The U.S. was selected because we sought geographically distant comparison market in which GLG could recruit a similarly sized panel of experts. Similar skin tone annotation research has sought annotations from dermatologists (44, but we did not want experts who were practiced reviewing images taken under highly controlled conditions for medical issues. Rather, we wanted experts who
5

---

were practiced in capturing and editing still images of people and examining skin in their photography. During an initial pilot we found that different types of experts may a) approach the task differently based on their discipline and b) this might result in differences in annotation. Specifically we found that photographers tended to approach the task as designed, assessing the skin tone in the image while correcting for extraneous image features. However, dermatologists interviewed in the pilot study were more likely to attempt to assess skin tone beneath clothing which was not depicted. This aligned with skin tone assessment in medical scenario, but not with our goal of achieving skin tone annotation in images that aligns with social perception. For this purpose we recruited photographers who self-re self-reported that people and portraits were the subjects of their work, had previous experience reviewing and editing skin tone in color correction software, and were most likely to approach the task as designed.
Experts were asked annotate a total of 50 images, 19 from the MST-F Golden Images or Challenging Image sets ($3) and 31 aesthetically similar distractor images. Images were presented in randomly assigned order and the participants were asked to view each image and select which skin tone on the MST best matched the person depicted in the image. The task was completed twice. Participants first annotated the Golden Images set and distractor images. Approximately one week later, participants annotated the Challenging Images set and new set of distractor images The experimental setup was designed to examine consistency of annotation across lighting conditions, with the time delay reducing the likelihood that participants would annotate the Challenging set based their memory of their previous selection.
Experts were shown an image and asked to select the skin-tone that matches most closely on the MST scale, to select which part of the person's 's body they used in their assessment, and to rate on a 5-point scale the ease/difficulyy of the annotation. An example of the task can be found in Appendix
Expert Intraclass Correlation (ICC)
1.0
0.6
thillenging
Golden
0.1
0.2
Inter-Rater Reliability We examine if the skin tone annotations are consistent among experts within each market using intraclass correlation (ICC), mea- sure of inter-rater reliability. Each expert reviewed each image, so we use a two-way random, single score ICC where the subject and annotator are viewed as random effect [56). We find that experts in the tested markets do consistently annotate skin tone on the MST. Figure shows significant, p <.05, intr- aclass correlation in both markets and in both image quality conditions.
0.0
(a) Photography Experts ICC
Crowdsourced Intraclass Correlation (ICC)
Per Sunject
the Golden.
02
Hnsegpar
Globel
Regional Differences in Annotation We con- ducted a series of Mann Whitney U tests [33] on each individual image determine if there were any significant differences in the annotation between the experts from the United States and those India (see Appendix E for full comparisons). Differences were significant using p- < 0.05 for three of the im- ages. Figure 4 shows the annotations for two of these in detail. The professional photographers in India tended to annotate the subject's 's skin as lighter (MST3-. MST5). Professional photographers U.S. tended to annotate the subject's 's skin as darker (MST5- MST7). The differences observed provide suggestive evidence that geographic location affects skin tone annotation, which we explore further and at scale using crowdsourced annotators.
(b) Crowdsource ICC
Figure 3: Inter-rater reliability (IRR) be-
tween annotators calculated via intraclass cor- relation (ICC). Figure [[a] shows the ICC scores for expert photographers in the US and India on the Challenging set and the Golden set. Figure 3(b] shows the ICC scores for crowdsourced annotators in each region, as well as the global pool. These ICC scores are calculated per subject, as well as over the Challenging and Golden sets.
the
'One of the MST-E subjects was excluded from this study due a programming error.

---

Disiitution frrm Aenoratoss
Ratro Drdritution Arrotaton
the
the
(a) Golden Image
(b) Challenging Image
Figure 4: Photography experts in India (top plots) and the US (bottom plots) MST annotation counts for Subject 14, Figure 4(a) shows Subject 14's Golden Image chosen by the MST scale expert ($3). Figure (bb shows Subject 14's matching Challenging Image. In large-scale experiments with trained we found consistently measurable differences in annotator behavior based on region.
annotators (
Distance from Intended Annotations In both markets we found that a high percentage of the consensus annotations were with 1-point of discrepancy of the oracle annotations, 88.9% in India and 83.4% in the United States. In addition, in both markets the average median distance from the oracle annotation was not above the 1-point guardrail established in previous research [3 34 0.78 in India and 0.84 in the United States. These results suggest that photography experts in India and the U.S. annotate skin tone in manner that aligns with the scale creator's intent.
6 Exploration of Crowdsourced Annotations
We collected annotations for the entire MST-E dataset and subset of the Open Images dataset from annotators in India, the Philippines, Brazil, Hungary, and Ghana. We followed standard processes for data annotation for computer vision-related work, using the services of professional data annotators without specific domain expertise outside of the instructions they received for this task. Annotators are compensated for their work (see Appendix K for more details). Each task was replicated 5 times in each region resulting in 25 annotations per image.
During task, annotators shown thumbnail of subject and the full image with box drawn around the location of the thumbnail. This allows annotators to understand the context and lighting of asked to select the best skin tone for the subject and report which part of
the image. Annotators the body they primarily used to make the selection (Face, Torso, Arms/Hands, Legs/Feet, or Other). Annotators could select that an image
difficult to annotate for skin tone, and skip the task if no skin was visible. See Appendix D for an example of the selection tool.
mixed into much larger set of similar images to prevent face recall among annotators. In the results section we look at the annotations Per Subject meaning annotations across all images of single subject as well as the annotations for the Golden Images and Challenging Images sets ($3)
Since the MST-E dataset contains many images for each subject, the images
Inter-Rater Reliability Five annotators were randomly selected from broader pool of annotators within each market to annotate each image. As the group of annotators was not consistent from image me- random, single score ICC to measure inter-rater reliability, where the subject a random effect for this analysis [56].
to image,
Consistent with our results from professional photographers, we find that non-expert annotators also consistently annotate skin tone on the MST, using shared mental model of the MST categories. Figure 3(b) shows significant, p < 0.05, ICC values for each region and annotation task. For every region, the ICC is very high with Per Subject ICC ranging from 0.86 to 0.94. In addition, we find some evidence that annotators remain consistent in skin tone annotation even when image quality is diminished. The Golden Images ICC ranged from 0.9 to 0.96, and Challenging Images ICC ranged from 0.82 to 0.95 a comparatively lower range but still significant and consistent.
7

---

Regional Differences in Annotation We found suggestive evidence for regional differences in annotation in Section and know that geographical region affects annotations other domains like opinion of the safety of chatbots [2]. Unlike the exploratory analysis of experts that used subset of the MST-E dataset, in this setting we have enough statistical power to explicitly test the effect of the annotator's s region on MST annotation. We performed multivariate analysis of variance (MANOVA) [28] with annotator region as the independent variable and MST annotations for each subject as the dependent variable. We found that the subject annotations were a significantly high predictor of the region of the annotators (Pillai's trace = 1.97, p < 0.05). This result suggests that the shared mental models of skin tone may vary significantly across different geographic regions.
Distance from Intended Annotations Despite measurable regional variation, we find high percentage of median MST annotations that are within point of oracle annotations. Using the global pool, the consensus annotation agreed within point for all but one subject (94.7%). In addition. we find the average median distance to oracle annotations to be less than for every region and image condition. The distance was 0.78 for both well lit and poorly lit images. This suggests that crowdsource annotators, similar to the expert photographers, arrive at consensus MST annotations that are similar to the scale creator's intent.
Replications The 25 annotations we acquired per image is larger than most practitioners collect. We use the Spearman- Broon prediction formula to determine inter-rater agreement over k replications. where & is the total number of replications over all regions (See Figure 5) 7 79 g3
We find that for datasets containing high quality images or annotations of single individual based on consensus across multiple images, one rater per region likely suffices (5 total annotators). For more challenging images, two annotators per region is beneficial. Therefore, if practitioner's budget is limited, we would suggest filtering for images with good lighting conditions (or even images with okay lighting conditions). If annotations on images with low quality lighting is necessary, practitioners can then annotate with 10 annotators just those low quality images.
Extension to In-the-Wild Annotations Open Im- ages is large-scale image dataset with number of image and object annotations [49]. We randomly selected subset of 41,471 images that contained at least one face. In the 41,471 images, 86,809 faces were present to annotate. The annotators used the same annotation tool as they did with the MST-E dataset (see Appendix D). Using this tool annotators could see the face thumbnail along with its location in the full image.
Spearman- -Boww Prediction over Rater Replication
2.00
0.98
0.96
0.92
Subject
0.90
Challenging
Golden
15 Rater Replication
Unlike the MST-E dataset where images were taken in a controlled setting, Open Images is an "in-the- wild" dataset containing many more unique individu- als and images of more varying quality. Even in these uncontrolled settings we expect that crowdsource an- notators will still consistently annotate skin tone on the Monk Skin Tone scale and that geographic region will have significant effect on the annotations.
Figure 5: Spearman Brownn inter-rater relia- bility calculations over replications of anno- tators on the MST-E dataset per subject, over the Challenging Images, and over the Golden Images.
Consistent with our hypotheses, we find high intraclass correlation across markets ranging from 0.875 to 0.91. In addition, the intraclass correlation from the Open Images dataset has no significant differences than crowdsource annotators intraclass correlation in the MST-E dataset (x² 0.26, 0.99 > 0.05). This suggests that crowdsource annotators are similarly consistent in MST annotations in controlled datasets and in-the-wild data sets. We also find significant differences between annotation distributions between regions (V²2 = 0.0766, p < 0.05), consistent with the MST-E annotations. Overall, this shows that recommendations based on annotator behaviors observed in our controlled studies likely translate real-world scenarios commonly faced by ML practitioners.

---

Percentage of MST value annotations
25
20
15
10
MSSTF
INSTE
MSTE
MSTE
INTT
MSTE
MSTE
MSTE
MSTTE
MATE
MST1
IMST2
MST3
MST4
MSTS
MST6
MST7
MST8
MST9
MST10
Figure 6: Comparison of the distribution of skin tone annotations in the MST-E dataset ("MSTE" in the plot) versus the "in-the-wild" Open Images dataset ("OI" in the plot). The Open Images dataset has strong skew towards lighter skin tones.
7 Discussion and Suggestions for Practitioners
One of the first design decisions practitioners have to make when starting skin tone annotation task the composition of the annotator pool. Following from common practices in literature [8 (5) we analyse .STT annotations from subject matter experts, professional photographers who focus on photographing people, and crowdsourced annotators who were provided training in MST skin tone annotations prior the annotation task. We find that both the photographers and trained annotators were able to reliably annotate for skin tone suggesting that subject matter experts are not necessarily needed for MST skin tone annotation tasks. This allows for more scalable annotation tasks using trained annotators on large datasets.
Additionally find that although trained annotators have high inter-rater reliability across all regions, there are regional differences between the MST annotations, suggesting that regional cultural contexts influence how people perceive skin tone. When combining the annotations across variety of regions, we find that the median of the annotations are consistently close to Dr. Monk's 's annotations. This suggests that geographically diverse annotator pool provides annotations that are in line with the intended use of the MST scale [59].
We find that annotators able to learn to be invariant to environmental factors such lighting quality, and find that they perform similarly on the mnllhe-wild subset of Open Images to the MST-E dataset, suggesting that performing skin tone annotation task on in-the-wild images is possible.
The MST-E dataset a valuable tool to help practitioners train their human annotators to annotate MST on images. But beyond that use case, the dataset can also be used to evaluate models. Particularly, the MST-E dataset can be used to evaluate fairness over the full MST scale as well as to evaluate performance of models over varying lighting conditions and poses. This is particularly useful since "in-the-wild" datasets like Open Images may have large skew in skin tone distribution (See 6igure
Limitations and Ethical Considerations This paper discusses the variance introduced by different geographical regions and types of annotators. However, does not consider other aspects of annotator identity and background, or other factors such as the language used in the task, different UT improvements, and annotators are affected by other people in the same image. Future work should look at these issues.
This work suggests the use of 10 annotators per image image quality is unknown. This very large jump from the status quo of 3 annotators per image, which could be limiting factor for practitioners with lower budget. A potential alternative for very restricted budgets would be to use large and geographically diverse pool of annotators but have only a subset of annotators provide explicit judgements for each image. One would then impute missing annotations based on a machine learned model personalized for each annotator. Similar approaches have been tested for other types of subjective annotations (in NL.P [18) and in face similarity (T] but not yet for skin tone. These methods should be investigated in future work.

---

The MST-E dataset has a small number of subjects (19), and while they do span the complete MST scale, they do not capture every variation in skin tone. Future work should extend this dataset to more subjects. In addition, the exploratory analyses of annotations from photography experts more limited than the scale achieved with crowdsourced annotators. That being said, with the additional analysis of the Open Images dataset, we believe our results will translate to other domains. While we use thresholds for agreement (c.g., 1-point discrepancy) that are in line with prior work when describing the resulting annotations as reliable, future work should focus on usefulness of the annotations in fairness analysis or mitigations leading to more inclusive and robust computer vision systems.
The MST-E dataset release enables the annotation of an aspect of person's 's appearance which has historically been the source of discrimination and continues to affect how people are treated in society, While there is the potential for malicious use of skin tone analysis, we believe the clear benefits for fairness research outweigh the potential risks. We chose not to collect or release other sensitive information about the subjects the dataset or our annotators, beyond general geographic region, to protect their privacy. While we have consent from participants to release the MST-E dataset for research and annotator training purposes, there remains a risk that the images could be used maliciously in model training. In order to mitigate these risks we clearly state the terms of use and do not allow for anonymous access to the image dataset.
Suggestions for practitioners Extrapolating from our results, we make the following suggestions for practitioners who are pursuing a task of MST scale annotation.
Diversity of annotators is important for subjective annotations. our case cultural dif- ferences across geographic regions influenced skin tone annotations. Other subjective annotations may require different types of diversity (i.c., gender in heteronormative commu- nities vs members of the LGBTQ+ community).
Given the wide range of annotations we suggest having more than three annotators per subject (the current status quo) and instead have least two annotators in at least five different geographical regions (10 ratings per image), particularly for challenging images.
If you know the geographical region of your data deployment scenario, it may be wise to use annotators from the same region. We recommend considering this design choice due to the difference behaviors across geographic pools we observed. We hypothesize that co-located annotators may have a better cultural context for certain annotation tasks that would provide the most relevant data, although more careful study of this is needed.
Train annotators with examples of individuals across the full MST spectrum in variety of conditions. The MST-F dataset can be used to perform this training.
8 Conclusion
Fairness signals/attributes are important to determine how well models, and eventually products using these models, work for every person. Skin tone is particularly important for computer vision models and applications. However, annotating skin tone (even annotating one's own skin tone) is a subjective task. We make the following contributions:
a) We show that skin tone can be annotated reliably on the MST scale by both trained crowd- sourced annotators and subject-matter experts such as professional photographers.
b) We show that annotators have an understanding of skin tone across conditions and, with training, can provide annotations invariant to lighting conditions.
c) We show that cultural contexts across geographical regions influence MST annotations. However, when collecting annotations from diverse geographical pool, annotators are able to provide annotations that match the original intention behind the MST scale.
d) We provide actionable suggestions to practitioners collecting subjective annotations.
c) We release the MST-E dataset to help practitioners train annotators to be invariant to lighting conditions and support further research.
Although fine-grained skin tone annotation a difficult task, it possible and it opens up the door to more nuanced understanding of bias in model performance.
10

---

Acknowledgements
We wish to thank our colleagues across Google working on fairness and inclusion in computer vision for their contributions to this work, especially Marco Andreetto, Parker Barnes, Ken Burke, Benoit Corda, Tulsee Doshi, Rachel Hornung. David Madras, Shrikanth Narayanan, Utsav Prabhu, Sagar Savla, Alex Siegman, Komal Singh and Biao Wang. We also would like to thank Annie Jean- -ann-Bpptstte, Florian Koenigsberger, Marc Repnyek, Maura O' Brien, and Dominique Mungin and the rest of the team who help supervise, fund, and coordinate our data collection.
References
[1] Jerone TA Andrews, Przemyslaw Joniak, and Alice Xiang. view from somewhere: Human-centric face representations. arXiv preprint 2331:23303.77766, 2023.
[2] Lora Aroyo, Mark Diaz. Christopher Homan, Vinodkumar Prabhakaran, Alex Taylor, and Ding Wang. The reasonable effectiveness of diverse evaluation data. arXiv preprint arXiv:2301.0944 2023.
[3] Solon Barocas, Anhong Guo, Ece Kamar, Jacquelyn Krones, Meredith Ringel Morris, Jennifer Wortman Vaughan, W Duncan Wadsworth, and Hanna Wallach. Designing disaggregated evaluations of ai systems: Choices, considerations, and tradeoffs. In Proceedings of the 2021 AAAT/ACM Conference Al, Ethics, and Sociery, pages 368-378, 2021.
[4] Joachim Baumann, Anikó Hannák, and Christoph Heitz. Enforcing group fairness in algorithmic decision making: Utility maximization under sufficiency. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 2315-2326, 2022.
[5] Hugo Berg, Siobhan Mackenzic Hall, Yash Bhalgat, Wonsuk Yang. Hannah Rose Kirk, Aleksandar Shtedritski, and Max Bain. A prompt array keeps the bias away: Debiasing vision-language models with adversarial learning, arXiv preprint arXiv:2203.1 1933, 2022.
[6] Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: misogyny, porno- raphy, and malignant stereotypes. arXiv preprint arXi::21 2021.
[7] William Brown Some experimental results in the correlation of mental abilities 1. British Journal of Psychology, 1904-1920, 33)):296-322, 1910.
[8] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77-91. PMLR, 2018.
[9] Eugene M Caruso, Nicole Mead, and Emily Balcetis. Political partisanship influences perception of biracial candidates' skin tone. Proceedings of the National Academy of Sciences, 206648)2001682011 2009.
[10] I. Elisa Celis and Vijay Keswani. Implicit diversity in image summarization. Proceedings of the ACM on Human- Conpuute- Interaction, 4(CSCW2):1-25 2020.
[11] Alain Chardon, Isabelle Cretois, and Colette Hourseau. Skin colour typology and suntanning pathways. International journal of cosmetic science, 13(4):191-208 1991.
[12] Jacqueline M Chen and Andrew Francis-Tan. Setting the tone: An investigation of skin color bias in asia. Race and Social Problems, 1402)11 2022.
[13] Xi Chen, Xiao Wang, Soravit Changpinyo, Piergiovanni. Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong. Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish Thapliyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhosseini, Chao Jia, Burcu Karagol Ayan, Carlos Riquelme, Andreas Steiner, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, and Radu Soricut. Pali: A jointly-scaled multilingual language-image model, 2022. URL URt.tttss::////e /tttp:://arxiv. org. abe/ 2w...00009200 06794
[14] Jaemin Cho. Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generative transformers. arXiv preprint arXiv:2202.04005 2022.
[15] Huang-Cheng Chou and Chi-Chun Lee. Every rating matters: Joint learning of subjective labels and indi- vidual annotators for speech emotion classification. In ICASSP 2019-2019 IEEE International Conference Acoustics. Speech and Signal Processing (ICASSP), pages 5886-5890
IEEE, 2019.
11

---

[16] Alexandra Chouldechova and Aaron Roth. The frontiers of fairness in machine learning. arXiv preprint urXiv.1810.08810. 2018.
[17] Nadia Craddock, Aisha Phoenix, Paul White, Caterina Gentili, Phillippa C Diedrichs, and Fiona Barlow. Understanding colourism in the uk: development and assessment of the everyday colourism scale. Ethnic and Racial Studies, pages 1-36, 2022.
[18] Aida Mostafazadch Davani, Mark Diaz, and Vinodkumar Prabhakaran, Dealing with disagreements: Look- ing beyond the majority vote in subjective annotations. Transactions of the Association for Computational Linguistics, 10:92-110, 2022.
[19] Sandra Del Bino and FJBJoD Bernerd. Variations in skin colour and the biological consequences of ultraviolet radiation exposure. British Journal of Dermatology, (69(s3)333-40, 2013.
[20] Mark Díaz, Ian Kivlichan, Rachel Rosen, Dylan Baker, Razvan Amironesei. Vinodkumar Prabbakaran. and Emily Denton. Crowdworksheets: Accounting for individual and collective identities underlying crowdsourced dataset annotation. In 2022 ACM Conference on Fairness, Accountability, and Trans- parency, FAceT '22, page 2342-2351, New York, NY, USA, 2022, Association for Computing Machinery, ISBN 4781450393522. doi: 10. 2011145/3331144 14635366477 3534647
[21] Djellel Difallah, Elena Filatova, and Panos Ipeirotis. Demographies and dynamics of mechanical turk workers. In Proceedings of the eleventh ACM international conference web search and data mining, pages 135-143, 2018.
[22] Chris Dulhanty and Alexander Wong. Auditing imagenet: Towards model-driven framework for annotating demographic attributes large-scale image datasets. arXiv preprint 1XXv:1105.0014// 2019.
[23] Steven Eilers, Daniel Q Bach, Rikki Gaber, Hanz Blatt, Yanina Guevara, Katic Nitsche, Roopal V Kundu, and June K Robinson. Accuracy of self-report in assessing fitzpatrick skin phototypes through vi. JAMA dermatology,
2013.
[24] European Parliament and Council of the European Union. Regulation (HU) 2016/679 of the European Parliament and of the Council, 2016. URL, wtt.tttts:://///t /tttps://daaaata /aapa////aaaaaaaaap.aaaaa wt//el///eee2216/79///1
[25] Haiwen Feng, Timo Bolkart, Joachim Tesch Michael J Black, and Victoria Abrevaya. Towards racially unbiased skin tone estimation via scene disambiguation. In Computer VisionEECC 2022: 17th European Conference, Tel Aviv, Israel, October 23 27, 2022, Proceedings, Part XIII, pages 72-90. Springer. 2022.
[26] Thomas B Fitzpatrick. Soleil et peau. Med. Esthet,, 2:33-34, 1975.
[27] Thomas B Fitzpatrick. The validity and practicality of sun-reactive skin types i through vi. Archives of dermatology. 124(6):869-8) 871, 1988.
[28] Aaron French, Marcelo Macedo, John Poulsen, Tyler Waterson, and Angela Yu. Multivariate analysis of variance (manova), 2008
[29] Gabriel R Galindo, Joni A Mayer, Donald Slymen, Delshad D Almaguer, Elizabeth Clapp. Latrice C Pichon, Katherine Hoerster, and John P Elder. Sun sensitivity in 5 us ethnoracial groups. CUTIS-NEW YORK-, 80(1):25, 2007.
[30] Denia Garcia and Maria Abascal. Colored perceptions: Racially distinctive names and assessments of skin color, American Behavioral Scientist, 60(4):420444 4411 2016.
[31] Arthur H Goldsmith, Darrick Hamilton, and William Darity. From dark to light: Skin color and wages among african- americans. Journal of Human Resources, 42(4):701 -388 2007.
[32] Nitesh Goyal, Ian D. Kivlichan, Rachel Rosen, and Lucy Vasserman. Is your toxicity my toxicity? exploring the impact of rater identity on toxicity annotation. Proc. ACM Hum, -Copptt Interact., 6(CSCW2), nov 2022. doi: 10. 155/3555088.
httphttts:///oo.rrrgg 10. 1145/3555088
[33] Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Konchek, and Omar Badri, Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition. pages 1820-1828, 2021.
[34] Matthew Groh, Caleb Harris, Roxana Daneshjou, Omar Badri, and Arash Koochek. Towards transparency in dermatology image datasets with skin tone annotations by experts, crowds, and an algorithm. arXiv preprint 27711:200 2022. 2022.
12

---

[35] Vishal Gupta and Vinod Kumar Sharma. Skin typing: Fitzpatrick grading and others. Clinics in dermatol- ogy, 3765)440-436, 2019.
[36] Margot Hanley, Solon Barocas, Karen Levy, Shiri Azenkot, and Helen Nissenbaum, Computer vision and conflicting values:
Describing people with automated alt text. In Proceedings of the 2021 AAAT/ACM Conference on AI, Ethics, and Society, pages 543-554, 2021.
[37] Hassan Hayat, Carles Ventura, and Agata Lapedriza. Modeling subjective affect annotations with multi-task learning. Senwors, 22(14), 2022. ISSN 1424-8220. doi: 10.3390//22145245. URL, https:/ttt://ww .ddp com/ 2tmm/1424-222/22///1 14/5245
[38] Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, and Cristian Canton Ferrer. Towards measuring faimess in the casual conversations dataset. IEEE Transactions on Biometrics, Behavior, and Identity Science, 4(3):324-33 332, 2021.
[39] Steven He, Charles McCulloch, W John Boscardin, Mary-Margaret Chren, Eleni Linos, and Sarah T Arron. Self-reported pigmentary phenotypes and race are significant but incomplete predictors of fitzpatrick skin phototype in an ethnically diverse population. Journal of the American Academy of Dermatology. 71 (4):731-737, 2014.
[40] The White House. Blueprint for an ai bill of rights, Mar 2023. URL S.Rl.tttpa://ww. httpp::/www..hiteehhuus.e whithousee gov ostp/ai-bill--aeeeeeeae bill-ofilllo--liieeeeeeeooolooll ofrtttt
[41] The White House. Red-ttemming large language models
identify novel risks,
https: //www. whitehouse gow http://stp///aauuueeeeee hupdates/2023/// /9/ red- tramine large- wanguage-modeee models to identidy- novel ai ai-risks/
Aug 2023.
URL
[42] John J Howard, Yevgeniy В Sirotin, Jerry L Tipton, and Arun R Vemury. Reliability and validity of image- and self-reported skin phenotype metrics. IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(4):550 560, 2021.
[43] Kimmo Karkkainen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1548-1558, 2021.
[44] Verna M Keith and Cedric Herring. Skin tone and stratification in the black community. American journal of sociology, 77(3):760-778, 1991.
[45] Newton M Kinyanjui, Timothy Odonga, Celia Cintas, Noel CF Codella, Rameswar Panda, Prasanna Sattigeri, and Kush R Varshney. Estimating skin tone and effects classification performance in dermatology datasets. arXiv preprint 1910v 2019.
[46] Brendan F Klare, Ben Klein, Emma Taborsky, Austin Blanton, Jordan Cheney, Kristen Allen, Patrick Grother, Alan Mah, and Anil K Jain. Pushing the frontiers of unconstrained face detection and recognition: larpa janus benehmark a. In Proceedings of IEEE conference on computer vision and pattern recognition, pages 1939,, 2015.
[47] Seema Kolkur, Dhananjay Kalhande, P Shimpi, C Bapat, and Janvi Jatakia. Human skin detection using rgb, hsv and yeber color models. arXiv preprint urXiv:1708.0069)) 2017.
[48] KS Krishnapriya, Gabriella Pangelinan, Michael C King, and Kevin W Bowyer. Analysis of manual and automated skin tone assignments. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 429-438, 2022.
[49] Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International Journal of Computer Vision (IJCV), pages 1-26, 2020.
[50] JC Lester, JL Jia, Zhang, GA Okoye, and E Linos. Absence images of skin of colour in publications of covid-19 19 skin manifestations. British Journal of Dermatology, 183(3):593 5955, 2020.
[51] Chunxi Liu, Michael Picheny, Leda San, Pooja Chitkara, Alex Xiao, Xiaohui Zhang, Mark Chou, Andres Alvarado, Caner Hazirhas, and Yatharth Saraf. Towards measuring faimess in speech recognition: casual conversations dataset transcriptions. In ICASSP 2022-2022 IEEE International Conference on Acousties, Speech and Signal Processing (ICASSP), pages 6162-6166. IEEE, 2022.
[52] Keith B Maddox. Perspectives on racial phenotypicality bias. Personality and Social Psychology Review, 8(4):383 401, 2004.
13

---

[53] Henry B Mann and Donald R Whitney. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, pages 50-60. 1947.
[54] Helen B Marrow, Dina G Okamoto, Melissa García, Muna Adem, and Linda R Tropp. Skin tone and mexicans' perceptions of discrimination in new immigrant destinations. Social Psychology Quarterly, 85 (4):374-385, 2022.
[55] Patrick Mason. Annual income, hourly wages, and identity among -mmirianss and other latinos. Industrial Relations: A Journal of Economy and Society, 43(4):817-834, 2004.
[56] Kenneth O McGraw and Seok P Wong. Forming inferences about some intraclass correlation coefficients. Psychological methods, 1(1):30, 1996.
[57] Nicole Meister, Dora Zhao, Angelina Wang, Vikram V Ramaswamy, Ruth Fong, and Olga Russakovsky. Gender artifacts in visual datasets. arXiv preprint arXiv:2206.0919) 2022.
[58] Margaret Mitchell, Dylan Baker, Nyalleng Moorosi, Emily Denton, Ben Hutchinson, Alex Hanna, Timnit Gebru, and Jamie Morgenstern. Diversity and inclusion metries in subset selection. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 1 1771223, 2020.
[59] Ellis Monk. Monk skin tone scale, 2022. URL
[60] Ellis P Monk The cost of color: Skin color, discrimination, and health among African-americaa American Journal of Sociology. 121(2):39 444, 2015.
[61] Ellis P Monk Jr. The consequences of "race and color" in brazil. Social Problems, 63(3):133 010:41334444 430, 2016.
[62] Ellis P Monk Jr. The unceasing significance of colorism: Skin tone stratification in the united states. Daedalus, 1 150(2):76-90, 2021.
[63] Edward Murguia and Edward E Telles. Phenotype and schooling among mexican americans. Sociology of education, pages 276-289, 1996.
[64] Preetam Nandy, Cyrus Diciccio, Divya Venugopalan, Heloise Logan, Kinjal Basu, and Noureddine El Karoui. Achieving faimess via post- pess-erocssing in web-scale recommender systems. In 2022 ACM Conference on Fairness, Accountailliy, and Transparency, pages 715-725, 2022.
[65] Margo Okazawa-Res Rey, Tracy Robinson, and Janie Victoria Ward. Black women and the polities of skin color and hair. Women & Therapy, ((1-2):89-103) 1987.
[66] UK Okoji, SC Taylor, and Lipoff. Equity in skin typing: why is time to replace the fitzpatrick scale. British Journal of Dermatology, 188(1):998-199 2021.
[67] European Parliament. Meps ready to negotiate first-ever rules for safe and transparent ai,
Jun 2023. URL https: //www wwwp://wwwwwwww.llcco/ www.earl.....rrre.eraa....
rores- safe- and transparent-
[68] Jordi Pont-Tuset, Jasper Uijlings, Soravit Changpinyo, Radu Soricut, and Vittorio Ferrari. Connecting vision and language with localized narratives. In Computer Vis 2020: 16th European Conference, Glasgow; UK, August 23-28, 2020, Proceedings, Part V 16, pages 647-664. Springer, 2020.
[69] Bilal Porgali, Vitor Albiero, Jordan Ryda, Cristian Canton Ferrer, and Caner Hazirbas. The casual conversations v2 dataset, 2023.
[70] Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. Data cards: Purposeful and transparent dataset documentation for responsible ai. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1776-1826, 2022.
[71] Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchin- son, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. Closing the ai accountability gap: Defining an end-to-end framework for internal algorithmic auditing. Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 33-44, 2020.
[72] Dennis Reidsma and Rieks op den Akker. Exploiting 'subjective" annotations. In Coling 2008: Proceedings of the workshop Human Judgements in Computational Linguistics, pages 8-16, 2008
[73] KIPS Robin, TRAN Loic, Emmanuel MALHERBE. and Matthieu PERROT. Beyond color correction: Skin color estimation in the wild through deep learning. Electronic Imaging, 32:1-8, 2020.
14

---

[74] Igor Ryabov. Colorism and educational outcomes of asian americans: evidence from the national longitu- dinal study of adolescent health. Social Psychology Education, 19:303-324, 2016
[75] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Thirty-sixth Conference on Neural Information Procexsing Systems Datasets and Benchmarks Track, 2022.
[76] Candice Schumann, Xuezhi Wang, Alex Beutel, Jilin Chen, Hai Qian, and Ed Chi. Transfer of machine learning fairness across domains. arXiv preprint 1ediv:9906006630 2019.
[77] Candice Schumann, Susanna Ricco, Utsav Prabhu, Vittorio Ferrari, and Caroline Pantofaru. step toward more inclusive people annotations for fairness. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 916-925, 2021.
[78] Marilyn S Sommers, Jamison D Fargo, Yadira Regueira, Kathleen M Brown, Barbara L Beacham, Angela R Perfetti, Janine S Everett, and David Margolis. Are the fitzpatrick skin phototypes valid for cancer risk assessment in a racially and ethnically diverse sample of women? Ethmicity & disease, 29(3):505, 2019.
[79] Charles Spearman. Correlation calculated from faulty data. British journal of psychology, 3(3):271, 1910.
[80] Felix von Luschan. Beiträge zur Völkerkunde der deutschen Schuizgebiete. D. Reimer. 1897.
[81] Angelina Wang, Alexander Liu, Ryan Zhang, Anat Kleiman, Leslie Kim, Dora Zhao, Iroha Shirai, Arvind Narayanan, and Olga Russakovsky. Revise: A tool for measuring and mitigating bias in visual datasets. International Journal of Computer Vision, 130(7):1790-B3)) 191011000010000,,, 2022.
[82] Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky, Towards fairness in visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 8919 8928, 2020.
[83] Ka Wong and Praveen Paritosh. k-rater reliability: The correct unit of reliability for aggregated human annotations. arXiv preprint arXiv:2203.129: 129113, 2022.
[84] Yu Yang. Aayush Gupta, Jianwei Feng, Prateek Singhal, Vivek Yadav, Yue Wu, Pradeep Natarajan, Varsha Hedau, and Jungseock Joo. Enhancing fairness in face detection in computer vision systems by demographic bias mitigation. In Proceedings of the 2022 AAAI/ACM Conference Al, Ethics, and Society, pages 813 822, 2022.
[85] Dora Zhao, Angelina Wang, and Olga Russakovsky. Understanding and evaluating racial biases image captioning. In International Conference on Computer Vision (ICCV), 2021.
[86] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai Wei Chang, Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv preprint 1707iv:7077.0444 2017.
15

---

A Expanded Related Work
A.1 Subjective Annotations
Soliciting annotations for subjective concepts is required for many modern machine learning tasks. As such, it has been a subject of considerable research across subfields, from NLP and affective computing to classical computer vision tasks such as attribute recognition. Our work adds to the growing literature that raises the importance of better understanding and modeling the individual perspectives of annotators for subjective tasks, especially as influenced by demographic cultural factors, and we highlight a few examples here from across disciplines. Goyal et al. (32 study differences in annotations of toxicity in online conversations when using specialized annotator pools comprised of individuals identifying as African American and LGBTQ, respectively. Aroyo et al. 2 annotate conversations for safety using annotator pools across two geographic regions. They significantly increased replications compared to standard practice, finding differences in annotator behavior between groups where single annotator's s judgements would not necessarily be stable across multiple views of the same content. Díaz et al. [20] argue for careful design of annotation tasks, accounting for the socio-cultural backgrounds of annotators and considering lived experiences as aspects of expertise. Given annotations for a subjective task from a diverse set of annotators, researchers are increasingly adopting model architectures that target individual annotators instead of or in addition to an overall consensus estimate [18 15 37]. Our work supports these findings and extends the general recommendations for practitioners by focusing specifically on how they apply to annotations for fairness in computer vision.
A.2 Skin Tone Research
There are a number of different ways to describe skin tone that have been used in the social sciences, dermatology, and computer vision. Here, we focus on the most common definitions used in computer vision fairness research, including more options and more detail than discussed in the main paper.
The most common objective method used by computer vision scientists and Al researchers is the individual typology angle (ITA), which is a metric based on the L* (lightness) and B* (yellow/blue)
components of the CIE L color space, defined as ITA = arctan L'-50) 180 Skin
6"
color then be classified using the ITA according to six categories, ranging from very light (category I) to dark (category VI) [1]. Measured ITA has been shown to be correlated with physical properties such as melanin content of skin [19]. The mathematical definition is conceptually appealing, with researchers describing it favorably as, for example, being "easily computed without subjective annotations." [43] However, ITA values computed from images captured in uncontrolled environments can vary due to the influence of scene lighting or other imaging artifacts (73). making it less reliable an annotation tool for wn-the-wild datasets. Furthermore, in practice the continuous ITA values are often mapped to different categorical scales using boundaries between categories that reintroduce elements of subjectivity [1] 19).
The Fitzpatrick Skin Type (FST) scale has become a gold standard of subjective skin tone measure- ment in dermatology. The six-point scale, initially conceived as four-points was developed dermatological tool to measure the response of different types of skin to phototherapy, exposure to ultraviolet light. For example Type I, the lightest skin type, is described as "Burss easily, never tans" and Type VI, the darkest skin type, is described as "Never burns, tans profusely" (27]. Outside of dermatology, computer vision scientists have also used the FST to label images for skin tone.
Most studies use either expert dermatologists or crowdsourced annotators to assign skin tone labels based on FST [46] 38). Though lighting and image quality may pose challenge in photo evaluation, experts and crowdsource annotators tend toward fairly accurate FST annotations where judgements expected to be within one FST value depending image conditions [50] 48). For example, an audit of the Fitzpatrick 17k dataset found their crowdsourced annotators had between 71 % and 85% % within one point accuracy to board-certified dermatologist ratings (33]
Despite its wide use, there are some notable limitations of FST for skin tone annotation. In particular, the scale was not developed to assess skin tone; rather, skin, hair, eyes, and history of sunburn were used to assess likelihood of skin burning during phototherapy. In addition, to make the scale more inclusive of people with Asian, Indian, and African heritage, Type V and Type VI were later
16

---

added to the four- cour-point scale [27]. These additions to the scale implied a strong correlation among race/ethnicity, skin tone, and sun reactivity; however, self-reported race/ethnicity and skin tone have been found to be significant but weak predictors of FST [39] That is, FST values may actually serve as a significant, but weak, correlate to one's actual skin tone.
FST also better represents a range of lighter skin tones than range of darker skin tones [27 29 One study found that only 42% of an ethnically diverse group of participants could be classified using the standard Fitzpatrick scale definitions [23] Sommers (2019) (78) summarized it saying, they [self-reported Fitzpatrick Skin Type] provide a restricted range of options for people with darker skin that does not capture variations in their skin color." This highlights& core limitation of the FST; it not inclusive of the full spectrum of skin tones. This limits the ability for the FST to faithfully represent people, and the ability for systems built upon human annotation to be fairly evaluated across skin types.
Researchers have called for alternative and finer-grained skin tone measurements both within der- matology [66] and more broadly in fairness for computer vision (59). The ability to draw on a finer-grained scale that reflects a broad range of skin tones provides flexibility to support careful design choices for disaggregated evaluations [3] and can play a crucial role in addressing performance disparities, especially those which disproportionately impact communities that have faced injustices and have been historically underrepresented [8]. Although fier-grained measurement is the goal, is not without its challenges and limitations.
Skin tone scales with too many shades, like the 36-point Felix von Luschan Skin Color chart [8)], can introduce additional cognitive load for annotators during an already challenging task, potentially reducing inter-rater agreement. In addition, it may be difficult for annotators to identify differences between skin tone shades at such fine-grained measurement, leading to lower annotator accuracy compared to an expert ground truth assessment.
We have chosen to have annotators use the 10-point Monk Skin Tone (MST) scale [59] (Figure D. The MST scale was designed through its color selection and gradations optimally capture ethnoracial diversity in skin tone across the Americas (c.g. North, South). This includes capturing skin tone variation within and across racial/ethnic categories in the United States (and potentially beyond). At 10 points, we hypothesized MST would give annotators a broader and more diverse range of skin tones compared the FST without overwhelming annotators with too many options. Although finer-grained scale adds more complexity and difficulty to achieve consensus at the annotation stage (compared to, for example, annotations for a binarized skin tone annotation task), we hypothesized that the MST allows for a more nuanced understanding of system behavior, better targeted disaggregated evaluations and audits, and more capable measures of representation.
A.3 Annotations for ML Fairness in Computer Vision
Perceived signals such as gender expression, skin tone, and age are commonly used in fairness evaluations and bias mitigation techniques in computer vision {2 g 84). There are four main methods currently used in collecting these annotations: third party crowdsourced annotations, annotations from captions, self reported annotations, and algorithmic or model generated annotations.
Crowdsourcing annotations for fairness evaluations is extremely common in computer vision. Zhao et al. [85] collect both Fitzpatrick Skin Type (26) and gender expression using Amazon Mechanical Turk (MTurk) with three replications per image. Amazon Mechanical Turk (AMT) is popular crowdsoureing platform that is commonly used across both computer vision and general machine learning practitioners. Difallah et al. [21 found that the majority of MTTurk workers are based in the US and India. They also found that MTurk workers tend to be younger than the overall population. Buolamwini and Gebru [8] collected Fitzpatrick Skin Type annotations for each individual through a beard-certified surgical dermatologist. This type of annotation is not scalable to larger datasets. Schumann et al. [77] show how annotation pipelines can introduce biases into annotations. They also provide a dataset Open Images Extended: More Inclusive Annotations for People (MIAP) with crowdsoureed annotations from annotators in India of perceived gender presentation and age range. In this paper we focus on crowdsourced annotations for skin tone across a wide geographic region.
Captions are often used to provide fairness annotations for images. For example, body of work on investigating and mitigating bias in image captioning builds on the dataset and definitions introduced by Zhao et al. (86). Here, images from MSCOCO are filtered into dataset slices depending on whether
17

---

the ground truth captions include only the word "man", or only the word "woman". Images with captions mentioning both terms are excluded. Recent work on understanding gendered bias vision datasets [57] adopts a similar strategy to identify dataset slices, with updates introduced by Zhao et al. [85] that expand the list of gendered terms included the filters. It is possible to construct sliced datasets of reasonable scale using this procedure because it is common for descriptions of people in the source datasets to use gendered terms. In fact, the annotation instructions in the Localized Narratives dataset includes the example "a man is flying kite" [68]. implicitly encouraging this practice.
In contrast, we find that captions in existing datasets rarely include description of an image subject's skin tone. We used Know Your Data to find and manually inspect images containing a person bounding box and captions that contain the word skin or skinned. Out of all 123,287 images, we find just 12 images with captions describing skin tone one "pale skin" and 11 "dark skin(ned)". Given that recent developments in large-scale vision-language models rely on paired text from rather than curated captions, we consider whether alt text datasets may exhibit different properties. Hanley et al. [36] provides alt text guidelines from two museums that recommend describing skin tone in images, and notes that nearly all of the concrete examples include these descriptors. However, web-crawled alt text from generic sources is of varying quality and does not adhere to these standards [6]. Even if we could ensure museum museum-quality alt text that follow these published guidelines, the suggested language simplifies the concept, resorting to more binarized language like light- or ttrk-skinned
web-crawled alt-text 13 75
Liu et al. [51 released the Casual Conversations Dataset which includes both self-reported gender and age. contrast, the authors of the dataset use third party annotations of Fitzpatrick Skin Type, not self-reported values, to categorize the skin tone of participants. Datasets with self-reported annotations are expensive to collect and tend to be very "clean" " and not in-the-wild images or videos. There is also an important distinction between subjective attributes such as skin tone, non-subjective attributes such as gender identity and age, and the perception these attributes. While self-reported values are extremely useful and important Liu et al. [51], the choice to use self-reported values vs third-party annotated values depends on the perspective practitioners want to understand: a) how models interact with the identities and non-subjective attributes of users or b) how models perform on different perceived attributes across a wide range of imagery (including generated imagery).
Finally, faimess annotations can be obtained through models as explored by Buolamwini and Gebru []. However, these models need to be trained on annotations acquired from one or more of the above techniques. Skin tone measures based off of ITA are notable exceptions. These measures are computed from pixel values from detected skin regions in the images with the objective of matching physical measurements captured under controlled lighting conditions in images captured under unknown lighting conditions are a function of both the scene lighting and the subject's true skin tone. Earlier work recognizes but does not always attempt correct for these effects [43 45 Others apply color correction by leveraging known properties of the scene or capture Recently, Feng et al. [25] introduce a learned method that jointly models scene illumination inferred from scene context and facial albedo, from which IT' can be computed. Cho et al. [14] present rare instance of method that estimates skin tone directly from pixel values but bypasses ITA computation. They detect skin regions in images and select the Monk Skin Tone category that most closely matches the average RGB value of pixels within this mask. We do not recommend this approach for MST annotations. See Appendix G for an illustration of limitations of this approach with unconstrained images.
[19 42 73). Raw pixel values
environment 48 42
B MST-E Golden and Challenging Image Sets
The MST-E dataset includes a set of Golden Images where each subject is represented by an image that best represents their skin tone. A Challenging Images set was also selected with worse lighting conditions.
https://knooyourdata-- tfds http://a.a...aaooo/ooooooooooto.
18

---

Golden Images
Challenging Images
C
Data Card
Publishers
Team
Contact Detail
Google LLC TONL.CO
Google Research, Perception and Responsible Al teams
Contact: Feedback form
Website: skintone, ciotone.googlee
Data Subject(s)
Data Snapshot
Data Description
Data about image search queries
Dataset size 5.21 GB
The primary goal of this dataset is to enable and test consistent skin tone annotation across environment capture conditions. It provides a starting point to support practi- tioners to teach their annotators to annotate skin tone consistently on the fine-grained MST scale and controlled set through which to study annotator behavior.
Subjects
19
Images
1515
Videos
31
Images by TONL. Copyright TONL.CO 2022 ALL RIGHTS RESERVED.
19

---

Primary Data Modality
Link to Data
Data Fields
Image Data
Kkinooneggoolle/sste-aaae
Data field for each image and video be found in with the following headers (Header de- scriptions follow the format Field Name, Example, Description)
Video Data
image_naan ___me, PXL 0,jpg, Name of the image
pose, side, Pose of the subject
lighting, well lit, Lighting
condition of the image
mask, false, Whether the subject is wearing mask
subject_nne _ame,
subject_0,
Name of the subject
MST, MST annotation for the subject
The Golden Image Set and Chal- lenging Image Set can be found in a csv with the following headers:
subject, subjecr_0,
Name of the subject
golden_ Balden_image__ id, PXL_0.jpg, Name the golden image
challenge @hallenge_imageeaee Pit_ Name of the challenging image
The Images and Videos for each subject can be found under the sub- ject's folder. For example, in the above examples image PXL_0.jpg can be found in the subject_0 folder.
20

---

Secondary Data Modal-
Link to Data
Data Fields Crowdsourced anno-
ity
https: googleapis.. ctte:/ecccmmsssssmmeemmeeeeeesse com/mste/ mste annotati CsV
tations can be found in a CSV with the following headers (Header de- scription follow the format Field Name, example, Description)
Image Data
Crowdsource
Annota-
tions
subject, subject_0, Name of the subject
image, PXL 0.jpg. Name of the image
region, Ghana, Region where the annotation came from annotation, MST/, Crowd-
sourced MST annotation.
Note that each image should have 5 annotations per region with total of 25 annotations per image. Some images were too hard annotate and were skipped by some annota- tors. In those cases there will be less than 25 annotations.
Purpose(s)
Domain(s) application
Motivating factor(s)
Fairness research
Research, Fairness, Computer Vision, Skin Tone, Model Evaluation
Subjects span the entire MST scale.
Trains annotators annotate im- ages on the MST scale.
Trains annotators to be invariant to lighting conditions.
Supports fairness research.
Dataset Use(s)
Intended and suitable use case(s)
Unsuitable use case(s)
Research Annotator training
Model training: The images cannot be used to train machine learning models.
Research:
Fairness re-
Model Evalutation
search
Annotator training: Can
be used train human an- notators to annotate skin tone.
Model Evaluation: Can be used to evaluate fairness in computer vision models.
21

---

Version Status
Dataset version
Maintenance plan
Actively Maintained No versions with new images will be made available, but this dataset will be actively main- tained, including but not limited to updates to the data if errors are re- ported.
Current Version
1.0
Storage:
The dataset is stored on Google Cloud.
Last Updated
05/2023
Release Date
05/2023
Versioning: The website skintone. hotpoonngoooolmmmmseddaaasssssse
will contain the latest version of the dataset.
Availability: The dataset will be available for download with a valid Google account. The dataset will remain available on Google Cloud.
Feedback: Feedback and errors can be submitted through the Feedback form
Provenance I Collection
Provenance Collection Methodology detail(s)
Provenance Collection Processing
Data
Method(s) used
Images sourced through
Source: TONL.CO and skin-
The subjects, and sub- sequent images, were sourced through TONL.CO.
Images:
TONL.CO
tone. googgoogle
Annotated by human expert.
Date Collection: 2022
Primary modality: Image Data
Timing: All images and videos of a subject were collected within a short timespan of hours elimi- nate skin tone variation across im- ages due to seasonal other tem- poral effects.
Update frequency: Static
Human attributes
Source(s) of human at- tributes
Rationale for collecting human attributes
Skin Tone
Skin Tone: The images in the dataset were reviewed and annotated using the MST scale by the MST creator, Dr. Ellis Monk, expert in social perception and inequality.
The primary goal of this dataset is enable and test consistent skin tone annotation across environment capture conditions. It provides a starting point support practition- ers to teach their annotators to an- notate skin tone consistently the fine nee-grained MST scale and a con- trolled through which to study annotator behavior.
Distribution of human attributes
MST
1
2
4
5
6
8
9
10
Subjects Images
2
3
2
2
2
179
218
112
187
179
165
76
143
142
111
Videos
2
5
3
4
5
22

---

Annotation workforce
Annotation characteristics
Annotation description
type
Number of unique annotations:
The images in the dataset were reviewed and annotated using the MST scale by the MST creator, Dr. Ellis Monk, expert in social per- ception and inequality. Each sub- ject was annotated with a single MST-E value.
Human Annotations Ex-
19
pert
Concepts and definitions referenced in this data card
MST
Definition: Monk Skin Tone
Source: Seintone-google
Interpretation: Inclusive Annotation workforce type
10 point skin tone scale.
Annotation characteristics
Annotation description
Number of unique annota- tions:
Crowdsourced annotations for the MST-E dataset has been made avail- able for replication purposes only. Each image was annotated 5 times different regions totalling 25 annotations per image.
Human Annotations
Crowdsource
23

---

D
Annotator Experience
Pick the skin-tone that matches most closely.
0
O
O
O
Which part the person's body you primarily use when making your selection?
Armss/Hands
Legs/Feet
O
O
O
How easy or difffcullt was it to assess this person's skin-tone?
Nelther
Somewhot
Somewhat difficult
Very difficult
Vary easy
easy difficult
O
The figure above shows the annotation task as seen by the professional photographers in India and the US. The task was presented English and all photographers were fluent English speakers. The photographers were sourced through GLG Gerson Lehrman Group. Each network member of GL.G, including the photographers from this study, accept the GLG Terms and Conditions prior to participation.

---

Which pand
pinaity asauttiin wwannadd seacoon
Pego MDT VLLDD www.thhoooaaaaaacooocoooooooo parsee, vilble allo, orgrayscal.
Scaer
The figure above shows the annotation task as seen by the trained crowdsourced annotators located in India, the Philippines, Brazil, Hungary, or Ghana.
E
Professional Photographers' Regional Differences
Figure 7 shows the complete Mann Whitney tests comparing annotations from professional photographers in the US to professional photographers in India. It includes the tests over the Golden Images set (Figure 7(a) and the Challenging Images set (Figure 7()).
MST
Subject
Value
p
MST
Subject
Value
P
MSTI
S18
7.5
0.179712
MSTI
S18
13.0
1.000000
MST2
S1
19.5
0.147499
MST2
SI
18.0
0.248213
MST2 S13
23.0
0.026888
MST2
S13
13.5
0.910670
MST2 S8
18.0
0.256839
MST2
S8
14.5
0.733730
MST3
18.5
0.218758
MST3
20.0
0.132622
MST3 S15
20.0
0.070701
MST3 S15
9.5
0.587594
MST4
12.5
1.000000
MST4
19.5
0.154424
MST4 S9
21.0
0.058758
MST4 S9
16.5
0.443194
MST5 S11
14.0
0.796253
MST5
SI1
20.5
0.110492
MSTS S6
11.0
0.824848
MST5 S6
13.0
1.000000
MST6 S14
24.0
0.018119
MST6 S14
22.0
0.048632
MST6 S3
8.5
0.408308
MST6 S3
6.0
0.186667
MST7
9.0
0.488422
MST7
13.0
1.000000
MST8
S17
9.0
0.512412
MST8 S17
4.0
0.058758
MST8
S2
6.5
0.204024
MST8
S2
6.0
0.193059
MST9
S10
12.5
1,000000
MST9
S10
10.0
0.607236
MST9
$4
6.0
0.175151
MST9
S4
8.0
0.344659
MSTIO
S12
9.5
0.588809
MSTI0
S12
11.0
0.817361
(a) Golden Images
(b) Challenging Images
Figure 7: Mann- Whitney U testing the difference in the annotations of professional photographers in the US versus India. Figure (aa)ttets tests annotations over the Golden Images set. Figure 7(b) tests annotations over the Challenging Images set.
Mann-Whitney
25

---

Distance from Intended Annotations
Figure shows confusion matrices of Dr. Monk's annotations compared the median annotations by the crowdsourced annotators.
G Pixel-Based Skin Tone
As mentioned in the related work, measuring skin tone from pixel values has limitations ($2) In this experiment we determine the skin tone through a) automated skin tone detection and b) manually cropped images. For each of these methods we look at closest MST skin tone by averaging all skin tone pixels and finding the closest MST point based on the L2 distance RGB space to the reference colors [4].
G.1 Automated skin tone detection
Following Cho et al. [14]. we mask out non-skin tone pixels using [47]. We find that this method performs poorly it picks up some background pixels, and does not capture all of the face particularly dark skin tones (see Figure 9.
Given that this method does not pick out skin tone well, the closest MST values do not accurately represent the subjects skin tone (See Figure 10).
G.2 Manually cropped images
In addition to the automated skin selection above, we also manually drew boxes around open portion of skin, and used the pixels from that crop (See Figure IT. As seen in Figure I0 cropped images with only skin tone pixels still do not produce accurate MST annotations.
H Maintenance plan
The dataset will be stored on Google Cloud indefinitely. The website skintone..eee htttttoon..ooolle/sseeetttteteee will remain updated with the latest version of the dataset. The dataset will be available for download with valid Google account. Any feedback found errors in the dataset the sskitoneegooggl website feedback form, Any errors will be fixed and the dataset will be updated accordingly. Currently we do not plan on releasing V2.0 version of the dataset.
be reported through
I Author statement and dataset use
Each subject provided consent for their images and videos be released. TONL has given permission for these images to be released and used for research or annotator training purposes only (Copyright TONL.CO 2022). The images cannot used train machine learning models. Annotations provided with the images can be used for research or annotator-training purposes only. When using the dataset for research, please cite this paper and credit TONI. for any images used.
The authors of this paper bear all responsibility in case of any violation of rights during the collection of the data other work, and will take appropriate action when needed, e.g. remove data with such issues.
Dataset format
The full dataset can be downloaded from (ooommskintnn..oom caairtoneggogllemstteetaaee The download will be zip folder containing images jpg format, videos in .m4 format, and annotations cav format. See the datacard in Appendix for more information about the fields found in the .csv file.
26

---

Broat Anrotation Confasion Muntle
Chana Armotation Corfision Matric
Mungary Armotation Confusion Mabik
:20
40
10
200
100
20
D
10
10
-00
0
10
20
10
20
0
20
10
20
20
10
0
10
25
20
20
20
20
40
0
20
40
10
50
20
20
50
10
Crowisourced Annotarions
Crowdsourced
Annotartons
India Ammotation Confasion Masti
Philopines Arnotatioo Confusion on Matrix
Glosal Annotation Confusion Mutrie
39
30
GOI
:00
10
0
300
20
D
105
D
20
D2
D
10
13
0
O
D
O
29
51
O
O
D
20
HP
-00
10
40
13
50
LO
0
20
20
10
10
O
2 17
24
17
25
11
0
0
40
40
Q
12
20
10
Crowissurced Annotations
Crovidourced Annotations
(a) Golden Images Set
Annotation Confision Mantl
Chana Annotation Corfazion
Annotation Condusion Manik
80
2D
20
80
30
50
90
10
0
0
0
30
30
5
0
0
0
30
10
0
10
11
22
20
10
50
40
0
20
0
92
20
20
10
10
50
20
Crowissurced Annotacions
Cronddsuurtdd Amnotations
Crowssoutoes roed Anrorstons
Corfusion
GO
19
ID
80
50
7
90
O
70
A
0
20
50
VI
O
0
O
12
M
25
5
D
90
25
GD
0
20
50
40
0
6
9
10
0
0
3
10
20
0
0
10
98
0
20
48
Crowidsurced Annotations
Cronidourced Annotations
Crowssourced Antotttonss
(b) Challenging Images Set
Brazl Anrotation Confusion Motrie
Chana Annotation Corfasion Matrix
Mungary Annotation Confusion Mubis
D D
:20
3T 0
D
320
23
O
D
100
3D
0
D
22
15
D
D
O
2
2
0
D
22
0
10
31
16
10
0
23
18
36
12
21
21
25
0
0
12
0
0
10
50
15
40
0
1
36
24
6
B
10
2
30
3O
India Annotation Confision Matris
Philopines Arnottttion Confusion on Matrix
Glozal Annotation Confusion Mutric
I9
HB
32
O
O
300
I85 HY
O
O
O
300
33
AE 18
1
O
D
109
D
17
D
D
29
57
O
O
D
O
38
D4
1G
O
D
R
25
100
60
0
2
25
22
6
5
24
25
O
20 20
0
17
27
L2
0
40
O
2
17
22
18
18
29
15
0
20
0
0
54
6
10
Croivbssured Annotations
Crovidoourced Annotations
Crowszourced Annotations
(c) Annotations per Subject
Figure 8: Comparisons between the intended MST value as annotated by the MST creator and the median value selected by crowdsource annotators across subsets of images and different annotator pools.
27

---

(a) Golden Images
(b) Challenging Images
Figure 9: Detected skin tone following 47
All pixels that are not skin tone are black.
K Annotator compensation
The compensation rates for the expert photographers and the crowdsourced annotators dictated by the relevant sourcing vendors. Payment rates are determined by fair market value in the different regions. The expert photographers were paid token honoraria not exceeding $500 by GLG for participation in two minute surveys. Crowdsourced annotators were contracted by the sourcing vendor work for monthly increments. The rough monthly payment for crowdsourced annotators per region is as follows:
Location
Per Rater/Mo
Estimated Hourly Wage
India
~$1,100.00
$6.00
Philippines
~$2,000.00
$11.00
Hungary
~$3,500.00
$19.00
Ghana
~$2,330000
$13.00
Brazil
~$2,30000
$13.00
Overall, spent approximately $780k crowdsourced annotations. The total compensation for the group of expert photographers who provided annotations through GLG was approximately $5000.
L Addressing previous reviews
This paper was previously submitted to FAceT 2023 titled, Consensus and Subjectivity of Skin Tone Annotation for Fairness. The reviewers agreed on the paper's relevance noting the nuance and exploration of skin tone annotation across regions and the implications of the work for practitioners, but also noted some objections and opportunities their rejection of the submission. We leveraged their feedback for stronger submission, and below is a summary of their feedback and how the feedback was addressed this submission.
Monk Skin Tone Scale In the prior submission reviewers raised objections about a missing citation for the Monk Skin Tone scale and who was chosen as Monk Skin Tone scale expert to provide ground truth annotations. Due to the timing, we were unable to cite the now published Monk Skin Tone scale white paper that describes the scale's development. We have cited it this paper in reference to the scale.
28

---

Closest MST from Detected Skin
5-
Actual
Detection Golcen
Detection Challenging
(a) Automated Skin Tone Detection
Closest MST from Cropped Image
165
Actual
Detection Golcen
Detecton Challenging
(b) Manually Cropped Images
Figure 10: Actual MST annotations by the MST creator compared to the closest MST RGB from the average RGB of the detected/selected skin tone pixels.
Relatedly, one reviewer raised question about the effect of scale length and why 10-point scale better or worse than other compositions. This an interesting question, but out of scope for this paper. In this submission, and the prior one, we focus people's ability to annotate skin tone on this particular scale due to the recent fairness literature adopting the Monk Skin Tone scale.
Monk Skin Tone Scale Expert In the prior submission reviewers raised objections about the unidentified Monk Skin Tone scale expert, (described as "a Monk Skin Tone scale expert"), and what credentials they possessed to establish ground truth on skin tone, a characteristic we highlight as subjective. the previous submission we wanted to remain double blind, but in this submission we highlight that Dr. Monk is the Monk Skin Tone scale expert and cite his credentials of academic expertise in skin tone research and the creator this particular skin tone scale.
Expert Annotation Exploration In the prior submission reviewers raised questions about the selection and limited sample size of experts in the expert annotation portion of the paper. In this submission we clarify why photographers were chosen instead of dermatologists the type of expert chosen in similar previous research. Photographers tended to assess skin tone in the image while some dermatologists tended to devise the skin tone under clothing which was not depicted in the image (or skin untouched by sun). We also more clearly delineate the expert portion of the paper exploratory and use the results from this work to establish hypotheses for the properly powered
29

---

(a) Manually Drawn Box
(b) Resulting Crop
Figure 11: Manually drawn box on an open selection of skin, and the resulting crop of the manual selection.
analyses later in the paper. Finally, we also note an opportunity to more expressly test annotation across various types of experts in the limitations for future consideration.
Analytic Clarifications In the prior submission reviewers sought some additional clarifications in the analyses. At high-level, we separated the analyses by sample in this version of paper to better delineate the separate analyses, and increase the clarity of the results. We also more clearly define "average median distance" and describe the "1-point discrepancy" threshold citing how they were used in previous research and their purpose in this work. Finally, we made some adjustments to the figures and footnotes where reviewers noted confusion or a preference for more information.
Clarification of Scope In the previous submission we reviewed a broad scope of research including subjective and objective forms of skin tone annotation and this submission we more clearly focused on subjective annotations to streamline the literature review and highlight the particular focus of this contribution.
Limitations and Ethical Considerations The previous submission did not include an explicit section dedicated to the limitations and ethical considerations of the submission. In this submission we included both to highlight further opportunities to investigate annotation ability quality and the proper usage of the MST-E dataset.
Monk Skin Tone Scale-Example Dataset One reviewer noted limitation of the MST-E dataset, only includes subjects. This limitation we accept and acknowledge the paper, but we unable to address in this iteration of the dataset. Another reviewer mentioned some additional demographic details like race/ethnicity of the MST-E subjects, but this data was explicitly not collected from the models. Finally, reviewer mentioned that releasing the annotations would increase the utility of the MST-E dataset for practitioners if we were able to make those publicly available well. We release the annotations with this paper. You can download the annotations from eetoragg.oooglla................ more information.
googleapis. wwwmnsttemmeeeeeeeeeeeee http://ssseemtt__nnotatttttott csv See Appendix
30


Reviews:
[{"review_id": "peSTTkggiL", "reviewer": "Reviewer_8mCy", "summary": "", "questions": "", "limitations": "The proposed dataset seems interesting, however there are many critical aspects, such as: \n- Missing Computer Vision perspective\n- Limited number of subjects;\n- Missing discussion about the possibility of increasing the number of subjects.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "- The presented dataset has been collected through a rigorous procedure. It has been validated also thanks to different group of experts.\n- It is a relevant paper for practitioners \n- it is very rigorous way for approaching the subjectivity of skin tone annotation.\n- The dataset is publicly available for further research.\n- The association of MSE-E to the MSE scale is a very good idea since it maps the annotations to a formal, and well known, scale.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "V5H3C1bWzu", "reviewer": "Reviewer_JyxF", "summary": "", "questions": "", "limitations": "The authors note that the dataset is limited in size. The authors note the potential for misuse of skin tone annotations, and limit the use cases of the dataset to prevent misuse.\n\nWhile the work is quite interesting, the range of impact is rather niche: it most directly lends it self to researchers hoping to build fairness vision datasets using annotators. ", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "The authors provide a very thorough and convincing analysis as to variation in perceived skin tone annotations. The MST-E dataset will be useful for researchers hoping to gather fairness related annotations for images and videos. The ground truth annotations created by Dr. Monk will create an excellent baseline to train annotators against, and could lead to more standardized annotations across datasets.\nThe paper offers and concrete and practical suggestions for researchers hoping to gather perceived skin tone annotations, and provides experimental evidence for their suggestions.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "ZQ94WCFvWo", "reviewer": "Reviewer_jgJ9", "summary": "", "questions": "", "limitations": "The lack of an explanation on how this work contributes to the advancement of the machine learning and the absence of description and experiments regarding its application in machine learning tasks are evident.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "Comprehensive experiments on annotations provided by annotators from different geographical regions.\nThe analysis of annotations of skin tone by annotators across conditions.\n\n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "ZhXnqnph9Q", "reviewer": "Reviewer_ph2V", "summary": "", "questions": "", "limitations": "- Dr Monk is the inventor of the Monk Skin Tone scale. However, I believe that Dr Monk being the sole annotator for ground truth is a significant limitation. In most instances, a robust dataset is annotated by numerous domain experts. I always have concerns when a dataset is labeled by a single individual. I would love to learn more about Dr Monk's annotation process, how long did it take him to annotate these photos, what consistent was Dr Monk at this task, etc.\n- I agree that photographers are very sensitive to skin tones. However, I believe that it would have strengthened the study if the authors included dermatologists in the annotation experiment.\n- The dataset is based on 18 subjects with memorable facial features. It is possible that annotation experts remembered their rating for each subject and tried to be consistent. More subjects and less photos per subject might have been a better study design.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "- A novel and more detailed skin tone scale (Monk Skin Tone) than Fitzpatrick skin types. \n- Interesting suggestions for practitioners in order to scale the data labeling process.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: Submission: Skin Tone Annotation Consensus (**#841**) Bottom line: accept (as poster)

**Summary**: Skin tone annotation is an incredibly challenging -- and increasingly important -- aspect of doing skin (disease related) research across populations of different skin types and tones. The current (often employed) standard is and remains the Fitzpatrick skin types/scale methodology, which however is flawed. The authors provide strong evidence for the validity of a novel and yet simple (10-category nominal) system, the Monk scale -- with Dr. Monk included on the list of authors, providing additional support for this publication. While several minor concerns remain, the AC believes that offering this annotation study and benchmarking methodology will drive the necessary improvements on skin type research forward significantly, and thus urges the SAC/PCs to accept the manuscript for (poster) publication.

Pros: two separate "user" groups (skin image experts in form of photographers with often greater care for color as an ontological/perceptual category, and novices) were asked to annotate the dataset of images, which comprise a specially constructed gold standard set, as well as a convenience sample of images of skin. Together with the analyses provided, the improvement on the original methodology (Monk scale, in collaboration with Dr. Monk) by creating patches showing the color under different lighting conditions, and the rigorous collection procedure, this dataset provides very good evidence and instructions for further research, creating a solid basis for future studies.

Cons: the dataset may be very niche (audience from the NeurIPS community is the relatively small overlap of automatic image processing *and* fairness *and* skin color as a hard-to-assess percept/concept), and from the title, the general difficulty that skin tone offers in dermatological research isn't as clear; **this paper might be better placed in an outlet that targets dermatologists with a strong research interest; so long as the SAC/PCs do not consider this to be a limiting factor, the recommendation to accept as a poster stands.**

Decision: given that this topic (skin tone, and lack of clear direction in the dermatology research domain with addressing the dearth of skin tone/color research) is of great interest and import **in the dermatology community**, the AC believes that publishing is warranted, albeit maybe better in a different outlet (this is difficult for me to assess, w.r.t. how much NeurIPS wishes to support smaller niche communities...). That notwithstanding, the manuscript and dataset as such **are important contributions**.

----

Reviewer 8mCy: the major concerns are poor representation of the subject to the overall computer vision (CV) audience; some of this has been addressed (by rewrites), and the reviewer seems largely satisfied; in addition the reviewer raised ethics concerns (consent of participants), which is of note; the AC believes the authors have covered this as best they could.

----

Reviewer JyxF: major concerns are around use cases, dataset size, and 10-annotator recommendations; in my reading of the rebuttals and the reviewed manuscript, these points are satisfactorily addressed (although the reviewer did not respond to the authors' rebuttal, and so I cannot say whether the reviewer would share this assessment).

----

Reviewer jgJ9: major concern is around a lack of clear descriptions of use cases for the machine learning (or CV) communities; I agree with the reviewer, and while the authors addressed this, the issue of "best outlet" for this publication remains (see above); the AC believes in the high value of the work, and supports publication, just unsure of outlet...

----

Reviewer ph2V: major concern is that Dr. Monk as the sole "ground truth" annotator is problematic; importantly, these ground truth annotations are **not** the major point of the publication (but the methodology and benchmarking method); as such the AC does not believe this to be grounds for rejecting the publication.

----

AC concerns:  the authors have addressed all concerns to my satisfaction (some concerns are limitations, which are equally addressed to my satisfaction).
Decision: Accept (Poster)