Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a study that compares disparities and bias between humans and machines in performing face verification and recognition in challenging conditions. The authors use two datasets named LFW and CelebA for these studies. Human experiment consists of randomly sampled subjects answering verification/identification questions and machine experiments were run using existing academic and commercial models.    
strength_and_weaknesses: Strengths:
- To the best of my knowledge, this is the first time such a study is done to compare human and machine bias on face verification/identification. 
- It looks like the authors have done a good job of curating the datasets by removing poor quality images and correcting the inconsistencies in its metadata. 

Weaknesses:
- Other than curating the datasets, I do not see much scientific contribution of this work. All the algorithms used for conducting experiments are existing algorithms.
- The results observed in the paper are expected results. I am not sure how these findings help in further improving the research in the face identification/verification area.

clarity,_quality,_novelty_and_reproducibility: Other than curating the datasets, I do not see much scientific contribution of this work. All the algorithms used for conducting experiments are existing algorithms. 

There are several grammatical errors in the paper. Also, it was hard to follow the writing in many sections of the paper.
summary_of_the_review: Overall I think the study presented in paper lacks novelty and the presented results are expected.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Bias in face recognition is a significant area of research that has/will have a meaningful impact on analyzing and managing disparities in downstream applications. The authors of this paper propose enhancements to the existing face datsets such as LFW and CelebA by improving the available metadata and also devise experiments to measure the bias exhibited by humans and models during identification and verification tasks.

strength_and_weaknesses: Strengths: 
1. Well-written and structured
2. Experimentally strong
3. Results 

Weaknesses: 
1. Missing technical novelty. 
2. No mention of how the existing model architectures were revised for the task on hand. 
3. It is not clear if the apparent bias in data (eg certain age groups were more populated than others) while training/validating and testing.
clarity,_quality,_novelty_and_reproducibility: Detailed and well-written. Adding in missing details will help with reproducibility.
summary_of_the_review: Overall, a well-written paper. The findings are interesting but may not be extrapolated as suggested by the authors.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors aim to answer two questions in this paper, "How and to what extent do humans exhibit
bias in their accuracy in facial recognition tasks?" and "How does this compare to machine learningbased models?"  To address these questions the authors present improvements to the LFW and CelebA dataets as well as present an analysis showing that both humans and models result in lower accuracy on dark skinned and female subjects and humans obtain higher accuracy on the tasks when their demographics match the subjects in the data.  They also find that while academic models are comparable in gender bias compared to humans, they are significantly more biased against darker skin tones than humans.
strength_and_weaknesses: 
The paper is clear and the work is well motivated.  It is certainly important to gauge model performance compared to humans.

It is great that the authors identify the weakness of these face datasets and present cleaned annotations and scrubbing of images that may not be desired in the set.

For some use cases, low quality images may be desired.  It may be preferable to include them but add a quality label to the annotations.  What are the specific thresholds used to decide to omit these images?

"old-timey" should be better defined so as not to be subjectively determined.

If black and white and old-timey images were removed due to the being too easy to recognize as paired then this should be validated but no quantitative check is presented.

"We found that many questions could be answered without considering face features at all, and these
were removed. For example if the subject is wearing identical attire and/or standing in front of an
identical background in two images."  Again, this needs to be shown to be true quantitatively if the authors are referring to the models ability to match faces.  Many models will not use background features making the issue not as important as presented.

"While the Fitzpatrick scale is not perfect, it is the best systematic option currently for ensuring a
broad representation."  Not sure this is still true with the existence of the Monk scale which was specifically created for this task while Fitzpatrick was not, although Fitzpatrick is far more commonly used.

What is the usefulness of the birth date if the date of image capture is not provided?

It would be helpful for future projects if more details on the process and workflow to manually check/verify the annotations in the data were provided.  Lessons learned and the final selected approach would be helpful for other researchers that are interested in a similar process.

It would be much better for the reader if results comparing the models and humans were presented in graphs, figures and tables to provide more detail than what is in Table 2.  This would also allow for high-level take-aways on the details to be better digestible.  A lot of specific statistical results are presented in the text but this should be reflected in figures as well.  Some details can be moved to an appendix if needed.

It would also be helpful to understand the per-human performance.  Some participants may be better at the tasks than others.  The human results would likely form a distribution of performance and this should be captured and compared to the per-model, and academic group as well as commercial group, model distribution.  The uncertainty from these distributions needs to be considered as we are comparing over multiple variables.

Odds ratio is not commonly used by the face rec community so a better description may be helpful

additional metrics such as FMR and FNMR, or FAR and FRR, should be presented.  It would help to understand where the errors are occurring and if the disparities are a result of one type of error or the other.  This is also useful for comparing human to model performance.

Typo at end of section 5: " In this case, we see a bias towards question gender in favor of males"
clarity,_quality,_novelty_and_reproducibility: The approach is clearly presented but lacks some details that would make the dataset scrubbing and annotation tasks fully reproducible.  This is a model vs human analysis paper so no training to replicate.  There have been other papers that have compared human to model performance so not completely novel but does present a larger study with more specific tasks than presented prior.

In general, the paper is lacking in quality that meets the standard of the conference as key metrics are missing and results should be more clearly presented and with more detail.
summary_of_the_review: The paper is clear and the problem is a good one to address but it lacks some key details and presentation limitations that reduce its score.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an benchmark (dataset and procedure) to evaluate the bias of human performance in the face recognition problem. To offer more information, the paper reports a set of similar tests on machine performance. 
strength_and_weaknesses: Strengths:
 - I totally agree with the paper that there is need in the research to evaluate the bias and compare the human performance with machine performance for the face recognition problem. Contribution here are welcomed
- release of curated database with reference methods evaluated should help the field.
- the paper is well presented and clear

Weaknesses:
1. The main issue of such paper is the following: how it was established the ground truth? By human annotation?! One key aspect of the paper is that it reports a 67%-83% accuracy of human observers accuracy. On other hand the limited number of persons in the authors team provides the ground truth. It is quite hard to take this as an established ground truth especially since there no is strong indication that experts (i.e. authors here) behave much better than normal persons. It is further said several time in the paper, that "one" of the authors revised the data to make sure that no errors are further passed. Yet a person has up to 83% accuracy and quite often 70%. Based on the data provided, curation by machine learning is preferred. 
At the end of the paper it is noted: "The labels and the means of collecting those labels is very important, and should almost always involve human review. We found that existing datasets’ labels (often which used computer-generated labeling methods) were widely insufficient and riddled with errors which have downstream implications on analysis." I believe that this is true but is also applicable to the ground truth of this data, since there is no information that consensus of large number of experts was used.
 
In other words, in my view, the main limitation of the paper is that ground truth is not established accurately. A large number of observers should have been used and average where there is consensus should be placed as labels. 

2. The paper does not report the number of incorrect labels in LFW. From the data reported in the paper and in the supplementary material, it seems that the amount of errors was small and most of curation was about removing non-public persons. Unfortunately, in this way the contribution w.r.t the dataset is harder to establish.
  
clarity,_quality,_novelty_and_reproducibility: The paper is clear and reproducible. There is novelty, but the problem lies in the accuracy of data.
summary_of_the_review: The paper aims to a direction where contributions are welcomed, and there is an auditorium at ICLR. The main  problem is that the accuracy of the data was not well established and in this case is critical.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a study on human bias in face recognition that is contrasted with model bias in machine learning. An elaborate human evaluation of two tasks, face identification and verification, has been conducted. A number of conclusions have been reported contrasting machine and human biases in terms of gender, and skin color. AC agrees with the reviewers that albeit it’s an important study, limited technical contribution (how to resolve existing model failures) and a narrow application domain (the paper studies face recognition and bias in face recognition) are two critical issues that place the contributions below the acceptance bar. We hope the reviews are useful to improve the manuscript. 
justification_for_why_not_higher_score: AC agrees with the majority of the reviewers.
Decision: Reject