  Recent advances in computer vision have led to the development of image
classification models that can predict tens of thousands of object classes.
Training these models can require millions of examples, leading to a demand of
potentially billions of annotations. In practice, however, images are typically
sparsely annotated, which can lead to problematic biases in the distribution of
ground truth labels that are collected. This potential for annotation bias may
then limit the utility of ground truth-dependent fairness metrics (e.g.,
Equalized Odds). To address this problem, in this work we introduce a new
framing to the measurement of fairness and bias that does not rely on ground
truth labels. Instead, we treat the model predictions for a given image as a
set of labels, analogous to a 'bag of words' approach used in Natural Language
Processing (NLP). This allows us to explore different association metrics
between prediction sets in order to detect patterns of bias. We apply this
approach to examine the relationship between identity labels, and all other
labels in the dataset, using labels associated with 'male' and 'female') as a
concrete example. We demonstrate how the statistical properties (especially
normalization) of the different association metrics can lead to different sets
of labels detected as having "gender bias". We conclude by demonstrating that
pointwise mutual information normalized by joint probability (nPMI) is able to
detect many labels with significant gender bias despite differences in the
labels' marginal frequencies. Finally, we announce an open-sourced nPMI
visualization tool using TensorBoard.
