  Social biases on Wikipedia, a widely-read global platform, could greatly
influence public opinion. While prior research has examined man/woman gender
bias in biography articles, possible influences of confounding variables limit
conclusions. In this work, we present a methodology for reducing the effects of
confounding variables in analyses of Wikipedia biography pages. Given a target
corpus for analysis (e.g. biography pages about women), we present a method for
constructing a comparison corpus that matches the target corpus in as many
attributes as possible, except the target attribute (e.g. the gender of the
subject). We evaluate our methodology by developing metrics to measure how well
the comparison corpus aligns with the target corpus. We then examine how
articles about gender and racial minorities (cisgender women, non-binary
people, transgender women, and transgender men; African American, Asian
American, and Hispanic/Latinx American people) differ from other articles,
including analyses driven by social theories like intersectionality. In
addition to identifying suspect social biases, our results show that failing to
control for confounding variables can result in different conclusions and mask
biases. Our contributions include methodology that facilitates further analyses
of bias in Wikipedia articles, findings that can aid Wikipedia editors in
reducing biases, and framework and evaluation metrics to guide future work in
this area.
