  The capabilities of natural language models trained on large-scale data have
increased immensely over the past few years. Downstream applications are at
risk of inheriting biases contained in these models, with potential negative
consequences especially for marginalized groups. In this paper, we analyze the
occupational biases of a popular generative language model, GPT-2, intersecting
gender with five protected categories: religion, sexuality, ethnicity,
political affiliation, and name origin. Using a novel data collection pipeline
we collect 396k sentence completions of GPT-2 and find: (i) The
machine-predicted jobs are less diverse and more stereotypical for women than
for men, especially for intersections; (ii) Fitting 262 logistic models shows
intersectional interactions to be highly relevant for occupational
associations; (iii) For a given job, GPT-2 reflects the societal skew of gender
and ethnicity in the US, and in some cases, pulls the distribution towards
gender parity, raising the normative question of what language models _should_
learn.
