  Dialogue systems in the form of chatbots and personal assistants are being
increasingly integrated into people's lives. These dialogue systems often have
the ability to adopt an anthropomorphic persona, mimicking a societal
demographic to appear more approachable and trustworthy to users. However, the
adoption of a persona can result in the adoption of biases. We define persona
biases as harmful differences in text (e.g., varying levels of offensiveness or
affirmations of biased statements) generated from adopting different
demographic personas. In this paper, we present the first large-scale study on
persona biases in dialogue systems and conduct analyses on personas of
different social classes, sexual orientations, races, and genders. Furthermore,
we introduce an open-source framework, UnitPersonaBias, a tool to explore and
aggregate subtle persona biases in dialogue systems. In our studies of the
Blender and DialoGPT dialogue systems, we show that the choice of personas can
affect the degree of harms in generated responses. Additionally, adopting
personas of more diverse, historically marginalized demographics appears to
decrease harmful responses the most.
