Learning to Generate Equitable Text in Dialogue from Biased Training Data (2023)

Abstract: The ingrained principles of fairness in a dialogue system’s decision-making process and generated responses are crucial for user engagement, satisfaction, and task achievement. Absence of equitable and inclusive principles can hinder the formation of common ground, which in turn negatively impacts the overall performance of the system. For example, misusing pronouns in a user interaction may cause ambiguity about the intended subject. Yet, there is no comprehensive study of equitable text generation in dialogue. Aptly, in this work, we use theories of computational learning to study this problem. We provide formal definitions of equity in text generation, and further, prove formal connections between learning human-likeness and learning equity: algorithms for improving equity ultimately reduce to algorithms for improving human-likeness (on augmented data). With this insight, we also formulate reasonable conditions under which text generation algorithms can learn to generate equitable text without any modifications to the biased training data on which they learn. To exemplify our theory in practice, we look at a group of algorithms for the GuessWhat?! visual dialogue game and, using this example, test our theory empirically. Our theory accurately predicts relative-performance of multiple algorithms in generating equitable text as measured by both human and automated evaluation.

## introduction machine learning models for text-generation in dialogue have trouble learning the "long tail" of a data distribution; i.e., the data concepts not frequently observed during training. for example, dataset biases like gender imbalance can induce a long tail in training data whereby important data relationships involving gender are underrepresented, like women in sports @xcite . when training, generative models often fail to learn these concepts in the long tail, and ultimately, learn in-figure 1 : examples from guesswhat?! dataset, which consists of game dialogues where a question-player queries an answer-player to identify a secret goal object known only to the answer-player. images containing men outnumber images with women 2 to 1, forming a "long tail" in the data distribution. in examples above, human annotations use visual (contextual) cues to agree on gender or balk when appropriate. traditional algorithms can be incorrect and overconfident, inheriting dataset bias towards male pronouns. algorithms motivated by learning theory (leather) are more robust, utilizing context in a human-like way. despite the multi-faceted impact of inequitable text generation in dialogue, we do not have a comprehensive and theoretically grounded framework for understanding how machines learn to generate inequitable text and when this outcome can be avoided. to provide a strong technical foundation for equitable generation in dialogue, we build on theories of computational learning @xcite . specifically, our theoretical contributions are as follows: 1. we define precise constraints that encapsulate diverse notions of equity in dialogue (def. 3.1). 2. we rigorously compare our proposals to traditional notions of equity in classification ( § 3.1). 3. we show computational learning theory models equitable learning well: algorithms from learning theory are easily adapted to learn equitable dialogue by augmenting data (thm. 3.1). 4. we prove algorithms based on learning theory can even learn to generate equitable text from some types of biased training data (thm. 3.2). loosely, thm. 3.2 is based on the idea that, when provided sufficient background, human text is not biased because it is typically context-aware (def. 3.4). for example, when the subject is a female scientist, a human will likely not use male pronouns in subject-referring conversation because humans tend to correctly employ dialogue context to inform their language use. instead, in many real-world datasets, bias is an aggregate property, arising from inequality of the proportions of protected attributes such as race or gender; e.g., more conversations about male than female doctors. the theoretical understanding we contribute is imperative because it informs algorithm design. in particular, using our theory, we can predict: 1. the most equitable algorithms for unseen data; 2. counter-intuitive properties of algorithms that lead to less equitable results. for example, consider algorithms which naïvely augment data to remove bias @xcite . through theoretical study, we identify cases where this practice can actually hurt an algorithm's chances at learning to be equitable. in fact, our experiments in § 4 confirm this. the remainder of the paper is organized as follows: § 2 provides background to position our contributions including discussion of related work, a brief tutorial on the employed learning theoretic framework, and a few running examples used throughout the text; § 3 provides our theoretical contributions including formulation of mathematical notions of equity in text generation and theoretical analysis of learning algorithms; § 4 conducts experiments which validate our theory in practice; and finally, § 5 concludes the work. code, data, and a python package will be made publicly available to promote further research. 1 1 https://github.com/anthonysicilia/equitable-dialogue2 background and related work ## experiments in section 3, we conclude by outlining algorithmic insights revealed by our theory. next, we test these theories on the guesswhat?! game corpus. ## conclusions in this paper, we provide a first in-depth study of equity in dialogue, formalizing mathematical notions of equity in dialogue and using computational learning theory to study how equity can be achieved through algorithm design. our empirical results show how our formal theoretical study of equity in dialogue can be used, with great benefit, to select and design algorithms in a task-oriented dialogue setting. in particular, we can: design algorithms that achieve both equity and humanlikeness, predict unexpected consequences of dataaugmentation, and provide proxy statistics that are useful in ranking the equity of algorithms. to promote further research, our code, data, and a python package will be made publicly available. 14 limitations while our theoretical work is broadly applicable to any protected attribute and any dialogue task, our empirical study has primarily tested gender bias on the guesswhat?! task. continued experimental study on a wider range of protected attributes and tasks can better support our mathematical findings. also, users of our theory should verify the assumptions of our theory when using it to draw insights on new datasets. specifically, as the type of data bias changes, it is possible the assumptions of thm. 3.2 may no longer be met. users of our theory should take care in ensuring context-awareness and context-preservation, for example, are reasonable assumptions on new data, prior to applying the insights of § 3.3. lastly, while all of our gender annotations come from human annotators, only a smaller subset come from annotators primed to judge correctness/equity of gender reference. so, more in-depth human evaluation can better support our theoretical results as well.