Robust metrics for the evaluation of fairness and safety in NLG

Johnny Tian­Zheng Wei Natural Language Processing Motivation. As language technology matures, we can expect natural language generation (NLG) systems to be more pervasive in our daily life. Companies routinely make headlines on successfully applying NLG to settings ranging from news personalization, customer service, and social media marketing. Startups such as Replika even specialize in developing chatbot technology to alleviate lonliness [1]. The rise in NLG applications is in part due to recent developments of end­to­end, data­driven neural generation models. These are expressive models that can learn to generate arbi­ trary sequences of words, and can be learned largely from data, significantly reducing engineering effort. As an example, it is now now easier than ever to prototype data­driven, state­of­the­art dia­ log models using publicly available conversational datasets and dialog­specific pretrained models trained on massive amounts of web conversations [2]. Incorporating neural generation techniques that leverage large amounts of data into NLG em­ pirically gives us better performance. However, we must recognize that our language found in the training data takes on social meaning. For instance, stereotypes, reflected in language, serve to spread beliefs about social groups to reinforce hierarchies, leading to social inequality. In extreme cases, language incites behavior such as self­harm, hate, or violence. It is known thatNLG systems can replicate, or even amplify the toxicity found in their training data [3]. As a commercial example, Tay [4], which Microsoft released to Twitter in 2016, tweeted racist and sexually­charged messages learned from its training data. For startups such as Replika, an insensitive utterance to a vulnerable individual could have severe consequences. Robust identification of unfair and un­ safe language is necessary in deployment and crucial to mitigation in training time to ensure our systems do not perpetuate societal inequalities or harm at­risk individuals. The methods to determine fairness and safety in NLG is converging on “output, classify, and count” ­ where 1) NLG systems are prompted for a set of outputs, then 2) an automatic classifier labels each output e.g. whether the output contains a microaggression, and finally 3) the number of positive labels are reported [5]. For language generation, the use of an automatic classifier is necessary, as the amount of equivalent human annotation implicated may be impractical in devel­ opment cycles. In the current paradigm, the introduction of an imperfect classifier is problematic, as the errors of the classifier should be taken into account for scientifically valid inference. This proposal seeks to address two critical robustness issues with current fairness and safety metrics in NLG deployment, and propose a mitigation technique based on robust metrics in training: (1) Bridging the gap betweenmetrics and quantification. Currently metrics research originating from the NLG communities does not engage with quantification research, which has its roots in statistics and social science. Fundamentally, both fields count with classifiers. These fields are complementary ­ quantification provides the statistical tools to adjust the observed counts based on classifier errors, and metrics provide the underlying models that can be used to detect the often complex unfair or unsafe phenomenon. There are two directions we can take to connect the two fields: 1) calibrating metric output probabilities with techniques such as Platt scaling so they may interplay with existing quantification tools [6], or 2) adopt structured losses derived from group­ level constraints in training metrics, with the goal to equalize false positive and false negative rates so metric errors may cancel out when used for counting [7]. (2) Training metrics robust to confounding features. For the underlying classifier used in quan­ tification, an assumption on its generalization breaks down when there are confounding features