Similarity Based Label Smoothing For Dialogue Generation (2022)

Abstract: Generative neural conversational systems are typically trained by minimizing the entropy loss between the training “hard” targets and the predicted logits. Performance gains and improved generalization are often achieved by employing regularization techniques like label smoothing, which converts the training “hard” targets to soft targets. However, label smoothing enforces a data independent uniform distribution on the incorrect training targets, leading to a false assumption of equiprobability. In this paper, we propose and experiment with incorporating data-dependent word similarity-based weighing methods to transform the uniform distribution of the incorrect target probabilities in label smoothing to a more realistic distribution based on semantics. We introduce hyperparameters to control the incorrect target distribution and report significant performance gains over networks trained using standard label smoothing-based loss on two standard open-domain dialogue corpora.

## introduction response generators rely heavily on language modelling for response generation. given a context comprising multiple conversation utterances, a response generator is formulated as a next utterance prediction problem, where the task is to generate a response conditioned on the context. with the advent of deep learning and availability of sufficient training data, parametric models like recurrent neural networks and transformers are generally used for language modelling. trained by minimizing the expected cross entropy between the training targets and the prediction logits, such models often overfit the training data and does not generalize well on the test set. label smoothing proposed by @xcite to improve the performance of inception net image classifier on the imagenet dataset has gained wide acceptance in natural language processing tasks as a regularization technique to enhance the generalization capability of deep neural networks. @xcite in his work "attention is all you need", where he proposed the state-of-the-art transformer architecture, had reported performance gains in machine translation using label smoothing during training. unlike other regularization techniques which constrain the model parameters and hidden representations, label smoothing augments the actual targets by reducing the target probability and assigning low probabilities to all classes, following a data independent uniform distribution. thus, preventing the model from predicting the correct labels overconfidently during training. however, as pointed out by @xcite and @xcite , the probabilities assigned to both the correct and incorrect classes constitute the knowledge of a network. in language modelling, incorporating label smoothing and assigning a uniform probability to all the incorrect classes can convey a false knowledge to the model. for example, as depicted in figure 1 , while generating "i am doing good" in response to the query "how are you doing ?", having generated the partial phrase "i am doing", both "great" and "awesome" conveys the same message as "good". although "bad" would convey an opposite yet meaningful message, a random word like "aeroplane" would be inappropriate. hence, instead of assuming a uniform distribution for the incorrect classes while using label smoothing, we can incorporate a weighing mechanism to present such knowledge to the model. here, we introduce simple ways of imparting such information by modifying the data independent uniform distribution in label smoothing with a more appropriate data dependent distribution proportional to the pre-trained word-embedding based similarity between the actual and incorrect targets. our primary contributions are follows (i) we propose a robust mechanism for augmenting the target labels in language modelling, which better reflects the real-world. (ii) we experiment our proposed framework with different hyper-parameter settings and perform thorough analysis of the observations 1 2 related work although numerous techniques have been introduced to enhance the generalizability of neural networks, as pointed out by @xcite , most work focus on regularizing model parameters, compared to external regularization techniques like label smoothing or target data augmentation. recent approaches for generalizable conversations can be broadly categorized as follows loss function augmentation @xcite proposed using maximum mutual information along with the cross entropy loss, in order to penalize generic responses like "i do not know", which are frequent in conversational datasets. @xcite attributed generic responses to the cross entropy loss function, which prefers frequent tokens. they proposed augmenting the loss function with a frequency based weighing mechanism dependent on the corpus for engendering diverse responses. @xcite experimented with using optimal transport to match sequences generated in the teacher and student modes, and increasing performance of student forced networks on the test dataset by reducing the gap between the two modes. @xcite proposed an adaptive label smoothing approach that can adaptively estimate a target label distribution at each time step for different contexts. compared to their approach, our proposed method is simpler with fewer parameters. data augmentation @xcite demonstrated that conversational datasets generally don't exhibit coherence in query response pairs, which affect the cross entropy loss. they propose a training data augmentation module, which can not only replace words in the actual target response with similar words using bert @xcite ), 1 code and dataset available here. but also augment the style of the response, preserving the meaning. they further introduced a neural weighting mechanism, which can assign weights or importance to the augmented and golden training data, and report significant performance gains. @xcite demonstrated that the log loss is not robust to noise, and hence proposed truncating the distribution of the training targets to achieve an easy to optimize and more robust loss function. @xcite introduced a network which can provide negative generated samples, and train the generation model to maximize the log likelihood of training data while minimizing the likelihood of negative samples. since instead of augmenting the training data, we adjust the probability of incorrect labels for each correct label, our proposed method belongs to the first category. ## methods and experiments we experiment with ways to augment the data independent uniform distribution enforced by label smoothing. let u i be an utterance consisting of words {w j } n @xmath0, where n is the number of words in the utterance. for each word w j , in label smoothing a probability 1s is assigned to the true label w j , and a probability of s (smoothing factor) is distributed uniformly among the rest of the k words in the vocabulary. we augment the distribution of the incorrect class by weighting the smoothing factor s according to the cosine similarity between the glove @xcite word embedding of the correct word in the training data and all the words in the vocabulary. thus, if the correct word to be predicted is "good", then the words "great" and "awesome" in the vocabulary would get a higher proportion of the smoothing factor s, compared to an unrelated word like "aeroplane"presenting more accurate knowledge to the model. mathematically, let ⃗ w j be the glove word embedding of word w j , w k be a matrix containing the glove word embedding for all the words in the vocabulary (including w j ), ⃗ w j_sim be the vector of cosine similarity between the word w j and all the words in the vocabulary. since glove word embeddings are learned representations, they can be noisy. hence, we introduce a binary mask mask j using a threshold t, below which we set the cosine similarity value in ⃗ w j_sim as 0, and multiply the similarity vector with the mask. the resulting vector is normalised to lie between 0 and 1, and finally multiplied by s. we treat t as a model hy-perparameter, and is tuned using grid search. we further reason that although glove embeddings are learned from text corpora, there are possibilities that dissimilar words can lie in close proximity in the embedding space, resulting in a high cosine similarity score, and presenting an incorrect knowledge to the model. to circumvent this problem, we further experiment with filtering out the cosine similarities of dissimilar words based on wordnet sysnets @xcite , which we achieve by implementing another mask mask ## results and analysis we compare the (i) sacrebleu score @xcite : a standardised version of the bleu score @xcite , (ii) rouge l score @xcite : which compares longest common subsequence (lcs), and automatically takes into account sentence level structure similarity and identifies longest co-occurring in sequence n-grams, (iii) meteor score @xcite : an improvement over bleu score, which incorporates stemming and synonymy matching along ## conclusion label smoothing has an undesirable property of assigning uniform probability to incorrect labels, which present an incorrect knowledge to learn from. in this paper we propose ways to convert the uniform distribution to a data dependent distribution by weighing the smoothing probability using cosine similarity of word embeddings between the correct and incorrect labels. we further experiment with wordnet synonyms as an additional filtering criteria, and report our findings. although we achieve significant improvements over all baselines, we notice a drawback where the proposed system is unable factor in context while weighing the distribution of the incorrect labels. as future research, we intend to address this drawback using more contextualised representations instead of static embeddings.