Triple-Hybrid Energy-based Model Makes Better Calibrated Natural Language Understanding Models (2023)

Abstract: Though pre-trained language models achieve notable success in many applications, it’s usually controversial for over-confident predictions. Specifically, the in-distribution (ID) miscalibration and out-of-distribution (OOD) detection are main concerns. Recently, some works based on energy-based models (EBM) have shown great improvements on both ID calibration and OOD detection for images. However, it’s rarely explored in natural language understanding tasks due to the non-differentiability of text data which makes it more difficult for EBM training. In this paper, we first propose a triple-hybrid EBM which combines the benefits of classifier, conditional generative model and marginal generative model altogether. Furthermore, we leverage contrastive learning to approximately train the proposed model, which circumvents the non-differentiability issue of text data. Extensive experiments have been done on GLUE and six other multiclass datasets in various domains. Our model outperforms previous methods in terms of ID calibration and OOD detection by a large margin while maintaining competitive accuracy.

## introduction since many industrial applications involve safety -critical domains such as healthcare @xcite @xcite @xcite , anticipating credit card defaults @xcite and selfdriving @xcite , it's essential for machine learning systems to provide not only accurate but also well-calibrated predictions @xcite , which can help to decide whether it can be trusted. however, models achieving high accuracy usually lead to overconfidence and miscalibration @xcite @xcite . this motivates an interesting and important area that attempts to achieve a better trade-off between accuracy and calibration. in addition to id calibration, it's more important for machine learning models to produce high uncertainty when ood data is observed, rather than to produce wrong yet wildly confident predictions. related works. to overcome the problem of miscalibration, numerous methods have been proposed. the natural way is post-hoc calibration that transforms the output of the original network into calibrated confidence scores while maintaining the network's accuracy @xcite @xcite . the second method to mitigate miscalibration is to add regularizations during training such as label smoothing @xcite , mixup @xcite . @xcite and @xcite further conveys that the aforementioned methods can be applied to improve the calibration of pre-trained language models on nlu tasks. the third way is to design a specific loss function to minimize the discrepancy between accuracy and confidence. for example, @xcite lately propose the id and ood regularizer to leverage the relationship between accuracy and uncertainty, and it obtains a significant improvement over previous methods in id calibration and ood detection. energy-based models. in another line of work, joint ebm (jem; @xcite has been shown great improvements on id calibration and ood detection for images without explicit calibration correction mechanism. the core idea is to reinterpret a joint distribution p θ (x, y) from a neural classifier p θ (y|x) in the perspective of ebms and jointly optimize the marginal distribution p θ (x) and a neural classifier p θ (y|x). @xcite further investigate the ood detection performance with different training approaches for p θ (x) such as stochastic gradient langevin dynamics (sgld; @xcite , sliced-score-matching (ssm; @xcite and variational entropy regularized approximate maximum likelihood (vera; @xcite . besides, @xcite propose an implicit generative models based on ebms (igebm) and apply sgld to optimize p θ (x|y). it performs significantly better ood detection than other generative models. however, as shown by @xcite , the accuracy of igebm has dropped dramatically to 49.1% on cifar10 while standard finetuning can achieve 95.8% accuracy. this result indicates that different loglikelihood factorization leads to great gaps in accuracy, id calibration and ood detection. moreover, these training methods such as sgld, ssm, vera need to calculate the gradients about inputs, the none differentiability of text data limits the application of these methods on both calibration and ood detection for nlu tasks. recently, @xcite proposes a joint training of classifier p θ (y|x) and marginal distribution p θ (x) based on residual ebm @xcite for nlu tasks. different from jem, their model is more flexible by designing various energy functions for marginal distribution without any restriction on joint distribution p θ (x, y). to estimate the parameters of marginal distribution p θ (x), they propose to apply noise contrastive estimation (nce; @xcite to train the energy model by discriminating the real data and the fake data generated by a noise distribution. to make the noise distribution as close as possible to the data distribution, they finetune a task-specific gpt-2 @xcite . though it achieves improvements on id calibration, it's often resource-intensive compared to previous methods to finetune gpt-2 @xcite . moreover, the quality and quantity of fake samples generated by noise distribution has great impacts for nce training @xcite . ## triple-hybrid energy-based model motivation. many works @xcite @xcite have shown that ebms could significantly reduce the expected calibration error and improve outof-distribution detection for image classification. specifically, the jem proposed in @xcite factorizes the joint distribution log p θ (x, y) into log p θ (x) + log p θ (y|x), where log p θ (y|x) is to maintain the classification performance and log p θ (x) is the generative term which contributes to better calibration and out-of-distribution detection. on the contrary, the igebm proposed in @xcite factorizes the joint distribution log p θ (x, y) into log p θ (y) + log p θ (x|y) for implicit generation and surprisingly find that it achieves better ood performance. however, lack of p θ (y|x) leads to terrible classification performance. it's shown in @xcite that the classification accuracy dropped dramatically to 49.1% on the cifar10 dataset, while the accuracy is 92.9% by jem. on the other hand, liu and abbeel (2020) proposed a hybrid discriminative-generative energybased model (hdge) for both classification and generation. the loss function consists of a discriminative conditional log-likelihood log p θ (y|x) and a generative conditional log-likelihood log p θ (x|y). compared to igebm, it includes log p θ (y|x) and thus achieves better classification performance. compared to jem, it includes the conditional generative model, rather than the marginal generative model. in other words, jem targets to reduce the energy for data from the population p θ (x), while hdge aims at reducing the energy for compatible pair (x, y). this motivates us to combine the benefits of both conditional and marginal generative model for better calibration and ood detection. ## experiments in this section, we conduct thorough experiments to investigate the empirical peformance of our proposed methods. we first introduce the criteria for id calibration and ood detection. ## conclusion in our work, we propose a triple-hybrid ebm with combination of classifier, conditional generative model and marginal generative model into a unified framework called them. to train ebms effectively and efficiently, we leaverage contrastive learning to approximate the log-likelihood of ebms with negligible computational resources. extensive experiments demonstrates that our model outperforms the state-of-art methods in terms of id calibration and ood detection with competitive accuracy. we further apply contrastive learning to jem and igebm without considering the generation ability to obtain jem(cl) and igebm(cl) respectively. compared to jem(cl) and hdge(cl), our model is more robust to the hyper-parameters of contrastive learning including the temperature and size of memory bank in terms of id calibration and ood detection.