Joint Energy-based Model Training for Better Calibrated Natural Language Understanding Models (2021)

Abstract: In this work, we explore joint energy-based model (EBM) training during the finetuning of pretrained text encoders (e.g., Roberta) for natural language understanding (NLU) tasks. Our experiments show that EBM training can help the model reach a better calibration that is competitive to strong baselines, with little or no loss in accuracy. We discuss three variants of energy functions (namely scalar, hidden, and sharp-hidden) that can be defined on top of a text encoder, and compare them in experiments. Due to the discreteness of text data, we adopt noise contrastive estimation (NCE) to train the energy-based model. To make NCE training more effective, we train an auto-regressive noise model with the masked language model (MLM) objective.

## introduction calibration refers to how well a classification model's confidence (reflected by its output posterior probability) aligns with its actual accuracy. as deep learning models achieve amazing accuracy in computer vision @xcite or natural language processing (nlp) @xcite , more research attention has been drawn to the calibration aspect of these models. as shown by @xcite , the high accuracy from deep models does not always lead to better calibration. this motivates an important line of works attempting to achieve a better trade-off between accuracy and calibration. most existing calibration methods @xcite @xcite generally rescale the posterior distribution predicted from the classifier after training. such post-processing methods require a held-out development set with a decent number of samples to be ˚bryan mccann contributed to this work while he was at salesforce research. available. to overcome this constraint, @xcite uses a penalty term to encourage better calibration during training. in another line of work, @xcite shows that one can jointly train an energy-based model (ebm) during the standard training of a neural classifier. although calibration is not explicitly addressed during ebm training, the calibration of the resulting model is shown to be greatly improved. some intuitions of the underlying reasons will be given in section 2.3. however, the training framework proposed by @xcite is designed for image classifiers, and it can not be readily applied to discrete text data. in this work, we propose a framework that uses noise contrastive estimation (nce) to jointly train an energy-based model during the finetuning of pretrained text encoders (e.g., bert @xcite or roberta @xcite ) for nlu tasks. we compare several variants of energy functions that can be defined on top of the encoder. our experiments show that the resulting models achieve competitive calibration results comparing to strong baselines, with little or no loss in accuracy. ## related works finally, we review applications of nce or energybased models in the nlp literature. due to its selfnormalizing property, nce training has been used for faster inference @xcite @xcite of auto-regressive language models. it has also been used in an attempt to train a sentence-level bi-directional lm @xcite . more closely related to our work, deng et al. (2020) adopts nce to train an ebm defined on top of a text encoder (the scalar variant), and uses it to improve language generation. ebm has also been recently used in non-autoregressive machine translation @xcite . ## conclusion in this work, we explore joint ebm training during the finetuning of pretrained text encoders with noise contrastive estimation. we find that joint ebm training can greatly improve the calibration of nlu models, with little or no loss on accuracy. text: q: what city north of new york was settled by huguenots? a: huguenot immigrants did not disperse or settle in different parts of the country, but rather, formed three societies or congregations; one in the city of new york, another 21 miles north of new york in a town which they named new rochelle, and a third further upstate in new paltz. label: 1 êθ pxq: -8.48 baseline: (.997, .003) ñ ebm: (.995, .005) text: q: what is the source of oxygen production through electrocatalytic means? a: a similar method is the electrocatalytic o2 evolution from oxides and oxoacids. label: 1 êθ pxq: 4.22 baseline: (.252, .748) ñ ebm: (.472, .527)