From: NeurIPS.cc/2022/Workshop/DistShift/Program_Chairs
title: Paper Decision
decision: Accept (Poster)
comment: The paper addresses the question of calibration in out-of-distribution evaluations. There are two main arguments: (1) the probability of the predicted label is not a good measure of calibration; (2) calibration and sharpness can be improved by temperature scaling. For a classifier trained with the standard ERM loss, it seems that the temperature-scaled probability ("max") is nearly identical to the entropy and superior to the margin, but the "out-of-the-box" version of "max" suffers from low sharpness.