From: NeurIPS.cc/2021/Workshop/DistShift/Program_Chairs
title: Paper Decision
decision: Accept (Poster)
comment: This paper shows how pre-training might help calibration only by a limited amount, depending on the distribution shift between the data during pre-training and fine-tuning. This paints a slightly more nuanced picture, challenging the general belief that pre-training helps calibration. The reviewers agree that a more quantitive characterization of the amount of "shift" would be important. Also, could you add intuition for why this phenomenon could be happening? Finally, please add references to related work that conduct similar studies as mentioned by Reviewer 4WtZ.