SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 14: Ablation study for Accuracy.
Metrics	MSE↓	MAE↓
Shot	All	Many	Med.	Few	All	Many	Med.	Few
FDS [49]	109.78	93.99	124.96	216.97	8.12	7.52	8.68	12.25
ENCODER-ONLY VIR (OURS)	95.99	81.89	121.78	157.92	7.57	6.97	8.72	10.03
DER [1]	106.81	91.32	122.45	209.76	8.11	7.36	9.03	12.69
PREDICTOR-ONLY VIR (OURS)	88.96	74.79	95.85	203.76	7.28	6.68	7.76	11.63
LDS+FDS	99.46	84.10	112.20	209.27	7.55	7.01	8.24	10.79
LDS + PREDICTOR-ONLY VIR (OURS)	87.48	73.72	107.64	161.69	7.17	6.63	8.06	9.80
LDS + ENCODER-ONLY VIR (OURS)	96.46	86.72	102.56	171.52	7.51	7.08	7.93	10.45
VIR (OURS)	81.76	70.61	91.47	142.36	6.99	6.39	7.47	9.51
Table 15: Ablation study for Uncertainty.
Metrics	NLL ↓	AUSE ↓
Shot	All	Many Med.	Few	All	Many Med.	Few
DER [1]	3.936	3.768	3.865	4.421	0.590	0.449
PREDICTOR-ONLY VIR (OURS)	3.887	3.755	3.854	4.394	0.443	0.387
LDS + PREDICTOR-ONLY VIR (OURS)	3.722	3.604	3.821	4.209	0.441	0.457
VIR (OURS)	3.703	3.598	3.805	4.196	0.434	0.456
effectiveness. Both verifies our claim that directly reweighting DER breaks NIG and leads to poor performance.
E.2 Ablation Study on  $\lambda$ .
In this section, we include ablation studies on the  $\lambda$  in our objective function. For  $\lambda \in \{10.0, 1.0, 0.1, 0.01, 0.001\}$ , we run our VIR model on the AgeDB dataset. Table 12 shows the results. We can observe that our model achieves the best performance when  $\lambda = 0.1$ .
E.3 Ablation Study on Number of Bins
In this section, we include ablation studies on the number of bins in our settings. For the cases with  $100 / 1 = 100$ ,  $100 / 3 \approx 33$ , and  $100 / 5 = 20$  bins, we run our VIR model on the AgeDB dataset. Table 13 shows the results. We can observe that our VIR remains consistent regardless of the number of bins.
E.4 Calibrated Uncertainty
Table 16: Calibration Uncertainty on AgeDB-DIR.
Metrics	NLL↓	AUSE↓
Shot	All	Many	Medium	Few	All	Many	Medium
DEEP ENS. [27]	5.311	4.031	6.726	8.523	0.541	0.626	0.466
[CALIBRATED] DEEP ENS. [27]	5.015	3.978	6.402	8.393	0.506	0.591	0.386
INFER NOISE [29]	4.616	4.413	4.866	5.842	0.465	0.458	0.457
[CALIBRATED] INFER NOISE [29]	4.470	4.183	4.756	5.622	0.404	0.426	0.410
DER [1]	3.918	3.741	3.919	4.432	0.523	0.464	0.449
[CALIBRATED] DER [1]	3.827	3.674	3.835	4.297	0.479	0.401	0.399
LDS + FDS + DER [1]	3.787	3.689	3.912	4.234	0.451	0.460	0.399
[CALIBRATED] LDS + FDS + DER [1]	3.708	3.636	3.807	4.032	0.417	0.364	0.269
VIR (OURS)	3.703	3.598	3.805	4.196	0.434	0.456	0.324
[CALIBRATED] VIR (OURS)	3.577	3.493	3.595	3.866	0.359	0.232	0.276
In a bid to enhance the evaluation of our model's uncertainty, we used our validation set to apply calibration techniques (specifically, variants of temperature scaling [14]) on different methods. We focused on each of the output distribution parameters  $\nu, \alpha, \beta$  as discussed in our main paper, introducing individual scalar weights for each parameter to calibrate uncertainty estimation. Upon deriving each weight  $\mathbf{w}_{\nu}, \mathbf{w}_{\alpha}, \mathbf{w}_{\beta}$  from the validation set, we incorporated them into the test dataset to ascertain the final performance. The data outlined in Table 16 indicates that following re-calibration, the uncertainty was further optimized. Notwithstanding such uncertainty calibration, our model persists in demonstrating superior performance compared to other benchmark methods.
E.5 Error bars on Accuracy
In order to further underscore the superiority of our methodology, we also included error bars for our proposed method (i.e., VIR) which are generated from five independent runs. Note that we did not report error bars for those second/third best baselines (e.g., RankSim, LDS+FDS) since we directly use the reported performance from their papers, and in NYUD2-DIR, all the error bars on VIR are approximately 0.001. Due to the width constraints of the paper, these are not included in Table 10. Results in Table 7, Table 8, and Table 9 demonstrate the effectiveness of our approach.