SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 4: Mean accuracy (standard deviation) in percentage on MNIST, Color-MNIST, and CIFAR-20 over the last ten epochs under support shift in case (iii) with DA baselines (5 trials). Best and comparable methods (paired  $t$ -test at significance level  $5\%$ ) are highlighted in bold. This result corresponds to the bottom row in Figure 3.
Data	Case	Val-only	Pretrain-val	CCSA	DANN	GIW
MNIST	(iii)	78.94 (0.99)	88.31 (1.02)	94.37 (0.46)	87.24 (1.58)	95.16 (0.20)
MNIST	(iv)	79.86 (0.26)	86.25 (0.95)	88.76 (0.86)	82.92 (1.89)	90.94 (0.91)
Color-MNIST	(iii)	31.19 (0.65)	80.40 (3.62)	80.13 (5.90)	64.34 (5.08)	93.87 (0.19)
CIFAR-20	(iii)	28.61 (0.66)	42.79 (0.41)	60.87 (0.39)	58.12 (0.44)	59.73 (0.47)
Table 5: Mean accuracy (standard deviation) in percentage on MNIST, Color-MNIST (C-MNIST), and CIFAR-20 over the last ten epochs under support-distribution shift in case (iii) with IW-like baselines (5 trials). Best and comparable methods (paired  $t$ -test at significance level  $5\%$ ) are highlighted in bold. LN/CS is short for label noise/class-prior shift. This result corresponds to Figure 5.
Data	Shift	Val-only	Pretrain-val	Reweight	MW-Net	R-DIW	DIW	GIW
MNIST	LN 0.2	78.94 (0.99)	86.34 (0.77)	89.28 (0.64)	82.29 (0.80)	82.70 (0.76)	80.12 (2.90)	94.60 (0.22)
LN 0.4	78.94 (0.99)	85.43 (0.97)	84.73 (1.27)	80.92 (0.52)	81.19 (0.39)	79.00 (0.33)	93.74 (0.53)
CS 10	78.94 (0.99)	86.43 (0.71)	80.99 (0.67)	79.76 (0.60)	80.15 (0.38)	79.73 (0.38)	93.43 (0.40)
CS 100	78.94 (0.99)	84.54 (1.61)	80.65 (0.71)	77.06 (0.71)	76.53 (0.45)	78.58 (0.96)	91.52 (0.59)
C-MNIST	LN 0.2	31.19 (0.65)	60.21 (4.47)	30.82 (1.04)	40.00 (0.44)	38.25 (0.46)	38.04 (1.83)	90.99 (0.61)
LN 0.4	31.19 (0.65)	57.13 (5.48)	27.26 (0.41)	39.98 (0.06)	39.08 (0.02)	39.02 (0.50)	90.09 (0.88)
CS 10	31.22 (0.69)	87.63 (1.97)	38.61 (0.62)	39.16 (0.19)	39.15 (0.19)	39.18 (0.19)	94.43 (1.87)
CS 100	31.22 (0.69)	79.70 (3.29)	37.70 (0.21)	36.81 (0.10)	36.75 (0.17)	36.98 (0.23)	88.34 (0.87)
CIFAR-20	LN 0.2	29.18 (0.30)	42.48 (0.55)	47.01 (0.87)	49.19 (0.31)	50.13 (0.29)	48.70 (0.36)	54.19 (0.69)
LN 0.4	29.18 (0.30)	39.46 (0.47)	38.94 (1.10)	43.29 (0.42)	43.10 (0.29)	43.23 (0.29)	45.48 (0.36)
CS 10	28.45 (0.27)	37.31 (0.28)	42.25 (0.68)	44.96 (0.21)	45.19 (0.20)	39.68 (0.14)	49.17 (0.40)
CS 100	28.45 (0.27)	36.55 (0.29)	31.95 (0.77)	34.66 (0.28)	34.87 (0.12)	26.39 (1.05)	40.27 (2.17)
Table 6: Mean accuracy (standard deviation) in percentage on MNIST, Color-MNIST (C-MNIST), and CIFAR-20 over the last ten epochs under support-distribution shift in case (iii) with DA baselines (5 trials). Best and comparable methods (paired  $t$ -test at significance level  $5\%$ ) are highlighted in bold. LN/CS is short for label noise/class-prior shift. This result corresponds to Figure 6.
Data	Shift	Val-only	Pretrain-val	CCSA	DANN	GIW
MNIST	LN 0.2	78.94 (0.99)	86.34 (0.77)	92.95 (0.20)	87.33 (3.22)	94.60 (0.22)
LN 0.4	78.94 (0.99)	85.43 (0.97)	89.83 (0.52)	65.56 (8.31)	93.74 (0.53)
CS 10	78.94 (0.99)	86.43 (0.71)	89.18 (0.33)	73.23 (0.66)	93.43 (0.40)
CS 100	78.94 (0.99)	84.54 (1.61)	72.67 (1.75)	49.28 (0.00)	91.52 (0.59)
C-MNIST	LN 0.2	31.19 (0.65)	60.21 (4.47)	84.32 (4.57)	60.65 (5.96)	90.99 (0.61)
LN 0.4	31.19 (0.65)	57.13 (5.48)	80.32 (0.93)	70.27 (9.09)	90.09 (0.88)
CS 10	31.22 (0.69)	87.63 (1.97)	67.39 (3.07)	49.65 (7.98)	94.43 (1.87)
CS 100	31.22 (0.69)	79.70 (3.29)	64.15 (6.80)	36.16 (5.80)	88.34 (0.87)
CIFAR-20	LN 0.2	29.18 (0.30)	42.48 (0.55)	52.49 (0.58)	48.31 (0.44)	54.19 (0.69)
LN 0.4	29.18 (0.30)	39.46 (0.47)	40.68 (0.70)	42.29 (0.70)	45.48 (0.36)
CS 10	28.45 (0.27)	37.31 (0.28)	46.17 (0.20)	41.58 (0.34)	49.17 (0.40)
CS 100	28.45 (0.27)	36.55 (0.29)	34.79 (0.35)	33.90 (0.14)	40.27 (2.17)
Table 7: Mean accuracy (standard deviation) in percentage on MNIST over the last ten epochs under support-distribution shift in case (iv) with IW-like baselines (5 trials). Best and comparable methods (paired  $t$ -test at significance level  $5\%$ ) are highlighted in bold. LN/CS means label noise/class-prior shift. This result corresponds to the top row in Figure 11.
Shift	Val-only	Pretrain-val	Reweight	MW-Net	R-DIW	DIW	GIW
LN 0.2	79.86 (0.26)	85.08 (0.71)	86.69 (0.80)	78.16 (1.30)	78.36 (1.08)	75.96 (1.39)	91.44 (0.85)
LN 0.4	79.86 (0.26)	83.26 (1.08)	81.01 (1.73)	75.60 (0.49)	75.76 (0.56)	72.12 (0.86)	88.84 (0.97)
CS 10	79.86 (0.26)	83.44 (1.24)	76.41 (1.28)	74.67 (0.84)	74.94 (0.59)	74.75 (0.69)	90.20 (0.66)
CS 100	79.86 (0.26)	82.09 (0.93)	74.84 (1.10)	71.42 (0.99)	71.11 (0.54)	73.66 (1.32)	87.14 (0.51)
Table 8: Mean accuracy (standard deviation) in percentage on MNIST over the last ten epochs under support-distribution shift in case (iv) with DA baselines (5 trials). Best and comparable methods (paired  $t$ -test at significance level  $5\%$ ) are highlighted in bold. LN/CS means label noise/class-prior shift. This result corresponds to the bottom row in Figure 11.
Shift	Val-only	Pretrain-val	CCSA	DANN	GIW
LN 0.2	79.86 (0.26)	85.08 (0.71)	88.06 (0.48)	85.32 (1.11)	91.44 (0.85)
LN 0.4	79.86 (0.26)	83.26 (1.08)	84.00 (0.70)	58.02 (6.63)	88.84 (0.97)
CS 10	79.86 (0.26)	83.44 (1.24)	83.60 (0.53)	64.30 (0.42)	90.20 (0.66)
CS 100	79.86 (0.26)	82.09 (0.93)	66.31 (0.92)	50.06 (0.00)	87.14 (0.51)