SlowGuess/NeurIPS_2023_OCR huggingface dataset:
ViT-S/4, 6 layers, full width
average	age	clothes	face structure	facial hair	gender	hair color	hair style	mouth
Scalarization
Uniform	91.23 ± 3.6e-04	86.70 ± 2.7e-03	92.79 ± 4.5e-04	85.16 ± 4.8e-04	95.45 ± 4.8e-04	98.10 ± 2.1e-04	92.99 ± 3.6e-04	91.68 ± 4.7e-05	86.95 ± 4.8e-04
PBT	91.21 ± 2.7e-04	86.79 ± 1.1e-03	92.77 ± 5.8e-04	85.18 ± 3.6e-04	95.42 ± 8.9e-04	98.02 ± 4.5e-04	92.92 ± 6.4e-04	91.66 ± 1.0e-03	86.95 ± 6.7e-04
PB2	91.28 ± 2.5e-04	86.96 ± 1.5e-03	92.87 ± 2.7e-04	85.17 ± 2.4e-04	95.47 ± 6.5e-05	98.10 ± 3.9e-04	92.90 ± 5.8e-04	91.81 ± 2.6e-04	86.94 ± 1.1e-03
MTO - Loss-based
Uncertainty [26]	91.22 ± 2.1e-04	86.60 ± 1.3e-03	92.85 ± 6.4e-04	85.24 ± 1.7e-04	95.43 ± 8.0e-05	98.07 ± 3.5e-04	92.91 ± 7.7e-04	91.72 ± 2.1e-04	86.93 ± 4.4e-05
IMTL-L [38]	91.21 ± 2.0e-04	86.65 ± 7.1e-04	92.83 ± 5.2e-04	85.17 ± 1.8e-04	95.42 ± 5.5e-04	98.01 ± 6.0e-04	92.99 ± 8.9e-04	91.70 ± 6.4e-04	86.93 ± 1.1e-04
MTO - Gradient-based
CAGrad [37]	91.25 ± 2.2e-04	86.79 ± 1.7e-03	92.88 ± 7.1e-05	85.12 ± 1.4e-04	95.46 ± 4.6e-04	98.17 ± 1.1e-04	92.91 ± 2.1e-04	91.74 ± 3.6e-05	86.88 ± 1.5e-04
GradDrop [8]	91.29 ± 2.3e-04	87.01 ± 5.0e-04	92.80 ± 1.0e-03	85.22 ± 8.0e-04	95.50 ± 2.7e-04	98.15 ± 6.0e-04	93.00 ± 9.1e-04	91.65 ± 1.5e-04	86.97 ± 4.0e-04
PCGrad [64]	91.21 ± 3.9e-04	86.55 ± 2.3e-03	92.81 ± 9.3e-04	85.15 ± 8.4e-04	95.44 ± 9.3e-04	98.16 ± 5.0e-04	92.92 ± 2.8e-04	91.75 ± 1.3e-03	86.87 ± 4.8e-04
Table D: (Table best seen zoomed in PDF) Results of MTL when training on all 8 tasks (subset of attributes) of CelebA for a depth of 9 layers. For PBT and PB2 we use slightly different parameters than DomainNet to account for the fact that CelebA contains more tasks, and hence has a larger search space: All PBT runs use a population size of  $N = 12$  models, such that every  $E_{\text{ready}} = 3$  epochs,  $Q = 40\%$  of the population triggers an exploit/explore step. For PB2 runs we use a population size of  $N = 8$  and otherwise the same  $Q$  and  $E_{\text{ready}}$  hyperparameters.
ViT-S/4, 9 layers, 0.25 width
average	age	clothes	face structure	facial hair	gender	hair color	hair style	mouth
Scalarization
Uniform	91.00 ± 2.5e-04	87.24 ± 1.6e-03	92.40 ± 7.6e-05	84.85 ± 4.5e-04	95.34 ± 5.7e-04	97.77 ± 5.0e-04	92.68 ± 3.5e-04	91.31 ± 5.8e-04	86.40 ± 8.0e-05
PBT	90.97 ± 1.7e-04	86.96 ± 1.1e-03	92.34 ± 2.5e-04	84.91 ± 8.4e-04	95.30 ± 8.1e-05	97.76 ± 2.9e-05	92.55 ± 1.1e-04	91.35 ± 6.3e-05	86.59 ± 2.1e-04
PB2	91.04 ± 2.7e-04	87.22 ± 1.9e-03	92.40 ± 2.5e-04	84.96 ± 4.2e-04	95.35 ± 3.5e-04	97.70 ± 4.0e-04	92.67 ± 1.4e-04	91.36 ± 3.1e-04	86.67 ± 4.8e-04
MTO - Loss-based
Uncertainty [26]	91.01 ± 2.3e-04	87.18 ± 1.4e-03	92.40 ± 4.5e-04	84.87 ± 3.5e-04	95.34 ± 6.6e-04	97.81 ± 5.7e-04	92.70 ± 3.9e-04	91.30 ± 4.5e-04	86.44 ± 1.1e-04
IMTL-L [38]	91.00 ± 2.4e-04	87.18 ± 1.5e-03	92.40 ± 4.6e-04	84.87 ± 4.0e-04	95.34 ± 6.6e-04	97.81 ± 5.0e-04	92.70 ± 3.3e-04	91.30 ± 4.7e-04	86.44 ± 6.2e-05
MTO - Gradient-based
CAGrad [37]	91.07 ± 1.3e-04	87.19 ± 4.3e-04	92.55 ± 3.5e-04	85.03 ± 4.8e-04	95.41 ± 2.7e-05	97.79 ± 6.0e-04	92.82 ± 1.8e-04	91.52 ± 3.5e-04	86.23 ± 2.3e-04
GradDrop [8]	90.83 ± 1.8e-04	86.91 ± 4.6e-04	92.23 ± 1.4e-04	84.66 ± 1.0e-03	95.26 ± 6.1e-04	97.66 ± 3.2e-04	92.58 ± 3.9e-04	91.14 ± 2.8e-04	86.18 ± 4.6e-04
PCGrad [64]	90.95 ± 2.4e-04	87.13 ± 1.2e-03	92.29 ± 4.2e-04	84.77 ± 4.7e-04	95.32 ± 3.6e-04	97.72 ± 3.5e-04	92.70 ± 8.0e-05	91.21 ± 1.6e-04	86.44 ± 1.3e-03
ViT-S/4, 9 layers, 0.5 width
average	age	clothes	face structure	facial hair	gender	hair color	hair style	mouth
Scalarization
Uniform	91.32 ± 3.2e-04	87.26 ± 2.3e-03	92.81 ± 2.4e-04	85.19 ± 9.4e-05	95.51 ± 3.4e-04	98.15 ± 7.1e-04	93.03 ± 8.9e-06	91.66 ± 4.0e-04	86.93 ± 7.3e-04
PBT	91.36 ± 2.3e-04	87.45 ± 1.5e-03	92.86 ± 6.5e-05	85.31 ± 5.2e-04	95.50 ± 2.8e-04	97.98 ± 2.2e-04	93.03 ± 2.4e-04	91.70 ± 5.4e-04	87.06 ± 5.4e-04
PB2	91.36 ± 1.4e-04	87.45 ± 7.8e-04	92.82 ± 2.7e-04	85.17 ± 3.1e-04	95.55 ± 4.4e-04	98.14 ± 3.4e-04	93.06 ± 1.2e-04	91.76 ± 4.3e-04	86.89 ± 1.7e-04
MTO - Loss-based
Uncertainty [26]	91.33 ± 1.4e-04	87.32 ± 4.3e-04	92.84 ± 2.0e-05	85.19 ± 2.1e-04	95.52 ± 4.9e-04	98.15 ± 7.8e-04	92.96 ± 2.0e-04	91.69 ± 6.5e-05	86.93 ± 2.5e-04
IMTL-L [38]	91.29 ± 1.5e-04	87.25 ± 5.0e-04	92.79 ± 2.0e-04	85.17 ± 7.1e-05	95.50 ± 4.9e-04	98.10 ± 8.5e-04	92.97 ± 1.4e-04	91.66 ± 4.8e-04	86.92 ± 9.7e-05
MTO - Gradient-based
CAGrad [37]	91.37 ± 1.8e-04	87.42 ± 2.1e-04	92.90 ± 4.1e-05	85.24 ± 1.3e-03	95.53 ± 2.3e-04	98.20 ± 1.1e-04	92.98 ± 2.6e-04	91.78 ± 2.7e-04	86.88 ± 3.4e-04
GradDrop [8]	91.27 ± 1.8e-04	87.50 ± 7.1e-05	92.71 ± 3.5e-05	85.10 ± 1.1e-04	95.46 ± 6.2e-05	98.04 ± 1.3e-03	92.94 ± 4.3e-04	91.57 ± 3.2e-04	86.85 ± 1.9e-04
PCGrad [64]	91.29 ± 1.6e-04	87.10 ± 3.2e-04	92.83 ± 4.3e-04	85.18 ± 6.1e-04	95.47 ± 1.1e-04	98.11 ± 6.4e-04	93.01 ± 8.9e-05	91.67 ± 6.3e-04	86.96 ± 2.6e-04
ViT-S/4, 9 layers, full width
average	age	clothes	face structure	facial hair	gender	hair color	hair style	mouth
Scalarization
Uniform	91.17 ± 1.7e-04	87.33 ± 5.7e-04	92.50 ± 8.1e-04	85.10 ± 4.2e-04	95.45 ± 1.9e-04	97.93 ± 1.1e-04	92.85 ± 2.9e-04	91.39 ± 1.2e-04	86.80 ± 6.9e-04
PBT	91.15 ± 2.7e-04	87.43 ± 3.1e-04	92.51 ± 3.6e-04	85.18 ± 1.1e-03	95.46 ± 3.1e-04	97.78 ± 1.5e-03	92.51 ± 1.4e-04	91.36 ± 8.0e-04	87.00 ± 5.8e-04
PB2	91.25 ± 2.3e-04	86.83 ± 1.2e-03	92.85 ± 4.8e-04	85.12 ± 7.3e-04	95.49 ± 5.6e-04	98.19 ± 6.0e-04	92.90 ± 5.6e-04	91.78 ± 4.5e-04	86.81 ± 3.7e-04
MTO - Loss-based
Uncertainty [26]	91.17 ± 1.8e-04	87.36 ± 6.4e-04	92.50 ± 8.0e-04	85.11 ± 3.0e-04	95.45 ± 1.6e-04	97.93 ± 1.4e-04	92.84 ± 3.8e-04	91.39 ± 5.3e-05	86.81 ± 8.3e-04
IMTL-L [38]	91.18 ± 1.6e-04	87.37 ± 4.3e-04	92.50 ± 8.0e-04	85.11 ± 2.8e-04	95.45 ± 1.7e-04	97.94 ± 2.1e-04	92.84 ± 3.7e-04	91.39 ± 6.5e-05	86.81 ± 7.8e-04
MTO - Gradient-based
CAGrad [37]	91.21 ± 1.4e-04	87.34 ± 4.3e-04	92.64 ± 3.7e-04	85.16 ± 4.0e-04	95.43 ± 4.3e-04	97.91 ± 1.1e-04	92.89 ± 4.3e-04	91.58 ± 3.6e-04	86.70 ± 5.5e-04
GradDrop [8]	91.20 ± 1.4e-04	86.55 ± 2.1e-04	92.81 ± 2.6e-04	85.19 ± 4.3e-04	95.40 ± 8.1e-04	98.09 ± 4.6e-04	92.91 ± 2.6e-04	91.71 ± 3.4e-04	86.90 ± 8.9e-06
PCGrad [64]	91.13 ± 3.5e-04	87.35 ± 2.4e-03	92.50 ± 1.1e-03	85.04 ± 5.3e-04	95.40 ± 3.5e-04	97.87 ± 0.0e+00	92.82 ± 2.8e-04	91.37 ± 1.2e-04	86.71 ± 5.0e-04