SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathcal {L} _ {i} = K L (\hat {\mathbf {y}} _ {i} \| \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}}) = \hat {\mathbf {y}} _ {i} \log \frac {\hat {\mathbf {y}} _ {i}}{\hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}}} \\ = \left(\mathbf {y} _ {i} + \boldsymbol {\sigma} _ {i}\right) \log \frac {\mathbf {y} _ {i} + \boldsymbol {\sigma} _ {i}}{\hat {\mathbf {p}} _ {i} ^ {\theta}} \tag {14} \\ = \left(\mathbf {y} _ {i} + \boldsymbol {\sigma} _ {i}\right) \log \left(\mathbf {y} _ {i} + \boldsymbol {\sigma} _ {i}\right) - \mathbf {y} _ {i} \log \hat {\mathbf {p}} _ {i} ^ {\theta} - \boldsymbol {\sigma} _ {i} \log \hat {\mathbf {p}} _ {i} ^ {\theta} \\ = - H \left(\mathbf {y} _ {i} + \boldsymbol {\sigma} _ {i}\right) + \mathcal {L} _ {\mathrm {c e}} \left(\mathbf {y} _ {i}, \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}}\right) - \boldsymbol {\sigma} _ {i} \log \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}} \\ \end{array}
$$
where  $\mathcal{L}_{\mathrm{ce}}(\mathbf{y}_i, \hat{\mathbf{p}}_i^\theta)$  is the cross entropy loss between  $\mathbf{y}_i$  and  $\hat{\mathbf{p}}_i^\theta$ . Because the gradient information is inaccessible from the deployed model, zeroth-order optimization (ZOO) is utilized to estimate gradients for the training of the data adaptor in SODA. To do this, the objective function  $f(\theta)$  in Eq. (1) is replaced with the training objective function  $\mathcal{L}_i$  in test-time data adaptation. Denote  $\mathcal{L}_i^\theta$  as the KL divergence loss computed by data adaptor with parameters  $\theta$ , the directional derivative approximation of ZOO is
$$
\begin{array}{l} \widehat {\nabla} _ {\boldsymbol {\theta}} \check {\mathcal {L}} _ {i} = \frac {1}{\mu q} \sum_ {j = 1} ^ {q} \left[ \left(K L (\hat {\mathbf {y}} _ {i} \| \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta} + \mu \mathbf {u} _ {j}}) - K L (\hat {\mathbf {y}} _ {i} \| \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}})\right) \mathbf {u} _ {j} \right] \\ = \frac {1}{\mu q} \sum_ {j = 1} ^ {q} \left[ \left(\left(\mathcal {L} _ {\mathrm {c e}} (\mathbf {y}, \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta} + \mu \mathbf {u} _ {j}}) - \boldsymbol {\sigma} _ {i} \log \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta} + \mu \mathbf {u} _ {j}}\right) - \left(\mathcal {L} _ {\mathrm {c e}} (\mathbf {y}, \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}}) - \boldsymbol {\sigma} _ {i} \log \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}}\right)\right) \mathbf {u} _ {j} \right] \\ = \frac {1}{\mu q} \sum_ {j = 1} ^ {q} \left[ \left(\mathcal {L} _ {\mathrm {c e}} (\mathbf {y}, \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta} + \mu \mathbf {u} _ {j}}) - \mathcal {L} _ {\mathrm {c e}} (\mathbf {y}, \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}})\right) \mathbf {u} _ {j} \right] + \frac {1}{\mu q} \sum_ {j = 1} ^ {q} \left[ \left(\boldsymbol {\sigma} _ {i} \log \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}} - \boldsymbol {\sigma} _ {i} \log \hat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta} + \mu \mathbf {u} _ {j}}\right) \mathbf {u} _ {j} \right] \\ = \widehat {\nabla} _ {\boldsymbol {\theta}} \mathcal {L} _ {\mathrm {c e}} + \frac {\sigma_ {i}}{\mu q} \sum_ {j = 1} ^ {q} \log \frac {\widehat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta}}}{\widehat {\mathbf {p}} _ {i} ^ {\boldsymbol {\theta} + \mu \mathbf {u} _ {j}}} \mathbf {u} _ {j}, \tag {15} \\ \end{array}
$$
where  $\widehat{\nabla}_{\pmb{\theta}}\mathcal{L}_{\mathrm{ce}} = \frac{1}{\mu q}\sum_{j = 1}^{q}\left[\left(\mathcal{L}_{\mathrm{ce}}(\mathbf{y}_i,\hat{\mathbf{p}}_i^{\pmb{\theta} + \mu \mathbf{u}_j}) - \mathcal{L}_{\mathrm{ce}}(\mathbf{y}_i,\hat{\mathbf{p}}_i^{\pmb{\theta}})\right)\mathbf{u}_j\right]$  is the ideal directional derivative approximation.
B.1 Implementation details of DINE and BETA
The implementations of DINE and BETA on CIFAR-10-C and CIFAR-100-C are kept the same, following their original work [22] and [46]. For DINE, the momentum hyperparameter  $\gamma = 0.6$  and the Mixup balancing hyperparameter  $\beta = 1$ . For BETA,  $\tau = 0.8$  for domain division,  $\alpha = 1.0$  for Mixup,  $\lambda_{mse} = 0$ , sharpening factor  $T = 0.5$ , and adversarial regularizer  $\gamma = 0.1$ . The training strategy of DINE and BETA are both SGD with learning rate  $= 0.001$  for target network backbones and 0.01 for MLP classifiers. momentum  $= 0.9$  and weight decay  $= 1\mathrm{e - }3$  are also adopted.
B.2 Software and hardware
In our paper, all models are implemented using PyTorch 1.13.1. The ImageNet pre-trained weights used in DINE and BETA are downloaded from TorchVision 0.14.1. The experiments are conducted using NVIDIA A100-PCIE-40GB GPU with CUDA 11.7.
B.3 Network structure of data adaptor
Figure 5 shows the network structure of the data adaptor used in our experiments. The basic structure of the data adaptor consists of two convolutional layers and an instance normalization layer in between. Multiple ResNet blocks and downsampling/upsampling layers can be inserted into the convolutional layers to form a deeper network as in [33]. For all methods except DA-Direct, the adapted data is generated by treating the network output as perturbation and adding it to the original data.
Figure 5: Network structure of data adaptor. The green block is the convolutional layer, the blue block is the instance normalization layer, the white block is the dropout layer, and the orange block is the downsampling/upsampling layer. The dashed blocks can be added or removed to form different network structures. The red dashed line means the network output is added to the original data to generate the adapted data.
C.1 Discussion about SODA and SODA-R
Compared with SODA, SODA-R uses computed first-order gradients and adopts several techniques to improve the performance, i.e., deeper data adaptor with 2 ResNet blocks, Adam optimizer, perturbation regularization, and dropout. The effect of network complexity has already been discussed in Section 4.3. In this subsection, we first introduce the perturbation regularization used in SODA-R, then evaluate the effect of perturbation regularization, different optimizers, and dropout on SODA and SODA-R.
C.1.1 Perturbation regularization in SODA-R
In SODA and SODA-R, the adapted data is computed by perturbing the original data with a generated perturbation. To further restrict the impact of generated perturbations on data  $\mathbf{X}_r = \{\mathbf{x}_{r_1},\mathbf{x}_{r_2},\dots,\mathbf{x}_{r_{l_r}}\}$  and  $\mathbf{X}_u = \{\mathbf{x}_{u_1},\mathbf{x}_{u_2},\dots,\mathbf{x}_{u_{l_u}}\}$ , perturbation regularization with  $l_{1}$  norm is used: let  $\mathbf{x}_{r_i}^{\theta}$  and  $\mathbf{x}_{u_i}^{\theta}$  be the corresponding adapted data of  $\mathbf{x}_{r_i}$  and  $\mathbf{x}_{r_u}$ ,
$$
\mathcal {R} (\mathbf {X}) = \mathbb {E} _ {\mathbf {x} _ {i} \in \mathbf {X} _ {r}} \left\| \mathbf {x} _ {r _ {i}} ^ {\boldsymbol {\theta}} - \mathbf {x} _ {r _ {i}} \right\| _ {1} + \mathbb {E} _ {\mathbf {x} _ {i} \in \mathbf {X} _ {u}} \left\| \mathbf {x} _ {u _ {i}} ^ {\boldsymbol {\theta}} - \mathbf {x} _ {u _ {i}} \right\| _ {1}. \tag {16}
$$
First-order gradients of the perturbation regularization are directly computed and back-propagated through the data adaptor. Hence, the training objective of SODA-R becomes:
$$
\mathcal {L} _ {\text {a l l}} (\mathbf {X}, \hat {\mathbf {Y}} _ {r}) = \mathcal {L} _ {\mathrm {i m}} \left(\mathbf {X} _ {u}\right) + \alpha \mathcal {L} _ {\mathrm {c e}} \left(\mathbf {X} _ {r}, \hat {\mathbf {Y}} _ {r}\right) + \beta \mathcal {R} (\mathbf {X}), \tag {17}
$$
where  $\beta$  is the weight of perturbation regularization and is set to be 0.005 for CIFAR-10-C and 0.01 for CIFAR-100-C.
C.1.2 Evaluation of perturbation regularization in SODA and SODA-R
We evaluate the effect of perturbation regularization in SODA and SODA-R on CIFAR-10-C and CIFAR-100-C. Except for the perturbation regularization term in the training objective, all other settings are kept the same as in the main experiments. The results are shown in Table 6 and Table 7.
It shows that perturbation regularization can improve the performance of SODA-R using first-order optimization, especially on CIFAR-100-C. However, it largely hinders the performance of SODA using zeroth-order optimization. The computed first-order gradients of perturbation regularization are more accurate than the estimated zeroth-order gradients of the main training objective. Thus, the data adaptor tends to optimize the perturbation regularization term first, resulting in perturbations with too small norms. The perturbations with too small norms do not have enough ability to modify the test data, which might be the reason for the worse performance achieved by SODA with perturbation regularization. One possible solution to this problem could be treating perturbation regularization as an optimization constraint and using constrained ZOO methods to train the data adaptor.
Table 6: Comparison of SODA and SODA-R with and without perturbation regularization on CIFAR-10-C.  $\beta = 0.005$  in experiments w/ regularization.
Methods	SODA	SODA-R
w/ regularization	73.40	88.39
w/o regularization	82.55	87.96
Table 7: Comparison of SODA and SODA-R with and without perturbation regularization on CIFAR-100-C.  $\beta = 0.01$  in experiments w/ regularization.
Methods	SODA	SODA-R
w/ regularization	42.27	60.31
w/o regularization	52.41	58.11
C.1.3 Evaluation of optimizers in SODA and SODA-R
We evaluate the effect of optimizers used in SODA and SODA-R on CIFAR-10-C and CIFAR-100-C. Except for the optimizer used to train the data adaptor, all other settings are kept the same as in the main experiments. The results are shown in Table 8 and Table 9. On CIFAR-10-C, SODA trained by SGD and Adam achieve almost the same accuracy, while SODA-R trained by Adam achieves  $3.3\%$  higher accuracy than SGD. On CIFAR-100-C, SODA-R trained by Adam still outperforms SODA-R trained by SGD, but SODA trained by Adam achieves even worse accuracy than SODA trained by SGD. It shows that Adam optimizer can improve the training of data adaptors using first-order gradients but fails when using the estimated zeroth-order gradients.
Table 8: Comparing of SODA and SODA-R using SGD and Adam optimizer on CIFAR-10-C.
Methods	SODA	SODA-R
SGD	82.55	84.95
Adam	82.75	88.39
Table 9: Comparison of SODA and SODA-R using SGD and Adam optimizer on CIFAR-100-C.
Methods	SODA	SODA-R
SGD	52.41	58.32
Adam	49.75	60.31
C.1.4 Evaluation of dropout in SODA and SODA-R
We also evaluate the effect of dropout on SODA and SODA-R. As depicted in Figure 5, a dropout layer can be inserted into the ResNet block. We conduct experiments using data adaptors with and without dropout layers for SODA and SODA-R. To keep the same network structure as SODA-R, the data adaptor used in SODA also has 2 ResNet blocks. The dropout ratio is set to be 0.5. All other settings are kept the same as in the main experiments. Table 10 and Table 11 show the results on CIFAR-10-C and CIFAR-100-C, respectively. For SODA-R, adding dropout layers can improve the accuracy by  $0.7\%$  on CIFAR-10-C and  $2\%$  on CIFAR-100-C. However, for SODA, adding dropout layers extremely hinders the performance, especially on CIFAR-100-C. This contrast indicates that dropout harms data adaptor optimized using estimated zeroth-order gradients while positively affecting data adaptor optimized using computed first-order gradients. The reason might be that the extra randomness introduced by dropout increases the difficulty of gradient estimation in zeroth-order optimization. Note that the accuracy of SODA using a data adaptor with 2 ResNet blocks on CIFAR-100-C is worse than that using a data adaptor with 0 ResNet blocks, which is consistent with the results on CIFAR-10-C as shown in Table 3.
Table 10: Comparing of SODA and SODA-R with and without dropout layers on CIFAR-10-C.
Methods	SODA	SODA-R
w/ dropout	32.19	88.39
w/o dropout	80.56	87.54
Table 11: Comparison of SODA and SODA-R with and without dropout layers on CIFAR-100-C.
Methods	SODA	SODA-R
w/ dropout	5.47	60.31
w/o dropout	43.96	58.27
To sum up, compared with SODA using zeroth-order optimization, SODA-R uses first-order optimization and adopts deeper network structure, perturbation regularization, Adam optimizer, and dropout to improve the performance. However, these techniques cannot improve or even hinder the performance of SODA. This comparison shows that the common boosting strategies used in first-order optimization cannot be directly applied to zeroth-order optimization, leading to the limited performance of methods using zeroth-order optimization.
C.2 Convergence of SODA
In Figure 3, the convergence speeds of SODA on CIFAR-10-C and CIFAR-100-C are slower than SODA-FO and SODA-R and do not achieve complete convergence after training with 150 epochs. We further train SODA on Gaussian noise in CIFAR-10-C and CIFAR-100-C for 300 epochs to show the complete convergence of SODA as depicted in Figure 6. With more training epochs, SODA can achieve higher accuracies on both datasets. However, training with more epochs means more adaptation processing time or more computing resources with parallel computation. For time and resource efficiency, we only report the accuracies achieved at 150 epochs in our main experiments, which already improves the deployed model by a large margin. If computing time and resources are not restricted, SODA can further improve the deployed model for higher accuracy.
(b)
Figure 6: Accuracy convergence on Gaussian noise in (a) CIFAR-10-C and (b) CIFAR-100-C for 300 epochs.
C.3 Hyperparameter analysis of reliable pseudo-label selection
We evaluate the hyperparameters in reliable pseudo-label selection, namely the confidence threshold  $\tau$ , the noise ratio  $\rho$ , and the balancing parameter  $\alpha$ .  $\tau$  and  $\rho$  controls the number of selected reliable pseudo-labels. With lower  $\tau$  and lower  $\rho$ , the number of selected reliable pseudo-labels increases. We evaluate  $\tau$  in  $\{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9\}$ , and  $\rho$  in  $\{0.1, 0.3, 0.5, 0.7, 0.9\}$ . Note that when  $\tau = 0$ , the number of selected pseudo-labels is not equal to  $(1 - \rho)n$ , where  $n$  is the total number of test data points, because the pseudo-labels are not evenly distributed across classes as depicted in Figure 8a. The inaccurate model prediction tends to bias towards a few classes, leading to more pseudo-labels belonging to those classes.  $\alpha$  controls the balance between the supervised training
objective  $\mathcal{L}_{\mathrm{ce}}$  and the unsupervised training objective  $\mathcal{L}_{\mathrm{im}}$ . We evaluate  $\alpha$  in  $\{0.01, 0.001, 0.0001\}$ . The results of SODA with different sets of hyperparameters on CIFAR-10-C Gaussian noise level 5 corruption are shown in Figure 7. The performance of SODA is stable across different hyperparameter settings. A common trend among different  $\alpha$  is that accuracy tends to increase when  $\tau$  and  $\rho$  decrease, i.e., the top-right corner of each figure. This trend shows that the performance of the data adaptor can be improved using more selected pseudo-labels, which further indicates the reliability of the selected pseudo-labels. There is a mild tendency of performance drop in the overall performance of SODA when unsupervised learning of test points with unreliable pseudo-labels is overwhelmed by supervised learning of reliable pseudo-labels with larger  $\alpha$ , indicating that learning on test points with unreliable pseudo-labels also has a contribution to the performance of SODA. To balance the supervised and unsupervised learning terms, we finally choose  $\alpha = 0.0001$  in our main experiments. Although better performance can be achieved by carefully fine-tuning  $\tau$  and  $\rho$  with a validation set to show the general performance of SODA and select the most reliable pseudo-labels for different corruptions, we set  $\tau = 0.9$  and  $\rho = 0.9$  for CIFAR-10/100-C tasks and  $\tau = 0.1$  and  $\rho = 0.9$  for ImageNet-C task in our main experiments without elaborated hyperparameter fine-tuning.
Figure 7: Evaluation of reliable pseudo-label selecting hyperparameters on CIFAR-10-C Gaussian noise corruption level 5. Numbers are prediction accuracies  $(\%)$  after adaptation.
C.4 Evaluation of queue size in SODA-O
We evaluate the effect of queue size in SODA-O. A larger queue size means more past reliable pseudo-labels and their corresponding test data points are stored and used in the adaptation process of the current mini-batch. Fixing batch size  $= 128$ , we conduct experiments on queue size  $\{500, 1000, 2000, 3000\}$ , and the results are shown in Table 12. The performance of SODA-O is stable across different queue sizes, especially when the queue size is smaller. When queue size increases, the ratio of reliable pseudo-labels used to train the data adaptor for the current mini-batch also increases. It makes the training of the data adaptor more biased towards the supervised training with the reliable pseudo-labels. Thus, the mild performance drop observed along with the larger queue size might indicate that the reliable pseudo-labels still have disturbance, and the unsupervised training of data points with unreliable pseudo-labels is useful to alleviate the negative effect caused by the remaining disturbance.
C.5 Results on challenging Office-Home tasks
We conduct experiments on three challenging Office-Home tasks to show the effectiveness of SODA and SODA-R on different kinds of distribution shifts. For SODA, the network structure of the
Table 12: Comparison of SODA-O with different queue sizes. Averaged accuracies  $(\%)$  over 19 corruptions are reported.
Queue Size	500	1000	2000	3000
CIFAR-10-C	78.78	78.79	78.22	77.73
CIFAR-100-C	47.18	47.21	46.67	45.63
data adaptor is the same as that in CIFAR-10/100-C tasks. For SODA-R, a deeper data adaptor is adopted with 2 downsampling/upsampling layers and 2 ResNet blocks. Both SODA and SODA-R are trained using SGD with learning rate  $= 1\mathrm{e} - 3$ , momentum  $= 0.9$ , and weight decay  $= 1\mathrm{e} - 5$  for 60 epochs. A warmup process is used to first train the data adaptor with pseudo-label supervision for 10 epochs. Following DINE [22] and BETA [46], MixMatch [2] and EMA pseudo-label updating are also adopted. The weights of cross-entropy loss and mutual information loss are set to be 0.1 and 0.01, respectively. The confidence threshold  $\tau$  is set to be 0.5. The noise ratio  $\rho$  is set to be 0. The query number is set to 5 for SODA. The results are listed in Table 13. The results show that SODA and SODA-R can improve the deployed model in challenging Office-Home tasks with initial prediction accuracies lower than  $50\%$
Table 13: Accuracies (%) on three challenging Office-Home domain adaption tasks. FO Grad. means the requirement of first-order gradient from the target model or deployed model. Model Mod. means the requirement of modifying the parameters of deployed models.
Methods	FO Grad.	Model Mod.	Art->Clipart	Product->Clipart	Realworld->Clipart
Deployed	-	-	44.67	40.53	47.12
SODA	X	X	46.53	41.47	48.71
SODA-R	✓	X	50.45	45.20	53.22
D Qualitative Evaluation of SODA
D.1 T-SNE Visualization of SODA features
To qualitatively evaluate the performance of SODA, we use T-SNE to visualize the feature embeddings of SODA, i.e., the input features of the last classification layer in the deployed model before and after adaptation in Figure 8. According to the visualization results, the feature embeddings are much more separated apart between classes after adaptation, showing the effectiveness of SODA.
(b) After adaptation
Figure 8: T-SNE visualization of SODA feature embeddings on CIFAR-10-C pixelate corruption level 5.
D.2 Examples of Adapted Data
Figure 9 shows examples of test and adapted data using SODA for 19 corruptions in CIFAR-10-C. Comparing original data without corruption, test data before adaptation, and adapted data after adaptation, it is obvious that the adapted data looks closer to the original data than the corresponding test data. This observation is consistent with the improved prediction accuracy using SODA and further illustrates that the distribution shifts between the test data and the training data are alleviated after applying SODA to test data. It also indicates that SODA adapts the test data to the deployed model by modifying them to "look like" the training data. Then, the distribution shifts between the test data and the training data are mitigated, leading to improved prediction of the deployed model.
E Detailed Results
There are 19 corruptions in CIFAR-10-C, CIFAR-100-C and ImageNet-C: Gaussian noise (GN), shot noise (ShN), impulse noise (IN), speckle noise (SpN), defocus blur (DB), glass blur (Glb), motion blur (MB), zoom blur (ZB), Gaussian blur (GaB), snow (SW), frost (FR), fog (FG), brightness (BR), contrast (CT), elastic transform (ET), pixelate (PX), jpeg compression (JC), spatter (SP) and saturate (SA). We report the accuracies of each method w.r.t. each corruption on CIFAR-10-C, CIFAR-100-C, and ImageNet-C in Table 14, Table 15, and Table 16. Except for SODA-R and MA-SO using first-order gradients from the deployed model, SODA outperforms all baselines on almost all corruptions. On CIFAR-10-C, SODA-R even outperforms MA-SO on all corruptions. On CIFAR-100-C, although the average accuracy of SODA-R is lower than that of MA-SO, SODA-R still outperforms MA-SO on 7 corruptions. On ImageNet-C, SODA improves the deployed model on most corruptions. SODA-R even outperforms or performs on par with MA-SO on 5 corruptions.
Table 14: Accuracies of 19 corruptions on CIFAR-10-C. For brevity, DA-PGD, DA-ZOO-Input, DA-PL, and DA-Direct are abbreviated as D-PG, D-Z-I, D-PL, and D-Di, respectively.
C	Deployed	DINE	BETA	D-PG	D-Z-I	D-PL	D-Di	SODA	SODA-R	MA-SO
GN	51.28	56.86	62.85	28.34	49.93	52.18	48.80	73.18	85.82	84.09
ShN	56.02	58.44	64.75	29.52	53.99	56.63	54.59	74.52	86.23	85.40
IN	42.98	47.25	53.36	22.23	42.57	44.38	41.12	57.48	83.20	75.06
SpN	57.15	59.41	65.61	27.36	54.33	57.07	56.04	73.77	85.52	84.55
DB	88.16	88.14	86.94	16.39	84.64	88.67	85.94	90.97	91.79	90.98
GIB	49.21	53.31	58.38	17.48	46.65	49.75	44.68	66.24	77.08	76.51
MB	76.62	77.25	79.27	17.76	72.05	77.35	75.16	86.43	91.13	87.37
ZB	89.14	89.37	88.86	17.76	85.93	89.73	87.52	90.96	92.28	92.46
GaB	84.59	84.66	84.65	15.87	80.24	85.74	84.38	91.25	93.13	90.93
SW	78.06	78.03	77.42	36.16	75.39	78.62	75.53	83.83	88.92	85.94
FR	71.75	72.39	72.96	23.15	68.13	72.24	70.41	82.92	87.63	87.45
FG	70.58	71.84	73.60	11.56	63.08	71.56	71.71	82.72	86.29	82.82
BR	92.98	92.85	91.28	41.57	91.46	92.75	90.18	92.84	93.15	92.14
CT	86.72	86.74	84.64	15.06	67.58	87.95	87.15	92.67	93.89	92.07
ET	76.64	77.35	78.02	18.32	72.30	77.04	71.99	79.50	82.70	81.99
PX	52.12	58.46	64.50	27.95	50.91	52.35	49.55	87.12	90.26	89.29
JC	80.55	80.93	80.70	29.20	78.51	81.03	78.60	86.06	88.08	87.04
SP	77.66	77.11	77.80	30.20	76.13	77.86	75.69	83.06	88.87	85.73
SA	93.13	92.90	92.98	42.09	91.56	92.68	90.02	92.94	93.62	92.43