Title: SODA: Robust Training of Test-Time Data Adaptors
Venue: NeurIPS 2023
Accepted: yes
Keywords: test-time data adaptation

zeroth-order optimization

out-of-distribution generalization
Abstract: Adapting models deployed to test distributions can mitigate the performance degradation caused by distribution shifts. However, privacy concerns may render model parameters inaccessible. One promising approach involves utilizing zeroth-order optimization (ZOO) to train a data adaptor to adapt the test data to fit the deployed models. Nevertheless, the data adaptor trained with ZOO typically brings restricted improvements due to the potential corruption of data features caused by the data adaptor. To address this issue, we revisit ZOO in the context of test-time data adaptation. We find that the issue directly stems from the unreliable estimation of the gradients used to optimize the data adaptor, which is inherently due to the unreliable nature of the pseudo-labels assigned to the test data. Based on this observation, we propose pseudo-label-robust data adaptation (SODA) to improve the performance of data adaptation. Specifically, SODA leverages high-confidence predicted labels as reliable labels to optimize the data adaptor with ZOO for label prediction. For data with low-confidence predictions, SODA encourages the adaptor to preserve data information to mitigate data corruption. Empirical results indicate that SODA can significantly enhance the performance of deployed models in the presence of distribution shifts without requiring access to model parameters.

SODA: Robust Training of Test-Time Data Adaptors

Abstract

Adapting models deployed to test distributions can mitigate the performance degradation caused by distribution shifts. However, privacy concerns may render model parameters inaccessible. One promising approach involves utilizing zeroth-order optimization (ZOO) to train a data adaptor to adapt the test data to fit the deployed models. Nevertheless, the data adaptor trained with ZOO typically brings restricted improvements due to the potential corruption of data features caused by the data adaptor. To address this issue, we revisit ZOO in the context of test-time data adaptation. We find that the issue directly stems from the unreliable estimation of the gradients used to optimize the data adaptor, which is inherently due to the unreliable nature of the pseudo-labels assigned to the test data. Based on this observation, we propose pseudo-label-robust data adaptation (SODA) to improve the performance of data adaptation. Specifically, SODA leverages high-confidence predicted labels as reliable labels to optimize the data adaptor with ZOO for label prediction. For data with low-confidence predictions, SODA encourages the adaptor to preserve data information to mitigate data corruption. Empirical results indicate that SODA can significantly enhance the performance of deployed models in the presence of distribution shifts without requiring access to model parameters.

1 Introduction

Deep neural networks have emerged as a dominant tool in solving artificial intelligence problems due to their exceptional performance across various tasks [19, 20, 42], thus being deployed to various environments. However, in practice, these models typically suffer performance degradation due to distribution discrepancies between training and test data [1, 13, 47, 35]. To mitigate this issue, TestTime Adaptation (TTA) is proposed as a promising solution, where unlabeled test data are leveraged to modify the parameters of deployed models to alleviate performance degradation [7, 21, 39].

In practice, the parameters of deployed models may be unmodifiable and inaccessible in many applications due to intellectual property protection, misuse prevention, or privacy concerns in healthcare and finance [22, 46]. The difficulty in modifying model parameters has hindered previous efforts aimed at adapting deployed models to the test distribution [15, 23, 29]. It is shown that training a data adaptor to modify test data offers an alternative solution to mitigate the performance degradation caused by distribution discrepancy [18, 37]. However, the difficulty in accessing model parameters makes gradient computation a challenging task, hindering these data adaptation methods.

One straightforward approach involves utilizing zeroth-order optimization (ZOO) [25] to train the data adaptor to adapt test data to fit deployed models. In particular, ZOO can be employed to estimate gradients for the optimization of data adaptors without modifying and accessing the parameters of deployed models. Therefore, test-time data adaptation with ZOO makes it possible to improve the performance of deployed models in many practical scenarios. However, introducing data adaptors

brightness

snow

test data

pixlelate

corrupted  data

adapted data

80

Test

Corrupted

Adapted

70

averaged accuracy (%)

60

50

20

10

0

CIFAR-10-C

CIFAR-100-C

trained with ZOO brings limited improvement. To endow data adaptors with reliability in improving model performance, we revisit ZOO in the context of test-time data adaptation. Training of the data adaptor depends heavily on the predicted labels. Thus, unreliable predicted labels will lead to unreliable gradient estimations in ZOO, which makes data features corrupted rather than adapted to deployed models. This is consistent with our observations as depicted in Figure 1.

Built upon the observation, we propose pseudo-label-robust data adaptation (SODA) to enhance deployed models with inaccessible parameters. SODA robustifies the training of data adaptors by splitting the test dataset into two subsets. One subset contains data with high-confidence predictions and is regarded as a reliable dataset. The other contains the remaining data points, considered to be an unreliable subset. For data in the reliable dataset, SODA trains the data adaptor using ZOO in a supervised manner. In the meantime, SODA encourages the data adaptor to preserve input information in an unsupervised manner for data corruption mitigation over unreliable data. To verify the efficacy of SODA, we evaluate it on three widely used benchmark datasets under various settings. Our experimental results demonstrate that SODA can effectively mitigate the performance degradation of deployed models in the presence of distribution shifts.

To sum up, our main contributions are presented as follows:

To enhance deployed models with unmodifiable and inaccessible parameters, we propose to shift from model adaptation to data adaptation, where zeroth-order optimization is employed to estimate gradients for the training of the data adaptor. Unfortunately, test-time data adaptation causes corruption of data features, leading to limited performance improvements.

To exploit the potential of test-time data adaptation, we find that the limited improvement issue stems from the unreliable nature of the predicted labels used in ZOO. Thus, we propose SODA to robustify the training of test-time data adaptation, where the data adaptor is trained to preserve input information for data with unreliable predictions.

We verify the effectiveness of SODA under various settings. Our experimental results demonstrate that SODA can directly enhance the deployed model under distribution shifts without accessing and modifying the model parameters.

2 Related Work

Test-time adaptation. Test-time adaptation is a machine learning technique that addresses the distribution shift problem using only unlabeled test data before making predictions. To mitigate the distribution discrepancy, most previous works adapt the pre-trained model to test data by modifying

the full or part of the model parameters. Some advanced works [7, 21] adapt the feature extractor to obtain more efficient feature representations of test data, while others [16, 17, 45] modify or replace the last linear classification layer to improve prediction over the extracted features. Batch normalization calibration [23, 34, 43, 49] is also exploited to adjust the statistics and affine parameters in batch normalization layers. Unlike previous works, our work focuses on situations where model parameters are unmodifiable and adapts test data to the pre-trained model as an alternative solution.

White-box input adaptation. Recently, several works have put effort into input-level optimization in test-time adaptation by changing the input data or features. Auxiliary auto-encoders [12, 18], amplitude features and Fourier-style calibration [50], label-preserving features along with a generative model [31], and generative diffusion model [10] are utilized to achieve input-level adaptation. These works require either modification of the model training process or gradients from the deployed model at test time. In contrast, our work focuses on test-time data adaptation without accessing the training process and gradients of the deployed model, making it more practical and broadly applicable.

Domain adaptation of black-box predictors. Domain Adaptation of Black-Box Predictors (DABP) is a subcategory of unsupervised domain adaptation [9, 30] that solves a more restricted and practical application setting. Under this setting, the pre-trained model is treated as a black box, and only the model's output is available during adaptation. Few works [22, 32, 46] have proposed solutions to this challenge using knowledge distillation to transfer knowledge from the black-box model to target models. In this work, we address the black-box setting in the context of test-time adaptation and propose to perform data adaptation without knowledge transfer. Besides, while previous DABP works require training on the entire unlabeled test dataset, our proposed method can also deal with online black-box settings where the test data arrive sequentially.

Zeroth-order optimization. Zeroth-order optimization (ZOO) [25] is a family of optimization methods that do not require gradient information to search for the optimal solution. Instead, it explores the searching space using various techniques [6, 24] to estimate the optimization direction. Although it can be less efficient than first-order optimization methods, ZOO methods can be helpful in scenarios where gradient information is unavailable or expensive to compute. For instance, recent studies have applied ZOO to perform adversarial attacks to black-box neural networks [48], hyperparameter optimization in federated learning without gradient descent [51], and transfer learning on black-box models [40]. In this work, we propose to leverage ZOO for test-time data adaptation on deployed models with inaccessible gradient information.

3 Methodology

We mainly focus on the C -way image classification task with a distribution shift between the training and test data, following previous works [26, 39, 43]. Given a deployed model M with inaccessible parameters, our purpose is to improve its prediction accuracy on unlabeled test data X = { x 1 , ..., x n } . Since the parameters and inner structures of M are unknown, only the output prediction probabilities are available from M during the entire adaptation process. Namely, the proposed method pseudolabel-robust data adaptation (SODA) aims to adapt X to M without requiring access to the parameters of M . The overall framework of SODA is shown in Figure 2.

3.1 Preliminary

Before elaborating on our proposed framework, we introduce zeroth-order optimization (ZOO). ZOO is a gradient-free alternative of first-order optimization (FOO), e.g., SGD, SCD, and Adam. Most ZOO methods follow the structure of FOO and consist of three fundamental steps [25]: gradient estimation, descent direction computation, and point updating. They utilize function-value-based gradient estimations to approximate the full or stochastic gradients computed in FOO.

One commonly used ZOO gradient estimation strategy is multi-point estimation [6, 24]. Given a continuously differentiable objective function f ( θ ) on a d -dimensional variable θ ∈ R d , multi-point estimation computes directional derivative approximation as follows:

Data adaptor

Deployed

models

Reliable pseudo-label selection

(c) Zeroth-Order Optimization

Eq. (1)

Eq. (2)

Deployed models

(a) Before Adaptation

ZOO

(b) During Adaptation

Deployed

models

where u 1 , ..., u q are q random direction vectors typically drawn from the standard multivariate normal distribution N ( 0 , I ) , and µ is the smoothing parameter. On a mini-batch of data points x 1 , ..., x l , the estimated gradient ˆ δ is the average of the multi-point estimations to all data points x 1 , ..., x l :

Various strategies are adopted to compute the descent direction. For zeroth-order stochastic gradient descent (ZO-SGD) utilized in our work, the descent direction is set as the current gradient estimation ˆ δ . The point updating rule is the same as in traditional SGD: with learning rate η ,

3.2 Zeroth-Order Optimization in Test-Time Data Adaptation

Let G with parameters θ be the data adaptor for test-time data adaptation. For each test data point x i ( i = 1 , ..., n ) , G transforms it to form the adapted data x θ i for inference as follows:

Ideally, the data adaptor G should be trained by minimizing the KL divergence between the predicted probabilities of the adapted data and the true labels of test data. Typically, the training process requires back-propagating the gradients from the deployed model to the data adaptor. A challenge arises as gradient computation is infeasible for the deployed model M with inaccessible parameters. In this regard, ZOO provides an effective approach to estimating gradients, as discussed in the previous section. Considering the parameters θ of the data adaptor G as the variables to be optimized, utilizing ZOO in test-time data adaptation is to replace the objective function f ( θ ) with the training objective function used to train the data adaptor G . Assuming that the true one-hot label y i of a test data point x i is given, and replacing the function f ( θ ) in Eq. (1) with KL divergence loss L ( · , · ) := KL( ·∥· ) , the directional derivative approximation w.r.t. L ( · , · ) and ( x i , y i ) is

In test-time adaptation, the true labels of test data are obviously unknown. A common strategy is to use the predicted pseudo-label ˆ y i as the substitute of the true label y i .

However, the pseudo-labels are unreliable due to the inaccurate model prediction under distribution shifts, causing the corrupted data features depicted in Figure 1. Let σ i denote the disturbance of pseudo-label ˆ y i , i.e., ˆ y i = σ i + y i , and ˆ p θ i = M ◦ G ( x i ; θ ) denote the predicted probability of the adapted data point x θ i = G ( x i ; θ ) , the KL divergence loss at test point x i becomes:

where L ce ( · , · ) is the cross-entropy loss. Then, replacing y i with ˆ y i in Eq. (5), the directional derivative approximation becomes

where ̂ ∇ θ L ce = 1 µq ∑ q j =1 [( L ce ( y i , ˆ p θ + µ u j i ) -L ce ( y i , ˆ p θ i ) ) u j ] is the ideal directional derivative approximation. The derivations of Eq. (6) and Eq. (7) are deferred to Appendix A. In Eq. (7), the last term is the disturbing term directly introduced by σ i , causing the difference between ̂ ∇ θ L ce and the unreliable directional derivative approximation ̂ ∇ θ ˇ L i . ̂ ∇ θ ˇ L i further leads to unreliable estimated gradients in Eq. (2), which hinders the optimization of θ and the training of G .

3.3 Pseudo-Label-Robust Training

A direct strategy to alleviate the impact of the disturbing term in Eq. (7) is to select pseudo-labels with small σ i . The selected pseudo-labels form reliable pseudo-label set ˆ Y r to train the data adaptor in a supervised manner. In particular, two basic criteria for reliable pseudo-label selection are adopted in SODA: 1) the prediction confidence should be higher than a threshold τ , indicating the selected pseudo-labels have small disturbances; 2) the number of selected reliable pseudo-labels for each class should be less than (1 -ρ ) n/C to maintain the balance among classes, where ρ is the noise ratio, and C is the number of classes. The test data points corresponding to the selected reliable pseudo-labels are considered as reliable data set X r , which is trained over with cross-entropy loss L ce and ˆ Y r .

To mitigate the data corruption over the remaining test data points X u with unreliable pseudo-labels, SODA trains over them in an unsupervised manner. Following previous works [21, 22], mutual information maximization [4, 36, 41] is a widely-used unsupervised loss that can encourage both global diversity and local certainty of model predictions by maximizing the mutual information between the input data sample and the predicted probabilities. Thus, it is adopted to preserve input information in X u as shown in Eq. (8), where x θ i = G ( x i ; θ ) and ˆ p i = M ◦ G ( x i ; θ ) .

3.4 Theoretical Analysis

To theoretically show the effectiveness of the proposed pseudo-label-robust training strategy, we analyze the expected gradient estimation error [5, 24] in the training of test-time data adaptor with zeroth-order optimization. For simplicity, we consider the special case where the estimated gradient equals directional derivative approximation with mini-batch size equal to 1. The expected gradient estimation error R X between the estimated gradient ̂ ∇ θ ˇ L i and the true gradient ∇ θ L i w.r.t. the whole test dataset X is:

Denoting h ( x i ) = -σ i log ˆ p θ i in Eq. (6), the gradient of the KL divergence loss is ∇ θ L i = ∇ θ L ce + ∇ θ h. Accordingly, the estimated gradient of the KL divergence loss is ̂ ∇ θ ˇ L i = ̂ ∇ θ L ce + ̂ ∇ θ h. Then, before applying pseudo-label-robust data adaptation, the upper bound of expected gradient estimation error is:

Algorithm 1 SODA framework

Input: test data X , data adaptor G with parameters θ , deployed model M , adaptation epochs N ,

In pseudo-label-robust training strategy, the test dataset X is separated into reliable set X r learned by cross-entropy loss with pseudo-labels, and unreliable set X u learned by mutual information loss, the expected gradient estimation error w.r.t. X becomes:

According to the previous study [3], minimizing the cross entropy loss L ce ( y i , ˆ p θ i ) is equivalent to maximizing the mutual information L im . Then, the upper bound of the expected gradient estimation error w.r.t. X is:

Comparing Eq. (10) and Eq. (12), the upper bound of the expected gradient estimation error is tightened after applying the pseudo-label-robust training strategy, justifying that the proposed pseudolabel-robust training strategy can alleviate the disturbance caused by the unreliable pseudo-labels in the gradient estimation in zeroth-order optimization of the test-time data adaptor.

3.5 SODA Framework

Up to this point, we have discussed zeroth-order optimization in test-time data adaptation and our proposed pseudo-label-robust training strategy. The overall training objective is shown in Eq. (13),

where α is the balancing hyperparameter between mutual information maximization loss L im and cross entropy loss L ce . The training algorithm of SODA is summarized in Algorithm 1.

4 Experiments

4.1 Experimental Setup

Datasets. We first evaluate our proposed framework SODA on two widely used out-of-distribution benchmarks, namely CIFAR-10-C and CIFAR-100-C [14], each containing 10,000 CIFAR-10/100 test images corrupted by 19 kinds of corruptions with 5 severity levels. We further evaluate SODA on a large scale dataset ImageNet-C [14] with 50,000 ImageNet validation images corrupted by the same corruptions as CIFAR10/100-C. We conduct our experiments on the highest level of each corruption and report the averaged accuracies over 19 corruptions.

Baselines. Deployed is the deployed model without adaptation. We compare our proposed SODA framework with two DABP baselines utilizing knowledge distillation. DINE [22] distills the knowledge from the deployed model to a target model by minimizing the KL divergence between smoothed pseudo-labels and target model predictions. BETA [46] divides the target domain into easy- and hard-to-adapt subdomains, then mutually distills twin networks with weak-strong augmentation on two subdomains. We further implement four vanilla baselines of test-time data adaptation using ZOO. DA-Direct directly generates adapted data instead of perturbations using the same network structure and initial pseudo-labels. DA-PGD adopts PGD [27] to directly generate perturbations using estimated gradients computed by ZOO and the training objective of SODA. DA-ZOO-Input uses the same data adaptor and training objective as SODA, except that the variables optimized in ZOO are the input data of the deployed model, i.e., the output adapted data of the data adaptor. The gradients of data adaptor parameters are computed based on the estimated gradients w.r.t. the adapted data. DA-PL trains the same data adaptor as SODA using initial pseudo-labels. Assuming gradient information is accessible, we also implement SODA-R using first-order gradients computed from the deployed model. To compare SODA with model adaptation, we implement MA-SO that modifies model parameters (except the last linear layer) using the same training objective as SODA.

Deployed model settings. For all experiments regarding CIFAR-10/100-C tasks, we adopt the CIFAR-10/100 pretrained ResNet-50 [11] model used in [26] and [38] as the deployed model. For experiments regarding the ImageNet-C task, we adopt the ImageNet pre-trained ResNet-50 model provided by TorchVision [28] as the deployed model. Except for SODA-R and MA-SO, the deployed model is frozen with only output probabilities accessible. For SODA-R, the deployed model is also frozen, but gradients can be computed and back-propagated through the model. For MA-SO, the deployed model is set in training mode, and all parameters can be modified.

Implementation details. For SODA, the data adaptor uses a small network with two convolutional layers and an instance normalization layer in between to generate perturbations added to the original test data. For SODA-R in the CIFAR-10/100-C tasks, two ResNet blocks as in [33] are inserted between the convolutional layers to form a larger data adaptor. For SODA-R in the ImageNet-C task, one ResNet block and a couple of downsampling/upsampling layers are inserted instead. Detailed data adaptor structure is described in Appendix B.3. For all methods except DINE, BETA, DA-PGD, and SODA-R, the data adaptor/model is optimized using SGD with learning rate = 1e-3, momentum = 0.9, and weight decay = 1e-5. For DINE and BETA, we follow the same settings as in [22] and [46], and the target models are both ResNet-50 initialized with ImageNet-pretrained weights downloaded from TorchVision [28]. For DA-PGD, the step size is also set to be 1e-3. SODA-R is optimized using Adam with learning rate = 1e-3 and weight decay = 1e-5. Batch size = 256 is fixed for all methods. The number of training epochs = 150 for all baselines except DINE and BETA. For DINE and BETA, we train them for 120 epochs and fine-tune them for 30 epochs. For all methods using ZOO, the query number q = 5 for CIFAR-10-C and ImageNet-C and q = 10 for CIFAR-100-C, smoothing parameter

µ = 1e-3. All experiments are repeated with three random seeds. The code implementation can be found at https://github.com/tmlr-group/SODA .

4.2 Experimental Results

Effectiveness of SODA. Table 1 shows the accuracies on CIFAR-10-C, CIFAR-100-C and ImageNetCaveraged over 19 corruptions. In the mostly restricted setting where first-order gradient computation and model modification are both not allowed, SODA improves the deployed model prediction accuracy by a large margin, 10% on CIFAR-10-C, 11% on CIFAR-100-C and 11% on ImageNet-C. SODA also outperforms the DABP baselines on CIFAR10-C and CIFAR100-C tasks. Note that DINE and BETA are excluded in experiments on ImageNet-C because they are required to initialize their target models by ImageNet pre-trained weights, which violates the TTA setting. Especially on CIFAR-100-C and ImageNet-C task with much lower initial prediction accuracy, all baselines except MA-SO fail to improve, while SODA and SODA-R make significant improvement.

SODA v.s. model adaptation. Further relaxing the restriction on the deployed model, when parameters of the deployed model are frozen, but gradient computation is feasible, SODA-R achieves comparable accuracy with completely unrestricted MA-SO on CIFAR-100-C and even better accuracy on CIFAR-10-C. It shows that data adaptation can be as effective as model adaptation for deployed models with inaccessible parameters. On ImageNet-C, SODA-R performs worse than MA-SO, suggesting that improvement is still needed on large-scale datasets.

Effect of pseudo-label-robust training strategy in SODA. The failures of DA-Direct show that directly generating data using unreliable pseudo-labels fails to adapt data to the deployed model due to the corrupted data features. By adding perturbations to the original test data, DA-PL improves from DA-Direct to a small degree but still fails to enhance the deployed model. With the same data adaptor as DA-PL, SODA can improve the model prediction to a large degree, indicating the effectiveness of the proposed pseudo-label-robust training strategy.

Effect of data adaptor parameter ZOO in SODA. Both using a perturbation generation strategy, DA-PGD fails by directly generating perturbations using estimated gradients without learning a data adaptor, and DA-ZOO-Input achieves worse results by directly optimizing the model input using ZOO. With the same training objective, the significant improvements made by SODA indicate the effectiveness of the data adaptor parameter ZOO adopted in SODA.

4.3 Discussion

Effect of query number in ZOO. To analyze the relation between query number q and adaptation accuracy in SODA, we conduct experiments with q ∈ { 2 , 5 , 10 , 20 , 50 } . The results in Table 2 show that SODA is not sensitive to query number used in ZOO. Especially on CIFAR-10-C, only a slight performance drop is observed when q = 2. On CIFAR-100-C, a larger query number contributes to accuracy improvement more observably, but when q > 10 , the contribution becomes less efficient, as the computation cost is also increased. Balancing the trade-off between accuracy and computation costs, we finally choose q = 5 for CIFAR-10-C and 10 for CIFAR-100-C.

Zeroth-order optimization v.s. first-order optimization. To better illustrate the effect of ZOO, we implement a comparing baseline as SODA-FO that shares the same setting with SODA but uses the first-order gradients back-propagated from the deployed model. Note that SODA-FO and SODA-R differ in data adaptor network structure, optimizer, and other strategies as discussed in Appendix C.1. As shown in Table 2, although SODA does not achieve the same accuracies as SODA-FO due to the unavoidable gradient estimation error, it still indicates competitive performance when gradient computation is infeasible. In Figure 3, SODA using ZOO has the slowest convergence speeds

0

25

50

75

100

125

150

50

55

60

65

70

75

80

85

SODA

SODA-FO

SODA-R

(a)

0

25

50

75

100

125

150

20

25

30

35

40

45

50

55

SODA

SODA-FO

SODA-R

(b)

50

100

500

1000

5000

10000

test data points

50

55

60

65

70

75

80

85

averaged accuracy (%)

Deployed

SODA

and lowest accuracies on both datasets. SODA-FO has similar convergence speeds as SODA-R but converges to lower accuracies, indicating that the strategies used in SODA-R with first-order optimization, i.e., deeper network and Adam optimizer, can boost the training of the data adaptor.

Effect of data adaptor network complexity. We also explore the effect of data adaptor network complexity on SODA and SODA-R. To increase network complexity, we change the number of ResNet blocks in the data adaptor. Results of data adaptor with {0, 1, 2, 3} ResNet blocks are reported in Table 3. For SODA, accuracy decreases as network complexity increases, indicating that complex networks hinder data adaptation using ZOO. However, the accuracy of SODA-R increases along with network complexity. This contrast illustrates that a more complex data adaptor can achieve higher accuracy but is restricted and even encumbered by zeroth-order gradient estimation.

SODAwith fewer test data points. Wefurther explore the effectiveness of SODA with fewer test data points. We randomly choose { 50 , 100 , 500 , 1000 , 5000 , 10000 } test data points evenly distributed across 10 classes in CIFAR-10-C for each corruption as smaller test datasets. The averaged accuracies are reported in Figure 4. The performance of SODA is better when training data adaptor over more test data points. Nevertheless, training over 5,000 data points achieves comparable accuracy with training over 10,000 data points, showing that SODA does not require an extremely large number of data points to achieve good performance. Besides, SODA can still improve with less than 500 test data points, providing promising insights for test-time data adaption with smaller test datasets.

Hyperparameter sensitivity. We conduct sensitivity analysis regarding the noise ratio ρ , the confidence threshold τ and the balancing parameter α . The detailed results illustrated in Appendix C.3 show that SODA is robust to different combinations of hyperparameters, but adapting an adapted threshold instead of a fixed threshold might further improve the performance of SODA.

4.4 SODA for Online Test-Time Adaptation

More practically, test data is not entirely available but arrives sequentially, i.e., online test-time adaptation. Hence, we further implement SODA-O as a variant of SODA under online settings. Given mini-batches of test data arrived sequentially { X 1 , ..., X T } , SODA-O adapts one mini-batch at a time before processing the next mini-batch with knowledge accumulated from the previous mini-batches.

The main difference between SODA-O and SODA lies in the reliable pseudo-label selection. Without access to the test data before adaptation, SODA-O maintains an ordered queue Q with maximum size S to store the selected reliable pseudo-labels Y r and their corresponding data points X r . Specifically,

*SODA is trained over the entire test dataset for 150 epochs for a mini-batch X t arrives at time t , reliable pseudo-labels Y r t with prediction confidences higher than τ are selected and pushed into Q along with their corresponding test data points X r t . To maintain the class balance in Q , the pseudo-labels with the smallest confidence for class k will be popped out once the number of pseudo-labels for k in Q is larger than S/C . Then, the remaining data points X u t in X t are considered as X ′ u t , all pseudo-labels and data points stored in Q are considered as Y ′ r t and X ′ r t . X ′ u t and X ′ r t form a small test dataset to train the data adaptor as in SODA, i.e. Step 2 in Algorithm 1. After adaptation, the inference of X t is given by the current data adaptor and the deployed model. Note that the optimization in SODA-O is not repeated after reaching the entire test dataset but only repeats for the current test data batch and the cached queue. During the adaptation of the current test data batch, the previous data batches are no longer available except for those saved in the queue. The data adaptor, hyperparameter setting, and training strategy of SODA-O are the same as SODA. The queue size S = 1,000 for both CIFAR-10-C and CIFAR-100-C.

Effect of epochs per mini-batch. We fix batch size = 256 and conduct experiments with different numbers of epochs per mini-batch, i.e., {5, 10, 30, 50, 100, 150} epochs/batch. As shown in Table 5, SODA-O is effective under online setting. As epochs/batch increase, the accuracy of SODA-O also increases and approaches the accuracy of SODA. But more training epochs means more processing time for each mini-batch, leading to a time-accuracy trade-off. With 10 epochs/batch, SODA-O can still improve the deployed model by 5% and 4% on CIFAR-10-C and CIFAR-100-C.

Effect of batch size. Fixing epochs/batch = 10, we also conduct experiments with different batch sizes, {32, 64, 128, 256}. The averaged accuracies are shown in Table 4. The results show that the performance of SODA-O is stable across different batch sizes. The best performance is achieved with batch size = 128. Only a slight performance drop is observed when batch size is reduced to 32, indicating that SODA-O can handle relatively small batch size.

5 Limitations

While providing valuable insights into robust test-time data adaptation for deployed models with inaccessible parameters, it is also essential to consider the limitations that may have affected the efficiency and effectiveness of our proposed framework. The main limitation of SODA is the time consumption brought by i) multiple queries required for gradient estimation in ZOO and ii) multiple passes required for training the random initialized data adaptor. In the future, one solution to alleviate this issue could be leveraging ZOO methods with higher query efficiency and less estimation error. Moreover, it is unclear whether SODA designed for OOD generalization can benefit OOD detection [8, 44]. Thus, we leave the exploration for OOD detection as our future work.

6 Conclusion

This paper focuses on two major challenges in adapting deployed machine learning models with inaccessible parameters at test-time: unmodifiable model parameters and infeasible gradient computation. Without modifying the model parameters, a data adaptor is adopted to adapt test data to the deployed model. Zeroth-order optimization is further leveraged to train the data adaptor with estimated gradients. Revisiting ZOO in test-time data adaptation, we discover that the unreliable gradient estimation in ZOO is due to the unreliable pseudo-labels assigned to test data. The proposed pseudo-label-robust data adaptation (SODA) addresses this issue with reliable pseudo-label selection and input information maximization. Our experiments on three widely used out-of-distribution benchmarks demonstrate the effectiveness of SODA in both offline and online settings.

Acknowledgments and Disclosure of Funding

This work was partially supported by the National Natural Science Foundation of China (No. 62372459, No. 62376282).

References

Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893 , 2019.

David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. In Advances in Neural Information Processing Systems , 2019.

Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Piantanida, and Ismail Ben Ayed. A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses. In European Conference on Computer Vision , 2020.

John Bridle, Anthony Heading, and David MacKay. Unsupervised classifiers, mutual information and'phantom targets. In Advances in Neural Information Processing Systems , 1991.

Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In ACM Workshop on Artificial Intelligence and Security , 2017.

John C Duchi, Michael I Jordan, Martin J Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE Transactions on Information Theory , 61(5):2788-2806, 2015.

Cian Eastwood, Ian Mason, Christopher KI Williams, and Bernhard Schölkopf. Source-free adaptation to measurement shift via bottom-up feature restoration. In International Conference on Learning Representations , 2022.

Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, and Feng Liu. Is out-of-distribution detection learnable? In Advances in Neural Information Processing Systems , 2022.

Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning , 2015.

Jin Gao, Jialing Zhang, Xihui Liu, Trevor Darrell, Evan Shelhamer, and Dequan Wang. Back to the source: Diffusion-driven test-time adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2016.

Yufan He, Aaron Carass, Lianrui Zuo, Blake E Dewey, and Jerry L Prince. Autoencoder based self-supervised test-time adaptation for medical image analysis. Medical image analysis , 72:102136, 2021.

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In International Conference on Computer Vision , 2021.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations , 2019.

Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. Model adaptation: Historical contrastive learning for unsupervised domain adaptation without source data. In Advances in Neural Information Processing Systems , 2021.

Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In Advances in Neural Information Processing Systems , 2021.

Minguk Jang and Sae-Young Chung. Test-time adaptation via self-training with nearest neighbor information. In International Conference on Learning Representations , 2023.

Neerav Karani, Ertunc Erdil, Krishna Chaitanya, and Ender Konukoglu. Test-time adaptable neural networks for robust medical image segmentation. Medical Image Analysis , 68:101907, 2021.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM , 60(6):84-90, 2017.

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature , 521(7553):436-444, 2015.

Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning , 2020.

Jian Liang, Dapeng Hu, Jiashi Feng, and Ran He. Dine: Domain adaptation from single and multiple black-box predictors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022.

Hyesu Lim, Byeonggeun Kim, Jaegul Choo, and Sungha Choi. Ttn: A domain-shift aware batch normalization in test-time adaptation. In International Conference on Learning Representations , 2023.

Sijia Liu, Jie Chen, Pin-Yu Chen, and Alfred Hero. Zeroth-order online alternating direction method of multipliers: Convergence analysis and applications. In International Conference on Artificial Intelligence and Statistics , 2018.

Sijia Liu, Pin-Yu Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III, and Pramod K Varshney. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine , 37(5):43-54, 2020.

Yuejiang Liu, Parth Kothari, Bastien Van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Ttt++: When does self-supervised test-time training fail or thrive? In Advances in Neural Information Processing Systems , 2021.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018.

TorchVision maintainers and contributors. Torchvision: Pytorch's computer vision library. https://github.com/pytorch/vision , 2016.

Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations , 2023.

Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering , 22(10):1345-1359, 2010.

Prashant Pandey, Mrigank Raman, Sumanth Varambally, and Prathosh Ap. Generalization on unseen domains via inference-time label-preserving target projections. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021.

Qucheng Peng, Zhengming Ding, Lingjuan Lyu, Lichao Sun, and Chen Chen. Toward better target representation for source-free and black-box domain adaptation. arXiv preprint arXiv:2208.10531 , 2022.

Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie. Generative adversarial perturbations. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2018.

Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In Advances in Neural Information Processing Systems , 2020.

Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624 , 2021.

Yuan Shi and Fei Sha. Information-theoretical learning of discriminative clusters for unsupervised domain adaptation. In International Conference on Machine Learning , 2012.

Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. In Advances in Neural Information Processing Systems , 2022.

Yongyi Su, Xun Xu, and Kui Jia. Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering. In Advances in Neural Information Processing Systems , 2022.

Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning , 2020.

Yun-Yun Tsai, Pin-Yu Chen, and Tsung-Yi Ho. Transfer learning without knowing: Reprogramming black-box machine learning models with scarce data and limited resources. In International Conference on Machine Learning , 2020.

Michael Tschannen, Josip Djolonga, Paul K Rubenstein, Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. In International Conference on Learning Representations , 2020.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems , 2017.

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations , 2020.

Qizhou Wang, Feng Liu, Yonggang Zhang, Jing Zhang, Chen Gong, Tongliang Liu, and Bo Han. Watermarking for out-of-distribution detection. In Advances in Neural Information Processing Systems , 2022.

Zehao Xiao, Xiantong Zhen, Ling Shao, and Cees GM Snoek. Learning to generalize across domains on single test samples. In International Conference on Learning Representations , 2021.

Jianfei Yang, Xiangyu Peng, Kai Wang, Zheng Zhu, Jiashi Feng, Lihua Xie, and Yang You. Divide to adapt: Mitigating confirmation bias for domain adaptation of black-box predictors. In International Conference on Learning Representations , 2023.

Yonggang Zhang, Mingming Gong, Tongliang Liu, Gang Niu, Xinmei Tian, Bo Han, Bernhard Schölkopf, and Kun Zhang. Adversarial robustness through the lens of causality. In International Conference on Learning Representations , 2022.

Yonggang Zhang, Ya Li, Tongliang Liu, and Xinmei Tian. Dual-path distillation: A unified framework to improve black-box attacks. In International Conference on Machine Learning , 2020.

Bowen Zhao, Chen Chen, and Shu-Tao Xia. Delta: degradation-free fully test-time adaptation. In International Conference on Learning Representations , 2023.

Xingchen Zhao, Chang Liu, Anthony Sicilia, Seong Jae Hwang, and Yun Fu. Test-time fourier style calibration for domain generalization. In International Joint Conference on Artificial Intelligence , 2022.

Yi Zhou, Parikshit Ram, Theodoros Salonidis, Nathalie Baracaldo, Horst Samulowitz, and Heiko Ludwig. Flora: Single-shot hyper-parameter optimization for federated learning. arXiv preprint arXiv:2112.08524 , 2021.

Appendix

A Derivation of Directional Derivative Approximation in SODA

In Section 3.2, given a deployed model M , the ideal objective function of training the data adaptor G with parameters θ in SODA is the KL divergence between the predicted probability ˆ p θ i = M ◦ G ( x i ; θ ) of the adapted data point x θ i = G ( x i ; θ ) and the true label y i of the original data point x i . Because y i is not available at test time, pseudo-label ˆ y i predicted by M is adopted as a substitute of y i . Due to the inaccurate model prediction under distribution shifts, there is a disturbance σ i in ˆ y i compared to y i , i.e. ˆ y i = σ i + y i . Hence, the KL divergence loss L ( · , · ) := KL( ·∥· ) at test data point x i is

where L ce ( y i , ˆ p θ i ) is the cross entropy loss between y i and ˆ p θ i . Because the gradient information is inaccessible from the deployed model, zeroth-order optimization (ZOO) is utilized to estimate gradients for the training of the data adaptor in SODA. To do this, the objective function f ( θ ) in Eq. (1) is replaced with the training objective function L i in test-time data adaptation. Denote L θ i as the KL divergence loss computed by data adaptor with parameters θ , the directional derivative approximation of ZOO is

where ̂ ∇ θ L ce = 1 µq ∑ q j =1 [( L ce ( y i , ˆ p θ + µ u j i ) -L ce ( y i , ˆ p θ i ) ) u j ] is the ideal directional derivative approximation.

B Implementation Details

B.1 Implementation details of DINE and BETA

The implementations of DINE and BETA on CIFAR-10-C and CIFAR-100-C are kept the same, following their original work [22] and [46]. For DINE, the momentum hyperparameter γ = 0.6 and the Mixup balancing hyperparameter β = 1. For BETA, τ = 0.8 for domain division, α = 1.0 for Mixup, λ mse = 0, sharpening factor T = 0.5, and adversarial regularizer γ = 0.1. The training strategy of DINE and BETA are both SGD with learning rate = 0.001 for target network backbones and 0.01 for MLP classifiers. momentum = 0.9 and weight decay = 1e-3 are also adopted.

B.2 Software and hardware

In our paper, all models are implemented using PyTorch 1.13.1. The ImageNet pre-trained weights used in DINE and BETA are downloaded from TorchVision 0.14.1. The experiments are conducted using NVIDIA A100-PCIE-40GB GPU with CUDA 11.7.

B.3 Network structure of data adaptor

Figure 5 shows the network structure of the data adaptor used in our experiments. The basic structure of the data adaptor consists of two convolutional layers and an instance normalization layer in between. Multiple ResNet blocks and downsampling/upsampling layers can be inserted into the convolutional layers to form a deeper network as in [33]. For all methods except DA-Direct, the adapted data is generated by treating the network output as perturbation and adding it to the original data.

ResNet

block n

ResNet

block 1

C Additional Analysis

C.1 Discussion about SODA and SODA-R

Compared with SODA, SODA-R uses computed first-order gradients and adopts several techniques to improve the performance, i.e., deeper data adaptor with 2 ResNet blocks, Adam optimizer, perturbation regularization, and dropout. The effect of network complexity has already been discussed in Section 4.3. In this subsection, we first introduce the perturbation regularization used in SODA-R, then evaluate the effect of perturbation regularization, different optimizers, and dropout on SODA and SODA-R.

C.1.1 Perturbation regularization in SODA-R

In SODA and SODA-R, the adapted data is computed by perturbing the original data with a generated perturbation. To further restrict the impact of generated perturbations on data X r = { x r 1 , x r 2 , ..., x r l r } and X u = { x u 1 , x u 2 , ..., x u l u } , perturbation regularization with l 1 norm is used: let x θ r i and x θ u i be the corresponding adapted data of x r i and x r u ,

First-order gradients of the perturbation regularization are directly computed and back-propagated through the data adaptor. Hence, the training objective of SODA-R becomes:

where β is the weight of perturbation regularization and is set to be 0.005 for CIFAR-10-C and 0.01 for CIFAR-100-C.

C.1.2 Evaluation of perturbation regularization in SODA and SODA-R

We evaluate the effect of perturbation regularization in SODA and SODA-R on CIFAR-10-C and CIFAR-100-C. Except for the perturbation regularization term in the training objective, all other settings are kept the same as in the main experiments. The results are shown in Table 6 and Table 7.

It shows that perturbation regularization can improve the performance of SODA-R using first-order optimization, especially on CIFAR-100-C. However, it largely hinders the performance of SODA using zeroth-order optimization. The computed first-order gradients of perturbation regularization are more accurate than the estimated zeroth-order gradients of the main training objective. Thus, the data adaptor tends to optimize the perturbation regularization term first, resulting in perturbations with too small norms. The perturbations with too small norms do not have enough ability to modify the test data, which might be the reason for the worse performance achieved by SODA with perturbation regularization. One possible solution to this problem could be treating perturbation regularization as an optimization constraint and using constrained ZOO methods to train the data adaptor.

C.1.3 Evaluation of optimizers in SODA and SODA-R

We evaluate the effect of optimizers used in SODA and SODA-R on CIFAR-10-C and CIFAR-100-C. Except for the optimizer used to train the data adaptor, all other settings are kept the same as in the main experiments. The results are shown in Table 8 and Table 9. On CIFAR-10-C, SODA trained by SGD and Adam achieve almost the same accuracy, while SODA-R trained by Adam achieves 3.3% higher accuracy than SGD. On CIFAR-100-C, SODA-R trained by Adam still outperforms SODA-R trained by SGD, but SODA trained by Adam achieves even worse accuracy than SODA trained by SGD. It shows that Adam optimizer can improve the training of data adaptors using first-order gradients but fails when using the estimated zeroth-order gradients.

C.1.4 Evaluation of dropout in SODA and SODA-R

We also evaluate the effect of dropout on SODA and SODA-R. As depicted in Figure 5, a dropout layer can be inserted into the ResNet block. We conduct experiments using data adaptors with and without dropout layers for SODA and SODA-R. To keep the same network structure as SODAR, the data adaptor used in SODA also has 2 ResNet blocks. The dropout ratio is set to be 0.5. All other settings are kept the same as in the main experiments. Table 10 and Table 11 show the results on CIFAR-10-C and CIFAR-100-C, respectively. For SODA-R, adding dropout layers can improve the accuracy by 0.7% on CIFAR-10-C and 2% on CIFAR-100-C. However, for SODA, adding dropout layers extremely hinders the performance, especially on CIFAR-100-C. This contrast indicates that dropout harms data adaptor optimized using estimated zeroth-order gradients while positively affecting data adaptor optimized using computed first-order gradients. The reason might be that the extra randomness introduced by dropout increases the difficulty of gradient estimation in zeroth-order optimization. Note that the accuracy of SODA using a data adaptor with 2 ResNet blocks on CIFAR-100-C is worse than that using a data adaptor with 0 ResNet blocks, which is consistent with the results on CIFAR-10-C as shown in Table 3.

To sum up, compared with SODA using zeroth-order optimization, SODA-R uses first-order optimization and adopts deeper network structure, perturbation regularization, Adam optimizer, and dropout to improve the performance. However, these techniques cannot improve or even hinder the performance of SODA. This comparison shows that the common boosting strategies used in first-order optimization cannot be directly applied to zeroth-order optimization, leading to the limited performance of methods using zeroth-order optimization.

C.2 Convergence of SODA

In Figure 3, the convergence speeds of SODA on CIFAR-10-C and CIFAR-100-C are slower than SODA-FO and SODA-R and do not achieve complete convergence after training with 150 epochs. We further train SODA on Gaussian noise in CIFAR-10-C and CIFAR-100-C for 300 epochs to show the complete convergence of SODA as depicted in Figure 6. With more training epochs, SODA can achieve higher accuracies on both datasets. However, training with more epochs means more adaptation processing time or more computing resources with parallel computation. For time and resource efficiency, we only report the accuracies achieved at 150 epochs in our main experiments, which already improves the deployed model by a large margin. If computing time and resources are not restricted, SODA can further improve the deployed model for higher accuracy.

0

50

100

150

200

250

300

55

60

65

70

75

(a)

0

50

100

150

200

250

300

20

25

30

35

40

(b)

C.3 Hyperparameter analysis of reliable pseudo-label selection

We evaluate the hyperparameters in reliable pseudo-label selection, namely the confidence threshold τ , the noise ratio ρ , and the balancing parameter α . τ and ρ controls the number of selected reliable pseudo-labels. With lower τ and lower ρ , the number of selected reliable pseudo-labels increases. We evaluate τ in { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 } , and ρ in { 0 . 1 , 0 . 3 , 0 . 5 , 0 . 7 , 0 . 9 } . Note that when τ = 0 , the number of selected pseudo-labels is not equal to (1 -ρ ) n , where n is the total number of test data points, because the pseudo-labels are not evenly distributed across classes as depicted in Figure 8a. The inaccurate model prediction tends to bias towards a few classes, leading to more pseudo-labels belonging to those classes. α controls the balance between the supervised training

objective L ce and the unsupervised training objective L im . We evaluate α in { 0 . 01 , 0 . 001 , 0 . 0001 } . The results of SODA with different sets of hyperparameters on CIFAR-10-C Gaussian noise level 5 corruption are shown in Figure 7. The performance of SODA is stable across different hyperparameter settings. A common trend among different α is that accuracy tends to increase when τ and ρ decrease, i.e., the top-right corner of each figure. This trend shows that the performance of the data adaptor can be improved using more selected pseudo-labels, which further indicates the reliability of the selected pseudo-labels. There is a mild tendency of performance drop in the overall performance of SODA when unsupervised learning of test points with unreliable pseudo-labels is overwhelmed by supervised learning of reliable pseudo-labels with larger α , indicating that learning on test points with unreliable pseudo-labels also has a contribution to the performance of SODA. To balance the supervised and unsupervised learning terms, we finally choose α = 0.0001 in our main experiments. Although better performance can be achieved by carefully fine-tuning τ and ρ with a validation set to show the general performance of SODA and select the most reliable pseudo-labels for different corruptions, we set τ = 0.9 and ρ = 0.9 for CIFAR-10/100-C tasks and τ = 0.1 and ρ = 0.9 for ImageNet-C task in our main experiments without elaborated hyperparameter fine-tuning.

α=0.01

α=0.001

α=0.0001

74.00

0

72.27

73.15

73.49

73.30

73.33

0

72.65

73.16

73.26

73.11

73.31

72.55

72.94

73.50

73.30

73.36

73.75

72.27

73.15

73.49

73.30

73.33

72.65

73.16

73.26

73.11

73.31

72.55

72.94

73.50

73.30

73.36

2.

72.27

73.15

73.49

73.30

73.33

2

72.65

73.16

73.26

73.11

73.31

2

72.55

72.94

73.50

73.30

73.36

73.50

0

0

0

3

72.27

72.95

73.44

73.15

73.31

72.65

73.12

73.21

73.34

73.19

3

72.55

73.34

73.30

73.32

73.44

0

0

0

73.25

72.27

72.93

73.29

73.19

73.00

72.65

73.22

73.48

73.44

73.25

4

72.55

73.30

73.75

73.45

73.24

0

73.00

5

72.27

72.90

73.07

73.10

73.09

0.5

72.65

73.25

73.35

73.25

73.29

5

72.55

73.14

73.32

73.26

73.14

72.27

72.91

73.15

73.25

72.92

0.6

72.65

73.15

73.31

73.10

73.52

72.55

73.23

73.22

73.21

73.40

72.75

72.27

72.95

73.21

72.94

73.02

0.7

72.65

72.99

73.13

73.03

73.05

72.55

73.04

73.27

73.02

73.09

72.50

0

0

8

72.27

72.71

73.18

73.01

72.87

72.65

73.08

73.09

73.01

73.00

8.

72.55

72.97

73.24

73.04

73.23

0

0

72.25

72.38

72.74

72.50

72.65

72.54

0.9

72.97

73.22

72.85

72.87

72.79

io

72.94

73.02

72.91

73.01

72.80

0

0

-72.00

0.9

0.7

0.5

0.3

0.1

0.9

0.7

0.5

0.3

0.1

0.9

0.7

0.5

0.3

0.1

d

d

d

C.4 Evaluation of queue size in SODA-O

We evaluate the effect of queue size in SODA-O. A larger queue size means more past reliable pseudo-labels and their corresponding test data points are stored and used in the adaptation process of the current mini-batch. Fixing batch size = 128, we conduct experiments on queue size { 500 , 1000 , 2000 , 3000 } , and the results are shown in Table 12. The performance of SODA-O is stable across different queue sizes, especially when the queue size is smaller. When queue size increases, the ratio of reliable pseudo-labels used to train the data adaptor for the current mini-batch also increases. It makes the training of the data adaptor more biased towards the supervised training with the reliable pseudo-labels. Thus, the mild performance drop observed along with the larger queue size might indicate that the reliable pseudo-labels still have disturbance, and the unsupervised training of data points with unreliable pseudo-labels is useful to alleviate the negative effect caused by the remaining disturbance.

C.5 Results on challenging Office-Home tasks

We conduct experiments on three challenging Office-Home tasks to show the effectiveness of SODA and SODA-R on different kinds of distribution shifts. For SODA, the network structure of the

data adaptor is the same as that in CIFAR-10/100-C tasks. For SODA-R, a deeper data adaptor is adopted with 2 downsampling/upsampling layers and 2 ResNet blocks. Both SODA and SODA-R are trained using SGD with learning rate = 1e-3, momentum = 0.9, and weight decay = 1e-5 for 60 epochs. A warmup process is used to first train the data adaptor with pseudo-label supervision for 10 epochs. Following DINE [22] and BETA [46], MixMatch [2] and EMA pseudo-label updating are also adopted. The weights of cross-entropy loss and mutual information loss are set to be 0.1 and 0.01, respectively. The confidence threshold τ is set to be 0.5. The noise ratio ρ is set to be 0. The query number is set to 5 for SODA. The results are listed in Table 13. The results show that SODA and SODA-R can improve the deployed model in challenging Office-Home tasks with initial prediction accuracies lower than 50%

D Qualitative Evaluation of SODA

D.1 T-SNE Visualization of SODA features

To qualitatively evaluate the performance of SODA, we use T-SNE to visualize the feature embeddings of SODA, i.e., the input features of the last classification layer in the deployed model before and after adaptation in Figure 8. According to the visualization results, the feature embeddings are much more separated apart between classes after adaptation, showing the effectiveness of SODA.

(a) Before adaptation

(b) After adaptation

60

40

20

0

-20

-40

-60

-60

-40

-20

0

20

40

60

80

60

40

20

0

-20

-40

-60

-80

-80

-60

-40

-20

0

20

40

60

80

D.2 Examples of Adapted Data

Figure 9 shows examples of test and adapted data using SODA for 19 corruptions in CIFAR-10-C. Comparing original data without corruption, test data before adaptation, and adapted data after adaptation, it is obvious that the adapted data looks closer to the original data than the corresponding test data. This observation is consistent with the improved prediction accuracy using SODA and further illustrates that the distribution shifts between the test data and the training data are alleviated after applying SODA to test data. It also indicates that SODA adapts the test data to the deployed model by modifying them to "look like" the training data. Then, the distribution shifts between the test data and the training data are mitigated, leading to improved prediction of the deployed model.

E Detailed Results

There are 19 corruptions in CIFAR-10-C, CIFAR-100-C and ImageNet-C: Gaussian noise (GN), shot noise (ShN), impulse noise (IN), speckle noise (SpN), defocus blur (DB), glass blur (GlB), motion blur (MB), zoom blur (ZB), Gaussian blur (GaB), snow (SW), frost (FR), fog (FG), brightness (BR), contrast (CT), elastic transform (ET), pixelate (PX), jpeg compression (JC), spatter (SP) and saturate (SA). We report the accuracies of each method w.r.t. each corruption on CIFAR-10-C, CIFAR-100-C, and ImageNet-C in Table 14, Table 15 ,and Table 16. Except for SODA-R and MA-SO using firstorder gradients from the deployed model, SODA outperforms all baselines on almost all corruptions. On CIFAR-10-C, SODA-R even outperforms MA-SO on all corruptions. On CIFAR-100-C, although the average accuracy of SODA-R is lower than that of MA-SO, SODA-R still outperforms MA-SO on 7 corruptions. On ImageNet-C, SODA improves the deployed model on most corruptions. SODA-R even outperforms or performs on par with MA-SO on 5 corruptions.

brightness

pixlelate

shot noise

impulse noise

speckle noise

defocus blur

glass blur

motion blur

frost

snow

fog

contrast

elastic transform

jpeg compression

spatter

saturate

test data

adapted data

test data

adapted data

test data

adapted data

test data

adapted data

original data

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SODA, a test-time data adaptor with the black-box source model leveraging Zeroth-Order Optimization for the adaptor, which involves a random perturbation on the adaptor model's parameters. It also considers the scenario, namely SODA-R when the gradient information is available and online setting SODA-O. The proposed method outperforms existing benchmarks, including BETA and DINE. The experiment is comprehensive with CIFAR10-C/CIFAR100-C. However, having more benchmark dataset experiments is better for achieving better confidence with the proposed framework for practical application.\n\nSTRENGTHS:\nTest-time adaptor problem is well-motivated, and leveraging the adaptor could potentially save the training cost in practice.\nUnder the limited available information scenario, the authors claim that they successfully tackle the problem better than other benchmarks, such as BETA and DINE. I value the simplicity of the approach only with two components; 1) mutual information maximization and 2) cross-entropy loss with pseudo labels by achieving a better performance. As a black-box scenario, overcoming the lack of gradient is another main ingredient of this paper, and it outperforms other benchmarks in CIFAR10-C/CIFAR100-C. Application of Zeroth-Order Optimization is another key ingredient, but the simplicity is better appealing to me.\n\nWEAKNESSES:\n1. I would like to understand the relationship between the model perturbation and the data augmentation in the proposed framework. See the more detailed question in the Questions section.\n2. A good combination of the choice $\\sigma, \\alpha, \\tau$ seems to be critical. I see this discussion at C3 in the Appendix, but I suggest that the authors make this more explicit in the main paper. If those parameters are dataset/or task-dependent, it's not yet applicable in practice.\n3. Why $\\sigma=0.5$ generally achieve a better performance? And a better performance with $\\alpha=0.0001$ implies that the mutual information term is the most critical. A further ablation study of each loss term would be necessary.\n\nMinor Comments:\n- L61: SOTA $\\to$ SODA?\n\nQUESTIONS:\n1. Do we need to apply data augmentations in the SODA framework? I guess not, but data augmentation information, including DINE and BETA, seems missing. In another sense, data augmentation conflicts (like Dropouts) with applying the model perturbation in mutual information calculation - Eq. $(7)$. So if we don't have to apply Data Augmentation, what is the primary source of the outperformance? I would like to see some convincing evidence of the hypothesis.\n2. Aligning with the first question, what is the source of the randomness in calculating Eq. $(7)$? Specifically, the perturbation on $\\theta$? or data $x_i$? I would like to have confirmation from the authors.\n\nLIMITATIONS:\nThe main limitation of this paper is the scope of the experiments, although an extensive study in CIFAR10-C/CIFAR100-C. There are many other benchmarks in the domain-adaptation, such as Office-31, VISDA, Office-Home, etc. It would be more convincing if the authors could demonstrate the out-performance with a fixed set of hyperparameters.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

comments: SUMMARY:
In this work, the authors aim to adapt unlabelled test data to a deployed model without access to its parameters and inner structures during the testing process. Specifically, the authors utilize a data adaptor during testing to map test data into the deployed model, which gradients are estimated via ZOO. Experiments are conducted on CIFAR-10C and CIFAR-100C to verify the effectiveness of the proposed method.

STRENGTHS:
1. This paper is well-written and easy-to-follow. 

2. The code is available with the submission which improves the reproducibility index of the paper.

WEAKNESSES:
1. Experiments are not convincing. The authors only use CIFAR-10C and CIFAR-100C to verify the effectiveness of their algorithm. I suggest more datasets with various types of distribution shift should be included such as Office-31, Office-Home, PAC, etc.

2. The necessity of ZOO is not clear. I can understand parameters of the deployed model are inaccessible, but why the data adaptor you generate during testing is still a black box? If the parameters of the data adaptor are known, why do not you simply fix the parameters of the deployed model as constants, and use FOO to calculate gradients of the data adaptor?

3. More baselines should be considered. There already exist some TTA algorithms that do not requiring access to parameters of the deployed model, such as T3A [1].

[1] Iwasawa Y, Matsuo Y. Test-time classifier adjustment module for model-agnostic domain generalization[J]. Advances in Neural Information Processing Systems, 2021, 34: 2427-2440.

QUESTIONS:
See the Weaknesses.

LIMITATIONS:
The authors provide the limitations of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes usage of zeroth order optimization (ZOO) for test-time adaptation (TTA) to ease several practical issues regarding accessing model parameters during TTA. Since the ZOO with pseudo label, which is a standard method in TTA, might cause the unreliable gradient, the paper proposes a sample selection method using the confidence of the prediction and class balance, and use only reliable sample to compute pseudo-label loss. The unreliable sample is used to compute another unsupervised loss to facilitate better adaptation. They show its effectiveness, CIFAR10-C and CIFAR100-C. No theory is provided.\n\nSTRENGTHS:\n1. The paper is generally well written and easy to follow. \n2. The usage of zeroth-order optimization in TTA is well motivated and interesting new problem. \n3. The proposed sample selection approach based on the confidence and class balance seems not to be revolutionary but sensible.\n\nWEAKNESSES:\n1. The experiment is limited to the CIFAR10-C and CIFAR100-C. As usual, I recommend adding experiments on ImageNet-C and some domain adaptation datasets. \n\n2. The technical novelty is not high. Besides, I found that the effectiveness of the selection method proposed in this paper is not fully validated. For example, the paper does not provide ablation about the sensitivity about the threshold parameter. I'm also curious why we should not compute information maximization loss for reliable samples. Besides, the necessity of information maximization loss is not experimentally validated. \n\n3. The selection of the information maximization loss is not well described. Why did you choose the specific loss function? \n\n4. Several details are unclear for me. Including, \n \n- What the difference between SODA-R and SODA-FO? \n- I'm confused by the table 5, since it says that SODA-O as a variant of SODA under *online* settings but seems to repeat the optimization multiple epochs. Do you repeat the optimization after you reach all test dataset? In the case, I have to say that it is not usual online setup. \n\n5. No theoretical results.\n\nQUESTIONS:\nSee weakness section\n\nLIMITATIONS:\nNone.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}

comments: SUMMARY:
To better adapt models to test distributions without changing model parameters, this paper utilizes the strategy that trains a data adaptor which can adjust the test data to fit the deployed models. To avoid the potential corruption of data features caused by the data adaptor, the proposed method treats the test-time adaptation process as a semi-supervised learning process. Specifically, the test data points are split into two subsets, including a high confidence set to perform regular cross entropy minimization and a low confidence set (as unlabeled set) to perform mutual information maximization.

STRENGTHS:
1. This paper studies a realistic problem in test time adaptation, i.e., the unreliable nature of the pseudo-labels assigned to the test data. 

2. This paper proposes solves the low-quality issue of pseudo-labels by transforming the adaptation process as a semi-supervised learning process, in which the data adaptor model is less impacted by the mislabeled data points.

3. The proposed method uses ZOO framework to estimate the gradients of the parameters and can efficiently the problem with a few queries.

WEAKNESSES:
1. The Pseudo-Label-Robust Data Adaptation module is the key contribution of this paper, but the design of this part is too simple. The problem here is actually noisy label learning problem and treating it as semi-supervised learning is a common strategy. 

2. The reliable pseudo-label selection process in Subsection 3.3 utilizes fixed threshold or ratio to select data points, which maybe not robust in real-world settings.

3. The experiments on large scale datasets are not presented in the paper. Moreover, the parameter analyses in terms of $\tau$ and $\rho$ are needed.

QUESTIONS:
Please refer to Weaknesses.

LIMITATIONS:
N/A.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-11