Test-Time Training with Masked Autoencoders
Authors: Yossi Gandelsman
Yu Sun
Xinlei Chen
Alexei A Efros
Venue: NeurIPS 2022 Accept (2022)
Keywords: Computer Vision
Test-Time Training
Masked Auto-Encoder

Abstract: Test-time training adapts to a new test distribution on the fly by optimizing a model for each test input using self-supervision.
In this paper, we use masked autoencoders for this one-sample learning problem.
Empirically, our simple method improves generalization on many visual benchmarks for distribution shifts.
Theoretically, we characterize this improvement in terms of the bias-variance trade-off.

Time Training with Masked Autoencoders
Yossi Gandelsman" UC Berkeley
Yu Sun" UC Berkeley
Xinlei Chen Meta
Alexei A. Efros UC Berkeley
Abstract
Test-time training adapts to a new test distribution the fly by optimizing model for each test input using self-supervision.
this paper, we use masked autoen- coders for this one-sample learning problem. Empirically, our simple method improves generalization on many visual benchmarks for distribution shifts. Theo- retically, characterize this improvement terms of the biri-variance trade-off.
Introduction
Generalization is the central theme of supervised learning, and a hallmark of intelligence. While most of the research focuses on generalization when the training and test data are drawn from the same distribution, this is rarely the case for real world deployment 43 and is certainly not true for environments where natural intelligence has emerged.
Most models today are fixed during deployment, even when the distribution changes. As consequence, a trained model needs to be robust to all possible distribution shifts that could happen in the future [22 14 46 33]. This turms out be quite difficult because being ready for all possible futures limits the model's capacity to be good at any particular one. But only one of these futures is actually going to happen.
This motivates an alternative perspective on generalization: instead of being ready for everything, simply adapts the future once it arrives. Test-time training (TTT) is one line of work that takes this perspective [42 32, 18 [. The key insight is that each test input gives a hint about the test distribution. We modify the model test time take advantage of this hint by setting up one-sample learming problem.
The only issue is that the test input comes without a ground truth label. But we can generate labels from the input itself thanks to self-supervised learning. At training time, TTT optimizes both the main task (e.g. object recognition) and the self-supervised task. Then at test time, it adapts the model with the self-supervised task alone for each test input, before making a prediction on the main task.
The choice of the self-supervised task is critical: it must be general enough to produce useful features for the main task on a wide range of potential test distributions. The celf-supervised task cannot be too easy or too hard, otherwise the test input will not provide useful signal. What is a general task at the right level of difficulty? We turn to fundamental property shared by natural images spatial smoothness, i.e. the local redundancy of information in the zy space. Spatial autoencoding removing parts the data, then predicting the removed content forms the basis of some of the most successful self-supervised tasks 47
37
19 49).
In this paper, show that masked autoencoders (MAE) 19 well-suited for test-time training.
Our simple method leads to substantial improvements on four datasets for object recognition. We also provide theoretical analysis of our method with linear models. Our code and models are available https://yossigandd
github. io/ttt_maq mae/ wiiiiiidx.... html
Equal contribution.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

2 Related Work
This paper addresses the problem of generalization under distribution shifts. We first cover work in this problem setting, as well as the related setting of unsupervised domain adaptation. We then discuss test-time training and spatial autocending, the two components of our algorithm.
2.1 Problem Settings
Generalization under distribution shifts. When training and test distributions are different, generalization is intrinsically hard without access training data from the test distribution. The robustness obtained by training or fine-tuning on one distribution shift (c.g. Gaussian noise) often does not transfer to another (e.g. salt-and peppee noise), even for visually similar ones [14 46). Currently the common practice is to avoid distribution shifts altogether by using a wider training distribution that hopefully contains the test distribution with more training data or data augmentation
Unsupervised domain adaptation. An easier but more restrictive setting is use some unlabeled training data from the test distribution (target), in addition to those from the training distribution 24 5 I7]. It is easier because, intuitively, most of the distribution shift happens on the inputs instead of the labels, so most of the test distribution is already known at training time through data. It is more restrictive because the trained model is only effective on the particular test distribution for which data is prepared in advance.
(source) 29 30
2.2 Test-Time Training
The idea of training on unlabeled test data has a long history. Its earliest realization is transductive learning, beginning in the 1990s [3] Vladimir Vapnik 45) states the principle of transduction as:
"Try to get the answer that you really need but not a more general one." This principle has been most commonly applied on SVMs [4 6 26 by using the test data to specify additional constraints on the margin of the decision boundary. Another early line of work is local learning [3] [5]]: for each test input, "local" model is trained on the nearest neighbors before a prediction made.
In the computer vision community, image bootstrap the more difficult faces in the same image. 35 creates a personalized generative model, by fine-tuning it on a few images of an individual person's 's face. [38] trains neural networks for super-resolution on each test image from scratch. (39] improves image retrieval by using one-against-all classifier on the query image. (34] makes video segmentation more efficient by using student model that learns online from a teacher model on the test videos. These are but few examples where vision researchers find it natural to continue training during deployment.
25
improves face detection by using the easier faces in a test
Test-time training (TTT) 42)
proposes this idea as solution to the generalization problem in supervised learning under distribution shifts. It produces a different model for every single test input through self- seff-supervision. This method has also been applied to several fields with domain-spectie self-supervised tasks: vision-basee reinforcement learning [8] legged locomotion [41] tracking and even medical imaging (2).
objects in videos [1] natural language question answering
The self-supervised pretext task employed by 422 is rotation prediction [5]: rotate each input the image plane by multiple of 90 degrees, and predict the angle as four-way classification problem. This task is limited in generality, because it can often be too casy or too hard. For natural outdoor scenes, rotation prediction scene understanding. On the other hand, for top-down views, it is too hard for many classes (c.g. frogs and lizards), since every orientation looks equally plausible.
be too easy by detecting the horizon's 's orientation alone without further
TTT 42 can be viewed alternatively as one-sample unsupervised domain adaptation (UDA) discussed in Subsection 2.1 UDA uses many unlabeled samples from the test distribution (target); TTT uses only one target sample and creates special case - this sample can be the test input itself. We find this idea powerful in two ways: 1) We no longer need to prepare any target data in advance. 2) The learned model no longer needs to generalize to other target data it only needs to "overfit" to the single training sample because it is the test sample.

---

Reconstruction: 0.63
Reconstruction: 0.60
Reconstruction: 0,58
Classification: 4.81
Classification: 2.88
Classification: 2.36
1/1
Original Image
Masked Image
Step
Step
Step 500
We train an MAE to reconstruct each test image at test time, masking 75% of the input patches. The three reconstructed images on the right visualize the progress of this one-sample learning problem. Loss of the main task (green) object recognition keeps dropping even after 500 steps of gradient descent, while the network continues optimize for reconstruction (red). The unmasked patches are not shown on the right since they are not part of the reconstruction loss.
Figure 1:
Other papers following 42 have worked related but different problem settings, assuming access an entire dataset (e.g. TTT+ (28)) batch (e.g. TENT (48) of test inputs from the same distribution. Our paper does not make such assumptions, and evaluates on each single test sample independently.
2.3 Self-Supervision by Spatial Autoencoding
Using autoencoders for representation learning goes back to 47 In computer vision, one of the earliest works is context encoders (37). which predicts a random image region given its context. Recent works [19 2 [4] combine spatial autoencoding with Vision Transformers (ViT) [I]] to achieve state-of-the performance. The most successful work is masked autoencoders (MAE) [19]: it splits each image into many (e.g. 196) patches, randomly masks out majority of them (e.g. 75%), and trains a model to reconstruct the missing patches by optimizing the mean squared error between the original and reconstructed pixels.
MAE pre-trains large encoder and a small decoder, both ViTs [ Only the encoder is used for downstream task, e.g. object recognition. The encoder features become inputs to task-specific linear projection head. There are two common ways to combine the pre-trained encoder and untrained head. 1) Fine-tuning: both the encoder and head are trained together, end-to-end, for the downstream task. 2) Linear probing: the encoder is frozen as a fixed feature extractor, only the head is trained. We refer back to these training options in the next section.
3 Method
At a high level, our method simply substitutes MAE [19 for the self-supervised part of TTT 42 In practice, making this work involves many design choices.
Architecture. Our architecture is Y-shaped (like in 42 feature extractor f simultancously followed by a self-supervised head and a main task head h. Here, f is exactly the encoder of MAE, and g the decoder, both ViTs. We intentionally avoid modifying them to make clean comparison with [19]. For the main task (e.g. object recognition) head h. (19] uses linear projection from the dimension of the encoder features to the number classes. The authors discuss being linear layer as mostly historic artifact. We also experiment with a more expressive main task head an entire ViT-Base to strengthen our baseline.
Training setup. Following standard practice, start from the MAE model provided by the authors of 19), with a ViT-Large encoder, pre-trained for reconstruction on ImageNet- 1k [9]. There are three ways to combine the encoder with the untrained main task head: fine-tuning, probing, and joint training. We experiment with all three, and choose probing with h being a ViT-Base, a.k.a. VIT probing, default setup, based the following considerations:
Fine-tuning: train o end-to-end. This works poorly with TTT because it does not encourage feature sharing between the two tasks. Fine-tuning makes the encoder specialize to recognition at

---

AdamW
SGD
bright
contrast
60
60
elastic
tog
Accuracy (56
50
gauss
glass
Impulse
20
30
motion
pixelate
shot
snow
zoom
Steps
Steps
We experiment with two optimizers for TTT. MAE 19 uses A for pre-training. But our results (left) show that A for TTT requires early stopping, which is unrealistic for generalization to unknown distributions without validation set. We instead use SGD, which keeps improving performance even after 20 steps (right).
Figure 2:
training time, but subsequently, TTT makes the encoder specialize to reconstruction at test time, and lose the recognition-onie features that had learned rely on. Fine-tuning used by [42 for the same reason.
2) VIT probing: train only, with frozen. Here, h is ViT-Base, instead of linear layer used for linear probing. ViT probing is much more lightweight than both fine-tuning and joint-training. trains 3.5 times fewer parameters than linear fine-tuning (86M 306M). It also obtains higher accuracy than linear fine-tuning without aggressive augmentations on the ImageNet validation set.
3) Joint training: train both h o and with rotation prediction. But with MAE, performs worse on the ImageNet validation set, to the best of our ability, than linear ViT probing. If future work finds reliable recipe for joint training baseline, our method can easily be modified to work with that.
by summing their losses together. This is used by [42
Training- time training. Denote the encoder produced by MAE pre-training as fo (and the decoder, used later for TTT, as go). Our default setup, ViT probing, produces a trained main task head hn:
ho arg minA Σ Im (h fo(x;), v.).
(1)
The summation is over the training set with n samples, each consisting of input x, and label y The main task loss 4m in case is the cross entropy loss for classification. Note that we are only optimizing the main task head, while the pre-trained encoder fo is frozen.
Augmentations. Our default setup, during training-time training, only uses image cropping and horizontal flips for augmentations, following the protocol in [19] for pre-training and linear probing. Fine-tuning in [19] (and 22 49)). however, adds aggressive augmentations including random changes in brightness, contrast, color and sharpness]] Many of them are in fact distribution shifts in evaluation benchmarks. Training with them analogous training on the test set, for the purpose of studying generalization to new test distributions. Therefore, we choose not use these augmentations,
top of the two above,
though they would improve results at face value.
Test-time training. At test time, we start from the main task head hy produced by VIT probing, as well as the MAE pre-trained encoder fu and decoder 9o- Once each test input arrives, optimize the following loss for TTT:
fz. 9r arg min, f.g 1.(g f(mask (a)ssk()))))))) (r))
'Oter augmentations used by fine-tuning in [19 are: interpolation, equalization, inversion, posterization, solarization, rotation, shearing, random erasing, and translation. These taken from RandAugment (8).

---

join Train (Sun al.)
TTT- iSun all
Accuracy (%)
Probing)
ITTMAE (VT Probing)
40
bright
contrast
defocus
elastic
tog
frast
gauss
giass
impuise
Ipeg
motion
pixelate
shot
snow
zoom
Figure 3: Accuracy (%) on ImageNet-C, level 5, Our method, TTT-MAE, significantly improves on top of our baseline, which already outperforms the method of 42). See Subsection 4.2 for details. Numbers for our baseline and TTT-MAE can be found in the last two rows of Table 2 Numbers for Sun et al. are taken from
The self-supervised reconstruction loss l, computes the pixel-wise mean squared error of the decoded patches relative to the ground truth. After TTT, we make a prediction on æ as h o fz (z)). Note that gradient-based optimization for Equation 2 Jawways starts from fo and go- When evaluating set, we always discard fa and 92 after making a prediction on each test input æ, and reset the weights to fo and go for the next test imput. By test-time training on the test inputs independently, we do not assume that they come from the same distribution.
test
Optimizer for TTT. In 422, the choice of optimizer for TTT is straightforward: it simply takes the same optimizer setting as during the last epoch of training- time training of the self-supervised task. This choice, however, is not available for us, because the learning rate schedule of MAE reaches zero by the end of pre- tre-training. We experiment with various learning rates for AdamW [3] and stochastic gradient descent (SGD) with momentum. Performance of both, using the best learning rate respectively, is shown in Figure
AdamW used in [9]. but for TTT it hurts performance with too many iterations. On the other hand, more iterations with SGD consistently improve performance on all distribution shifts. Test accuracy keeps improving even after 20 iterations. This is very desirable for TTT: the single test image is all we know about the test distribution, so there is no validation set to tune parameersss or monitor performance for early stopping. With SGD, we can simply keep training.
4 Empirical Results
4.1 Implementation Details
In all experiments, we use MAE based on the ViT-Large encoder [19,, pre-trained for 800 epochs on ImageNet-Ik 99. Our ViT-Base head h takes input the image features from the pre-trate pre-trained MAE encoder. There is linear layer in between that resizes those features to fit as inputs to the head, just like between the encoder and the decoder in [19]. A linear layer is also appended the final class token of the head to produce the classification logits.
TTT is performed with SGD, as discussed, for 20 steps, using momentum of 0.9, weight decay of 0.2, batch size of 128, and fixed learning rate of 5e-3. The choice of 20 steps is purely computational: more steps will likely improve performance marginally, judging from the trend observed in Figure 2 Most experiments are performed on four NVIDIA A100 GPUs; an industrial cluster with V100 GPUs.
sweeps are ran on
Like in pre- pre-training, only tokens for non- non-masked patches are given to the encoder during TTT. whereas the decoder takes all tokens. Each image in TTT batch has different random mask. We use the same masking ratio as in MAE 19 75%. We do not use any augmentation on top of random masking for TTT. Specifically, every iteration of TTT is performed on the same 224 x 224 center crop of the image as we later use make a prediction; the only difference being that predictions are made images without masking.
5

---

Jellyfish
Holster
Chiton
Fire salamander
Spaghetti squash
Frost
JPEG
Brighiness
Elastic
Ours
42
Ours
42
Ours
42
Ours
Jellyfish
-3
0
-6
5
2
-2
Holster
-1
4
-2
6
14
Chiton
-1
3
-3
8
-1
2
-3
Fire salamander 1
3
-1
3
2
5
Spaghetti squash
-3
-2
Changes after TTT (-Rot and -MAE) number of correctly classified images, out of 50 total for each category. On these rotation invariant classes, TTT-Rot [42) hurts performance, while TTT-MAE still helps, thanks to the generality of MAE as a self-supervised task. The four corruptions selected for being the most accurate categories of the TTT-Rot baseline.
Table 1:
We optimize both the encoder and decoder weights during TTT, together with the class token and the mask token. We have experimented with freezing the decoder and found that the difference, even for multiple iterations of SGD, is negligible; see i the appendix for detailed comparison. This consistent with the observations in [4]. We also tried reconstruction loss both with and without normalized pixels, done in [19]. Our method works well for both, but slightly better for normalized pixels because the bascline is slightly better. We also include these results the appendix.
4.2 ImageNet-C
ImageNet-C [22] a benchmark for object recognition under distribution shifts. contains copies of the ImageNet (
validation set with 15 types of corruptions, each with 5 levels of severity. Due to space constraints, results in the main text are on level 5, the most severe, unless stated otherwise. Results the other four levels are in the appendíx. Sample images from this benchmark also in the appendix in Figure 4
In the spirit of treating these distribution shifts truly unknown, do not training data, prior knowledge, or data augmentations derived from these corruptions. This complies with the stated rule of the benchmark, that the corruptions should be used only for evaluation, and the algorithm being evaluated should not be cerrutiionsppeciin [22]. This rule, in our opinion, helps community progress: the numerous distribution shifts outside of research evaluation cannot be anticipated, and an algorithm cannot scale it relies on information that is specific to a few test distributions.
Main results. Our main results on ImageNet-C appear in Figure Following the convention in 42 we plot accuracy only on the level-5 corruptions. Results on the other levels are in the appendix. results here the default training setup discussed in Section 3 take pre-trained MAE encoder, then perform ViT probing for object recognition on the original ImageNet training set. Our baseline (green) applies the fixed model to the corrupted test sets. TTT-MAE (red) on top of our baseline significantly improves performance.
Comparing reconstruction vs. rotation prediction. The baseline for TTT-Rot, taken from 42 is called Joint Train in Figure This is ResNet (20] with 16-layers, after joint training for rotation prediction and object recognition. Because our baseline is much more advanced than the ResNet baseline, the former already outperforms the reported results of Tot-Rot, for all corruptions except contrast. ? So comparison to [42] is more meaningful in relative terms: TTT-MAF has higher performance gains in all corruptions than TTT-Rot, on top of their respective baselines.
It turns out that for the contrast corruption type. the ResNet baseline of TTT-Rot is much better than our VIT baseline, even though the latter is much larger; TTT-MAE improves the VIT baseline nevertheless.

---

brigh
cont
defoc
elast
Tog
frost
gauss
glass
impul
jpeg
motn
pixel
shot
zoom
Joint Train
62.3
4.5
26.7
39.9
25.7
30.0
5.8
16.3
5.8
45.3
30.9
45.9
7.1
25.1
31.8
Fine-Tune
67.5
7.8
33.9
32.4
36.4
38.2
22.0
15.7
23.9
51.2
37.4
51.9
23.7
37.6
37.1
ViT Probe
68.3
6.4
24.2
31.6
38.6
38.4
17.4
18.4
18.2
51.2
32.2
49.7
18.2
35.9
32.2
TTT-MAE
69.1
34.4
50.7
44.7
50.7
30.5
36.9
32.4
63.0
41.9
63.0
33.0
42.8
45.9
Table 2: Accuracy (%) on ImageNet-C, level 5. The first three rows are fixed models without test- time training, comparing the three design choices discussed in Section [] The third row, ViT probing, our default baseline throughout the paper; it has the same numbers as the baseline in Figure [ The last row is our method: TTT-MAE after ViT probing. This achieves the best performance across all corruption types; it has the same numbers as TTT-MAE in Figure
Rotation invariant classes. As discussed in Section 2 rotation prediction [5] often too hard to
be helpful, in contrast to a more general task like spatial autoencoding. In fact. we find entire classes that are rotation invariant, and show random examples from them in Table |
Not surprisingly, these
images are usually taken from top-down views; rotation prediction on them can only memorize the auxiliary labels, without forming semantic features as intended. This causes TTT-Rot of [42] to hurt performance on these classes, as seen in Table [ Also not surprisingly, TTT-MAE is agnostic to rotation invariance and still helps on these classes.
Training- time training. In Section B we discussed our choice of ViT probing instead of fine- tuning or joint training. The first three rows of Table 2 compare accuracy of these three designs. As discussed, joint training does not achieve satisfactory performance on most corruptions. While fine-tuning initially performs better than ViT probing, it is not amenable to TTT. The first three rows only for training-time training, after which fixed model is applied during testing. The last row is TTT-MAE after ViT probing, which performs the best across all corruption types. Numbers in the last two rows the same as, respectively, for Baseline (ViT Probing) and TTT-MAE in Figure 3
4.3 Other ImageNet Variants
We evaluate TTT-MAE on two other popular benchmarks for distribution shifts. ImageNet-A 23 contains real-world, unmodified, and naturally occurring examples where popular ImageNet models perform poorly. ImageNet-R [2 contains renditions of ImageNet classes, e.g. art, cartoons and origami. Both TTT-MAE and the baseline use the same models and algorithms, with exactly the same
as for ImageNet-C (Subsection 4.2) Our method continues to outperform the
baseline by large margins: see T.ble
4.4 Portraits Dataset
The Portraits dataset [16] contains American high school yearbook photos labeled by gender, taken over more than a century. It contains real-world distribution shifts in visual appearance over the years. We sort the entire dataset by year and split it into four equal parts, with 5062 images each, Between the four splits, there are visible low-level differences such as blurriness and contrast, as well as high-level change like hair-style and smile.
We create three experiments: for each, train on of the first three splits and test the fourth. Performance is measured by accuracy in binary gender classification. As expected, baseline performance increases slightly as the training split gets closer to test split in time.
Our model is the same ImageNet pre-trained MAE + VIT-Base (with sigmoid for binary classification), and probing is performed on the training split. We use exactly the same parrm-parameters for TTT. Our results are shown Table Our method improves on the baseline for all three experiments.
We find of these classes through the following: 1) Take the ImageNet pre- vreitrained ResNet- 18 from the official Py PoTch website [36). 2) Run the original validation set, and copy rotated 90-dege 3) Eliminate classes for which the original accuracy is lower than 60%. 4) For each class still left. compare accuracy on the original vs. rotate images, and choose the 5 classes for which the normalized difference the smallest.
"The Portraits dataset only accessed by the UC Berkeley authors.
7

---

ImageNet-A
ImageNet-R
Bascline
TTT-MAE
Bascline
TTT-MAE
15.3
21.3
31.3
38.9
Table 3: Accuracy (%) on ImageNet-A and ImageNet-R [1];; see Subsection 4.3 for details. Baseline is ViT Probing the default trained on the original ImageNet training set; it is in fact the same model as for ImageNet-C. Our method improves on the baseline for both datasets, using the same hyper-parameters as the rest of the paper.
Train Split
1
3
Baseline TTT-MAE
76.1
76.5
78.2
76.4
76.7
79.4
Table 4: Accuracy (%) for binary classification on the Portraits dataset; see Subsection 4.4 for details. For each column, we train on the indicated split, and test on the fourth. Baseline is still VIT Probing (except with sigmoid in the end). Our method improves the baseline for all three training splits. We perform both regular training and TTT using the same pyper-parameters as the rest of the paper.
Theoretical Results
Why does TTT help? The theory of 42 gives an intuitive but shallow explanation: when the self-supervised task happens to propagate gradients that correlate with those of the main task. But this evasive theory only delegates one unknown the test distribution, to another the magical self-supervised task with correlated gradients, without really answering the question, or showing any concrete example of such self- superviee task.
In this paper, we show that autoencoding, i.e. reconstruction, is a self-supervised task that makes TTT help. To do with minimal mathematical overhead, restrict ourselves the linear world, where our models are linear and the distribution shifts are produced by linear transformations. We analyze autoencoding with dimensionality reduction instead of masking, as we believe the two closely related in essence.
The most illustrative insight, in our opinion, is that under distribution shifts, TTT finds a better bias- variance rrade-off than applying fixed model. The fixed model is biased because it is completely based on biased training data that do not represent the new test distribution. The other extreme is to completely forget the training data, and train a new model from scratch on each test input; this is also undesirable because the single test input is high variance, although unbiased by definition.
sweet spot of the variance trade-off is to perform TTT while remembering the training data in some way. In deep learning, this is usually done by initializing with a trained model for SGD, like for our paper and [42]. In this section, we retain memory by using part the covariance matrix derived from training data.
is well known that linear autoencoding equivalent to principle component analysis (PCA). This equivalence simplifies the mathematics by giving us closed form solutions to the optimization problems both during training and test-time training. For the rest of the section, use the term PCA instead of linear autoencoding, following convention of the theory community.
Problem setup. Let 2, P. the training distribution, and assume that the population covariance matrix Σ = Cov(x) is known. PCA performs spectral decomposition on Σ = DDT,, and takes the top & eigenvectors #1,... Ug to project x E R to R Throughout this section, we denote u, as the äth column of the matrix U, and likewise for other matrices.
Assume that for each æ, the ground truth y is linear function of the PCA projection with k: 1:
y wu x,
33)

---

for some known weight w E R. For mathematical convenience, we also assume the following about the eigenvalues, i.c. the diagonal entries of D:
0 03 Jd =
(4)
At test time, nature draws a new T, y P. but we can only see न. corrupted version of x. We model a corruption as an unknown orthogonal transformation Rt and I = Rr.
Algorithms.
The baseline is to blithely apply fixed model and predict
1 ww 2,
(5)
as an estimate of y. Intuitively, this will be inaccurate if corruption is severe i.e. is far from I. Now we derive the PCA version of TT.. We first construct new (and rather trivial) covariance matrix with I, the only piece of information we have about the corruption. Let a [0, 1] be a hyper-parameter.
We then form linear combination of Σ with our new covariance matrix:
M(a) = (1 - a) -+ TT
(6)
Denote its spectral decomposition as ((aa (a) -(a) V (a). We then predict
ý wv
Bias-variance trade-off. a = O is equivalent to the baseline, and a = means that we exclusively use the single test input and completely forget about the training data. statistical terms, a presents a bias- variance trade-off: a 0 means more bias, a 1 means more variance.
Theorem. Define the prediction risk as E[D vIJ. where the expectation is taken over the corrupted test distribution. This risk strictly dominated when a = O. That is, TTT with some hyper-parameter choice > 0 is, average, strictly better than the baseline.
The proof is given in the appendix.
Remark on the assumptions. The assumption in Equation 3 mild; it basically just says that PCA least helpful on the training distribution. It is also mild assume that E and context, since the estimated covariance matrix and weight asymptotically approach the population ground truth as the training set grows larger. The assumption the eigenvalues in Equation greatty simplifies the math, but a more complicated version of our analysis should exist without it. Lastly, believe that our general statement should still hold for invertible linear transformation instead of orthogonal transformations, since they share the same intuition.
known in this
Limitations
Our method is slower at test time than the baseline applying fixed model. Inference speed has not been the focus this paper. It might improved through better syper-parammters, optimizers, training techniques and architectural designs. Since most of the deep learning toolbox has been refined for regular training on large datasets, it still has much room to improve for test-time training.
While spatial autoencoding is general self-supervised task, there is no guarantee that it will produce useful features for every main task, on every test distribution. Our work only covers one main task object recognition, and handful of the most popular benchmarks for distribution shifts.
In the long run, believe that human-like generalization is most likely emerge in human-lite like environments. Our current paradigm of evaluating machine learning models with i.dd. images collected into test set is far from the human perceptual experience. Much closer the human experience would be a video stream, where test-time training naturally has more potential, because seff-supervised learning can be performed on not only the current frame, but also past ones.
Acknowledgments and Disclosure of Funding
The authors would like to thank Yeshwanth Cherapanamjerii for help with the proof of our theorem, and Assaf Shocher for proofreading. Yu Sun would like to thank his co-advisor, Moritz Hardt, for his continued support. Yossi Gandelsman funded by the Berkeley Fellowship. The Berkeley authors funded, in part, by DARPA MCS and ONR MURI.

---

References
[1] Pratyay Banerjee, Tejas Gokhale, and Chitta Baral. Self-supervised test-time learning for reading compre- hension. arXiv preprint arXiv:2103. //263, 2021.
[2] Hangbo Bao, Li Dong. and Furu Wei. abs/2106.08254, 2021.
Beit: BERT pre-training of image transformers. CoRR.
[3] Léon Bottou and Vladimir Vapnik. Local learning algorithms. Neural computation, 4(6):888-900, 1992.
[4] Minmin Chen, Kilian Q Weinberger, and John Blitzer. Co-training for domain adaptation. In Advances in neural information processing systems, pages 2456-2464, 2011.
[5] Xilun Chen, Yu Sun, Ben Athiwaratkun, Claire Cardie, and Kilian Weinherger. Adversarial deep averaging networks for cross-lingual sentiment classification. Transactions of the Association for Computational Linguistics, 6:557-577 2018.
[6] Ronan Collobert, Fabian Sinz, Jason Weston, Léon Bottou, and Thorsten Joachims. Large scale transductive svms. Journal of Machine Learning Research, 7(8), 2006.
[7] Gabricla Csurka. Domain adaptation for visual applications: A comprehensive survey. arXiv preprint arXiv: 1702.05374. 2017.
[8] Ekin D. Cubuk, Barret Zoph. Jonathon Shlens, and Quoc V. Le. Randaugment: Practical data augmentation with no separate search. CoRR, abs//1999 13719,, 2019.
[9] Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. pages 248 255. leee, 2009.
[10] Samuel Dodge and Lina Karam. Quality resilient deep neural networks. arXiv preprint 1703.000 19, 2017.
[11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. /9999, 2020.
[12] Yang Fu, Sifei Liu, Umar Iqbal, Shalini De Mello, Humphrey Shi, and Jan Kautz. Learning to track instances without video annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8680-8689, 2021.
[13] A. Gammerman, V. Vovk, and V. Vapnik. Learning by transduction. In In Uncertainty in Artificial 155. Morgan Kaufmann, 1998.
Intelligence, pages 148-155,
[14] Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt,, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. In Advances in Neurul Information Processing Systems, pages 7538 7550, 2018.
[15] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv: 1803.07728, 2018.
[16] Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexei A. Efros. A century of portraits: A visual historical record of american high school yearbooks. CoRR, abs/151 1, 1.2575, 2015.
[17] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE Conference on Compuder Vision and Pattern Recognition, pages 2066 2073. IEEE, 2012.
[18] Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Alenyà, Pieter Abbeel, Alexci A Efros, Lerrel Pinto, and Xiaolong Wang. Self-supervised policy adaptation during deployment. arXiv preprint wwXiv:2007.043000 2020.
[19] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. Masked autoencoders scalable vision learners. CoRR, abs/2111 1222111006377 2021.
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
[21] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distributio generalization. JCCV, 2021.
[22] Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corrup- tions and perturbations. CoRR, abss1903.11266 2019.
[23] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. CVPR, 2021.
10

---

[24] Judy Hoffman, Eric Tzeng. Taesung Park, Jun- Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, and Trevor Darrell. Cyeada: Cycle-consistent adversarial domain adaptation. arXiv preprini 1.1.0:1711.05211. 2017.
[25] Vidit Jain and Erik Learned-Miller. Online domain adaptation of pre-trained cascade of classifiers. In CVPR 201 I, pages 577-584. IEEE, 2011.
[26] Thorsten Joachims. Learning to classify text using support vector machines, volume 668. Springer Science & Business Media, 2002.
[27] Neerav Karani, Ertnne Erdil, Krishna Chaitanya, and Ender Konukoglu. Test-time adaptable neural networks for robust medical image segmentation. Medical Image Analysis, 68:101907, 2021.
[28] Yuejiang Liu, Parth Kothari, Bastien van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Tu++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34, 2021.
[29] Mingsheng Long, Yue Can, Jianmin Wang, and Michael I Jordan, Learning transferable features with deep adaptation networks. arXiv preprint N0M2vv/502.02799 2015.
[30] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael Jordan. Unsupervised domain adaptation with residual transfer networks. In Advances in Neural Information Processing Systems, pages 136-144, 2016.
[31] Nyy Loshchilov and Frank Hutter. arXiv:1711.05101 2017.
Decoupled weight decay regularization.
arXiv preprint
[32] Xuan Luo, Bia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation. ACM Transactions Graphics (ToG), 39(4):71-1, 2020.
[33] Chengzhi Mao. Lu Jiang, Mostafa Dehghani, Carl Vondrick, Rahul Sukthankar, and Irfan Essa. Discrete representations strengthen vision transformer robustness. arXiv preprint ar 111:22/11.1049. 2021.
[34] Ravi Teja Mullapudi, Steven Chen, Keyi Zhang. Deva Ramanan, and Kayvon Fatahalian. Online model distillation for efficient video inference. arXiv preprint arXiv:1812.02699 2018.
[35] Yotam Nitzan, Ktir Aberman, Qiurui He, Orly Liba, Michal Yarom, Yossi Gandelsman, Inbar Mosseri, Yael Pritch, and Daniel Cohen-Or. Mystyle: A personalized generative prior. arXiv preprint arXiv:2203.17272. 2022.
[36] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché- Auché-Buc.F Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024 8035. Curran Associates, Inc., 2019.
[37] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2536- 2234,254,,22 2016.
[38] Assaf Shocher, Nadav Cohen, and Michal Irani. "zero-shot" super-resolution using deep internal learning. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 3118-3126.
2018.
[39] Abhinav Shrivastava, Tomasz Malisiewicz, Abhinav Gupta, and Alexei A. Efros. Data-driven visual similarity for cross-domain image matching. ACM Transaction of Graphics (TOG) (Proceedings of ACM SIGGRAPH ASIA), 30(6), 2011.
[40] Yu Sun, Erie Tzeng, Trevor Darrell and Alexei A Efros. Unsupervised domain adaptation through self-supervision. arXiv preprint, 2019.
[41] Yu Sun, Wyatt L Ubellacker, Wen-Loong Ma, Xiang Zhang, Changhao Wang. Noel V Csomay-Shanklin, Shanklin, Masayoshi Tomizuka, Koushil Sreenath, and Aaron D Ames. Online learning of unknown dynamics for model-based controllers in legged locomotion. IEEE Robotics and Automation Letters, 6(4)8442 8449, 2021.
[42] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Eiros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning, pages 9229-9248. PMLR, 2020.
[43] Antonio Torralba and Alexei Efros. Unbiased look dataset bias. In CVPR 2011, pages 1521-1528. IEEE, 2011.
[44]
Vladimir Vapnik. The nature statistical learning theory. Springer science & business media, 2013.
[45] Vladimir Vapnik and S. Kotz. Estimation of Dependences Based on Empirical Data: Empirical Inference Science (Information Science and Statisties). Springer-Verlag, Verlag. Berlin, Heidelberg, 2006.

---

[46] Igor Vasiljevic, Ayan Chakrabarti, and Gregory Shakhnarovich. Examining the impact of blur recognition by convolutional networks. arXiv preprint urXiv:1611.0555 2016.
[47] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. ICML '08, page 1096 1103, New York, NY, USA, 2008. Association for Computing Machinery.
[48] Dequan Wang. Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006. 10726, 2020.
[49] Zhenda Xic, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In International Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[50] Hao Zhang, Alexander C Berg, Michael Maire, and Jitendra Malik. Svm-knn:
Discriminative nearest neighhor classification for visual category recognition. 2006 Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06) volume 2, pages 2126-2136. IEFE, 2006.
[51] Stephan Zheng, Yang Song, Thomas Leung, and Ian Goodfellow. Improving the robustness of deep neural networks stability training. In Proceedings of the ieee conference on computer vision and pattern recognition, pages 4480-4488, 2016.
12


Reviews:
[{"review_id": "WWKikAS5Nzm", "reviewer": "Reviewer_QNmJ", "summary": "The paper proposes to use test time training on the test images to improve generalization under distribution shifts. Using the establishe MAE method, the method add an additional classifier branch on the encoder for classification. To train the model, the paper studies three ways: 1) finetuning classifier and encoder, 2) ViT probing 3) joint training MAE and classifier, where ViT probing works the best. The test time training only trains the MAE part. Results are shown on ImageNet-C, R, and A.\n\n", "questions": "1. Does the proposed work on single test images? Or it has to be trained on many test images from the same domain, with batchsize>1.\n\n2. The theoretical part is not linked to the accuracy, why the accuracy gets better? Why the proposed MAE outperforms past method, would simple PCA reconstruction also improve accuracy based on the theory? ", "limitations": "None.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Using MAE as test-time training improves generalization on shifted distributions is effective from the presented results. Results from 4 datasets have been shown, with significant improvement.\n\n2. The paper is well presented.\n\n3. The reviewer appreciates the theoretical explanation, which is interesting.\n\nWeakness:\n1. On ImageNet-C's level 1-4, baseline results are missing. What is the standard mCE number on ImageNet-C following the standard metric? Would the approach's number better than other non-test-time training ViT method? Such as the Discrete ViT [1], which uses VQVAE for encoding, similar to the MAE task for reconstruction, but does not require test time training.\n\n2. Should add more intuition for theoretical justification part, such that without reading the equation this part is also clear.\n\n[1] Discrete Representations Strengthen Vision Transformer Robustness. ICLR 2022. https://arxiv.org/abs/2111.10493", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "ULNCiLay5Ft", "reviewer": "Reviewer_ZRRX", "summary": "This paper presents a test-time training algorithm to address the degradation of classification performance typically seen in deep learning models. The paper combines two ideas— self-supervised learning via masked autoencoders, and test-time training (TTT) using a self-supervised objective. Specifically, the paper applies the recently proposed masked auto-encoder (MAE) framework for self-supervised training, towards test-time training. To do so, the authors propose to freeze the feature network of this model and learning a classifier head on top during training, and then fine-tuning the feature network to minimize the reconstruction loss on the test sample to address distribution shift. The authors show a significant improvement in performance across different datasets using their approach over the existing state-of-the-art using test-time training method.", "questions": "Questions and suggestions:\n\n- Line 53: “One way to turn extrapolation into interpolation again is to make the training distribution wider, and the training set larger, to cover the test distribution.”. This statement is misleading because it assumes that it is always possible for a single hypothesis function to perform well on both training and test distributions. But this may not be necessarily true (e.g. label shift where p(y|x) changes). This needs to be either clarified as the scope of this statement, or removed all together.\n\n- Line 177: Not very clear. My understanding is that the baseline is the pre-trained MAE model which is kept fixed, and only a classifier head is trained using labels.\n\n- Foot note #3 below Line 196: You mean much worse? Fig 3 shows TTT-Rot is worse than ViT baseline.\n\n- Paragraph of Line 199: Not clearly stated what is the difference between Probing and TTT-MAE in table 2. If the goal is to talk about training-time training, and the 1st 3 rows are without TTT, then it should be explicitly mentioned, because mentioning TTT-MAE results alongside is confusing.", "limitations": "Limitations:\n\n- As mentioned above, the theoretical justification is too simplistic and disconnected from the proposed method and the real world setting. I think it is better to remove it from the paper.\n\n- Some ablation studies would be great that provide some insight into why the proposed framework works better compared to the existing rotation prediction based TTT framework. Some intuition is discussed by authors along the lines that rotation prediction is not a well defined task for certain class of images (e.g. top or bottom view images where rotation has no semantic meaning). But a clearer ablation in terms of the differences in the features learned by the two models would be more helpful.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- simple approach.\n- The experimental results show significant improvement across different datasets in the test-time training paradigm.\n- ablation analysis on test-time training optimizer AdamW vs SGD.\n\nWeaknesses:\n\n- theoretical justification is largely disconnect from the proposed method. I do not see how the data matrix affine transform as a proxy for distribution shift applies in the real world setting. It does not provide any insight into why TTT really helps, or how the specific strategy chosen in this work is better than the existing TTT algorithm that uses the rotation prediction task.\n- The bias-variance terminology is confusing and should be removed, as it s trying to convey a different meaning from what it traditionally means.\n- The writing is not clear in a couple of places, and needs improvement.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Hlnpo31Iwqw", "reviewer": "Reviewer_vzDf", "summary": "Summary:\nThe paper performs training using test set images using the idea of masked autoencoder to further improve the performance. Experiments show that this approach improves the performance on out-of-distribution images. They also present a theoretical analysis suggesting the reasons for the observed improvements under distribution shifts. ", "questions": "Please see the strengths and weakness section\n\n** Update after the authors response **\n\nUpdating my scores to lean towards acceptance.", "limitations": "Yes, the limitations have been included in the paper.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths and Weakness:\n\n- The paper leverages a relatively new idea of self-supervised pre-training to obtain good results on out of distribution image classification tasks.\n\n- I feel that the paper falls short in the contributions part. Additionally, the novelty seems to be quite limited in the paper. The idea of masked autoencoders is quite known and the paper just applies it to the pre-existing task of test-time training.  As such, there are very limited technical contributions that the paper makes. In my opinion, the paper is looking more suitable for a workshop.\n\n- I felt the theoretical analysis part may not be adding much value and can be moved to the appendix. That would help in saving space, which could be used to write up more interesting analysis or new experiments. Some suggestions to make the contributions stronger include doing ablations with different types of MAE architectures showcasing the pros and cons of each, impact of masking ratio in MAE on the test-time training training.\n\n- Instead of always using the same pre-trained MAE checkpoint, the authors can train or use different MAE models trained on varied datasets apart from ImageNet and then perform experiments as to which kind of datasets are more robust to test-time training and offer more improvements.\n\n- The code is not submitted with the paper, raising reproducibility concerns.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "66yvO1dwbE", "reviewer": "Reviewer_evQW", "summary": "The authors propose to improve generalization and robustness under distribution shift by cloning the network and training the clone for on multiple masked copies of each incoming example at evaluation time using an auxiliary masked reconstruction loss.", "questions": "The paper would greatly benefit from including more self-supervised losses as baselines.", "limitations": "The authors have clearly outlined the additional computational cost incurred by the use of their approach.\n", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* The paper is easy to follow and well written.\n* The experimental protocol is built around multiple large scale datasets.\n* The method is novel, simple, and effective.\n\nWeaknesses:\n* An explicit mathematical formulation of the training and test-time training losses would make the paper easier to read.\n* Compares reconstructions solely to a rotation prediction self-supervised loss.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept