Tent: Fully Test-Time Adaptation by Entropy Minimization

1 INTRODUCTION . Deep networks can achieve high accuracy on training and testing data from the same distribution , as evidenced by tremendous benchmark progress ( Krizhevsky et al. , 2012 ; Simonyan & Zisserman , 2015 ; He et al. , 2016 ) . However , generalization to new and different data is limited ( Hendrycks & Dietterich , 2019 ; Recht et al. , 2019 ; Geirhos et al. , 2018 ) . Accuracy suffers when the training ( source ) data differ from the testing ( target ) data , a condition known as dataset shift ( Quionero-Candela et al. , 2009 ) . Models can be sensitive to shifts during testing that were not known during training , whether natural variations or corruptions , such as unexpected weather or sensor degradation . Nevertheless , it can be necessary to deploy a model on different data distributions , so adaptation is needed . During testing , the model must adapt given only its parameters and the target data . This fully test-time adaptation setting can not rely on source data or supervision . Neither is practical when the model first encounters new testing data , before it can be collected and annotated , as inference must go on . Real-world usage motivates fully test-time adaptation by data , computation , and task needs : 1 . Availability . A model might be distributed without source data for bandwidth , privacy , or profit . 2 . Efficiency . It might not be computationally practical to ( re- ) process source data during testing . 3 . Accuracy . A model might be too inaccurate without adaptation to serve its purpose . To adapt during testing we minimize the entropy of model predictions . We call this objective the test entropy and name our method tent after it . We choose entropy for its connections to error and shift . Entropy is related to error , as more confident predictions are all-in-all more correct ( Figure 1 ) . Entropy is related to shifts due to corruption , as more corruption results in more entropy , with a strong rank correlation to the loss for image classification as the level of corruption increases ( Figure 2 ) . To minimize entropy , tent normalizes and transforms inference on target data by estimating statistics and optimizing affine parameters batch-by-batch . This choice of low-dimensional , channel-wise feature modulation is efficient to adapt during testing , even for online updates . Tent does not restrict or alter model training : it is independent of the source data given the model parameters . If the model can be run , it can be adapted . Most importantly , tent effectively reduces not just entropy but error . ∗Equal contribution . †Work done at Adobe Research ; the author is now at DeepMind . 1Please see the project page at https : //github.com/DequanWang/tent for the code and more . Our results evaluate generalization to corruptions for image classification , to domain shift for digit recognition , and to simulation-to-real shift for semantic segmentation . For context with more data and optimization , we evaluate methods for robust training , domain adaptation , and self-supervised learning given the labeled source data . Tent can achieve less error given only the target data , and it improves on the state-of-the-art for the ImageNet-C benchmark . Analysis experiments support our entropy objective , check sensitivity to the amount of data and the choice of parameters for adaptation , and back the generality of tent across architectures . Our contributions • We highlight the setting of fully test-time adaptation with only target data and no source data . To emphasize practical adaptation during inference we benchmark with offline and online updates . • We examine entropy as an adaptation objective and propose tent : a test-time entropy minimization scheme to reduce generalization error by reducing the entropy of model predictions on test data . • For robustness to corruptions , tent reaches 44.0 % error on ImageNet-C , better than the state-ofthe-art for robust training ( 50.2 % ) and the strong baseline of test-time normalization ( 49.9 % ) . • For domain adaptation , tent is capable of online and source-free adaptation for digit classification and semantic segmentation , and can even rival methods that use source data and more optimization . 2 SETTING : FULLY TEST-TIME ADAPTATION . Adaptation addresses generalization from source to target . A model fθ ( x ) with parameters θ trained on source data and labels xs , ys may not generalize when tested on shifted target data xt . Table 1 summarizes adaptation settings , their required data , and types of losses . Our fully test-time adaptation setting uniquely requires only the model fθ and unlabeled target data xt for adaptation during inference . Existing adaptation settings extend training given more data and supervision . Transfer learning by fine-tuning ( Donahue et al. , 2014 ; Yosinski et al. , 2014 ) needs target labels to ( re- ) train with a supervised loss L ( xt , yt ) . Without target labels , our setting denies this supervised training . Domain adaptation ( DA ) ( Quionero-Candela et al. , 2009 ; Saenko et al. , 2010 ; Ganin & Lempitsky , 2015 ; Tzeng et al. , 2015 ) needs both the source and target data to train with a cross-domain loss L ( xs , xt ) . Test-time training ( TTT ) ( Sun et al. , 2019b ) adapts during testing but first alters training to jointly optimize its supervised loss L ( xs , ys ) and self-supervised loss L ( xs ) . Without source , our setting denies joint training across domains ( DA ) or losses ( TTT ) . Existing settings have their purposes , but do not cover all practical cases when source , target , or supervision are not simultaneously available . Unexpected target data during testing requires test-time adaptation . TTT and our setting adapt the model by optimizing an unsupervised loss during testing L ( xt ) . During training , TTT jointly optimizes this same loss on source data L ( xs ) with a supervised loss L ( xs , ys ) , to ensure the parameters θ are shared across losses for compatibility with adaptation by L ( xt ) . Fully test-time adaptation is independent of the training data and training loss given the parameters θ . By not changing training , our setting has the potential to require less data and computation for adaptation . 3 METHOD : TEST ENTROPY MINIMIZATION VIA FEATURE MODULATION . We optimize the model during testing to minimize the entropy of its predictions by modulating its features . We call our method tent for test entropy . Tent requires a compatible model , an objective to minimize ( Section 3.1 ) , and parameters to optimize over ( Section 3.2 ) to fully define the algorithm ( Section Section 3.3 ) . Figure 3 outlines our method for fully test-time adaptation . The model to be adapted must be trained for the supervised task , probabilistic , and differentiable . No supervision is provided during testing , so the model must already be trained . Measuring the entropy of predictions requires a distribution over predictions , so the model must be probabilistic . Gradients are required for fast iterative optimization , so the model must be differentiable . Typical deep networks for supervised learning satisfy these model requirements . 3.1 ENTROPY OBJECTIVE . Our test-time objective L ( xt ) is to minimize the entropy H ( ŷ ) of model predictions ŷ = fθ ( xt ) . In particular , we measure the Shannon entropy ( Shannon , 1948 ) , H ( ŷ ) = −∑c p ( ŷc ) log p ( ŷc ) for the probability ŷc of class c. Note that optimizing a single prediction has a trivial solution : assign all probability to the most probable class . We prevent this by jointly optimizing batched predictions over parameters that are shared across the batch . Entropy is an unsupervised objective because it only depends on predictions and not annotations . However , as a measure of the predictions it is directly related to the supervised task and model . In contrast , proxy tasks for self-supervised learning are not directly related to the supervised task . Proxy tasks derive a self-supervised label y′ from the input xt without the task label y . Examples of these proxies include rotation prediction ( Gidaris et al. , 2018 ) , context prediction ( Doersch et al. , 2015 ) , and cross-channel auto-encoding ( Zhang et al. , 2017 ) . Too much progress on a proxy task could interfere with performance on the supervised task , and self-supervised adaptation methods have to limit or mix updates accordingly ( Sun et al. , 2019b ; a ) . As such , care is needed to choose a proxy compatible with the domain and task , to design the architecture for the proxy model , and to balance optimization between the task and proxy objectives . Our entropy objective does not need such efforts . 3.2 MODULATION PARAMETERS . The model parameters θ are a natural choice for test-time optimization , and these are the choice of prior work for train-time entropy minimization ( Grandvalet & Bengio , 2005 ; Dhillon et al. , 2020 ; Carlucci et al. , 2017 ) . However , θ is the only representation of the training/source data in our setting , and altering θ could cause the model to diverge from its training . Furthermore , f can be nonlinear and θ can be high dimensional , making optimization too sensitive and inefficient for test-time usage . Published as a conference paper at ICLR 2021 < latexit sha1_base64= '' FGMSn1olAms3UkJ+mUM6lRBkJrw= '' > AAAB6HicbVDLSgNBEOyNryS+oh69DAZBEMKuKHoMevGYgHlgsoTZSW8yZvbBzKwYlnyBFw+K5OoP+C/e/BqdJB40saChqOqmu8uLBVfatj+tzNLyyupaNpdf39jc2i7s7NZVlEiGNRaJSDY9qlDwEGuaa4HNWCINPIENb3A18Rv3KBWPwhs9jNENaC/kPmdUG6l63CkU7ZI9BVkkzg8plnPx+Pb94avSKXy0uxFLAgw1E1SplmPH2k2p1JwJHOXbicKYsgHtYcvQkAao3HR66IgcGqVL/EiaCjWZqr8nUhooNQw80xlQ3Vfz3kT8z2sl2r9wUx7GicaQzRb5iSA6IpOvSZdLZFoMDaFMcnMrYX0qKdMmm7wJwZl/eZHUT0rOaemsatK4hBmysA8HcAQOnEMZrqECNWCA8AjP8GLdWU/WqzWetWasn5k9+APr7RuTUJCF < /latexit > < latexit sha1_base64= '' 8eHH7cr25vA7s0zJYYCDPQNSaT0= '' > AAAB7XicbVDLSgNBEOyNrxhfUY+KDAbBU9gVQb0FvXhMwDwgWcLsZDYZM7OzzMwKYcnRuxcPinj1F/Id3vwGf8LJ46CJBQ1FVTfdXUHMmTau++VklpZXVtey67mNza3tnfzuXk3LRBFaJZJL1QiwppxFtGqY4bQRK4pFwGk96N+M/foDVZrJ6M4MYuoL3I1YyAg2Vqq1ulgI3M4X3KI7AVok3owUSoejyvfj0ajczn+2OpIkgkaGcKx103Nj46dYGUY4HeZaiaYxJn3cpU1LIyyo9tPJtUN0YpUOCqWyFRk0UX9PpFhoPRCB7RTY9PS8Nxb/85qJCS/9lEVxYmhEpovChCMj0fh11GGKEsMHlmCimL0VkR5WmBgbUM6G4M2/vEhqZ0XvvHhVsWlcwxRZOIBjOAUPLqAEt1CGKhC4hyd4gVdHOs/Om/M+bc04s5l9+APn4wd3ypLI < /latexit > < latexit sha1_base64= '' r9CoIRh1LwyAxszWUWZZpZEIYvU= '' > AAAB7XicbVA9TwJBEJ3DL8Av1NLmIjGxIndGoyXRxhIT+YhwIXvLHqzs7V5254yE8B9sLDDG1tL/Yuev0QUsFHzJJC/vzWRmXpgIbtDzPp3M0vLK6lo2l1/f2NzaLuzs1oxKNWVVqoTSjZAYJrhkVeQoWCPRjMShYPWwfznx6/dMG67kDQ4SFsSkK3nEKUEr1VrIY2bahaJX8qZwF4n/Q4rlXDK+fX/4qrQLH62OomnMJFJBjGn6XoLBkGjkVLBRvpUalhDaJ13WtFQSuyQYTq8duYdW6biR0rYkulP198SQxMYM4tB2xgR7Zt6biP95zRSj82DIZZIik3S2KEqFi8qdvO52uGYUxcASQjW3t7q0RzShaAPK2xD8+ZcXSe245J+UTq9tGhcwQxb24QCOwIczKMMVVKAKFO7gEcbw7CjnyXlxXmetGednZg/+wHn7Btf2kwo= < /latexit > < latexit sha1_base64= '' icKTvSnYuWAwxCN4MXaVcPxJrUE= '' > AAAB7HicbVBNS8NAEN34WetX1aMiwSJ4KokI6q3oxWMLpi20oWy2k3bpZhN2J0IJPXr24kERr/6G/g5v/gb/hNuPg7Y+GHi8N8PMvCARXKPjfFlLyyura+u5jfzm1vbObmFvv6bjVDHwWCxi1QioBsEleMhRQCNRQKNAQD3o3479+gMozWN5j4ME/Ih2JQ85o2gkrxUA0nah6JScCexF4s5IsXw0qn4/Ho8q7cJnqxOzNAKJTFCtm66ToJ9RhZwJGOZbqYaEsj7tQtNQSSPQfjY5dmifGqVjh7EyJdGeqL8nMhppPYgC0xlR7Ol5byz+5zVTDK/8jMskRZBsuihMhY2xPf7c7nAFDMXAEMoUN7farEcVZWjyyZsQ3PmXF0ntvORelK6rJo0bMkWOHJITckZccknK5I5UiEcY4eSJvJBXS1rP1pv1Pm1dsmYzB+QPrI8ftLWSVw== < /latexit > < latexit sha1_base64= '' 6pSYsGji0D9Bm0vY9by0e43+pZo= '' > AAAB6HicbVDLSgNBEOyNrxhfUY9eBoPgxbArAfUW9OIxAfOAZAmzk95kzOzsMjMrhJAv8OJBEa9+kjf/xkmyB00saCiquunuChLBtXHdbye3tr6xuZXfLuzs7u0fFA+PmjpOFcMGi0Ws2gHVKLjEhuFGYDtRSKNAYCsY3c381hMqzWP5YMYJ+hEdSB5yRo2V6he9Ysktu3OQVeJlpAQZar3iV7cfszRCaZigWnc8NzH+hCrDmcBpoZtqTCgb0QF2LJU0Qu1P5odOyZlV+iSMlS1pyFz9PTGhkdbjKLCdETVDvezNxP+8TmrCa3/CZZIalGyxKEwFMTGZfU36XCEzYmwJZYrbWwkbUkWZsdkUbAje8surpHlZ9irlm3qlVL3N4sjDCZzCOXhwBVW4hxo0gAHCM7zCm/PovDjvzseiNedkM8fwB87nD3htjL0= < /latexit > < latexit sha1_base64= '' KLNiQjydwC+UjsLtIanox9T+rq8= '' > AAAB63icbVBNS8NAEJ3Ur1q/qh69LBbBU0mkoN6KXjxWsB/QhrLZbNqlu5uwuymU0L/gxYMiXv1D3vw3btoctPXBwOO9GWbmBQln2rjut1Pa2Nza3invVvb2Dw6PqscnHR2nitA2iXmsegHWlDNJ24YZTnuJolgEnHaDyX3ud6dUaRbLJzNLqC/wSLKIEWxyaRCy6bBac+vuAmideAWpQYHWsPo1CGOSCioN4Vjrvucmxs+wMoxwOq8MUk0TTCZ4RPuWSiyo9rPFrXN0YZUQRbGyJQ1aqL8nMiy0nonAdgpsxnrVy8X/vH5qohs/YzJJDZVkuShKOTIxyh9HIVOUGD6zBBPF7K2IjLHCxNh4KjYEb/XlddK5qnuN+u1jo9a8K+IowxmcwyV4cA1NeIAWtIHAGJ7hFd4c4bw4787HsrXkFDOn8AfO5w8aWY5N < /latexit > < latexit sha1_base64= '' lbHwl5bkUbenc+Yo+u8yNzpxsy0= '' > AAAB6nicbVDLSgNBEOyNrxhfUY+KDAbBU9gVQb0FvXhM0DwgWcLsZDYZMjO7zMwKYcnRoxcPinj1I/Id3vwGf8LJ46CJBQ1FVTfdXUHMmTau++VklpZXVtey67mNza3tnfzuXk1HiSK0SiIeqUaANeVM0qphhtNGrCgWAaf1oH8z9usPVGkWyXsziKkvcFeykBFsrHTXEkk7X3CL7gRokXgzUigdjirfj0ejcjv/2epEJBFUGsKx1k3PjY2fYmUY4XSYayWaxpj0cZc2LZVYUO2nk1OH6MQqHRRGypY0aKL+nkix0HogAtspsOnpeW8s/uc1ExNe+imTcWKoJNNFYcKRidD4b9RhihLDB5Zgopi9FZEeVpgYm07OhuDNv7xIamdF77x4VbFpXMMUWTiAYzgFDy6gBLdQhioQ6MITvMCrw51n5815n7ZmnNnMPvyB8/EDTj2RiQ== < /latexit > < latexit sha1_base64= '' xnrzB72KzfqBMQ17s1zlsxQWR+k= '' > AAAB7XicbZDLSgMxFIbP1Fsdb1WXboJFcFVmRFAXYtGNywr2Au1QMmmmjU0yQ5IRytB3cONCETcufBT3bsS3Mb0stPWHwMf/n0POOWHCmTae9+3kFhaXllfyq+7a+sbmVmF7p6bjVBFaJTGPVSPEmnImadUww2kjURSLkNN62L8a5fV7qjSL5a0ZJDQQuCtZxAg21qq1NOsK3C4UvZI3FpoHfwrFiw/3PHn7civtwmerE5NUUGkIx1o3fS8xQYaVYYTTodtKNU0w6eMubVqUWFAdZONph+jAOh0Uxco+adDY/d2RYaH1QIS2UmDT07PZyPwva6YmOg0yJpPUUEkmH0UpRyZGo9VRhylKDB9YwEQxOysiPawwMfZArj2CP7vyPNSOSv5x6ezGK5YvYaI87ME+HIIPJ1CGa6hAFQjcwQM8wbMTO4/Oi/M6Kc05055d+CPn/Qf/xpJs < /latexit > For stability and efficiency , we instead only update feature modulations that are linear ( scales and shifts ) , and low-dimensional ( channel-wise ) . Figure 4 shows the two steps of our modulations : normalization by statistics and transformation by parameters . Normalization centers and standardizes the input x into x̄ = ( x− µ ) /σ by its mean µ and standard deviation σ . Transformation turns x̄ into the output x′ = γx̄+ β by affine parameters for scale γ and shift β . Note that the statistics µ , σ are estimated from the data while the parameters γ , β are optimized by the loss . For implementation , we simply repurpose the normalization layers of the source model . We update their normalization statistics and affine parameters for all layers and channels during testing . 3.3 ALGORITHM . Initialization The optimizer collects the affine transformation parameters { γl , k , βl , k } for each normalization layer l and channel k in the source model . The remaining parameters θ \ { γl , k , βl , k } are fixed . The normalization statistics { µl , k , σl , k } from the source data are discarded . Iteration Each step updates the normalization statistics and transformation parameters on a batch of data . The normalization statistics are estimated for each layer in turn , during the forward pass . The transformation parameters γ , β are updated by the gradient of the prediction entropy∇H ( ŷ ) , during the backward pass . Note that the transformation update follows the prediction for the current batch , and so it only affects the next batch ( unless forward is repeated ) . This needs just one gradient per point of additional computation , so we use this scheme by default for efficiency . Termination For online adaptation , no termination is necessary , and iteration continues as long as there is test data . For offline adaptation , the model is first updated and then inference is repeated . Adaptation may of course continue by updating for multiple epochs .

Summary: This paper tackles an interesting problem setting — fully test-time adaptation with only target data. The proposed method is to minimize the test-time entropy, and the loss is used to update the feature modulation layer only. The proposed method compares favorably with the state of the arts, on the ImageNet-C benchmark and unsupervised domain adaptation tasks.