Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attacks
Year: 2021

Abstract:
Adversarial attacks optimize against models to defeat defenses. Existing defenses 1 are static, and stay the same once trained, even while attacks change. We argue 2 that models should fight back, and optimize their defenses against attacks at test 3 time. We propose dynamic defenses, to adapt the model and input during testing, 4 by defensive entropy minimization (dent). Dent alters testing, but not training, for 5 compatibility with existing models and train-time defenses. Dent improves the 6 robustness of adversarially-trained defenses and nominally-trained models against 7 white-box, black-box, and adaptive attacks on CIFAR-10/100 and ImageNet. In 8 particular, dent boosts state-of-the-art defenses by 20+ points absolute against 9 AutoAttack on CIFAR-10 at ∞ = 8/255. 10

text: Adversarial attacks optimize against models to defeat defenses. Existing defenses1 are static, and stay the same once trained, even while attacks change. We argue2 that models should fight back, and optimize their defenses against attacks at test3 time. We propose dynamic defenses, to adapt the model and input during testing,4 by defensive entropy minimization (dent). Dent alters testing, but not training, for5 compatibility with existing models and train-time defenses. Dent improves the6 robustness of adversarially-trained defenses and nominally-trained models against7 white-box, black-box, and adaptive attacks on CIFAR-10/100 and ImageNet. In8 particular, dent boosts state-of-the-art defenses by 20+ points absolute against9 AutoAttack on CIFAR-10 at ∞ = 8/255.10
1 Introduction: Attack, Defend, and Then?11
Deep networks are vulnerable to adversarial attacks: input perturbations that alter natural data to12 cause errors or exploit predictions [54]. As deep networks are deployed in real systems, these attacks13 are real threats [63], and so defenses are needed. The challenge is that every new defense is followed14 by a new attack, in a loop [56]. The strongest attacks, armed with gradient optimization, update to15 circumvent defenses that do not. Such iterative attacks form an even tighter loop to ensnare defenses.16 In a cat and mouse game, the mouse must keep moving to survive.17
Current defenses, deterministic or stochastic, stand still: once trained, they are static and do not adapt18 during testing. Adversarial training [18, 30] learns from attacks during training, but cannot learn19 from test data. Stochastic defenses alter the network [11] or input [20, 7], but their randomness is20 independent of test data. Static defenses do not adapt, and so they may fail as attacks update.21
Our dynamic defense fights adversarial updates with defensive updates by adapting during testing22 (Figure 1). In fact, our defense updates on every input, whether natural or adversarial. Our defense23 objective is entropy minimization, to maximize model confidence, so we call our method dent for24 defensive entropy. Our updates rely on gradients and batch statistics, inspired by test-time adaptation25 approaches [53, 43, 28, 29, 58]. In pivoting from training to testing, dent is able to keep changing, so26 the attacker never hits the same defense twice. Dent has the last move advantage, as its update always27 follows each attack.28
Dent connects adversarial defense and domain adaptation, which share an interest in the sensitivity of29 deep networks to input shifts. Just as models fail on adversarial attacks, they fail on natural shifts30 like corruptions. Adversarial data is a particularly hard shift, as evidenced by the need for more31 parameters and optimization for adversarial training [30], and its negative side effect of reducing32
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.
(a) train (b) static defense (c) dynamic defense
✔ ✔ ✔
<latexit sha1_base64="2rOUSCw5QBt4CukzW0lP28fPN44=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Rjw4jGCeUCyhNnJbDJmdmaZ6RXCkn/w4kERr/6G3+DNv3HyOGhiQUNR1U13V5RKYdH3v72V1bX1jc3CVnF7Z3dvv3Rw2LA6M4zXmZbatCJquRSK11Gg5K3UcJpEkjej4c3Ebz5yY4VW9zhKeZjQvhKxYBSd1OjggCPtlsp+xZ+CLJNgTsowR61b+ur0NMsSrpBJam078FMMc2pQMMnHxU5meUrZkPZ521FFE27DfHrtmJw6pUdibVwpJFP190ROE2tHSeQ6E4oDu+hNxP+8dobxdZgLlWbIFZstijNJUJPJ66QnDGcoR45QZoS7lbABNZShC6joQggWX14mjfNKcFnx7y7K1ehzFkcBjuEEziCAK6jCLdSgDgwe4Ale4NXT3rP35r3PWle8eYRH8Afexw/oPZAN</latexit> ✓
<latexit sha1_base64="AhwyGtIuXiSBnj0qtFaaFkQkbRE=">AAACM3icbVBLSwMxGMz6rPVV9eglWARBKdla1/ZW8CKeKtgHbJeSTdM2NPsgyYpl2f/kxbu/wYMgHhTx6n8w2/ZgWwc+GGYmyZdxQ86kQujNWFpeWV1bz2xkN7e2d3Zze/sNGUSC0DoJeCBaLpaUM5/WFVOctkJBsedy2nSHV6nfvKdCssC/U6OQOh7u+6zHCFZa6uRu2uM7bNF3nRgV0Nl0EvhwGs96lQoqmZY2LxAqWpU0dV4sW1bS7lKucNLJ5bU0Blwk5pTkwRS1Tu6l3Q1I5FFfEY6ltE0UKifGQjHCaZJtR5KGmAxxn9qa+tij0onHOyXwWCtd2AuEHl/Bsfr3RIw9KUeeq5MeVgM576Xif54dqV7ZiZkfRor6ZPJQL+JQBTAtEHaZoETxkSaYCKZ3hWSABSZK15zVJZjzX14kjWLBtArotpSvus+TOjLgEByBE2CCS1AF16AG6oCAR/AKPsCn8WS8G1/G9yS6ZEwrPAAzMH5+AUAHpoE=</latexit> x +
<latexit sha1_base64="2rOUSCw5QBt4CukzW0lP28fPN44=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Rjw4jGCeUCyhNnJbDJmdmaZ6RXCkn/w4kERr/6G3+DNv3HyOGhiQUNR1U13V5RKYdH3v72V1bX1jc3CVnF7Z3dvv3Rw2LA6M4zXmZbatCJquRSK11Gg5K3UcJpEkjej4c3Ebz5yY4VW9zhKeZjQvhKxYBSd1OjggCPtlsp+xZ+CLJNgTsowR61b+ur0NMsSrpBJam078FMMc2pQMMnHxU5meUrZkPZ521FFE27DfHrtmJw6pUdibVwpJFP190ROE2tHSeQ6E4oDu+hNxP+8dobxdZgLlWbIFZstijNJUJPJ66QnDGcoR45QZoS7lbABNZShC6joQggWX14mjfNKcFnx7y7K1ehzFkcBjuEEziCAK6jCLdSgDgwe4Ale4NXT3rP35r3PWle8eYRH8Afexw/oPZAN</latexit> ✓ <latexit sha1_base64="2rOUSCw5QBt4CukzW0lP28fPN44=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Rjw4jGCeUCyhNnJbDJmdmaZ6RXCkn/w4kERr/6G3+DNv3HyOGhiQUNR1U13V5RKYdH3v72V1bX1jc3CVnF7Z3dvv3Rw2LA6M4zXmZbatCJquRSK11Gg5K3UcJpEkjej4c3Ebz5yY4VW9zhKeZjQvhKxYBSd1OjggCPtlsp+xZ+CLJNgTsowR61b+ur0NMsSrpBJam078FMMc2pQMMnHxU5meUrZkPZ521FFE27DfHrtmJw6pUdibVwpJFP190ROE2tHSeQ6E4oDu+hNxP+8dobxdZgLlWbIFZstijNJUJPJ66QnDGcoR45QZoS7lbABNZShC6joQggWX14mjfNKcFnx7y7K1ehzFkcBjuEEziCAK6jCLdSgDgwe4Ale4NXT3rP35r3PWle8eYRH8Afexw/oPZAN</latexit> ✓ <latexit sha1_base64="2rOUSCw5QBt4CukzW0lP28fPN44=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Rjw4jGCeUCyhNnJbDJmdmaZ6RXCkn/w4kERr/6G3+DNv3HyOGhiQUNR1U13V5RKYdH3v72V1bX1jc3CVnF7Z3dvv3Rw2LA6M4zXmZbatCJquRSK11Gg5K3UcJpEkjej4c3Ebz5yY4VW9zhKeZjQvhKxYBSd1OjggCPtlsp+xZ+CLJNgTsowR61b+ur0NMsSrpBJam078FMMc2pQMMnHxU5meUrZkPZ521FFE27DfHrtmJw6pUdibVwpJFP190ROE2tHSeQ6E4oDu+hNxP+8dobxdZgLlWbIFZstijNJUJPJ66QnDGcoR45QZoS7lbABNZShC6joQggWX14mjfNKcFnx7y7K1ehzFkcBjuEEziCAK6jCLdSgDgwe4Ale4NXT3rP35r3PWle8eYRH8Afexw/oPZAN</latexit> ✓
attacker
defender
<latexit sha1_base64="ppjlPDcyu4VwLDDqZq2kE4xaNjg=">AAACV3icbVHLTgIxFO2MqIgv1KWbRmJioiEdRISdiRuXmIiYwEg6pWBjpzNp7xjJZP7NbzBu+BU32gEW+DjJTU7Oub29PQ1iKQwQMnXclcLq2npxo7S5tb2zW97bvzdRohnvsEhG+iGghkuheAcESP4Qa07DQPJu8Hyd+90Xro2I1B1MYu6HdKzESDAKVhqUVX82o6fHgZ+SKjlbVIZfT9OfXqtV8y4azUbtsn5BWk3vDJNqvdUk50t6ri035WbWH3IJ9DGFLBuUK3b4DPgv8RakghZoD8rv/WHEkpArYJIa0/NIDH5KNQgmeVbqJ4bHlD3TMe9ZqmjIjZ/O9s7wsVWGeBRpWwrwTF0+kdLQmEkY2M6QwpP57eXif14vgVHTT4WKE+CKzS8aJRJDhPOQ8VBozkBOLKFMC7srZk9UUwb2K0o2BO/3k/+S+1rVa1TJbb1yFbzN4yiiQ3SETpCHLtEVukFt1EEMfaBPZ8UpOFPny11zi/NW11lEeIB+wN37Buxpqzg=</latexit>
x + t <latexit sha1_base64="nBInVXJwAgBXrV95v+J1waqiBYM=">AAACW3icbVHLSgMxFM1MfdTxVRVXboJFENQy04dtdwU3LhWsLbRjyaRpG8w8SO6IZZh/8xtc6cJfEdNpQateuHByzr25NydeJLgC2343zNzK6tp6fsPa3Nre2S3s7d+rMJaUtWkoQtn1iGKCB6wNHATrRpIR3xOs4z1ezfTOE5OKh8EdTCPm+mQc8BGnBDQ1KMh+dkdPjj03sUv2+SJT/HyWLGtNp35Zrdu1RqNcqZWb1XNslyrfR0fLtRnnLFVp2kr7QyaAPCRw4aTpoFDUA7LAf4GzAEW0iJtB4bU/DGnsswCoIEr1HDsCNyESOBUstfqxYhGhj2TMehoGxGfKTbLdU3yimSEehVJnADhjf3YkxFdq6nu60icwUb+1Gfmf1oth1HATHkQxsIDOB41igSHEM6PxkEtGQUw1IFRyvSumEyIJBf0dljbB+f3kv+C+rN0s2bfVYst7mduRR0foGJ0iB9VRC12jG9RGFL2hT2PdyBsfZs60zK15qWksLDxAS2EefgGe+Krr</latexit> x + t 1 <latexit sha1_base64="aXlGNmGpUtm/2tH4c4aPucVYgsk=">AAACXHicbVHLahsxFNVM08ZxmsZpoJtuRE2h0NRI4/E8doFuukwgTgL21Ghk2RHRPJDulJph/q3f0F02+ZVWHnuR14ELh3OudK+O0lJJA4TcOe6rnddvdjt73f23B+8Oe0fvL01RaS7GvFCFvk6ZEUrmYgwSlLgutWBZqsRVevt97V/9EtrIIr+AVSmSjC1zuZCcgZVmPTNt75joZZrUZEBOttXg31/rx17kD+kw9EZ+TGgckOgEkwENosAL/RGJIzKMPRq2YhR5w5EX+5SGgR+SoNtM50IB+1nDN69pZr2+ndACPyd0S/poi7NZ7+90XvAqEzlwxYyZUFJCUjMNkivRdKeVESXjt2wpJpbmLBMmqdvlG/zZKnO8KLStHHCrPjxRs8yYVZbazozBjXnqrcWXvEkFiyipZV5WIHK+GbSoFIYCr5PGc6kFB7WyhHEt7a6Y3zDNONj/6NoQ6NMnPyeXno14QM79/mn6ZxNHB31En9AXRFGITtEPdIbGiKM79M/pOHvOvbvj7rsHm1bX2UZ4jB7B/fAfKhyrJg==</latexit> x + t 2 <latexit sha1_base64="ppjlPDcyu4VwLDDqZq2kE4xaNjg=">AAACV3icbVHLTgIxFO2MqIgv1KWbRmJioiEdRISdiRuXmIiYwEg6pWBjpzNp7xjJZP7NbzBu+BU32gEW+DjJTU7Oub29PQ1iKQwQMnXclcLq2npxo7S5tb2zW97bvzdRohnvsEhG+iGghkuheAcESP4Qa07DQPJu8Hyd+90Xro2I1B1MYu6HdKzESDAKVhqUVX82o6fHgZ+SKjlbVIZfT9OfXqtV8y4azUbtsn5BWk3vDJNqvdUk50t6ri035WbWH3IJ9DGFLBuUK3b4DPgv8RakghZoD8rv/WHEkpArYJIa0/NIDH5KNQgmeVbqJ4bHlD3TMe9ZqmjIjZ/O9s7wsVWGeBRpWwrwTF0+kdLQmEkY2M6QwpP57eXif14vgVHTT4WKE+CKzS8aJRJDhPOQ8VBozkBOLKFMC7srZk9UUwb2K0o2BO/3k/+S+1rVa1TJbb1yFbzN4yiiQ3SETpCHLtEVukFt1EEMfaBPZ8UpOFPny11zi/NW11lEeIB+wN37Buxpqzg=</latexit> x + t <latexit sha1_base64="nBInVXJwAgBXrV95v+J1waqiBYM=">AAACW3icbVHLSgMxFM1MfdTxVRVXboJFENQy04dtdwU3LhWsLbRjyaRpG8w8SO6IZZh/8xtc6cJfEdNpQateuHByzr25NydeJLgC2343zNzK6tp6fsPa3Nre2S3s7d+rMJaUtWkoQtn1iGKCB6wNHATrRpIR3xOs4z1ezfTOE5OKh8EdTCPm+mQc8BGnBDQ1KMh+dkdPjj03sUv2+SJT/HyWLGtNp35Zrdu1RqNcqZWb1XNslyrfR0fLtRnnLFVp2kr7QyaAPCRw4aTpoFDUA7LAf4GzAEW0iJtB4bU/DGnsswCoIEr1HDsCNyESOBUstfqxYhGhj2TMehoGxGfKTbLdU3yimSEehVJnADhjf3YkxFdq6nu60icwUb+1Gfmf1oth1HATHkQxsIDOB41igSHEM6PxkEtGQUw1IFRyvSumEyIJBf0dljbB+f3kv+C+rN0s2bfVYst7mduRR0foGJ0iB9VRC12jG9RGFL2hT2PdyBsfZs60zK15qWksLDxAS2EefgGe+Krr</latexit> x + t 1 <latexit sha1_base64="aXlGNmGpUtm/2tH4c4aPucVYgsk=">AAACXHicbVHLahsxFNVM08ZxmsZpoJtuRE2h0NRI4/E8doFuukwgTgL21Ghk2RHRPJDulJph/q3f0F02+ZVWHnuR14ELh3OudK+O0lJJA4TcOe6rnddvdjt73f23B+8Oe0fvL01RaS7GvFCFvk6ZEUrmYgwSlLgutWBZqsRVevt97V/9EtrIIr+AVSmSjC1zuZCcgZVmPTNt75joZZrUZEBOttXg31/rx17kD+kw9EZ+TGgckOgEkwENosAL/RGJIzKMPRq2YhR5w5EX+5SGgR+SoNtM50IB+1nDN69pZr2+ndACPyd0S/poi7NZ7+90XvAqEzlwxYyZUFJCUjMNkivRdKeVESXjt2wpJpbmLBMmqdvlG/zZKnO8KLStHHCrPjxRs8yYVZbazozBjXnqrcWXvEkFiyipZV5WIHK+GbSoFIYCr5PGc6kFB7WyhHEt7a6Y3zDNONj/6NoQ6NMnPyeXno14QM79/mn6ZxNHB31En9AXRFGITtEPdIbGiKM79M/pOHvOvbvj7rsHm1bX2UZ4jB7B/fAfKhyrJg==</latexit> x + t 2
<latexit sha1_base64="hRRmoBqyNB/m3sMmuTH2eEiEI1k=">AAACYXicbVHLbtswEKTUtE3cJlHTYy5EjAIFmhqU/JJvAdpDjykQJwEs1aDotU2EeoBcBTEE/Vu/IbdceumPlLJ9yKMDLDCYWXKXw6RQ0iBjD477auf1m7e7e6137/cPDr0PR5cmL7WAschVrq8TbkDJDMYoUcF1oYGniYKr5OZb41/dgjYyzy5wVUCc8kUm51JwtNLUu4vWd0z0Iokr1mGn26pphEtA/qV62hD4/nDQG7J+GAbdfjDqnVLWGQTDXp+NQtYdBX5/MGq0hoSPjG6rjr6DQv6rwq9BXU+9th2zBn1J/C1pky3Op959NMtFmUKGQnFjJj4rMK64RikU1K2oNFBwccMXMLE04ymYuFovX9NPVpnRea5tZUjX6uMTFU+NWaWJ7Uw5Ls1zrxH/501KnIdxJbOiRMjEZtC8VBRz2sRNZ1KDQLWyhAst7a5ULLnmAu2ntGwI/vMnvySXQccfdNjPXvss+b2JY5cckxPymfhkSM7ID3JOxkSQP86Os+8cOH/dPddzjzatrrON8CN5Avf4H21BrDk=</latexit> ✓ + t 2 <latexit sha1_base64="A9WiZ+bK/7LW5jA/gnGgQJ1REHg=">AAACYHicbVHLbhMxFPVMeYTQ0rTsYGMRISFRInsyz10lWLAsEmkrZYbI4ziJVc9D9h2qaDT/xjewY8GGL8F5LGjLka50dM617/VxXitpgJBfjnvw6PGTp71n/eeHRy+OByenl6ZqNBcTXqlKX+fMCCVLMQEJSlzXWrAiV+Iqv/m48a++C21kVX6FdS2ygi1LuZCcgZVmg9t0e8dUL/OsJSNytq8Op7ASwN63dxs8Mk48GoRx6EV+QJL4DJNRGJEgjr1x4CU+pVG40YJx5AV+QmgSkij2x/0u/SQUsG8tfKBdNxsM7ZQt8ENC92SI9riYDX6m84o3hSiBK2bMlJIaspZpkFyJrp82RtSM37ClmFpaskKYrN3u3uG3VpnjRaVtlYC36r8nWlYYsy5y21kwWJn73kb8nzdtYBFnrSzrBkTJd4MWjcJQ4U3aeC614KDWljCupd0V8xXTjIP9k74Ngd5/8kNy6Y1oOCJf/OF5/mMXRw+9Rm/QO0RRhM7RZ3SBJoij386Bc+gcOX/cnnvsnuxaXWcf4Ut0B+6rv9kLq/o=</latexit> ✓ + t 1 <latexit sha1_base64="C+y4zG3KVrZ+ZpUXBqcomtU3aGw=">AAACXnicbVFNa9tAEF2pbZIqTeO2l0AvS02h0GBWsmTZt0By6DGFOglYqlmtx/aS1Qe7o4IR+m/9Db2VXvpTsv441EkfDDzem92ZfZtVShpk7LfjPnv+4uDw6KV3/Ork9WnnzdsbU9ZawFiUqtR3GTegZAFjlKjgrtLA80zBbXZ/ufZvf4A2siy+4aqCNOeLQs6l4GilaadONndM9CJLG9Zj57tqaYJLQP652W/wRwMWD8O+34+DKByx4JyyXsT25GitxX48CGMWDYdBPwpGodcmV6CQf2+wbaedrp2xAX1K/B3pkh2up51fyawUdQ4FCsWNmfiswrThGqVQ0HpJbaDi4p4vYGJpwXMwabPZvKUfrTKj81LbKpBu1H9PNDw3ZpVntjPnuDSPvbX4P29S43yYNrKoaoRCbAfNa0WxpOus6UxqEKhWlnChpd2ViiXXXKD9Ec+G4D9+8lNyE/T8QY99DbsX2c9tHEfkPflAPhGfxOSCfCHXZEwE+eM4juccO3/dA/fEPd22us4uwndkD+7ZA690q38=</latexit> ✓ + t
Experiments evaluate dent against white-box attacks (APGD, FAB), black-box attack (Square), and35 adaptive attacks that are aware of its updates. Dent boosts state-of-the-art adversarial training defenses36 on CIFAR-10 by 20+ points against AutoAttack [9] at ∞ = 8/255. Ablations inspect the effects of37 iteration, parameterization, and batch size. Our code is included in the supplement.38
Our contributions39
• We highlight an opportunity for dynamic defense: the last move advantage.40
• We propose the first fully test-time dynamic defense: dent adapts both the model and input41 during testing without needing to alter training.42
• Dent augments state-of-the-art adversarial training methods, improving robustness by 30%43 relative, and tops the AutoAttack leaderboard by 15+ points.44
• We devise two adaptive attacks against dent: denying updates and mixing batches.45
2 Related Work46
Adversarial Defense For adaptive adversaries, which change in response to defenses, it is natural47 to consider dynamic defenses, which adapt in turn. Evans et al. [14] explain dynamic defenses are48 promising in principle but caution they may not be effective in practice. Their analysis concerns49 randomized defenses, which do change, but their randomization does not adapt to the input. We argue50 for dynamic defenses that depend on the input to keep adapting along with the attacks. Goodfellow51 [17] supports dynamic defenses for similar reasons, but does not develop a specific defense. We52 demonstrate the first defense to optimize the model and input during testing for improved robustness.53
Most defenses for deep learning focus on first-order adversaries [18, 30] that are equipped with54 gradient optimization but constrained by `p-norm bounds. Adversarial training and randomization55 are the most effective defenses against such attacks, but are nevertheless limited, as they are fixed56 during testing. Adversarial training [18, 30] trains on attacks, but a different or stronger adversary57 (by norm or bound) can overcome the trained defense [46, 55]. Randomizing the input [37, 7, 32]58 or network [11] requires the adversary to optimize in expectation [3], but can still fail with more59 iterations. Furthermore, these defenses gain adversarial robustness by sacrificing accuracy on natural60 data. Dent adapts during testing to defend against various attacks without more harm to natural61 accuracy.62
Generative, self-supervised, and certified defenses try to align testing with training but are still63 static. Generative defenses optimize the input w.r.t. autoregressive [50], GAN [42], or energy [23]64 models, but the models do not adapt, and may be attacked by approximating their gradients [3].65 Self-supervised defenses optimize the input w.r.t. auxiliary tasks [49], but again the models do not66 adapt. Certified defenses [7, 66] guarantee robustness within their training scope, but are limited67 to small perturbations by specific types of attacker during testing. Changing data distributions or68
adversaries requires re-training all of these defenses. Dent adapts during testing, without requiring69 (re-)training, and is the only method to update the model itself against attack.70
Domain Adaptation Domain adaptation mitigates input shifts between the source (train) and target71 (test) to maintain model accuracy [34, 41]. Adversarial attacks are such a shift, and adversarial72 error is related to natural generalization error [51, 15]. How then can adaptation inform dynamic73 defense? Train-time adaptation is static, like adversarial training, with the same issues of capacity,74 optimization, and re-computation when the data/adversary changes. We instead turn to test-time75 adaptation methods.76
Test-time adaptation keeps updating the model as the data changes. Model parameters and statistics77 can be updated by self-supervision [53], normalization [43], and entropy minimization [58]. These78 methods improve robustness to natural corruptions [22], but their effect on adversarial perturbations79 is not known. We base our defense on entropy minimization as it enables optimization during testing80 without altering model architecture or training (as needed for self-supervision). For defense, we (1)81 extend the parameterization of adaptation with model and input transformations, (2) optimize for82 additional iterations, and (3) investigate usage on data that is adversarial, natural, or mixed. We are83 the first to report test-time model adaptation improves robustness to adversarial perturbations.84
Dynamic Inference A dynamic model conditionally changes inference for each input, while a85 static model unconditionally fixes inference for all inputs. There are various dynamic inference86 techniques, with equally varied goals, such as expressivity with more parameters or efficiency with87 less computation. All static models are alike; each dynamic model is dynamic in its own way.88
Selection techniques learn to choose a subset of components [1, 57]. Halting techniques learn to89 continue or end computation [19, 59]. Mixing techniques learn to combine parameters [47, 33, 62].90 Implicit techniques learn to iteratively update [6, 4]. While these methods learn to adapt during91 training, our method keeps adapting by directly optimizing during testing.92
3 Dynamic Defense by Test-Time Adaptation93
Adversarial attacks optimize against defenses at test time, so defenses should fight back, and counter-94 optimize against attacks. Defensive entropy minimization (dent) does exactly this for dynamic95 defense by test-time adaptation.96
In contrast to many existing defenses, dent alters testing, but not training. Dent only needs differen-97 tiable parameters for gradient optimization and probabilistic predictions for entropy measurement.98 As such, it applies to both adversarially-trained and nominally-trained models.99
3.1 Preliminaries on Attacks and Defenses100
Let x ∈ Rd and y ∈ {1, . . . , C} be an input sample and its corresponding ground truth. Given a101 model f(·; θ) : Rd → RC parameterized by θ, the goal of the adversary is to craft a perturbation102 δ ∈ Rd such that the perturbed input x̃ = x+ δ causes a prediction error f(x+ δ; θ) 6= y.103
A targeted attack aims for a specific prediction of y′, while an untargeted attack seeks any incorrect104 prediction. The perturbation δ is constrained by a choice of `p norm and threshold : {δ ∈ Rd |105 ‖δ‖p < }. We consider the two most popular norms for adversarial attacks: `∞ and `2.106
Adversarial training is a standard defense, formulated by Madry et al. [30] as a saddle point problem,107
argmin θ E(x,y) max δ L(f(x+ δ; θ), y), (1)
which the model minimizes and the adversary maximizes with respect to the loss L(ŷ, y), such as108 cross-entropy for classification. The adversary iteratively optimizes δ by projected gradient descent109 (PGD), a standard algorithm for constrained optimization, for each step t via110
δt = Πp(δ t−1 + α · sign(∇δt−1L(f(x+ δt−1; θ), y))), (2)
for projection Πp onto the norm ball for `p < , step size hyperparameter α, and random initialization111 δ0. The model optimizes θ against δ to minimize the loss of its predictions on perturbed inputs. This112 is accomplished by augmenting the training set with adversarial inputs from PGD attack.113
Adversarial training is state-of-the-art, but static. Dynamic defenses offer to augment its robustness.114
3.2 Defensive Entropy Minimization115
Defensive entropy minimization (dent) counters attack updates with defense updates. While adver-116 saries optimize to cross decision boundaries, entropy minimization optimizes to distance predictions117 from decision boundaries, interfering with attacks. As the adversary optimizes its perturbation δ, dent118 optimizes its adaptation ∆,Σ. Figure 2 shows dent’s model (∆) and input (Σ) updates.119
Dent is dynamic because both ∆,Σ depend on the testing data, whether natural x or adversarial120 x+ δ. On the contrary, static defenses depend only on training data through the model parameters θ.121 Figure 3 contrasts static and dynamic defenses across the steps of attack optimization.122
Entropy Objective Test-time optimization requires an unsupervised objective. Following tent [58],123 we adopt entropy minimization as our adaptation objective. Specifically, our defense objective is to124 minimize the Shannon entropy [45] H(ŷ) of the model prediction during testing ŷ = f(x; θ) for the125 probability ŷc of class c:126
H(ŷ) = − ∑
c∈1,...,C p(ŷc) log p(ŷc) (3)
Adaptation Parameters Dent adapts the model by ∆ and input by Σ (Figure 2). For the model, dent127 adapts affine scale γ and shift β parameters by gradient updates and adapts mean µ and variance σ2128 statistics by estimation. These are a small portion of the full model parameters θ, in only the batch129 normalization layers [25]. However, they are effective for conditioning a model on changes in the130 task [33] or data [43, 58]. For the input, dent updates Gaussian smoothing g by gradient updates131 of the parameter Σ, while adjusting the filter size for efficiency [48]. This controls the degree of132 smoothing dynamically, unlike defense by static smoothing [7].133
In standard models the scale γ and shift β parameters are shared across inputs, and so adaptation134 updates batch-wise. For further adaptation, dent can update sample-wise, with different affine135 parameters for each input. In this way, it adapts more than prior test-time adaptation methods with136 batch-wise parameters [58, 43].137
Our model and input parameters are differentiable, so end-to-end optimization coordinates them138 against attacks as layered defenses. This coordination is inspired by CyCADA [24], for domain139 adaptation, but dent differs in its purpose and its unified loss. CyCADA also optimizes input and140 model transformations but does so in parallel with separate losses. Our defensive optimization is141 joint and shares the same loss.142

heading: ACCURACY(%) NATURAL ADVERSARIAL
text: Update Algorithm In summary, when the adversary attacks with perturbation δt, our dynamic143 defense reacts with Σt,∆t. The parameters of the model f and smoothing g are updated by144 argminΣ,∆H(f(g(x+ δ; Σ); θ + ∆)) through test-time optimization. At each step, dent estimates145 the normalization statistics µ, σ and then updates the parameters γ, β,Σ by the gradient of entropy146 minimization. Figure 3 contrasts static defenses and dynamic defenses that update like dent.147
Dent adapts on batches rather than samples. Batch-wise adaptation stabilizes optimization for entropy148 minimization. The defense parameters reset between batches.149
Discussion The purpose of a dynamic defense is to move when the adversary moves. When the150 adversary submits an attack x+ δt, the defense counters with ∆t. In this way, the defense has the151 last move, and therefore an advantage.152
Our dynamic defense changes the model, and therefore its gradients, but differs from gradient153 obfuscation [3]. Our defense does not rely on (1) shattered gradients, as the update does not cause154 non-differentiability or numerical instability; (2) stochastic gradients, as the update is deterministic155 given the input, model, and prior updates; nor (3) exploding/vanishing gradients, as the update156 improves robustness with even a single step (although more steps are empirically better).157
Dent forces the attack to rely on a stale gradient, as δt follows ∆t−1, while the model adapts by ∆t.158
4 Experiments159
We evaluate dent against white-box, black-box, and adaptive attacks with a variety of static defenses160 and datasets. For attacks, we choose the AutoAttack [9] benchmark, which includes four attack types161 spanning white-box/gradient and black-box/query attacks. For static defenses, we choose strong and162 recent adversarial training methods, and we also experiment with nominally trained models. For163 datasets, we evaluate dent on CIFAR-10/CIFAR-100 [27], as they are popular datasets for adversarial164 robustness, and ImageNet [40], as it is a large-scale dataset.165
We ablate the choice of model/input adaptation, parameterization, and the number of updates.166
4.1 Setup167
Metrics We score natural accuracy on the regular test data x and adversarial accuracy on the perturbed168 test data x+ δ. Each is measured as percentage accuracy (higher is better). We report the worst-case169 adversarial accuracy across attacks.170
Test-time Optimization We optimize batch-wise ∆ (dent) and sample-wise ∆ (dent+). Dent updates171 by Adam [26] with learning rate 0.001. Dent+ updates by AdaMod [13] with learning rate 0.006. Σ172
updates use learning rate 0.25. All updates use batch size 128 and no weight decay. Dent+ regularizes173 updates by information maximization [16, 29]. We tuned update hyperparameters against PGD174 attacks. Please see the code for exact settings.175
Architecture For comparison with existing defenses, we keep the architecture and training the same,176 and simply load the public reference models provided by RobustBench [10]. For analysis and ablation177 experiments, we define a residual net with 26 layers and a width multiplier of 4 (ResNet-26-4) [21, 64],178 following prior work on adaptation [53, 58].179
4.2 Attack Types & Threat Model180
We evaluate standard white-box and black-box attacks with adversarially-trained models (Section 4.3)181 and nominally-trained models (Section 4.4), as well as dent-specific adaptive attacks (Section 4.5).182
We primarily evaluate against AutoAttack’s ensemble of:183
1. APGD-CE [30, 9], an untargeted white-box attack by cross-entropy,184
2. APGD-DLR [9], a targeted white-box attack with a shift and scale invariant loss,185
3. FAB [8], a targeted white-box attack for minimum-norm perturbation,186
4. Square Attack [2], an untargeted black-box attack with square-shaped updates.187
These attacks are cumulative, so a defense is only successful if it holds against each type. Following188 convention, we evaluate `∞ attacks with ∞ = 8/255 and `2 attacks with 2 = 0.5. This is the189 standard evaluation adopted by the popular RobustBench benchmark [10].190
We devise and experiment with two adaptive attacks against dent and its dynamic updates. The first191 interferes with adaptation by denying updates: it optimizes offline against θ without ∆,Σ updates.192 The second interferes with adaptation by mixing data: it combines adversarial data and natural data193 in the same batch. Both are specific to dent to complement our general evaluation by AutoAttack.194
These attacks fall under the usual white-box threat model. The adversary has full access to the195 classifier, including its architecture and parameters, and the defense, such as dent’s adaptation196 parameters and statistics. With this access the adversary chooses an attack for each input, but it197 cannot choose the inputs (the test set is fixed).198
We include one additional requirement: dent assumes access to test batches rather than individual199 test samples. While independent, sample-wise defense is ideal for simplicity and latency, batch200 processing is not impractical. For example, cloud deployments of deep learning batch inputs for201 throughput efficiency, and large-scale systems handle many inputs per unit time [31].202
The supplementary material covers more attacks, including AutoAttack Plus and Boundary, to confirm203 that AutoAttack is a sufficient measure of robustness.204
4.3 Dynamic Defense of Adversarial Training205
We extend static adversarial training defenses with dynamic updates by dent. Compared to nominal206 training, adversarial training achieves higher adversarial accuracy but lower natural accuracy. The207 purpose of dent is to improve adversarial accuracy without further harming natural accuracy.208
Dent improves state-of-the-art defenses. Table 1 shows state-of-the-art adversarial training defenses209 [5, 44, 39, 38, 60, 12] with and without dynamic defense by dent. Note that dent does not specialize210 to the choice of norm or bound, unlike adversarial training, but instead adapts to each attack during211 testing. In each case, dent significantly improves adversarial accuracy and maintains natural accuracy.212
Dent updates batch-wise for 30 steps. Dent+ is more robust in fewer steps by sample-wise adaptation.213 With sample-wise (γ, β) parameters, dent+ needs only six steps to reach an adversarial accuracy214 within 90% of the natural accuracy. These experiments only include model adaptation of ∆, without215 input adaptation of Σ, as we found it unnecessary when combined with adversarial training.216
Dent helps across attack types. Table 2 evaluates dent against each attack in the AutoAttack217 ensemble. Dent improves robustness to each attack type. We report the worst case across these types218 in the remainder of our experiments.219
Dent helps across datasets and architectures. We experiment on ImageNet to check scalability.220 We evaluate the defense of Wong et al. [60], one of few defenses that scales to this dataset, against221 strong `∞-PGD attacks with 30 iterations, step size of 0.1, and five random starts. Dent improves the222 adversarial accuracy by 14 points against PGD at ∞ = 4/255 and natural accuracy by 23 points.223
Table ?? in the supplement confirms improvement across more defenses, architectures, and datasets.224
4.4 Dynamic Defense of Nominal Training225
Dent improves the adversarial accuracy of off-the-shelf, nominally-trained models. As dent does not226 assume adversarial training, it can apply to various models at test time.227
For nominal training, we exactly follow the CIFAR reference training in pycls [35, 36] with ResNet-228 26-4/ResNet-32-10 architectures. Briefly, we train by stochastic gradient descent (SGD) for 200229 epochs with batch size 128, learning rate 0.1 and decay 0.0005, momentum 0.9, and a half-period230 cosine schedule.231
We evaluate against `∞ and `2 AutoAttack attacks on CIFAR-10. As the nominally-trained models232 have no static defense, we constrain the adversaries to smaller perturbations.233
Dent defends nominally-trained models. Table 3 inspects how each part of dent affects adversarial234 accuracy and natural accuracy. When applying dent to nominally-trained models, model adaptation235 through ∆ is further helped by input adaptation through Σ. In just a single step, the ∆ update236 improves adversarial accuracy without affecting natural accuracy. from 8.8% to 15.0% against `∞237 attacks with just a single step. With 10 steps, and Σ adaptation, dent improves the model’s adversarial238 accuracy to 45.4% against `∞ attacks and 36.5% against `2 attacks. In total, dent boosts `∞ and `2239 adversarial accuracy by almost 40 and 30 points while only sacrificing 3 points of natural accuracy.240 Dent delivers this boost at test-time, without re-training.241
Input adaptation helps preserve natural accuracy. Gaussian smoothing significantly improves242 adversarial accuracy. This agrees with prior work on denoising by optimization [20] or randomized243 smoothing [7]. Tuned as a fixed hyperparameter, smoothing helps adversarial accuracy but hurts244 natural accuracy. Optimized end-to-end, our dynamic smoothing reduces the natural accuracy gap.245 On natural data, the learned Σ for the blur decreases to approximate the identity transformation.246
4.5 Adaptive Attacks on Dent Updates247
We adaptively attack dent through its use of adaptation by (1) denying updates and (2) mixing batches.248 To deny updates, we attack the static model offline by optimizing against θ without ∆,Σ updates,249 then submit this attack to dent. This attempts to short circuit adaptation by disrupting the first update250 with a sufficiently strong perturbation. To mix batches, we mix adversarial and natural data in the251 same batch. This attempts to prevent adaptation by aligning batch statistics with natural data.252
Denying Updates The aim of this attack is to defeat adaptation on the first move, before dent can253 update to counter it. We optimize against the static model alone to prevent defensive optimization254 until adversarial optimization is complete. Under this attack, the input to dent is the final perturbation255 derived by adversarial attack against the static model.256
We examine whether these offline perturbations can disrupt adaptation. Table 4 shows that dent can257 still defend against this attack. This suggests that updating, and having the last move, remains an258 advantage for our dynamic defense.259
Mixing Batches Dent adapts batch-wise, with the underlying assumption that one shared transfor-260 mation can defend the whole batch. We challenge this assumption by evaluating mixed batches of261 adversarial and natural data. In Table 5, we vary the ratio of adversarial and natural data in each batch262 and measure accuracy on the adversarial portion.263
At the extreme, we consider an adaptive attack with only one adversarial input per batch. Specifically,264 we batch one adversarial input with 15 natural inputs randomly chosen from the test set. This adaptive265 attack aims to reduce adaptation by the dynamic defense, as natural inputs do not need adaptation.266
Dent is generally robust to batch mixing, and improves over adversarial training in 10 steps or less.267
4.6 Ablations & Analysis268
More updates deliver more defense. The number of steps can balance defense and computation.269 Table 6 shows that more steps offer stronger defense for both dent and dent+. However, more steps270 do nevertheless require more computation: ten-step optimization takes 25.9× more operations than271 the static model (Table 3). As a plus, dent+ is not only more robust, but also more efficient in needing272 fewer steps. Note that the computational difference between dent and dent+ is negligible, as the273 adaptation parameters are such a small fraction of the model.274
Model adaptation updates depend on the attack type. Dent adapts by adjusting normalization275 statistics and affine transformation parameters. Dent can fix or update the normalization statistics276 (µ, σ) by using static training statistics (×) or dynamic testing statistics ( √ ); Dent can fix or update277
the affine parameters (γ, β) by not taking gradients (×) or applying gradient updates ( √
). Table 7278 compares each combination: affine updates always help, but both updates together hurt `2 robustness.279
Batch size We analyze dent’s sensitivity to batch size and focus on small batch sizes. Some real-world280 tasks, such as autonomous driving, naturally provide a small batch of inputs (from consecutive video281
frames or various cameras, for example), and so we confirm that dent can maintain robustness on282 such small batches. Table 8 varies batch sizes to check dent’s natural and adversarial accuracy.283
5 Discussion284
In advocating for dynamic defenses, we hope that test-time updates can help level the field for attacks285 and defenses. Our proposed defensive entropy method takes a first step by countering adversarial286 optimization with defensive optimization over the model and input. While more test-time computation287 is needed for the back-and-forth iteration of attacks and defenses, the cost of defense scales with the288 cost of attack, and some use cases may prefer slow and strong to fast and wrong.289
Limitations Dent depends on batches to adapt, especially for fully test-time defense without ad-290 versarial training. It also relies on a particular choice of model and input parameters. A different291 objective could possibly lessen its dependence on batch size and reliance on constrained updates.292 More generally, dynamic defenses may present difficulties for certification or deployment, as they293 could drift. Along with how to update, improved defenses could investigate when to reset, or how to294 batch inputs for joint optimization.295
Benchmarking Standardized benchmarking, by AutoAttack and RobustBench for example, drives296 progress by competition and empirical corroboration. Dent brings adversarial accuracy on their297 benchmark within 90% of natural accuracy for three of the most accurate methods tested [5, 61, 12].298 This is encouraging, but more research is needed to fully characterize dynamic defenses like dent.299 However, RobustBench is designed for static defenses, and disqualifies dent by its rule against300 test-time optimization. Continued progress could depend on a new benchmark to standardize rules301 for how attacks and defenses alike may adapt.302
By fighting gradients with gradients, dent shows the potential for dynamic defenses to update and303 counter adversarial attacks. The next steps—by attacks and defenses—will tell.304
References305 [1] Andreas, J., Rohrbach, M., Darrell, T., and Klein, D. Neural module networks. In CVPR, 2016.306
[2] Andriushchenko, M., Croce, F., Flammarion, N., and Hein, M. Square attack: a query-efficient black-box307 adversarial attack via random search. In ECCV, 2020.308
[3] Athalye, A., Carlini, N., and Wagner, D. Obfuscated gradients give a false sense of security: Circumventing309 defenses to adversarial examples. In ICML, 2018.310
[4] Bai, S., Kolter, J. Z., and Koltun, V. Deep equilibrium models. In NeurIPS, 2020.311
[5] Carmon, Y., Raghunathan, A., Schmidt, L., Duchi, J. C., and Liang, P. S. Unlabeled data improves312 adversarial robustness. In NeurIPS, 2019.313
[6] Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D. Neural ordinary differential equations. In314 NeurIPS, 2018.315
[7] Cohen, J., Rosenfeld, E., and Kolter, Z. Certified adversarial robustness via randomized smoothing. In316 ICML, 2019.317
[8] Croce, F. and Hein, M. Minimally distorted adversarial examples with a fast adaptive boundary attack. In318 ICML, 2020.319
[9] Croce, F. and Hein, M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-320 free attacks. In ICML, 2020.321
[10] Croce, F., Andriushchenko, M., Sehwag, V., Flammarion, N., Chiang, M., Mittal, P., and Hein, M.322 Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670, 2020.323
[11] Dhillon, G. S., Azizzadenesheli, K., Bernstein, J. D., Kossaifi, J., Khanna, A., Lipton, Z. C., and Anandku-324 mar, A. Stochastic activation pruning for robust adversarial defense. In ICLR, 2018.325
[12] Ding, G. W., Sharma, Y., Lui, K. Y. C., and Huang, R. Mma training: Direct input space margin326 maximization through adversarial training. In ICLR, 2020.327
[13] Ding, J., Ren, X., Luo, R., and Sun, X. An adaptive and momental bound method for stochastic learning.328 arXiv preprint arXiv:1910.12249, 2019.329
[14] Evans, D., Nguyen-Tuong, A., and Knight, J. Effectiveness of moving target defenses. In Moving target330 defense, pp. 29–48. Springer, 2011.331
[15] Gilmer, J., Ford, N., Carlini, N., and Cubuk, E. Adversarial examples are a natural consequence of test332 error in noise. In ICML, 2019.333
[16] Gomes, R., Krause, A., and Perona, P. Discriminative clustering by regularized information maximization.334 In NeurIPS, 2010.335
[17] Goodfellow, I. A research agenda: Dynamic models to defend against correlated attacks. arXiv preprint336 arXiv:1903.06293, 2019.337
[18] Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. arXiv338 preprint arXiv:1412.6572, 2014.339
[19] Graves, A. Adaptive computation time for recurrent neural networks. arXiv preprint arXiv:1603.08983,340 2016.341
[20] Guo, C., Rana, M., Cisse, M., and van der Maaten, L. Countering adversarial images using input342 transformations. In ICLR, 2018.343
[21] He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, 2016.344
[22] Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and345 perturbations. In ICLR, 2019.346
[23] Hill, M., Mitchell, J. C., and Zhu, S.-C. Stochastic security: Adversarial defense using long-run dynamics347 of energy-based models. In ICLR, 2021. URL https://openreview.net/forum?id=gwFTuzxJW0.348
[24] Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., Efros, A., and Darrell, T. Cycada:349 Cycle-consistent adversarial domain adaptation. In ICML, 2018.350
[25] Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal351 covariate shift. In ICML, 2015.352
[26] Kingma, D. and Ba, J. Adam: A method for stochastic optimization. In ICLR, 2015.353
[27] Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, University of354 Toronto, 2009.355
[28] Liang, J., Hu, D., and Feng, J. Do we really need to access the source data? source hypothesis transfer for356 unsupervised domain adaptation. In ICML, 2020.357
[29] Liang, J., Hu, D., Wang, Y., He, R., and Feng, J. Source data-absent unsupervised domain adaptation358 through hypothesis transfer and labeling transfer. arXiv preprint arXiv:2012.07297, 2020.359
[30] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant360 to adversarial attacks. In ICLR, 2018.361
[31] Olston, C., Fiedel, N., Gorovoy, K., Harmsen, J., Lao, L., Li, F., Rajashekhar, V., Ramesh, S., and Soyke, J.362 Tensorflow-serving: Flexible, high-performance ml serving. In NeurIPS Workshop, 2017.363
[32] Pang*, T., Xu*, K., and Zhu, J. Mixup inference: Better exploiting mixup to defend adversarial attacks. In364 ICLR, 2020.365
[33] Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A. Film: Visual reasoning with a general366 conditioning layer. In AAAI, 2018.367
[34] Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. Dataset shift in machine368 learning. The MIT Press, 2009.369
[35] Radosavovic, I., Johnson, J., Xie, S., Lo, W.-Y., and Dollár, P. On network design spaces for visual370 recognition. In ICCV, 2019.371
[36] Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., and Dollár, P. Designing network design spaces. In372 CVPR, 2020.373
[37] Raff, E., Sylvester, J., Forsyth, S., and McLean, M. Barrage of random transforms for adversarially robust374 defense. In CVPR, 2019.375
[38] Rice, L., Wong, E., and Kolter, Z. Overfitting in adversarially robust deep learning. In ICML, 2020.376
[39] Rony, J., Hafemann, L. G., Oliveira, L. S., Ayed, I. B., Sabourin, R., and Granger, E. Decoupling direction377 and norm for efficient gradient-based l2 adversarial attacks and defenses. In CVPR, 2019.378
[40] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A.,379 Bernstein, M., et al. ImageNet large scale visual recognition challenge. IJCV, 2015.380
[41] Saenko, K., Kulis, B., Fritz, M., and Darrell, T. Adapting visual category models to new domains. In381 ECCV, 2010.382
[42] Samangouei, P., Kabkab, M., and Chellappa, R. Defense-GAN: Protecting classifiers against adversarial383 attacks using generative models. In ICLR, 2018.384
[43] Schneider, S., Rusak, E., Eck, L., Bringmann, O., Brendel, W., and Bethge, M. Improving robustness385 against common corruptions by covariate shift adaptation. In NeurIPS, 2020.386
[44] Sehwag, V., Mahloujifar, S., Handina, T., Dai, S., Xiang, C., Chiang, M., and Mittal, P. Improving387 adversarial robustness using proxy distributions. arXiv preprint arXiv:2104.09425, 2021.388
[45] Shannon, C. A mathematical theory of communication. Bell system technical journal, 27, 1948.389
[46] Sharma, Y. and Chen, P.-Y. Attacking the madry defense model with l_1-based adversarial examples.390 arXiv preprint arXiv:1710.10733, 2017.391
[47] Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. Outrageously large392 neural networks: The sparsely-gated mixture-of-experts layer. In ICLR, 2017.393
[48] Shelhamer, E., Wang, D., and Darrell, T. Blurring the line between structure and learning to optimize and394 adapt receptive fields. arXiv preprint arXiv:1904.11487, 2019.395
[49] Shi, C., Holtz, C., and Mishne, G. Online adversarial purification based on self-supervised learning. In396 ICLR, 2021. URL https://openreview.net/forum?id=_i3ASPp12WS.397
[50] Song, Y., Kim, T., Nowozin, S., Ermon, S., and Kushman, N. Pixeldefend: Leveraging generative models398 to understand and defend against adversarial examples. In ICLR, 2018.399
[51] Stutz, D., Hein, M., and Schiele, B. Disentangling adversarial robustness and generalization. In CVPR,400 June 2019.401
[52] Su, D., Zhang, H., Chen, H., Yi, J., Chen, P.-Y., and Gao, Y. Is robustness the cost of accuracy?–a402 comprehensive study on the robustness of 18 deep image classification models. In ECCV, 2018.403
[53] Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A. A., and Hardt, M. Test-time training for out-of-distribution404 generalization. In ICML, 2020.405
[54] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing406 properties of neural networks. In ICLR, 2014.407
[55] Tramer, F. and Boneh, D. Adversarial training and robustness for multiple perturbations. In NeurIPS, 2019.408
[56] Tramer, F., Carlini, N., Brendel, W., and Madry, A. On adaptive attacks to adversarial example defenses.409 In NeurIPS, 2020.410
[57] Veit, A. and Belongie, S. Convolutional networks with adaptive inference graphs. In ECCV, 2018.411
[58] Wang, D., Shelhamer, E., Liu, S., Olshausen, B., and Darrell, T. Fully test-time adaptation by entropy412 minimization. In ICLR, 2021.413
[59] Wang, X., Yu, F., Dou, Z.-Y., Darrell, T., and Gonzalez, J. E. Skipnet: Learning dynamic routing in414 convolutional networks. In ECCV, 2018.415
[60] Wong, E., Rice, L., and Kolter, J. Z. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.416
[61] Wu, D., Xia, S.-T., and Wang, Y. Adversarial weight perturbation helps robust generalization. In NeurIPS,417 2020.418
[62] Yang, B., Bender, G., Le, Q. V., and Ngiam, J. Condconv: Conditionally parameterized convolutions for419 efficient inference. In NeurIPS, 2019.420
[63] Yuan, X., He, P., Zhu, Q., and Li, X. Adversarial examples: Attacks and defenses for deep learning.421 TNNLS, 2019.422
[64] Zagoruyko, S. and Komodakis, N. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.423
[65] Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., and Jordan, M. I. Theoretically principled trade-off424 between robustness and accuracy. In ICML, 2019.425
[66] Zhang, H., Chen, H., Xiao, C., Gowal, S., Stanforth, R., Li, B., Boning, D., and Hsieh, C.-J. Towards426 stable and efficient training of verifiably robust neural networks. In ICLR, 2020.427
Checklist428
1. For all authors...429
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s430 contributions and scope? [Yes] 1. The last move advantage is described in Section 3.2431 under “Discussion”. 2. Dent is unique in its adaptation of the model parameters during432 testing without altering training. Section 2 explains that generative and self-supervised433 defenses optimize the input, but require auxiliary models, and their parameters are fixed.434 3. Table 1 is our main result showing that dent improves state-of-the-art adversarial435 training defenses. 4. Adaptive attacks are explained and reported in Section 4.5.436
(b) Did you describe the limitations of your work? [Yes] Section 4.2 describes the threat437 model for our work and its requirement of test batches. This is discussed more in438 Section 5.439
(c) Did you discuss any potential negative societal impacts of your work? [No] Our project440 is about defense, not attack, and so it has less potential for direct negative impact.441
(d) Have you read the ethics review guidelines and ensured that your paper conforms to442 them? [Yes]443
2. If you are including theoretical results...444
(a) Did you state the full set of assumptions of all theoretical results? [N/A]445 (b) Did you include complete proofs of all theoretical results? [N/A]446
3. If you ran experiments...447
(a) Did you include the code, data, and instructions needed to reproduce the main experi-448 mental results (either in the supplemental material or as a URL)? [Yes] The experiment449 setup (Section 4.1) and code in the supplementary material specify experiment details450 and reproduce our main results for dent+.451
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were452 chosen)? [Yes] Our method alters testing, not training, but its test-time optimization453 details are given by the above.454
(c) Did you report error bars (e.g., with respect to the random seed after running experi-455 ments multiple times)? [No] We evaluate with the standard adversarial benchmarks or456 AutoAttack and RobustBench, which do not report variability in this way.457
(d) Did you include the total amount of compute and the type of resources used (e.g., type458 of GPUs, internal cluster, or cloud provider)? [No] Our project does not require exotic459 resources, but standard deep learning hardware. In particular, we use V100 and Titan460 Xp GPUs on local servers.461
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...462
(a) If your work uses existing assets, did you cite the creators? [Yes] Libraries, datasets,463 and models are all cited in Experiments (Section 4.464
(b) Did you mention the license of the assets? [No] The licenses are mentioned at the given465 references and URLs.466
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]467 Our code is included in the supplemental material.468
(d) Did you discuss whether and how consent was obtained from people whose data you’re469 using/curating? [N/A] We did not collect new data.470
(e) Did you discuss whether the data you are using/curating contains personally identifiable471 information or offensive content? [N/A] We did not collect new data.472
5. If you used crowdsourcing or conducted research with human subjects...473
(a) Did you include the full text of instructions given to participants and screenshots, if474 applicable? [N/A]475
(b) Did you describe any potential participant risks, with links to Institutional Review476 Board (IRB) approvals, if applicable? [N/A]477
(c) Did you include the estimated hourly wage paid to participants and the total amount478 spent on participant compensation? [N/A]479

Review summaries:
This paper proposed a new defense, namely, defensive entropy minimization (dent), to improve adversarially robustness. The proposed method alters testing but not the training phase, which is different from most existing defenses that are static and do not adapt during testing. The improvements of model robustness evaluated by different strong attacks and datasets have confirmed its effectiveness. 

This paper studies the problem of dynamic defenses agaisnt the potential adversarial attacks during the test time. The work introduces a new method derived from "defensive entroy minimization" approach that "fights back" and counter-optimize against attacks. In general, the proposed method updates the parameters of DNN slightly for every round of inferences, so as to balance the accuracy  and fuzziness of models. The basic idea was clear while experiment results demonstrated the potentials of proposed methods. During the experiments, authors aimed at establishing three major claims (1) Dynamic Defense of Adversarial Training, (2) Dynamic Defense of Nominal Training, and (3) Adaptive Attacks on Dent Updates, with sub-claims in details. Experiment results support these claims in part. Ablation studies also demonstrate the indispensibility of and contributed made made by every component.  

The paper proposes a novel test-time dynamic defense method, which adapts the model and input during testing by defensive entropy minimization (named dent). Given an input, the proposed method minimizes the entropy of the model prediction to adapt the input and updates parameters in batch normalization layers. Empirical results show that the proposed method improves the robustness of adversarially/nominally-trained models against various attacks including AutoAttack.