text: 226,55,1217459930954981376,53464710,Eric Wong,"1/ New paper on an old topic: turns out, FGSM works as well as PGD for adversarial training!*  *Just avoid catastrophic overfitting, as seen in picture Paper: <LINK> Code: <LINK> Joint work with @_leslierice and @zicokolter to be at #ICLR2020 <LINK> 2/ Summary: Changing the initialization to be uniformly random is the main contributor towards successful FGSM adversarial training. Generated adversarial examples need to be able to actually span the entire threat model, but otherwise don't need to be that strong for training. 3/ Save your valuable time with cyclic learning rates and mixed precision! These techniques can train robust CIFAR10 and ImageNet in 6 min and 12 hrs using FGSM adv training. Super easy to incorporate (just add a 3-4 lines of code), and can accelerate any training method. 4/ Did you try FGSM before and it didn't work? It probably failed due to ""catastrophic overfitting"": plotting the learning curves reveals that, if done incorrectly, FGSM adv training learns a robust classifier, up until it suddenly and rapidly deteriorates within a single epoch.",https://arxiv.org/abs/2001.03994,"Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD). In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice. Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost. Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy to PGD attacks with $\epsilon=8/255$ in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at $\epsilon=2/255$ in 12 hours, in comparison to past work based on ""free"" adversarial training which took 10 and 50 hours to reach the same respective thresholds. Finally, we identify a failure mode referred to as ""catastrophic overfitting"" which may have caused previous attempts to use FGSM adversarial training to fail. All code for reproducing the experiments in this paper as well as pretrained model weights are at this https URL ",Fast is better than free: Revisiting adversarial training,4,"['1/ New paper on an old topic: turns out, FGSM works as well as PGD for adversarial training!* \n\n*Just avoid catastrophic overfitting, as seen in picture\n\nPaper: <LINK>\nCode: <LINK>\n\nJoint work with @_leslierice and @zicokolter to be at #ICLR2020 <LINK>', ""2/ Summary: Changing the initialization to be uniformly random is the main contributor towards successful FGSM adversarial training.\n\nGenerated adversarial examples need to be able to actually span the entire threat model, but otherwise don't need to be that strong for training."", '3/ Save your valuable time with cyclic learning rates and mixed precision! These techniques can train robust CIFAR10 and ImageNet in 6 min and 12 hrs using FGSM adv training.\n\nSuper easy to incorporate (just add a 3-4 lines of code), and can accelerate any training method.', '4/ Did you try FGSM before and it didn\'t work? It probably failed due to ""catastrophic overfitting"": plotting the learning curves reveals that, if done incorrectly, FGSM adv training learns a robust classifier, up until it suddenly and rapidly deteriorates within a single epoch.']",20,01,1081