Question: Are all methods trained to convergence?
Evidence: Datasets. We conduct experiments on multiple imbalanced training datasets artificially created from three benchmark image datasets CIFAR10, CIFAR100 and SVHN with diverse imbalanced distributions. Specifically, we consider two different imbalance types: Exponential (Exp) imbalance and Step imbalance. For Exp imbalance, the number of training examples of each class will be reduced according to an exponential function n = n i τ i , where i is the class index, n i is the number of training examples in the original training dataset for class i and τ ∈ (0, 1). We categorize half classes with most frequent example sizes in the imbalanced training dataset as well-represented classes and the remaining half classes as under-represented classes. For Step imbalance, we follow the similar process adopted in Section 2.1. Moreover, we denote imbalance ratio K as the ratio between training example sizes of the most frequent and least frequent class. We construct different imbalanced datasets "Step-10", "Step-100", "Exp-10" and "Exp-100", by adopting different imbalanced types (Step or Exp) with different imbalanced ratios (K = 10 or K = 100) to train models, and evaluate model's performance on the original uniformly distributed test datasets of CIFAR10, CIFAR100 and SVHN correspondingly.
More detailed information about imbalanced training sets can be found in Appendix A.5.
Baseline methods. We implement several representative and state-of-the-art imbalanced learning methods (or their combinations) into adversarial training as baseline methods. These methods include: (1) Focal loss (Focal); (2) LDAM loss (LDAM); (3) Class-balanced reweighting (CB-Reweight), where each example is reweighted proportionally by the inverse of, where a deferred reweighting training schedule is applied based on the CB-LDAM loss. In addition, we also include the original PGD adversarial training method using cross entropy loss (CE) in our experiments.
Our proposed methods. We evaluate three variants of our proposed SRAT method 3 with different implementations of the prediction loss L(•, •) in Eq. (), i.e., CE loss, Focal loss and LDAM loss. The variant utilizing CE loss is denoted as SRAT-CE, and, similarly, other two variants are denoted as SRAT-Focal and SRAT-LDAM, respectively. For all these three variants, Class-balanced method) is adopted to set weight values within the deferred reweighting training schedule.
Implementation details. All aforementioned methods are implemented using a Pytorch library DeepRobust. For CIFAR10/CIFAR100 based datasets, the adversarial examples used in training are calculated by PGD-10, with a perturbation budget ϵ = 8/255 and step size γ = 2/255; in evaluation, we report robust accuracy under l ∞ -norm 8/255 attacks generated by PGD-20 on Resnet-18 models. For SVHN based datasets, the settings are similar, excepts we set step size γ to 1/255 in both training and evaluation, as suggested in. We set the total training epochs to 200 and the initial learning rate to 0.1, and decay the learning rate at epoch 160 and 180 with the ratio 0.01. The deferred reweighting strategy will be applied starting from epoch 160.
We set the total training epochs to 200 and the initial learning rate to 0.1, and decay the learning rate at epoch 160 and 180 with the ratio 0.01. The deferred reweighting strategy will be applied starting from epoch 160.
Answer: Yes. Based on the ResNet18 architecture, we set 200 training epochs to train all methods with initial learning rate 0.1 and apply two times learning rate decay at epoch 160 and 180 with the ratio 0.01 to make sure the trained can be trained converge.