## 3 Materials and Methods
### Switching Logit Weights for Improved Classifier Generalization  
The idea of switching logit weights stems from the fact that, in principle, training of a DNN can be categorized into two critical phases. [PERSON] et al. [57] in their study observe that the early stages of DNN training is a crucial time in the development of skills necessary for satisfactory classifier performance. They show that relative to the quality of input data, neural networks in the first few epochs develop strong connections that appear not to change even with additional training. Further, they outline two distinct phases in the training life-cycle of a DNN; first, the _memorization phase_ during which rapid growth of information about the data is experienced together with a significant increase in the strength of the connections, followed by the _compression_ or _forgetting phase_ phase during which redundant connections are eliminated and non-relevant variability in the data is discarded. Refs. [58; 59] both study the evolution of the loss landscape during optimization by analyzing the Hessian spectrum of DNNs. Their study reveals that the curvature of the loss landscape changes rapidly during the early phase of training and is later on reshaped such that a few large eigenvalues emerge with a majority of the remaining tending to zero while the negatives become very small. In addition, [PERSON] et al. [59] observe that after a short period of training, the gradient converges to a very small subspace spanned by the top \(k\) eigenvectors of the Hessian where \(k\) is the number of classes in the dataset. Two phases of training emerge from [59]; the early phase when the loss landscape around the network state appears to change rapidly with the Hessian splitting slowly into two varying subspaces, and the second phase when learning appears to concentrate in a small subspace with all [PERSON] et al. [60] uncover three sub-phases, the early phase where gradient magnitudes are anomalously large and motion is rapid followed by the second phase where gradients overshoot to smaller magnitudes before leveling off while performance increases rapidly, and lastly, the third phase where learning slowly begins to decelerate. Following the categorization put forward in [57], two critical phases from the training life-cycle of a DNN emerge. We propose the use of two distinct types of weight vectors during training, one for each phase to help achieve improved classifier generalization over the entire training dataset. The first phase of training is the _memorization phase_, which is characterized by the rapid growth of information about the data and an increase in the formation of strong network connections. From the observations we made in Section 3.4, the _ideal logit weight_ vector \(\mathbf{\tilde{q}}\) allows us to condition the DNN to focus on the minority samples. This type of conditioning, if applied at this crucial learning stage of the DNN, ensures that most of the information captured and connections formed are relevant to our samples of interest from the minority. The second phase of training, the _compression phase_, is characterized by the elimination of the redundant connections and discarding of non-relevant variability in data. We observe that the benefits attained in this phase are useful to all the classes in the dataset. On the other hand, the _ideal logit weight_ vector \(\mathbf{\tilde{q}}\) as composed gives an advantage only to samples from the minority classes during training. To transform this training strategy into one that provides equal opportunity across all classes, we switch to \(\bar{\mathbf{q}}\), a _uniform logit weight_ vector modeled after a uniform distribution, hence allocating equal weighting to all the classes involved. For the remainder of the training cycle, we use the _uniform logit weight_ vector \(\bar{\mathbf{q}}\).