# Gradient Agreement Hinders the Memorization of Noisy Labels
## 4 Experiments
### Ablation Study  
#### 4.3.1 Ablation Study on Components  
GAL synchronously trains two networks with gradient agreement updating and uses pseudo-labels to supervise the learning of a third network. To evaluate the contribution of each part, we perform ablation studies in this section. The evaluation results are presented in Table 4. The experiment named \"\(\theta_{A}+\theta_{B}+\text{Cross Entropy}\)\" refers to training two nets using standard cross-entropy loss and using the average prediction results of the two models to calculate the accuracy. In the experiments named \"Random Updating\", \begin{table}
\begin{tabular}{l l c c c c c c c} \hline \hline
**Dataset** & \multicolumn{6}{c}{**CIFAR-10**} & \multicolumn{6}{c}{**CIFAR-100**} \\ \hline
**Method/Noise Ratio** & \multicolumn{1}{c}{**Sym 20\%**} & \multicolumn{1}{c}{**Sym 50\%**} & \multicolumn{1}{c}{**Sym 80\%**} & \multicolumn{1}{c}{**Asym 40\%**} & \multicolumn{1}{c}{**Sym 20\%**} & \multicolumn{1}{c}{**Sym 50\%**} & \multicolumn{1}{c}{**Sym 80\%**} \\ \hline \multirow{3}{*}{Cross-Entropy} & Best & 86.8 & 79.4 & 62.9 & 85.0 & 62.0 & 46.7 & 19.9 \\  & Last & 82.7 & 57.9 & 26.1 & 72.3 & 61.8 & 37.3 & 8.8 \\  & Best-Last & 4.1 & 21.5 & 36.8 & 12.7 & 0.2 & 9.4 & 11.1 \\ \hline \multirow{3}{*}{MentorNet [6]} & Best & 86.6 & 82.4 & 63.1 & - & 61.8 & 47.3 & 22.8 \\  & Last & 85.2 & 81.3 & 41.9 & - & 61.1 & 38.5 & 11.6 \\  & Best-Last & 1.4 & 1.1 & 21.2 & - & 0.7 & 8.8 & 11.2 \\ \hline \multirow{3}{*}{Co-teaching [8]} & Best & 87.9 & 83.5 & 64.4 & 87.1 & 63.4 & 50.3 & 25.6 \\  & Last & 86.5 & 81.9 & 44.2 & 85.2 & 62.8 & 48.5 & 13.2 \\  & Best-Last & 1.4 & 1.6 & 20.2 & 1.9 & 0.6 & 1.8 & 12.4 \\ \hline \multirow{3}{*}{Co-teaching+ [9]} & Best & 89.5 & 85.7 & 67.4 & 87.8 & 65.6 & 51.8 & 27.9 \\  & Last & 88.2 & 84.1 & 45.5 & 86.4 & 64.1 & 45.3 & 15.5 \\  & Best-Last & 1.3 & 1.6 & 21.9 & 1.4 & 1.5 & 6.5 & 12.4 \\ \hline \multirow{3}{*}{JoCoR [10]} & Best & 90.2 & 85.3 & 69.2 & 88.3 & 65.4 & 52.5 & 27.1 \\  & Last & 88.9 & 83.8 & 45.1 & 86.6 & 64.3 & 50.4 & 15.7 \\  & Best-Last & 1.1 & 1.5 & 24.1 & 1.7 & 1.1 & 2.1 & 11.4 \\ \hline \multirow{3}{*}{Co-learning [11]} & Best & 93.1 & 88.6 & 76.2 & 88.0 & 69.5 & 59.8 & 38.6 \\  & Last & 92.5 & 87.6 & 62.4 & 86.5 & 68.9 & 59.1 & 35.2 \\  & Best-Last & 0.6 & 1.0 & 13.8 & 1.5 & 0.6 & 0.7 & 3.4 \\ \hline \multirow{3}{*}{GAL} & Best & 93.7 & 90.5 & 78.9 & 91.6 & 70.5 & 61.3 & 40.8 \\  & Last & 93.5 & 90.1 & 78.3 & 91.2 & 70.1 & 60.8 & 40.2 \\ \cline{1-1}  & Best-Last & 0.2 & 0.4 & 0.6 & 0.4 & 0.4 & 0.5 & 0.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance comparison with previous methods. Models are trained on CIFAR-10 and CIFAR-100 with different ratios and types of label noise as PreAct-18 resnet being the backbone and tested on a clean testing set. The best test accuracy in the whole training process and the test accuracy of the last epoch are reported. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Method** & **CE** & **Decoupling** & **Co-Teaching** & **Co-Teaching+** & **JoCoR** & **Co-Learning** & **GAL** \\ \hline Best & 82.68 & 79.22 & 82.43 & 50.66 & 82.88 & 82.95 & 83.33 \\ Last & 81.1 & 78.24 & 81.52 & 48.52 & 81.06 & 81.18 & 82.91 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance comparison on Animal-10N. the network is warmed up with 80 epochs and then we randomly update a certain proportion of parameters in every iterations. In the experiments named \"\(\theta_{A}+\theta_{B}+\text{Random}\) Updating\", two nets are trained synchronously. After warming up, a certain proportion of parameter pairs in the two nets are randomly selected for updating. For the above two settings, 80%, 50%, 20% and 60% of parameters are randomly selected under 20%, 50%, 80% symmetric noise and 40% asymmetric noise respectively. As shown in Table 4, when applying gradient agreement updating to the two nets, the performances in different experimental settings sharply increase, manifesting its effectiveness.