# MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification
## 4 Experiments
### Experiments Results and Analysis
#### 4.6.2 Ablation Study and Analysis  
In order to explore the versatility of MLRNet across different backbone networks, we substituted the backbone network with Bert-base-3l, which comprises only three layers of attention layers, and conducted further experiments on the AG News dataset. We configured imbalance factors for bias data types as [1, 10, 20], employed label noise of the UNIFORM type, and varied noise levels within the range [0.0, 0.2, 0.4, 0.6]. Building upon these settings, we conducted a detailed investigation into our methodology to ascertain its adaptability to diverse bias scenarios. The outcomes of these experiments are presented in Table 4.  
**Effects of the number of Meta-Samples.** Given the inherent errors in data collection and the substantial cost associated with expertly annotating unbiased datasets, we systematically varied the quantity of meta-samples during training to further validate the robustness of our method. As delineated in Table 5, our approach demonstrates noteworthy performance, even when the number of meta-samples per class is as low as 10, resulting in a notable improvement of 1.24% (imbalance factor set at 20, noise level at 0.4, and DF feature dimensionality at 128). Interestingly, as the quantity of meta-samples per class gradually increases, this improvement becomes even more pronounced. It is worth noting, however, that when the number of meta-samples per class reaches 500, there is a decline in accuracy,  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Imbalance-Ratio** & \multicolumn{3}{c}{**10**} & \multicolumn{3}{c}{**20**} \\ \hline
**Noise-Rate** & **0.2** & **0.4** & **0.6** & **0.2** & **0.4** & **0.6** \\ \hline Bert-base & 89.41 & 88.77 & 87.43 & 87.53 & 86.18 & 84.88 \\ MW-Net & 90.26 & 89.62 & 88.36 & 88.65 & 88.12 & 86.71 \\ MLRNet (Ours) & **91.23** & **90.18** & **88.82** & **89.77** & **89.71** & **88.39** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Accuracy of our model with Bert-base-I3 on AG News.  
as illustrated in Figure 4. We attribute this phenomenon to the fact that an excessively large meta-dataset exacerbates the bias within the training set, ultimately leading to a decrease in accuracy. Hence, we emphasize the importance of an appropriately sized meta-dataset in achieving optimal results.  
**Effects of discriminative feature dimensionality.** Given that the dimensionality of discriminative features directly impacts the computational workload and training speed of the meta-model, we conducted experiments by adjusting the dimensionality of discriminative feature embeddings. The results are presented in Table 6 (imbalance factor set at 20, noise level at 0.4, and 200 meta-samples per class). Concurrently, as evident from Figure 5, it is apparent that with an increase in DF dimensionality, the average accuracy steadily improves, underscoring the pivotal role of discriminative features. However, the escalation in DF dimensionality also leads to a substantial increase in model parameters, resulting in extended training times and heightened training costs. Consequently, taking into account factors both performance and training time, we opted for a DF dimensionality of 128.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**DF Dimensionality** & **32** & **64** & **128** & **256** & **768** \\ \hline
**Accuracy (\%)** & 88.50 & 88.36 & 89.71 & 90.04 & **90.35** \\ \hline \hline \end{tabular}  
* The best results are highlighted in **bold**.  
\end{table}
Table 6: Accuracy under varying dimensionality of DF on AG News.  
Figure 4: Impact of meta-samples per-class.  
Figure 5: Impact of DF dimensionality.