# MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification
## 4 Experiments
### Comparison Method  
Due to factors such as scale, manual construction, and language differences, AG news is a large-scale English dataset without biased data, while the other datasets are small-scale biased Chinese datasets. Therefore, we use the state-of-the-art comparison methods of the current dataset for comparison.  
For the AG News dataset, we conducted experiments with different setting of bias, which included cases of extreme class imbalance and extreme label noise. For example, in Table 1, an imbalance factor of 200 means that the class with the highest number of samples has 200 times the number of samples compared to the class with the lowest number. The main comparison was made with two SOTA methods: [36] used meta-learning for sample reweighting (referred to as MW-Net), and [26] used meta-learning for label correction (referred to as MLC). The label correction model represented by MLC performs poorly on high class imbalance, so we conducted additional experiments under complete label noise settings. The values in Table 2 represent the average accuracy of 10 noise levels for each of the two bias construction methods (UNIFORM and FLIP); for example, UNIFORM noise level 0.2 and FLIP noise level 0.8 indicates whether to use UNIFORM or FLIP to construct noise, and 20% or 80% of sample labels are replaced with noisy labels. GLC [5] and MLC are the best models for handling label noise in weakly supervised label correction models.  
Result on the Chinese Benchmark Datasets  
We further evaluated our approach on the Chinese benchmark datasets, and the results are presented in Table 3. The experimental outcomes affirm that our methodology also yields commendable results on the Chinese dataset, thus underscoring its versatility and efficacy across diverse tasks.  
For other Chinese benchmark datasets, we also follow the work of previous researchers [42,43] and use their baseline for comparison, such as RoBERTa-I3 [39], Bert-wwm-ext [46] and RoBERTa-wwm-ext [46]. BERT-WWM is the first pre-training scheme designed for the Chinese language, which uses a whole word mask (WWM) strategy based on Chinese words. Compared to individual Chinese character-based masks, word-based masks enable the model to learn more semantic information. BERT-wwm-ext adopts the same model structure as BERT-wwm, consisting of 12 layers of bert layers. In fact, it is able to increase the training dataset and training steps to improve the effect. RoBERTa changed the pre-training strategy and removed the Next Sentence Prediction (NSP) task; it is trained with dynamic masking. Thus, RoBERTa-wwm-ext is a  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Bert-Base** & **MW-Net** & **GLC** & **MLC** & **MLRNet (Ours)** \\ \hline
**Accuracy(\%)** & 74.65 & 75.91 & 83.88 & 85.27 & **85.41** \\ \hline \hline \end{tabular} The best results are highlighted in **bold**.  
\end{table}
Table 2: Mean accuracies on AG News under two noise types and 10 noise levels.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **RoBERTa-I3** & **Bert-wwm-ext** & **RoBERTa-wwm-ext** & **MW-Net** & **MLRNet (Ours)** \\ \hline THNEWS & 55.43 & 56.84 & 56.94 & 57.03 & **59.30** \\ IFLYTEK & 58.67 & 59.43 & 60.31 & 59.74 & **61.45** \\ CIC & 84.45 & 84.90 & 84.70 & 84.69 & **85.35** \\ \hline \hline \end{tabular} The best results are highlighted in **bold**.  
\end{table}
Table 3: Performance comparisons on Chinese benchmark datasets.  
the WWM strategy and increases the training dataset and training steps. RoBERTa-I3 is a RoBERTa-wwm-ext model but with three Bert layers.  
In addition, MW-Net [36], as the state-of-the-art meta-learning reweighting method with strong universality and robustness, runs through all comparison methods.