# MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification
## 3 The Proposed Meta-Loss Reweighting Net Method  
### Optimization Objective in Meta-Learning  
Following the work of [36], we perform model learning on two sets of data: a large set containing biased data and a small set of clean and unbiased data. Due to the high cost of expert labeling, the clean and unbiased dataset is much smaller than the biased dataset.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Imbalance Ratio** & **10** & **20** & **50** & **100** & **200** & & **1** \\ \hline
**Noise Rate** & & & **0** & & **0** & **0.2** & **0.4** \\ \hline Bert-base & 90.58 & 89.33 & 85.21 & 81.18 & 69.01 & 93.44 & 92.19 & 90.13 \\ MW-Net & 91.48 & **90.20** & 87.79 & 84.58 & 80.37 & 93.44 & 93.27 & 91.23 \\ MLC & 91.91 & 88.75 & 78.63 & 75.41 & 73.87 & **94.57** & 93.70 & 91.76 \\ MLRNet(Ours) & **92.15** & 89.95 & **90.07** & **88.47** & **83.22** & 94.21 & **94.10** & **92.38** \\ \hline \hline \end{tabular} The best results are highlighted in **bold**.  
\end{table}
Table 1: Performance comparisons on AG News with varying noise rates and imbalance factors.  
Using the biased dataset as the entire training set is not optimal, since it contains a large amount of biased data. On the other hand, training solely on the clean dataset leads to severe overfitting due to the limited number of samples. To address this, we adopt the idea of meta-learning, where we use the small unbiased dataset as the meta-dataset to guide the parameter updates of both the classifier and the meta-network.  
Let us represent the training set containing biased data as \(D^{train}=\{x_{i},y_{i}\}_{i=1}^{N}\) and the meta-dataset as \(D^{meta}=\{x_{j},y_{j}\}_{j=1}^{M}\), where \(N\) and \(M\) represent the sample sizes of the training set and the meta-dataset, respectively, and \(N>>M\).  
We use \(X^{train}\) and \(Y^{train}\) to denote all the samples and label vectors in the training set \(D^{train}\), respectively. Similarly, \(X^{meta}\) and \(Y^{meta}\) represent all the samples and label vectors in the meta-dataset \(D^{meta}\). The classifier is denoted by \(F\), and its learnable parameters are represented as \(\omega\). The regular parameters can be obtained by minimizing the following loss function:  
\[\omega^{*}=arg\min_{\omega}\frac{1}{N}L\Big{(}Y^{train},F\big{(}X^{train}; \omega\big{)}\Big{)} \tag{1}\]  
Here, we use \(L^{train}(\omega)\) as shorthand for \(L(Y^{train},F(X^{train};\omega))\). However, due to the presence of biased data, we need to assign weights to sample loss values to adjust the impact of biased data and improve the model's robustness. We use \(V(l;\theta)\) to represent the weight net, where \(\theta\) represents its learnable parameters. Clearly, once \(\theta\) is set to be optimal, we can obtain the optimal \(w^{*}\) value, and Equation (1) can be written in the following weighted loss form:  
\[\omega^{*}=arg\min_{\omega}\frac{1}{N}V\Big{(}L^{train}(\omega); \theta\Big{)}L^{train}(\omega) \tag{2}\]  
The optimal \(\theta\) can be obtained by minimizing the following loss function:  
\[\theta^{*}=arg\min_{\theta}\frac{1}{M}L\big{(}Y^{meta},F\big{(}X^{meta}; \omega^{*}(V(\theta))\big{)}\big{)} \tag{3}\]