# A Small-Sample Text Classification Model Based on Pseudo-Label Fusion Clustering Algorithm
## 3 Methodology
### EPiDA Data Augmentation  
In order to improve the classification effect of the method in this paper, we also focus on the enhancement of the labeled data. The augmentation method is represented by the expansion of a small amount of labeled data, and the quality and diversity of the expanded data significantly affect the performance of the model. We balance data diversity and quality by using the EPiDA framework to generate expanded data with a high level of synthesis.  
EPiDA consists of three components: data augmentation, sample evaluation and selection (REM + CEM), and classifier. The data augmentation module is selected from the currently used methods: back translation and context enhancement. Moreover, the sample evaluation and selection module and the classifier are used to evaluate the enhanced samples generated by the data augmentation methods comprehensively through mutual feedback. The relative entropy maximization (REM) is used to consider diversity, i.e., try to select those augmented samples that differ the most from the original samples; the conditional entropy minimization (CEM) considers quality, i.e., try to select augmented samples that can minimize the uncertainty of the model. the flow chart of EPiDA data augmentation is shown in Figure 3.  
Firstly, in order to generate diverse and rich samples, it is necessary to make the samples generated by data augmentation as different as possible from the original samples; therefore, for the augmented samples \(a_{i}^{j}\) loss should be maximum, and the objective function is  
\[\text{max}L_{R}\Big{(}w,\phi\Big{(}a_{i}^{j}\Big{)}\Big{)}=H(p(y_{i}))+\frac{1 }{hn}\sum_{i=1}^{n}\sum_{j=1}^{h}D\Big{(}p\Big{(}w^{T}\phi\Big{(}a_{i}^{j} \Big{)}\Big{)},p(y_{i})\Big{)}, \tag{7}\]  
Figure 3: The illustration of the EPiDA data augmentation module.  
where \(y_{i}\) is the true label; \(\phi:\mathbb{R}^{d}\rightarrow\mathbb{R}^{D}\) is a finite-dimensional feature mapping, which is not affected by data augmentation; \(w\in\mathbb{R}^{D}\) is a learnable parameter; \(H\) is the Shannon entropy, which is used to measure the information gain of the augmented samples; \(D\) is the relative entropy, which indicates the difference with \(p\Big{(}w^{T}\phi\Big{(}a_{i}^{j}\Big{)}\Big{)}\) and \(p(y_{i})\); and \(p\) is the probability distribution. Since \(p(y_{i})\) is the one-hot vector, \(H(p(y_{i}))=0\).  
According to Equation (7), we can understand that the difference between the original sample and the augmented sample is the largest when the relative entropy \(D(p(w^{T}\phi(a_{i}^{j}))\), \(p(y_{i}))\) is maximized so that the diversity of the augmented data can be achieved. Therefore, the diversity of the augmented samples is scored as  
\[s_{1}^{ij}=D\Big{(}p\Big{(}w^{T}\phi\Big{(}a_{i}^{j}\Big{)}\Big{)},p(y_{i}) \Big{)}. \tag{8}\]  
Secondly, in order to generate high-quality augmented data, the semantic deviation of the augmented samples from the original samples is constrained by minimizing the conditional entropy, and the objective function is  
\[\text{min}L_{\text{C}}\Big{(}w\phi\Big{(}a_{i}^{j}\Big{)}\Big{)}=H\Big{(}p \Big{(}w^{T}\phi\Big{(}a_{i}^{j}\Big{)}\Big{)}\Big{|}p\Big{(}w^{T}\phi\Big{(} x_{i}^{j}\Big{)}\Big{)}\Big{)}. \tag{9}\]  
Thus, the quality score of the augmented sample is  
\[s_{2}^{ij}=-H\Big{(}p\Big{(}w^{T}\phi\Big{(}a_{i}^{j}\Big{)}\Big{)}\Big{|}p \Big{(}w^{T}\phi\Big{(}x_{i}^{j}\Big{)}\Big{)}\Big{)}. \tag{10}\]  
Finally, taking into account the diversity and quality of the augmented samples, the final score is  
\[s_{\tau}^{ij}=s_{1}^{ij}+s_{2}^{ij}. \tag{11}\]  
The loss of the EPiDA data augmentation framework is  
\[L_{E}=L_{\text{C}}+L_{R}. \tag{12}\]  
Based on the final scores, the enhanced samples generated by back translation and context augmented are evaluated, and a reasonably augmented sample \(X_{a}\) is selected and merged with a small amount of unlabeled data \(X_{l}\) to realize the expansion of a small amount of labeled data.