# Text Classification Model Enhanced by Unlabeled Data for LaTeX Formula
## 4 Experiments and Discussion
### Experiments and Discussion
#### 4.3.2 Ablation Experiments  
Several related models are chosen for verifying the role of pretraining and pseudo-labeling. BERT, BERT with Task pretrained, BERT with Domain pretrained, and TP-BERT \begin{table}
\begin{tabular}{c c c} \hline \hline
**Class** & **Number** & **Example** \\ \hline Derivative & 415 & \(\frac{d\phi^{j}}{dt}=-\phi^{j}\frac{\partial\omega}{\partial\phi^{j}}\) \\  & & \(\frac{1}{8\pi}\partial_{\omega}\sum\omega_{ij}^{(2)}+\frac{1}{8\pi}\partial_{ \Omega}\sum\Omega_{ijkl}^{(3)}=\frac{1}{2}\) \\ \hline Integral & 946 & \(\delta(x,y)=\int\frac{d^{4}k}{(2\pi)^{4}}e^{k^{\prime}\
abla_{y}\mathcal{L}(x,y)}\) \\ \hline  & & \(S=\int x\Big{(}\frac{1}{2}\partial_{\mu}\phi\cdot\partial_{\mu}\phi+\frac{m^{ 2}}{\beta^{2}}\sum_{\alpha}^{\prime}n_{\alpha}e^{\beta_{\alpha}(\cdot)\cdot \phi}\Big{)}^{(2)}\) \\ \hline Series & 193 & \(\sum_{S_{p+1}}\theta_{\sigma(i_{0})}\theta_{\sigma(i_{1})}\cdots\theta_{ \sigma(i_{p})}=0\) \\ \hline  & & \(\sum_{\mu=1}^{m}F_{\mu}\big{(}v_{1}^{\prime},\cdots,v_{m-1}^{\prime}\big{)}=0\) \\ \hline Matrix & 305 & \(p^{0}=\begin{pmatrix}0&-i\\ i&0\end{pmatrix}\) \\  & & \(\begin{pmatrix}z^{\prime}\\ z^{\prime}\end{pmatrix}=\begin{pmatrix}u&0\\ 0&\bar{a}\end{pmatrix}\Big{(}\begin{matrix}2\\ \end{pmatrix}\) \\ \hline Others & 2834 & \(V_{\text{eff}}(r)=-\frac{15}{16}\frac{r^{4}}{r^{7}}+Q\Big{(}\frac{r^{6}}{r^{ 11}}\Big{)}\) \\ \hline  & & \(f(z,\xi)=\sum_{b}(\xi-k)ln\Big{(}\frac{c_{b}}{(a_{b}-c_{b}\xi)}\frac{[(a_{b}-c_ {b}\xi)\bar{c}]+(b_{b}-d_{b}\xi)]}{(c_{b}\bar{c}+d_{b})}\Big{)}+c^{\prime}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The ‘LaTeX formulas of advanced mathematics’ labeled data. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Class** & **Precision/\%** & **Recall/\%** & **F1/\%** \\ \hline Derivative & 86.36 & 95.00 & 90.47 \\ Integral & 95.83 & 88.46 & 92.00 \\ Series & 95.83 & 100.00 & 97.87 \\ Matrix & 93.75 & 100.00 & 96.77 \\ Others & 96.04 & 96.74 & 96.39 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results of UL-BERT. are the different pretrained BERT models for finetune and further self-training. The results are shown in Table 4. 'A-B' means B is used to obtain pseudo-labels, and A uses the obtained pseudo-labels for further learning. For example, BERT-CNN means a CNN is used to generate pseudo-labels for the training of BERT. BiLSTM, BiGRU, and CNN with GloVe [22] word vectors are chosen as classification models without pretrained. We use three kinds of models to obtain pseudo-labels: classification models (e.g., CNN and LSTM), generative models (e.g., Transformer), and the Bert based self-training model. The results are shown in Table 5.  
In Table 6, we fine-tuned MathBERT,TP-BERT and UL-BERT three times with different random seeds, and record the mean, with standard deviations as subscripts. In Tables 4 and 5, precision, recall and F1 scores of UL-BERT are the highest among all models, which illustrates the effectiveness of our approach to domain pretraining and self-training. * The role of BERT Comparing with traditional models, such as BiGRU and CNN with GoI \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Precision/\%** & **Recall/\%** & **F1/\%** \\ \hline MathBERT & 93.29\({}_{0.38}\) & 92.87\({}_{0.30}\) & 92.97\({}_{0.30}\) \\ TP-BERT & 94.26\({}_{0.44}\) & 94.10\({}_{0.53}\) & 94.14\({}_{0.50}\) \\ UL-BERT & 95.11\({}_{0.14}\) & 94.98\({}_{0.18}\) & 94.98\({}_{0.16}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results of different models, with standard deviations as subscripts.