## 4 Results and Discussion
### Evaluation Metrics  
This study assesses the effectiveness of specific methods by utilizing precision (P), recall (R), and F-measure (F1) as metrics [42]. The classifier tree models used in this study differ in their training and testing stages, leading to the division of these metrics into three categories based on specific algorithms: micro-label, macro-averaging [11; 43], and micro-tree. These categories are used to evaluate the performance of the model during the training and testing stages. We compare these three types of metrics and use the micro-label and macro-averaging of the precision P, recall R, and F1 to determine the overall average performance of all classifiers in the classifier tree in each class. In contrast, we use the micro-tree of precision P, recall R, and F1 to observe the performance of the classifier tree in the classification results. Given a classifier \(i\in\mathbb{C}\) (\(\mathbb{C}\) is the classifier set for a classifier tree) and category \(j\in\mathcal{L}_{i}\) (\(\mathcal{L}_{i}\) is the category set of classifier \(i\)), let \(TP_{i}^{j}\), \(FP_{i}^{j}\), and \(FN_{i}^{j}\) be the values of true positives, false positives, and false negatives, respectively. Then, the \(P_{i}\), \(R_{i}\), and \(F1_{i}\) metrics for classifier \(i\) are defined as follows [11; 28]:  
\[ \begin{split} P_{i}&=\frac{\sum_{j\in\mathcal{L}_{i }}TP_{i}^{j}}{\sum_{j\in\mathcal{L}_{i}}TP_{i}^{j}+\sum_{j\in\mathcal{L}_{i}} FP_{i}^{j}},\\ R_{i}&=\frac{\sum_{j\in\mathcal{L}_{i}}TP_{i}^{j}}{ \sum_{j\in\mathcal{L}_{i}}TP_{i}^{j}+\sum_{j\in\mathcal{L}_{i}}FN_{i}^{j}},\\ F1_{i}&=\frac{2\times P_{i}\times R_{i}}{P_{i}+R_{i }}\end{split}  \tag{20}\]  
To combine the results measured by all classifiers in the classifier tree, we use the macro-averaging of the precision \(P_{macro}\), recall \(R_{macro}\), and \(F1_{macro}\)[11]:  
\[ \begin{split} P_{macro-train}&=\frac{\sum_{i=1}^{ \mathcal{L}_{i}}P_{i}}{n}\\ R_{macro-train}&=\frac{\sum_{i=1}^{\mathcal{L}_{i }}R_{i}}{n}\\ F1_{macro-train}&=\frac{\sum_{i=1}^{\mathcal{L}_{i }}F_{i}}{n}\end{split}  \tag{21}\] \begin{table}
\begin{tabular}{c c c c} \hline
**Ex.** & **Model** & **W-Embeddings** & **D-Embeddings** & **Transfer Learning** \\ \hline \multirow{4}{*}{Ex. I} & XLNet + HA + FC & XLNet & HA & / \\  & BERT + HA + FC & BERT & HA & / \\  & WWM + HA + FC & BERT-WWM & HA & / \\  & ERNIE3.0 + HA + FC & ERNIE3.0 & HA & / \\ \hline \multirow{4}{*}{Ex. II} & XLNet + HA + FC & XLNet & HA & / \\  & XLNet + FT + FC & XLNet & FT & / \\  & XLNet + BiLSTM + HA + FC & XLNet & Bi-LSTM + HA & / \\ \hline \multirow{4}{*}{Ex. III} & XLNet + HA + FC & XLNet & HA & / \\  & XLNet \(-\) HA \(-\) FC \(-\) TL & XLNet & HA & TL \\  & XLNet + BiLSTM + HA + FC & XLNet & BiLSTM + HA & / \\  & XLNet + BiLSTM + HA + FC + TL & XLNet & BiLSTM + HA & TL \\ \hline \multirow{4}{*}{Ex. IV} & XLNet + BiLSTM + HA + FC + TL & XLNet & BiLSTM + HA & TL \\  & XLNet + BiLSTM + HA + FC & XLNet & HA & / \\ \cline{1-1}  & WWM + HA + FC & BERT-WWM & HA & / \\ \cline{1-1}  & BERT + HA + FC & BERT & HA & / \\ \cline{1-1}  & ERNIE3.0 + HA + FC & ERNIE3.0 & HA & / \\ \cline{1-1}  & XLNet + FT + FC & XLNet & FT & / \\ \hline \end{tabular}
\end{table} 
Table 2: Baseline models in each experiment. Given a label \(k\in\mathrm{K}\) (K is the set of leaf nodes on the multi-label tree), let \(TP_{k}\), \(FP_{k}\), and \(FN_{k}\), be the values of true positives, false positives, and false negatives, respectively, in the test stage. Subsequently, the \(P_{k}\), \(R_{k}\), and \(F1_{k}\) metrics are defined as  
\[P_{k}=\frac{TP_{k}}{TP_{k}+FP_{k}},R_{k}=\frac{TP_{k}}{TP_{k}+FN_{k}},F1_{k}= \frac{2\times P_{k}\times R_{k}}{P_{k}+R_{k}} \tag{22}\]  
\[P_{macro-test}=\frac{\sum_{i=1}^{m}P_{i}}{m},R_{macro-test}=\frac{\sum_{i=1}^{ m}R_{i}}{m},F1_{macro-test}=\frac{\sum_{i=1}^{m}F1_{i}}{m} \tag{23}\]  
\[