# Long Text Truncation Algorithm Based on Label Embedding in Text Classification
## 3 Method
### Long Text Filtering Based on Similarity Calculation
#### 3.2.2 Sentence Vector Filtering and Text Reorganization
The LTTA-LE algorithm proposed by us makes use of sentence vector filtering and text reorganization technology to filter out a large number of redundant texts by calculating the similarity between text fragments and topics so as to improve the performance of the model. 3. For max_len = 32 or even smaller, since our experiment focuses on long text, the data we select are all long text data. When max_len is extremely small, only a few important sentence segments can be retained and part of the head and tail information is ignored, which results in slightly worse performance of the BERT-base + LTTA-LE (ours) than that of the BERT-base + head & tail truncated method. #### 4.3.4 Comparison of Different Label-Embedding Methods

The core improvement of the LTTA-LE algorithm is the introduction of a pretraining prefix template and a label word mapping prefix template. The pretraining prefix template is used to realize the interaction between long text and label words, and the label word mapping prefix template is used to limit the semantic scope of label words. Compared with other traditional label word embedding methods, the LTTA-LE algorithm can better utilize context information to extract the vector representation of label words, and it can effectively alleviate the problem of word polysemy. In order to verify the performance of the LTTA-LE algorithm, we conduct some experiments to compare the long text classification effects under different label word embedding algorithms in this section. The label word embedding methods compared are shown as follows. **One-hot embedding**: Use the one-hot embedding method to encode the label words, and filter the long text according to the label encoding vector to obtain shorter text data, which are then input into the model for training and prediction. **Word2 vec embedding**: Use the word2 vec method to vectorize label words and then filter long text data. **Pretraining without prefix**: A pretraining parameter package is used to encode the label word vector directly without introducing the prefix template. **LTTA-LE (ours)**: The LTTA-LE method proposed in this paper. The comparison experiments in this section will be performed on the Arxiv dataset with max_len = 256. Table 4 shows the performance of various label-embedding strategies on the Arxiv dataset. As shown in Table 4, LTTA-LE (ours) achieved the highest \(F_{1}\), which is improved by 3.65% (one-hot embedding), 1.35% (word2 vec embedding), and 1.14% (pretraining without prefix), respectively. The reasons for the differences in the scores of different label-embedding strategies on the Arxiv dataset can be summarized into the following four points. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Method** & **Precision** & **Recall** & \(\mathbf{F_{1}}\) \\ \hline One-hot embedding & 0.8201 \(\pm\) 0.0227 & 0.8012 \(\pm\) 0.0318 & 0.8105 \(\pm\) 0.0267 \\ Word2 vec embedding & 0.8471 \(\pm\) 0.0132 & 0.8116 \(\pm\) 0.0337 & 0.8289 \(\pm\) 0.0191 \\ Pretraining without prefix & 0.8489 \(\pm\) 0.0122 & 0.8131 \(\pm\) 0.0297 & 0.8306 \(\pm\) 0.0175 \\ LTTA-LE (ours) & **0.8512 \(\pm\) 0.0116** & **0.8292 \(\pm\) 0.0163** & **0.8401 \(\pm\) 0.0133** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Experimental results of different label-embedding methods on the Arxiv dataset (\(\pm\)indicates standard deviation). One-hot embedding uses a simple and independent one-hot vector to represent label word information, resulting in the loss of potential label information. Filtering and extracting long text data based on the results of one-hot label embedding will mistakenly miss important information. 2 Word2 vec embedding is a static word vector, which cannot be dynamically adjusted and optimized for specific datasets and tasks, so it cannot capture rich semantic information between label words and long texts. 3 Pretraining without a prefix can capture contextual semantic information to a certain extent.