# Long Text Truncation Algorithm Based on Label Embedding in Text Classification
## 3 Method
### Long Text Filtering Based on Similarity Calculation
#### 3.2.2 Sentence Vector Filtering and Text Reorganization
#### 4.1.2 20 NewsGroup

20 NewsGroup includes 18,000 news text documents covering 20 different news text categories. Since we pay more attention to the performance of the model on the long text dataset, we filter 8000 long text data from it, and the ratio of training set and test set in the dataset is 9:1. #### 4.1.3 IMDB Review

IMDB Review is a well-known movie evaluation dataset, which contains a total of 50,000 annotated text data. The dataset has two types of labels: positive and negative. We filter 20,000 long texts with annotations. The ratio of the training set and test set in the dataset is 9:1. ### Experimental Details

We have completed the key part of the code development with the help of the transformer toolkit. The hyperparameter settings of the model are shown in Table 1. Some symbols and setting are explained as follows:

* Epoch: one epoch refers to training once with all the samples in the training set. * Batch_size: the number of samples grabbed for one training iteration. * Learning rate: the rate at which the model learns the parameters. * Model optimization: Adam optimizer [31]. In addition, in order to minimize the interference caused by randomness, five-fold cross-validation is used for model training and performance evaluation. ### Model Performance Evaluation

#### 4.3.1 Evaluation Metrics

We utilize three evaluation metrics, precision, recall, and \(F_{1}\), to measure the performance of long text classification methods. Among them, precision refers to the proportion of correctly predicted positive samples to all predicted positive samples. Recall represents the proportion of correctly predicted positive samples to all actual positive samples. \(F_{1}\) is the harmonic mean of precision and recall, which can take into account both precision and recall. Furthermore, they can be described by the following equation. \[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}, \tag{5}\]

\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}, \tag{6}\]

\[F_{1}=\frac{2\times\text{Precision}\times\text{Recall}}{\text{Precision}+ \text{Recall}}. \tag{7}\]

where TP = true positive, FP = false positive, TN = true negative, and FN = false negative. #### 4.3.2 Experimental Results

In order to verify the excellent performance of the LTTA-LE algorithm, this section will compare different algorithms on three public datasets, setting the hyperparameter of max_len as 256. An overview of each model approach involved in the comparison is shown below. **BERT-base + head truncation**: Keep sentence segments of max_len length from the beginning of the text. **BERT-base + tail truncation**: Keep sentence segments of max_len length from the end of the text. **BERT-base + head & tail truncation**: Keep a part at the beginning and end of the text; in this experiment, keep half at the beginning and end of the text. **BERT-base + max pooling**: The long text is first segmented into multiple short sentence segments using the same method as LTTA-LE, and then, the sentence vector information for each short sentence segment is extracted based on the BERT-based model. Finally, max pooling is applied to the obtained [CLS] modules. After splicing, the text classification results are obtained through a layer of a fully connected network. **CogLTX**: Identify key sentences by training a judgment model, concatenate key sentences for reasoning, and enable multi-step reasoning via rehearsal and decay [32]. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Stage** & **Epoch** & **Batch_Size** & **Learning Rate** \\ \hline Pretraining & 50 & 64 & \(2\times 10^{-5}\) \\ Finetuning & 10 & 32 & \(2\times 10^{-5}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Hyperparameter settings for our proposed model. **BERT-base + LTTA-LE (ours)**: The LTTA-LE method proposed in this paper. Table 2 shows the performance of different methods on three public datasets, and the last three columns of Table 2 show the average scores of precision, recall, and \(F_{1}\) on the three datasets.