# Long Text Truncation Algorithm Based on Label Embedding in Text Classification
###### Abstract
However, if a prompt template is added in front of the word that needs to be represented, for example, a prompt template of \"the brand of this electronic product is\" is added in front of \"apple\", then the word mapping of \"apple\" we obtain is obviously related to electronic products. By introducing a prompt template, the problem of \"polysemy\" will be subtly alleviated. For problem one, [PERSON] and [PERSON] proposed a general paradigm ULMFiT [19]. The ULMFiT paradigm mainly divided model training into three stages. The first stage trained the language model on a large number of general corpora, the second stage continued to train the language model on the specific domain data, and the third stage carried out specific tasks on the specific domain data. After the first stage, we can obtain a pretrained model, which made the convergence of the second stage much faster. In the second stage, domain-specific data were used as the pretraining object, so the model can quickly adapt to the characteristics of the target data, so as to fully learn the context information of the specific domain data. For problem two, [PERSON] and [PERSON] proposed a semi-supervised model: pattern-exploiting training (PET) [20]. By introducing a prompt template, PET reformulated the model input into cloze-style phrases, which can help the language model understand the given task. At the same time, PET added a prompt template before the user input, which was equivalent to limiting the semantic range corresponding to the user input in a prompt manner. This method enabled word embeddings to accurately represent the user input, thus effectively alleviating the problem of polysemy. Inspired by this, we propose the LTTA-LE algorithm. First, two prefix templates are constructed to prompt user input, which effectively limits the semantic range corresponding to user input. Then, an adaptive filtering threshold equation is designed, which dynamically determines the filtering amount of each long text. Next, the similarity between the label embedding and each segment in the long text is calculated, effectively filtering out invalid information in the long text. Finally, a three-stage model training architecture is introduced, which significantly improves the classification performance and generalization ability of the model. The main contributions of this paper are as follows. 1. **Unique prefix template design, which can alleviate the problem of polysemy. ** This paper constructs a pretraining prefix template and a label word mapping prefix template. By constructing the pretraining prefix template, the joint training of label words and long texts is realized. During the pretraining process, the long text and the label words are fully interacted to avoid the gap between them. With the help of the label word mapping prefix template, the label word embedding can more accurately describe the real semantics expressed by the user input, which effectively alleviates the problem of polysemy. 2. **More efficient long text noise filtering scheme. ** Considering that there are a large number of redundant sentence segments irrelevant to the topic in long texts, we can filter invalid information in long texts by calculating the similarity between the label embedding and each sentence segment in long texts. On the one hand, it can meet the limitation of the BERT model on the length of the input corpus. On the other hand, it can effectively filter the noise and improve the classification performance of the model. 3. **Adaptive filtering threshold equation. ** By designing an adaptive filtering threshold equation, the filtering quantity of each long text is dynamically determined, which avoids the problem of incomplete information filtering caused by too long text or serious information loss caused by too short text after filtering. 4. **A general and efficient three-stage model training scheme. ** This paper introduces a three-stage model training scheme, which mainly includes: in-domain pretraining, finetuning, and prediction.