## 2 Related Work  
Currently, text classification methods can be broadly classified into shallow learning methods and deep learning methods [10]. [PERSON] et al. [4] consider the characteristics of co-occurrence matrices that are easy to increment and can better cope with the content of open and changing domains by constructing a background network to learn different categories and combining multiple background networks into a classifier. However, shallow learning-based methods still require feature engineering and usually ignore the natural order structure or contextual information in text data, making it difficult to learn the semantic information of words. Deep learning models are a class of machine learning models based on neural network structures with multiple hidden layers, which perform excellently in natural language processing. [PERSON] et al. [5] used Word2 Vec [11] to characterize the text of goods. They generated HS code class center vectors through unsupervised clustering, extracted features of the filing elements during the prediction stage, and calculated the angular similarity with the class center vectors to determine the probability of the sample belonging to the class. The model based on the convolutional neural network (CNN) can capture the local features in the text through convolutional operation, which has a strong local perception ability [12]. However, the convolutional operation of CNN usually uses a fixed-size receptive field, which may lead to information loss or information redundancy when dealing with texts of different lengths. [PERSON] et al. [6] proposed a shallow structured convolutional neural network (SSCNN) with an auxiliary network to reduce the semantic fusion in merchandise classification effectively solves the problem of neighboring feature fusion and redundancy in the text of merchandise descriptions but does not consider the global associative relationships in the text. Long Short-Term Memory (LSTM) [13] effectively solves the recurrent neural network gradient vanishing and long-time dependency problems by introducing three gating mechanisms, which can effectively capture the sequential information and contextual relationships in the text, while the bidirectional Long Short-Term Memory Network (BiLSTM) deals with the sequence forward and reverse information, which can build a richer contextual representation. [PERSON] et al. [14] used the n-gram technique to convert natural text sentences into bag-of-words-like features and then fed these features into bidirectional LSTM, which can make better use of multi-scale feature representation and contextual information. [PERSON] et al. [7] constructed a graph network of declared elements through graph modeling techniques and used LSTM to learn sequence information from them, and used graph neural networks to fuse the two parts of the features for HS code classification, which improved the classification effect of the model. However, word embedding models such as Word2 vec can only generate static word vectors, which cannot capture the polysemy of words and the complex semantic relationships between words, limiting the classification performance of the model. To further optimize the model classification effect, many scholars have introduced the attention mechanism into the text classification task [15; 16; 17; 18; 19]. The core idea of the attention mechanism is to simulate the human attention mechanism, i.e., to be able to focus on certain key information in a targeted manner when processing a large amount of information. [PERSON] et al. [18] proposed the Transformer model with self-attention as the core, which demonstrated excellent performance in many natural language processing tasks.