# SA-SGRU: Combining Improved Self-Attention and Skip-GRU for Text Classification
## 1 Introduction  
With the continuous development of the Internet, Internet text data is growing exponentially. Now it is an urgent problem to apply text classification technology to classify a huge number of diverse pairs of texts scientifically and effectively. As one of the most efficient natural language processing technologies, text classification has been widely used in the fields of public opinion analysis [1], text content retrieval [2], sentiment analysis of online comments [3], etc. While relevant research has attracted attention, more and more people are looking for ways to make insight discoveries and improve decision-making by utilizing the ability provided by machine learning big data. Big data can be defined as \"huge data sets beyond the storage, capture, management, or analysis capabilities of traditional database software\" [4]. The potential of this data depends on the ability to extract value from vast amounts of data through data analytics. Testing big data is challenging because of its diversity and complexity [5].  
Deep learning is a subfield of machine learning that enables performance improvements through data insight [6]. Machine learning algorithms are rarely challenged by big data in acquiring knowledge. Big data provides a large amount of data and information, which can be used by machine learning algorithms to extract patterns or build analysis models [7]. Methods based on deep neural networks, like the convolutional neural network (CNN) and the recurrent neural network (RNN), can extract deep-level features of text and have been widely known and used. The main task of the text classification method based on deep learning is to extract the features of the text and categorize the extracted text features through the neural network.  
In recent years, with the in-depth study of text classification algorithms by researchers, it has been proven that deep learning methods are superior to traditional machine learningmethods and have achieved good results in the field of natural language processing. However, these algorithms have some defects in the field of text classification, such as CNN's inability to tackle the long-distance dependence problem in the text information, and RNN always reads the entire text input, resulting in a lengthy input process for longer sequences and insensitivity to key pattern information in the text. Therefore, it is imperative to solve the problems inherent in the above methods.  
In this paper, a method called SA-SGRU is presented to address the shortcomings of the aforementioned methods. The SA-SGRU improves GRU and self-attention, making up for their shortcomings while effectively utilizing their advantages. In addition, the optimized CNN is effectively combined to extract richer text features and improve the performance of the model text classification.  
The rest of this paper is organized as follows: Section 2 introduces related work. Section 3 presents the proposed method. Section 4 describes the performance of the experiment using the proposed method. Section 5 discusses our contributions and shortcomings, as well as future work. Section 6 summarizes our research.