###### Abstract
It involves taking a weighted average of all inputs, mapping them to a range between zero and one using softmax, and ensuring that the sum of all weights equals one. By assigning greater weights to important tokens, the model converges faster and exhibits better robustness. In recent years, with the integration of Self-Attention in the field of text sentiment analysis [18; 19; 20], related algorithms have achieved promising results in sentiment analysis tasks [21]. The performance of deep learning in sentiment analysis varies with the granularity of the input. When larger granularity sentences are used as input, the effectiveness is not as good as when more fine-grained inputs like word vectors or character vectors are used [22]. To deal with large volumes of text data, Google proposed Word2 Vec word embedding tools in 2013. In 2014, [PERSON] et al. [23] trained word embeddings using Word2 Vec to transform words in text into fixed dimensional word embeddings, which were then used as inputs for convolutional neural networks (CNNs). Various kernel sizes were employed to extract local features, effectively validating the usefulness of word embeddings. Meanwhile, the [PERSON] team [24] designed K-Max Pooling based on Max Pooling principles to extract the k highest-ranking features in each sliding process. This method has gradually been applied in various fields. In 2015, the [PERSON] team [25] proposed a solution to the lack of context information in CNN by incorporating semantic information from context. By combining the strengths of CNN and LSTM [26], this approach was applied to text sentiment analysis, yielding promising results. In order to enhance the ability to capture textual information, Dilated Convolution Networks [27] were subsequently applied to text sentiment analysis. In 2019, [PERSON] et al. [28] combined BiLSTM with the Attention mechanism, focusing on the hierarchical structure of text, and achieved good results across different datasets. In 2021, [PERSON] [29], using a multi-channel CNN-BiLSTM with attention for sentiment analysis in Chinese text, involved leveraging both the original contextual features and multi-scale higher-level contextual features in a joint architecture of CNN and RNN. This was achieved by incorporating different scales of high-level features in the multi-channel architecture, which allowed for capturing both the original contextual characteristics and the multi-scale higher-level contextual characteristics. [PERSON] [30] proposed DualGCN, which consists of two modules: SynGCN and SemGCN. SynGCN module was used as a generator, while SemGCN and Attention were combined to form a dual-channel model that simultaneously considers the complementarity between syntactic structures and semantic relationships. However, since the attention mechanism mainly focuses on the output layer, it fails to leverage the relationships between different parts of a sequence when dealing with inputs of mixed granularity. While these methods have achieved good results, there still remains a problem of underutilizing the correlations between different granularities when using only the BiLSTM model connected with attention at the character level. In addition, BERT has also achieved great success in text sentiment analysis [31]. BERT aims to pre-train deep bidirectional representations of unlabeled text by jointly conditioning on both left and right context across all layers. Therefore, fine-tuning the pre-trained BERT model with just an additional output layer allows for the creation of state-of-the-art models for various tasks without requiring extensive architectural modifications for specific tasks. [PERSON] et al. [32] provided a comprehensive comparison of different BERT models in the context of sentiment analysis. However, BERT-based models also face the issue of not being able to directly handle excessively long texts and can only process a maximum of 512 tokens consecutively.