## 1 Introduction  
Nowadays, people express their feelings and opinions to exchange their views using social media, such as Twitter, Facebook, Weibo, LinkedIn, and WeChat. Data gathered from these media has motivated researchers to explore opinion mining and public views. With massive amounts of user-generated text on social media, sentiment analysis (SA) has become an essential part of NLP with many applications, such as information storage and retrieval techniques, web grading, and many more. [1]. Text processing is necessary to remember the ultimate goal of analyzing it and unexpectedly extracting information. Although the amount of data in social media repositories increases exponentially, the traditional algorithms often fail to extract the sentiments from such big data. Researchers recently started to use deep learning (DL) approaches based on the distributed representation to deal with data specifications during training datasets, feature engineering, and other meeting problems on traditional techniques [2]. Studies show that CNN and RNN perform effectively in DL for several cases, especially in sentiment detection [3]. CNN convolutional applies in NLP to perform the feature mapping and pooling layer applied over the sequence dimension to obtain the fixed-length output. This process enables capturing local features but losing the context information [4]. However, it is also tricky for CNN networks to extract the long-distance features, and the pooling layer cannot capture word location information. RNN is the state-of-the-art algorithm for sequential data that can solve variable-length sequence input, make the feature representation more valuable, and convert to text vectors, forming a matrix consisting of feature vector and sequence dimensions [5]. However, RNN networks have problems with large-scale parallel computing [6]. Long short-term memory (LSTM) is a variant of RNN with a collection of gates to control the learning flow at each time step to capture long-range dependencies in long sentences [7]. Therefore, LSTM can address long-term information preservation and vanishing gradient text [8]. Compared with the LSTM model, the Bidirectional Long Short Term Memory(BiLSTM) model can significantly analyze a large amount of contextual information from context. The enhanced structure of BiLSTM and bidirectional gated recurrent units (BiGRU) [9] remains the original effect while making the network more straightforward. Furthermore, since [PERSON] et al. [10] introduced the Transformer-based method, text classification research has made great strides. To examine this model behavior and provide some human-understandable analysis, the weights of the attention mechanism inherent in these structures have often been considered. Similarly, [PERSON] et al. [11] proposed constructing an auxiliary sentence from the aspect and converting ABSA to a sentence-pair classification task that utilized the BERT transformation model. However, the attention RNN models are still generic to deal with both global and local features since the attention mechanism is computed to be determined based on the whole input sequence of the last RNN unit outputs. Meanwhile, fine-tuning strategies for the approaches directly used by the Transformer model remain a critical scientific challenge. This paper presents a novel attention-based CNN and Bi-LSTM neural network to locate an attention region covering the key sentiment words iteratively to address the problems. First, term frequency-inverse document frequency (TF-IDF) feature weighting and pre-trained Glove ( [[https://github.com/stanfordnlp/GloVe](https://github.com/stanfordnlp/GloVe)]([https://github.com/stanfordnlp/GloVe](https://github.com/stanfordnlp/GloVe)) accessed on 20 October 201), word embedding approaches extracted vital information from textual data. Next, CNN is used to learn high-level feature context from input representation. Simultaneously, the attention mechanism is applied at the CNN output layers to pay suitable attention to different documents.