# Transformer-Based Graph Convolutional Network for Sentiment Analysis
## 2 Related Work  
### Sentiment Analysis  
The origin of sentiment analysis refers to the sciences of psychology, sociology and anthropology which focus on human emotions [19; 20; 21]. Scholars have conducted extensive related research because of its usefulness in online review monitoring and business competitive intelligence. To date, several methods have been used for such analysis. They can be classified into two broad groups: the traditional methods based on feature engineering, which essentially use dictionaries and machine learning approaches, and modern methods based on deep learning methods. Early models performed sentiment analysis based on a set of rules, relying on sets of emotion dictionaries, and a large amount of labeled data was required for feature engineering. [PERSON] et al. [22] defined emotion as a tuple of (holder, target, polarity, time) where holder represents the opinion's author, target refers to the related subject, polarity is the category of the expressed emotion, and time means the time of the evaluation. Another method by [23] classifies sentiments by combining the individual word-levelsentiment. [PERSON]. [24] introduced a generative model that jointly models emotion words, subject words and emotion polarity in a sentence as a triple. The main drawback of this method is the resulting high dimensional feature space. For addressing this problem, many works have used feature selection techniques [25; 26] applying various machine learning approaches. Of the various machine learning classification methods used to classify users' sentiments from a text, decision tree, LDA, Naive Bayes, Support Vector Machine (SVM), and artificial neural networks are the most common and have achieved a higher performance [9; 22; 27; 28]. However, these methods need massive training data and are often slow. To approach these problems, unsupervised lexicon-based methods were proposed, making use of both supervised and lexicon-based approaches [29; 30]. Following this idea, many other methods [31; 32] have been introduced. In recent years, many researchers have applied deep neural networks for classifying sentiment. Unlike traditional machine learning methods, they can automatically complete the feature generation step and learn more extensive representation. [PERSON]. [33] used a convolutional neural network (CNN) based model and connected a max-pooling layer after each convolution to extract features from the text. The emotion polarity is determined after inputting the fully connected layer. [PERSON]. [34] adopted a dynamic max-pooling to capture fine-gained features. The authors learn the embedding of text regions by applying CNNs to high-dimensional text data. Later, [PERSON]. [35] used the CNN model based on letter-level features, combining six convolutional layers and three fully connected layers for large-scale text classification datasets. Although CNN models are faster than RNNs because of parallelization, they can only extract the local features in the filter region. A memory unit is introduced with recurrent neural networks (RNNs) to make the network have memory ability. Hence, RNN can consider the long-distance dependency within texts. However, original RNNs suffer from gradient dispersion and gradient disappearance, which affect the learning process [3]. To solve this problem, the long short term memory (LSTM) model has been used [36]. LSTMs use a gate mechanism which can keep the connection within instances and capture the relationship between words. Recently, attention-based sentiment analysis models have been used and outperform previous methods. [PERSON] et al. [37] propose an attention-based model that mirrors the hierarchical structure of documents before applying two attention mechanism layers at the sentence and word level. More recently, graph neural networks (GNNs) have become a powerful approach for industries and academies. GNNs have been widely used in NLP tasks [38; 39; 40]. Ref.