# A New Sentiment-Enhanced Word Embedding Method for Sentiment Analysis
## 2 Related Work
### Sentiment Enhancement  
In recent times, some scholars have used the sentiments as external knowledge. [PERSON] et al. [44] proposed a sentiment-enhanced neural graph recommender. This model applied an attention mechanism [30] to extract a representation of a user's review and an item review representation. They further extracted another user representation and item representation based on a graph convolutional network (GCN) [45]. The model obtained the final user and item representations by concatenating the two user representations and item representations. Then, the model adopted a fully connected network to extract an auxiliary sentiment representation from the output of the self-attention network. Finally, they trained their sentiment-enhanced neural graph recommender with a loss between sentiment auxiliary representation and user-item representation. Their experiments achieved the best performance on Toys, Kindle, Yelp2017, and Yelp2018 datasets. However, this model only adopted sentiment as an additional feature to implement a recommendation system. It did not enhance word embedding with sentiments. [PERSON] et al. [46; 47] proposed an unsupervised system with sentiment propagation across dependencies (USS-PAD) model [48]. Their method found the relationship between each emoj's description and sentiment orientation. The classification results of their model are close to those of manual annotations. [PERSON] et al. [15] used k-nearest neighbors (KNN) to zoom in on positive samples and pushed out negative samples in a sentiment lexicon. Their refinement method outperformed original baseline models on SemEval-2013 and SST-2 datasets. [PERSON] et al. [49] used sentiment lexicon as external sentiment knowledge and injected this knowledge into the bidirectional long short-term memory (BiLSTM) to analyze implicit sentiment. They achieved the best results on SMP2019-ECISA, COAE (2015), and SemEval (2013-2017) datasets. [PERSON] et al. [6] injected sentiment lexicon and labeled sentiment corpora as the pre-training corpora for two bidirectional gate recurrent unit (BiGRU) layers. They pre-trained their model with three objectives: predicting target words, predicting word sentiment, and predicting sentence sentiment. After pre-training, they concatenated word embeddings and the hidden states of two BiGRUs, and obtained the sentiment-enhanced word embedding. The experimental results show that their model achieved comparable results to contextualized word embedding models (such as BERT) on three datasets with only 1-7 M parameters. However, these models considered sentiment an additional feature or combined other features to analyze sentiment, but fewer scholars injected sentiment information into word embeddings.  
In the field of injecting sentiment into word embedding, Sent2 Vec [33] used SIF [34] or uSIF [35] to encode a sentence into a vector. The vector is further input into a fully connected network to learn sentimental embedding. This method achieved better results than other word embeddings on Dbpedia and Yahoo datasets. [PERSON] and [PERSON] [14] proposed Approaches 1 and 2 for enhancing sentiment. Specifically, Approach 1 adopted an MLP to find the mapping relationships between words and their sentiment intensity; and Approach 2 trained Word2 Vec [1] and GloVe [2] with an additional sentiment vector if a target word is a sentiment word. The two methods achieved better results than Word2 Vec and GloVe on SemEval-2013 and SST datasets. Therefore, few studies are focused on injecting sentiment information into word vectors.