## 2 Related Work
### Word Embeddings-Based Techniques and Deep Learning Models  
Recently, word embeddings-based vector representation techniques are playing an important role in natural language processing [28]. According to [PERSON] [29], research in word embedding feature selection gained momentum in 2013. The main word embeddings algorithms are Word2 Vec [15], Glove [16] and FastText [17; 30], which are used to convert words to vectors. Recently, bidirectional encoder representations from the transformers (BERT) model [18] has received much attention due to its bidirectional and attention mechanisms. Consequently, use of BERT embedding-based models outperforms other models, thus showing remarkable performance in sentiment analysis tasks [31; 32]. Word embeddings are better than the normal bag of words representation, since they cater for synonyms and produce vectors with lower dimensionality than the bag of words [14; 15]. [PERSON] [33] did research on word embeddings and established that Word2 Vec embeddings performed better than the other word-embedding algorithms. Currently, most researchers use pre-trained word embeddings vectors as inputs of machine learning classifiers in their sentiment analysis research since they are more accurate and compatible with deep learning neural networks [22]. However, pre-trained word embeddings ignore sentiment orientation of words and their context, hence affecting sentiment classification accuracy [14; 28]. This is because they use word distances and synonyms to calculate word vectors. [PERSON] [34] studied use of pre-trained Word2 Vec vectors as inputs to convolutional neural networks and improved their performance by hyper parameter tuning of the CNN model. [PERSON] et al. [35] used pre-trained Glove vectors as inputs for attention-based LSTM models for aspect-level sentiment analysis. [PERSON] et al. [21] used pre-trained Word2 Vec in idiom recommendation model in essay writing. [PERSON] et al. [36] used pre trained Word2 Vec model and improved them for cross-domain classification by extending the vector to include domain information. Recently [PERSON] and [PERSON] [37] used FastText pre-trained word embeddings and neural networks to classify Konkani texts. [PERSON] et al. [38] used BERT to integrate mental features and short text vector to improve topic classification and false detection in short text. Although their work showed better performance, they did not compare their proposal with other word embedding models. They also suggested more research to be performed on application of BERT in other contexts of text classification. [PERSON] et al. [31] did a study to compare Word2 Vec, Glove, FastText, and BERT. They demonstrated that transformer architectures, such as BERT models, are the state-of-the-art models for text representation and play a crucial role in sentiment analysis. The superiority of BERT is that it can read series of words in either direction, unlike other word embedding algorithms. Further, BERT employs the attention mechanism of the transformer that assigns a word its vector, depending on the surrounding words. This mechanism enhances the semantic representation of the target text. However, the series of input words to be read by the BERT algorithm maintains the entire words of the target text. We propose that the performance of BERT algorithm can be enhanced by focusing the input series to a few words, which contain sentiment information and their neighbours of the target text. This can be guided by utilization of sentiment lexicon and word N-grams. In a recent study [13], the researchers investigated a text representation technique using sentiment lexicon and N-grams where a Lexicon-pointed hybrid N-gram feature extraction model (LeNFEM) was proposed and investigated. A three-word N-gram was identified, which contains a sentiment word by use of a sentiment lexicon. The N-gram was then expanded to form a hybrid vector containing words, POS tags, and sentiments.