# Re-Engineered Word Embeddings for Improved Document-Level Sentiment Analysis
## 5 Comparative Analysis  
In this section, we conduct a comparative evaluation of the proposed method with some related techniques. Representative results are listed in Table 2 (the results which were obtained using the proposed method are highlighted in bold). Discussions are also provided to point out the pros and cons of the related works, as well as emphasize the particular characteristics of the proposed system. In order to boost the classification performance for sentiment analysis, one intuitive strategy could be combining multiple approaches and fusing the results. Indeed, such a heterogeneous stacking ensemble framework was proposed and tested using a range of databases, and some promising results were reported in [26]. In their work, a series of popular word embedding techniques have been used to extract the features of the text separately; namely, Word2 Vec [17], GloVe [27], and BERT [28]. The resulting embeddings were subsequently fed into an ensemble of classifiers, including long short-term memory (LSTM) networks [29], gated recurrent unit (GRU) [30], and their variants. F1-score performances reported for Sentiment140 [24] and IMDB Review datasets are 0.8435 and 0.9222, respectively (Table 2). In [26], a computational overhead analysis was also reported, which indicated the existence of a trade-off between the classification performance and computational time. It is worth noting, however, that this system required each embedding vector to have 200+ dimensions in order to achieve the reported performance, which took about 40 to 60 min for model training for IMDB Reviews and Sentiment140, respectively. Compared to the proposed system in this work which generated vectors with lengths of less than 100 dimensions, the training and test in total took only a few seconds. A recent advance towards solving the five-star rating classification problem was reported by [PERSON] et al. [31], in which a chain of complex deep nets were employed to discriminate between the classes of the five-way database. They proposed to firstly use a pretrained BERT model for the feature vector extraction, then connected it with an LSTM network, further followed by a CNN. The ReLU activation function was used in the final stage of the model. Compared to Word2 Vec, BERT includes some particular advances over Word2 Vec, for example, it supports out-of-vocabulary words. The Amazon Fine Food Review dataset was used to test their algorithm and the F1-score of about 80% was reported [31] (Table 2). Such performance, however, was achieved using a model with a rather convoluted architecture, and the optimization of the model parameters would be costly [9]. Compared to the proposed method, no pretrained model is needed after the embedding re-engineering. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Dataset/Metric** & **Precision** & **Recall** & **F1-Score** \\ \hline \multirow{4}{*}{2-way} & IMDB Reviews & 91.65\% [26] & 92.79\% [26] & 0.9222 [26] \\  & & **97.50\%** & **97.50\%** & **0.9740** \\ \cline{2-5}  & Sentiment140 & 85.39\% [26] & 83.33\% [26] & 0.8435 [26] \\  & & **98.50\%** & **99.00\%** & **0.9867** \\ \hline \multirow{4}{*}{5-way} & Amazon Reviews & **96.53\%** & **96.46\%** & 0.8024 [31] \\  & & & & **0.9647** \\ \cline{1-1} \cline{2-5}  & \multirow{2}{*}{Yelp Reviews} & 5-way & 5-way & 0.7570 (3-way) [32] \\ \cline{1-1}  & & **93.93\%** & **94.96\%** & **0.9439** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparative analysis of the proposed method (shown in bold) and a few recent reports using the four adopted datasets. Note that some results were achieved using combined categories. The 5-way classification is considered a more challenging classification problem and its intermediate ranks are sometimes combined to form more distinctive categories. For example, the results reported in [32] (Table 2) were achieved by grouping ratings 1 and 2 as \"negative\" sentiments, ratings 4 and 5 as \"positive\" sentiments, and the rating 3 as \"neutral\", hence simplifying the classification problem.