## 6 Discussions  
The state-of-the-art machine learning and deep learning models coupled with word embedding techniques have been explored for the classification of citations into positive, negative, and objective (neutral). Each word embedding technique has been evaluated using standard evaluation measures, such as accuracy, precision, recall, and F1 score. Figure 4 presents the performance comparison of classifiers for sentiment analysis. Results reveal that the deep learning model, CNN, outperforms when trained on blended features.  
Figure 4a shows the accuracy comparison of models using fastText, fastText subword, and GloVe. It can be observed that CNN achieved the best results with every word embedding technique, as compared to the other models. On the other hand, LSTM has shown the highest accuracy result with combined features and the lowest accuracy with GloVe. Machine learning models and the voting classifier has also shown moderate performance results when trained on fastText. RF, LR, and SGD showed improved results when trained on fastText subword, and the lowest results when trained on GloVe.  
Figure 4b presents the performance comparison of models in terms of precision. As demonstrated in the results, CNN achieved the highest values of precision with each word embedding technique. The highest precision achieved by CNN is 94.24% when it is trained on GloVe. LSTM has shown the lowest value for precision, with 70% when trained on fastText, whereas LSTM achieved 84% precision with fastText subword. RF achieved the highest precision score of 81% with GloVe. SGD and LR showed similar values for precision with fastText and fastText subword.  
Figure 4c shows the recall comparison of models using fastText, fastText subword, GloVe embedding, and combined features. SGD, LR, and the voting classifier achieved the highest recall, of 96.18%, when used with combined features. Machine learning models achieved the highest score for recall when trained on fastText subword, with 87%. However, CNN achieved the highest recall of 89% when trained on GloVe. LSTM achieved a good score for recall with fastText subword, and achieved the lowest recall score with fastText and GloVe.  
Figure 4: Performance comparison of machine and deep learning models with different features, (**a**) accuracy, (**b**) precision, (**c**) recall, and (**d**) F1 score.  
Figure 4d manifests the F1 score comparison of models for citation sentiment analysis. Results reveal that the highest F1 score is obtained by CNN with 95% when used with the combined feature. For fastText and fastText subword, the F1 scores are 86% and 83%, respectively. LSTM achieved the lowest F1 score with fastText and GloVe. Machine learning models have shown almost similar results in terms of F1 score for the sentiment analysis of in-text citations.  
GloVe has around 840 billion crawl words, which is relatively higher than fastText and fastText subword embedding. However, GloVe also considers global stats about word occurrence and shows better performance as compared to word2 vec. However, GloVe did not perform well on citation sentiment analysis, as it only considers word co-occurrence. As a citation text is from multiple domains, fastText improves the result by considering parts of words by using n-grams. This ability makes fastText more generalized for unknown words. The above discussion illustrates that CNN performs better with word embedding as compared to other models. We choose CNN coupled with blended features and PCA as the best approach to perform sentiment analysis on citation texts. The dataset used to train models is imbalanced, with less negative and positive instances than those in the neutral class. CNN has convolutional layers that extract useful features from the text and can find the relationship among features. Word embedding represents text in the form of vectors. In this study, we explored the suitable word combination of classifiers and word embedding techniques to classify imbalanced datasets of citation texts.