# GBSVM: Sentiment Classification from Unstructured Reviews Using Ensemble Classifier
## 4 Results and Discussions  
This section contains the details of the experimental results conducted in this research along with the discussion of the results. Experiments are conducted using SVM, GBM, RF and LR classifiers to classify sentiments into positive, negative and neutral classes. The train-test split is done as 70:30 for training and testing, respectively. Although the proposed GBSVM is constituted of GBM and SVM, these techniques are tested individually as well to analyze their performance. Consequently, four techniques including GBM, SVM, RF, and LR are analyzed for their efficacy against the proposed approach. For the experiments, we did not use the cross-validation. Although k-fold is more precise from a theoretical perspective, yet more computationally complex. In the light of the research [59] that states that cross-validation is the common exercise but sometimes it is better not to use it, we did not use it. TF and uni-, bi- and tri-gram variants of TF-IDF are used as features for classification. Results obtained using TF features are shown in Table 3.  
Experiment results demonstrate the proposed GBSVM performs well when used with TF features. The underlying reason is the combination of GBM and SVM where GBM works on weak learners to make it strong for prediction. The learning rate applied is 0.1 which fairly gives accurate results and SVM is used with linear kernel and so performs faster and more accurately on the categorical data. As the dataset contains a higher number of positive class instances so, the precision rate for the positive class is quite good. \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Classifier**} & \multirow{2}{*}{**Accuracy**} & \multicolumn{2}{c}{**Negative Class**} & \multicolumn{2}{c}{**Neutral Class**} & \multicolumn{2}{c}{**Positive Class**} \\ \cline{3-10}  & & **Precision** & **Recall** & **F1** & **Precision** & **Recall** & **F1** & **Precision** & **Recall** & **F1** \\ \hline GBSVM & 0.93 & 0.92 & 0.85 & 0.88 & 0.86 & 0.94 & 0.90 & 0.95 & 0.96 & 0.96 \\ GBM & 0.91 & 0.91 & 0.80 & 0.85 & 0.78 & 0.95 & 0.86 & 0.95 & 0.95 & 0.95 \\ SVM & 0.92 & 0.87 & 0.87 & 0.87 & 0.86 & 0.92 & 0.89 & 0.95 & 0.94 & 0.95 \\ LR & 0.92 & 0.91 & 0.84 & 0.87 & 0.84 & 0.90 & 0.87 & 0.95 & 0.96 & 0.95 \\ RF & 0.87 & 0.85 & 0.71 & 0.78 & 0.80 & 0.87 & 0.83 & 0.90 & 0.94 & 0.92 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Accuracy of classifiers using Term Frequency (TF) features. Similarly, GBSVM outperforms other machine learning classifiers when TF-IDF features are used for sentiment classification and gives an accuracy of 92%. Like before, precision results for the positive class are comparatively higher than that of negative and neutral classes primarily on account of the higher training data samples for positive class. F1 score which considers both precision and recall is also high for GBSVM than that of other classifiers. Results shown in Table 4 are for uni-gram TF-IDF, however, bi-gram TF-IDF has also been used and results are shown in Table 5.  
Results with bi-gram TF-IDF features reveal that the performance of all classifiers has been highly degraded. Theoretically, a higher-order n-gram model contains more information on a word's context which can lead to a model overfit. This happens when the data is sparse where we have a relatively large number of tokens but the frequency of the tokens is low. In such scenarios, a low order n-gram model can perform better than a high order n-gram model which is the case with the current dataset. This can be further corroborated from the results when TF-IDF tri-gram features are used for sentiment classification. Results with tri-gram features are shown in Table 6.  
As we can see the accuracy values obtained from different classifiers have been further decreased with tri-gram features. The most probable reason is the nature of the data that has been used during the training phase.