# GBSVM: Sentiment Classification from Unstructured Reviews Using Ensemble Classifier
## 3 Materials and Methods
### Proposed Approach
#### 3.2.2 Proposed GBSVM  
The proposed classifier is an ensemble model also called a meta-classifier that performs classification by combining the probability scores from different base models. The final predicted class is based on the aggregate results from the base models. The VC (GBSVM) presented in this study is the ensemble of two classifiers GBM and SVM which are used for the final prediction of the target class. GBM is an efficient algorithm that uses a boosting method that converts the weak learner into a strong learner and trains many models in a gradual and sequenced manner. GBM is the extended version of a decision tree algorithm, that does not work well because it generates random trees and one tree has no co-relation with the other, i.e., it builds random independent trees. While the trees in the GBM are built to eliminate the short-coming (error-residuals) of the previous weak tree. The SVM, on the contrary, is a linear model used to solve classification and regression problems. It deals with both linear and non-linear problems and works well for different applications [48]. SVM separates data into different classes by using a line or hyper-plane. SVM can perform complex types of classification based on the labels provided to the algorithm. A kernel is used in SVM that converts the low-dimension spaces into high-dimension spaces as it converts the problems into separate problems that are not separable. The final result from the VC can be based on either the hard voting or soft voting. Hard voting is a case of majority voting where the class prediction from each based model is considered. Here, the class label \(\widehat{Y}\) can be predicted by majority voting of each classifier \(C\). \[\widehat{Y}=mode[c_{1}(x),c_{2}(x),...,c_{m}(x)]. \tag{2}\]  
For example, if the predictions from \(c_{1}\), \(c_{2}\), and \(c_{3}\) are 'positive', 'negative', and 'positive', respectively, then the final prediction will be 'positive' by the majority vote. \[\widehat{Y}=mode[positive,negative,positive]=positive. \tag{3}\]  
Figure 4: Steps followed in the pre-processing of the data. The soft voting, on the other hand, considers the probability score from each classifier of a specific class that the current sample belongs to. At that point, soft voting criteria determine the class with the highest probability which it gets by averaging the individual values of the classifiers [49]. The proposed GBSVM takes the advantage of the advantages of both GBM and SBM and combines their predicted probability of a particular class to make the final decision. \(M_{GBM}\) and \(M_{SvM}\) are trained on the training data set and then used to predict the probability for positive, neutral and negative classes separately. Using the predicted probability from the two classifiers, an average probability for each class is computed for a given review. The decision function is then used to decide the final prediction/label of the review which is based on the highest average probability for a class. The working mechanism of the GBSVM is given in Algorithm 3. ```
1:input data \((x,y)_{i=1}^{N}\)
2:\(M_{GBM}\) = Trained_GBM
3:\(M_{SVM}\) = Trained_SVM
4:for\(i=1\) to \(M\)do
5:if\(M_{GBM}\
eq 0\) & \(M_{SVM}\
eq 0\) & \(training\_set\
eq 0\)then
6:\(ProbSVM-Pos=M_{SVM}\).\(probability(Pos-class)\)
7:\(ProbSVM-Neu=M_{SVM}\).\(probability(Neu-class)\)
8:\(ProbSVM-Neg=M_{SVM}\).\(probability(Neg-class)\)
9:\(ProbGBM-Pos=M_{GBM}\).\(probability(Pos-class)\)
10:\(ProbGBM-Neu=M_{GBM}\).\(probability(Neu-class)\)
11:\(ProbGBM-Neg=M_{GBM}\).\(probability(Neg-class)\)
12: Decision function = \(max(\frac{1}{N_{classifier}}\sum_{classifier}(Avg_{(ProbSVM-Pos,ProbGBM-Pos)}\), \(Avg_{(ProbSVM-Neu,ProbGBM-Neu)},Avg_{(ProbSVM-Neg,ProbGBM-Neg)}))\)
13:endif
14: Return final label \(\widehat{p}\)
15:endfor
```  
**Algorithm 3** Gradient Boosted Support Vector Machine (GBSVM). In this study, the soft voting technique is used.