## 4 Proposed Sentiment Analysis Methodology
### Feature Selection  
The feature selection stage aims to reduce the input parameters or features to predict the target values efficiently. In addition, some predictive models contain many variables that can affect the efficiency of the memory or can reduce the system performance due to the incompatibility between the input and the target features. Supervised and unsupervised methods are the main key features for predicting target features. The selection process in unsupervised methods removes redundant features and eliminates the target variable, while supervised methods focus on the target features by removing any insignificant input features.  
Other feature selection methods, such as wrapper and filter methods, can be applied by evaluating the model performance on the corpus data \(c_{d}\) and sentiment analysis documents. Regarding the wrapper method for feature selection, the input features are divided into different subsets. The method applies several models on the subsets to select the best model that achieves the highest performance. The filter method for feature selection applies several statistical techniques to estimate the relationship between input and target features. In addition, the filter method scores each resulting value between the input and the output features and then filters the best models based on the recorded scores.  
This paper applied the filter method for feature selection by dividing the input features into a different subset of feature vectors and then selecting the subsets that are highly associated or related to the target features. The correlation and selection of the subsets are applied using three feature importance methods: Pearson correlation, Chi\({}^{2}\), and Random Forest (RF).  
The Pearson method explores the correlation score between the input and target features, where the score ranges from \(-1\) to \(+1\). If the correlation score is close to \(+1\), then the relationship to the target is high and vice versa. Firstly, the covariance between each word term feature \(w_{t}\) and the target feature of the expected polarity term \(p_{t}\) is calculated, and then the Person correlation is measured by dividing the covariance value in Equation (5) by the multiplication of the standard deviation of both word term feature \(w_{t}\) and polarity term features \(p_{t}\) as explained in Equation (6).  
\[Cov\;(w_{t},\;p_{t})=\frac{1}{n}\times\sum_{i=1}^{n}((w_{ti}-\overline{w}_{t}) \times(p_{ti}-\overline{p}_{t})\;) \tag{5}\]  
where:  
\(w_{ti}\): each input word term feature in the vector.  
\(\overline{w}_{t}\): the mean of the overall word term features.  
\(p_{ti}\): each target polarity term feature.  
\(\overline{p}_{t}\): the mean of the overall polarity term features.  
\(n\): the length of the word terms and polarity terms.  
\[PC\;(w_{t},\;p_{t})=\frac{Cov\;(w_{t},\;p_{t})}{sd_{wt}\times sd_{pt}} \tag{6}\]  
where:  
\(sd_{wt}\): the standard deviation of word terms.  
\(sd_{pt}\): the standard deviation of polarity terms.  
The Chi\({}^{2}\) method for feature selection is used to measure the independence degree between the observed values of the word term features \(w_{t}\) and the expected values of the polarity term features \(p_{t}\). Therefore, the Chi\({}^{2}\) method selects the features of both word terms and polarity terms that are highly correlated as shown in Equation (7).  
\[Chi^{2}=\sum\frac{(w_{ti}-p_{ti})^{2}}{p_{ti}} \tag{7}\]  
The RF is considered a predictive method with high performance and low overfitting value. In addition, the RF combines both filter and wrapper methods and contains several decision trees. Each tree is based on a random extraction of the word term features \(w_{t}\) and a random extraction of the polarity term features \(p_{t}\). Each tree cannot trace all the word term features and polarity term features to reduce the overfitting.