# Stock Market Prediction Using Microblogging Sentiment Analysis and Machine Learning
## 4 Results
### Performance  
This section shows the outcomes of employing VADER and TextBlob as Twitter and StockTwits SA tools, respectively. Two measures were utilized to assess the proposed model: F-score and AUC. The F-score is a measure of a model's accuracy and predictability. AUC is defined as the area under the curve, and is formed by contrasting the true-positive rate against the false-positive rate. It describes the model's discriminating power, as well as its ability to predict whether the stock price will increase or decline [33]. According to Tables 6-9 we make the following observations. The F-scores of our ML models vary from 53.8% to 68.7% for the 'StockTwits with TextBlob' dataset, while the AUC areas range from 40% to 53.3% (Table 6). We are equally concerned about true positives and true negatives, since we want to anticipate both positive and negative increases in Microsoft's share value. As a result, we use the F-score and the AUC as assessment measures, which are profoundly different. SVM has the highest F-score of 68.7% and the highest AUC of 53.3%. Similarly, NB performed well, with an F-score of 66.7% and an AUC of 51%. Figure 8: Sentiment & Stock Change for ‘Twitter with VADER’ data set. Figure 7: Sentiment & Stock Change for ‘Twitter with TextBlob’ data set. According to Table 7 we observe that the F-score fluctuates from 45.5% to 68%, while the AUC ranges from 40% to 55%. SVM and LR achieve the best F-score, which is 68%. Their AUC is 55% and 54.7%, respectively. The next two best ML algorithms are NB and MLP, presenting an F-score equal to 66.7% and an AUC equal to 50%. In the case of the 'Twitter with TextBlob' dataset (Table 8), the F-score varies from 54.1% to 75%. The best F-score is 75%, achieved by SVM; the second best is 74.3%, achieved by MLP; and the third best is 72%, achieved by KNN and DT. In addition, AUC values can be as high as 68%. Their percentages vary from 44% to 68%. The best AUC values are observed in KNN and DT, reaching 68%. Finally, the F-score for the 'Twitter with VADER' dataset varies from 58% to 76.3% (Table 9). SVM has the greatest performance (76.3%), with LR coming in second with 70.1%. AUC, on the other hand, varies from 46% to 67%. SVM has the highest discriminating power of 67%, whereas LR has the second highest at 63.2%. \begin{table}
\begin{tabular}{c c c} \hline
**Model** & **F-Score (\%)** & **AUC (\%)** \\ \hline KNN & 53.8 & 40 \\ SVM & 68.7 & 53.3 \\ LR & 59.9 & 44.6 \\ NB & 66.7 & 51 \\ DT & 61.5 & 50 \\ RF & 61.5 & 50 \\ MLP & 56 & 45 \\ \hline \end{tabular}
\end{table} 
Table 6: Algorithmic performance for ‘StockTwits with TextBlob’ dataset. \begin{table}
\begin{tabular}{c c c} \hline
**Model** & **F-Score (\%)** & **AUC (\%)** \\ \hline KNN & 45.5 & 40 \\ SVM & 68 & 55 \\ LR & 68 & 54.7 \\ NB & 66.7 & 50 \\ DT & 52.2 & 45 \\ RF & 58.8 & 45 \\ MLP & 66.7 & 50 \\ \hline \end{tabular}
\end{table} 
Table 7: Algorithmic performance for ‘StockTwits with VADER’ dataset. \begin{table}
\begin{tabular}{c c c} \hline
**Model** & **F-Score (\%)** & **AUC (\%)** \\ \hline KNN & 72 & 68 \\ SVM & 75 & 44 \\ LR & 54.1 & 50 \\ NB & 70.3 & 50 \\ DT & 72 & 68 \\ RF & 69.6 & 64 \\ MLP & 74.3 & 50 \\ \hline \end{tabular}
\end{table} 
Table 8: Algorithmic performance for ‘Twitter with TextBlob’ dataset. Next, it should be noted that Tables 10 and 11 and Figure 9 refer to the average values of training/validation score, training/validation learning curve and training/validation time of all datasets ('StockTwits with TextBlob', 'StockTwits with VADER', 'Twitter with TextBlob' and 'Twitter with VADER') for all models (KNN, SVM, LR, NB, DT, RF and MLP) using F-score for training and AUC for validation. According to Table 10, SVM exhibits the best results, with a 72% average training score and a 54.83% validation score. Finally, according to Table 11, it is observed that the LR resulted in the quickest training (2.26 s) and validation times (0.45 s), followed by the SVM with 3.55 s for training and 0.89