## 5 Experimental Results
### ANN
#### 5.5.3 Results  
We have also performed a DM-test to verify that the forecasting performance of SVM within the intervals denoted by the rules is superior to the outcome of the same method for random intervals. We used the dataset of both sentiments and tweets to conduct the evaluations for all stock prices. Again, the null hypothesis was considered to be that the two forecasting models (rule vs. random intervals) have equal accuracy.  
Table 7 presents the \(p\)-value and DM-statistic metrics. Recall that when the \(p\)-value is close to zero, the null hypothesis is rejected. Furthermore, negative values of the DM-statistic depict that the squared errors of the model listed first (the rule-based intervals) are lower than those of the model listed last.  
As seen from that table, for all companies, \(p\)-values are close to zero and the DM-statistic is negative, denoting that not only the null hypothesis does not hold, but the intervals identified by the rules have lower squared errors.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Sentiment and Tweets** & **Sentiment Only** & **Tweets Only** \\ \hline Random Interval \#5 & 0.30 & 14.56 & 14.56 \\ Random Interval \#6 & 17.86 & 3.97 & 17.86 \\ Random Interval \#7 & 40.63 & 42.07 & 42.71 \\ \hline
**MSFT SVM** & & & \\ \hline Random Interval \#1 & 21.59 & 33.89 & 7.25 \\ Random Interval \#2 & 59.45 & 54.10 & 50.23 \\ Random Interval \#3 & 20.03 & 57.34 & 0.00 \\ Random Interval \#4 & 39.64 & 51.13 & 27.63 \\ Random Interval \#5 & 45.47 & 34.48 & 4.25 \\ Random Interval \#6 & 22.45 & 22.54 & 10.11 \\ Random Interval \#7 & 91.55 & 67.40 & 92.20 \\ \hline
**ORCL Linear** & & & \\ \hline Random Interval \#1 & 67.89 & 69.39 & 69.39 \\ Random Interval \#2 & 0.00 & 13.29 & 0.00 \\ Random Interval \#3 & 0.00 & 0.00 & 0.00 \\ Random Interval \#4 & 0.00 & 4.77 & 4.77 \\ Random Interval \#5 & 58.34 & 58.34 & 58.34 \\ Random Interval \#6 & 0.00 & 0.00 & 0.00 \\ Random Interval \#7 & 14.63 & 30.23 & 30.02 \\ \hline
**ORCL SVM** & & & \\ \hline Random Interval \#1 & 0.00 & 19.61 & 0.00 \\ Random Interval \#2 & 60.23 & 56.87 & 20.32 \\ Random Interval \#3 & 50.80 & 0.00 & 28.25 \\ Random Interval \#4 & 60.24 & 62.06 & 38.49 \\ Random Interval \#5 & 76.92 & 73.75 & 18.81 \\ Random Interval \#6 & 0.00 & 0.00 & 0.00 \\ Random Interval \#7 & 54.75 & 49.19 & 52.59 \\ \hline \hline \end{tabular}
\end{table}
Table 6: _Cont._The fact that rules have been found in which the sentiment score, the tweets or both can improve the prediction is a very encouraging result for further future study.  
In Figures 13-15, all texts have been clustered in topics, using the LDA algorithm. The latter could help to improve our method by incorporating a better filtering of news data by using the topic of each text. In other words, we could choose the texts that are more relevant to the stock market, based on the results of the topic modeling.  
\begin{table}
\begin{tabular}{l l l} \hline \hline  & \multicolumn{2}{l}{**Null Hypothesis: Both**} \\  & \multicolumn{2}{l}{_Forecasts Have the Same Accuracy_} \\ \hline \multirow{2}{*}{**Company Index**} & **Rules vs. Random Intervals** & \\ \cline{2-3}  & \(p\)**-Value** & **DM Statistic** \\ \hline AAPL (116.29 \$) & \(0.1107\) & \(-1.6839\) \\ \hline GE (29.23 \$) & \(0.149\) & \(-1.5968\) \\ \hline IBM (142.25 \$) & \(0.081\) & \(-2.2125\) \\ \hline MSFT (50.81 \$) & \(0.09359\) & \(-2.4001\) \\ \hline ORCL (37.89 \$) & \(0.1367\) & \(-1.5357\) \\ \hline \hline \end{tabular}
\end{table}
Table 7: DM-test between two forecasting models, based on SVM and using sentiments and tweets as additional features. The leftmost model is the one considering the intervals denoted by rules, while the rightmost represents the model of random intervals.  
Figure 13: Topic modeling for AAPL and GE stocks. The \(y\)-axis represents the number of texts in each topic, and the \(x\)-axis represents the topicId.