## 3 Results  
As mentioned in Section 2.1.1, this research collected the data of car sales volumes and multi-channel online sentiment volume from 2014 to 2019. For training and evaluating the three prediction models (classical, sentimental, and CNN-LSTM) proposed in Section 2.2, the collected data from 2014 to 2018 were used as the training dataset, and the data of 2019 were used as the test dataset. This section reviews and compares the forecasting performances of the three models to verify the effects of the online sentiment data and the CNN-LSTM model on forecasting of car sales movement directions in Taiwan. The performances of the classical, sentimental, and the CNN-LSTM models proposed in Section 2.2 were evaluated with four indices, including accuracy, precision, recall, and F1-score. These measures are defined in Equations (9)-(14) based on the confusion matrix shown in Table 6.  
\[Accuracy=\frac{DD+FF+UU}{N} \tag{9}\]  
\[Precision\ (x)=\left\{ \begin{array}{ll}\frac{DD}{P(U)},\ if\ x=D\\ \frac{FF}{P(F)},\ if\ x=F\\ \frac{UU}{P(U)},\ if\ x=U\end{array} \right. \tag{10}\]  
\[Precision=\frac{P(D)\times Precision(D)+P(F)\times Precision(F)+P(U)\times Precision (U)}{N} \tag{11}\]  
\[Recall\ (x)=\left\{ \begin{array}{ll}\frac{DD}{A(D)},\ if\ x=D\\ \frac{FF}{A(F)},\ if\ x=F\\ \frac{UU}{A(U)},\ if\ x=U\end{array} \right. \tag{12}\]  
\[Recall=\frac{A(D)\times Recall(D)+A(F)\times Recall(F)+A(U)\times Recall(U)}{N} \tag{13}\]  
\[F1-score\ =\ \frac{2\times Precision\times Recall}{Precision+Recall} \tag{14}\]  
Accuracy is an intuitive measure of the performance of a prediction model. As defined in Equation (9), accuracy is the ratio of the correct predictions to the total number of predictions. However, accuracy is great only when the dataset is balanced. Since the dataset for forecasting car sales trends in Taiwan collected in this research had an uneven \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{2}{c}{**Number of Predictions for Car Sales**} & \multicolumn{2}{c}{**Predicted**} & \multicolumn{2}{c}{**Total**} \\
**Movement Direction*** & **D** & **F** & **U** & **Total** \\ \hline
1-6 & D & DD & DF & DU & A(D) \\ Actual & F & FD & FE & FU & A(F) \\  & U & UD & UF & **UU** & A(U) \\ \hline Total & & P(D) & P(F) & P(U) & N \\ \hline \hline \end{tabular}  
* DD, FF, UU: The number of correct predictions for car sales movement directions D, F, and U, respectively; FD: The number of actual directions F predicted as direction D; UD: The number of actual directions U predicted as direction D; DF: The number of actual directions D predicted as direction F; UF: The number of actual directions U predicted as direction F; DU: The number of actual directions D predicted as direction U; FU: The number of actual directions F predicted as direction U.  
\end{table} 
Table 6: The confusion matrix for car sales movement direction prediction. class distribution, three other measures: precision, recall, and F1-score, also needed to be reviewed for evaluation of the performance of the three prediction models. Precision was used to evaluate the correctness of the prediction of a model for each kind of car sales movement direction. As shown in Equation (10), the precision of a specific direction is the ratio of the correct predictions within the total predictions of that direction. That is, it removes a type of prediction for U, F, and D, and evaluates the accuracy of the prediction. To evaluate the overall precision of a model, Equation (11) defines the precision as the weighted average of the precision of individual directions. On the other hand, recall is used to evaluate the ability of a model to correctly identify the observations in each kind of car sales movement direction in the dataset. As defined in Equation (12), the recall of a specific direction is the ratio of the correct predictions among the observations with actual values of that direction. To evaluate the overall recall of a model, the recall is defined as Equation (13), as the weighted average of the recall of individual directions.