## 3 Results
In some situations, precision or recall will be maximized at the expense of the other metric. To take both precision and recall into account, Equation (14) defines the F1-score as the harmonic mean of precision and recall. Therefore, the F1-score is an overall measure of a model's accuracy that combines precision and recall [40]. The F1-score is usually more useful than accuracy, especially if the dataset has an uneven class distribution. As mentioned in Sections 1 and 3, the classical, sentimental, and the CNN-LSTM models were designed to verify the effects of the online sentiment data and the CNN-LSTM model on the forecasting of car sales movement directions in Taiwan. Since the sentimental model was designed by adding the sentimental data to the classical model as the predictor variables for that purpose, the effects of multi-channel online sentiment data on car sales prediction accuracy could be reviewed by comparing the performances of the classical and sentimental models. Furthermore, since the CNN-LSTM model adopted a deep learning method integrating the CNN and LSTM networks instead of the MLR method in the sentimental model, the effects of the CNN-LSTM model on car sales prediction accuracy could be verified by comparing the performances of the two models. Consequently, the remainder of this section will first review the confusion matrix of the testing results. Tables 7-9 show the aggregate confusion matrices of the classical, sentimental, and the CNN-LSTM models, respectively, for the six car brands. Based on the confusion matrices, the four performance indices of accuracy, precision, recall, and F1-score were then computed and compared model by model to evaluate the effects of online sentiment data and the CNN-LSTM model on the forecasting of the car sales movement directions. Figure 4 shows the accuracy of the classical, sentimental, and the CNN-LSTM models for six car brands. For all but TOYOTA, their prediction accuracies of the sentimental model were higher than those of the classical model. TOYOTA had accuracies of 50% in both the classical and sentimental models. On average, the sentimental model (51.39%) provided an improvement in accuracy of 9.72% over that of the classical model (41.67%) due to the addition of multi-channel online sentiment data as the predictor variables to predict car sales movement directions. Furthermore, the accuracies of the CNN-LSTM model for all six brands were higher than those of the sentimental model. On average, the CNN-LSTM model (69.45%) achieved accuracy that was 18.06% higher than that of the sentimental model (51.39%) due to the adoption of the CNN-LSTM network instead of the MLR method. In total, the CNN-LSTM model demonstrated an average improvement in accuracy that was 27.78% higher than that of the classical model. Consequently, according to the above comparison, the accuracy of forecasting of car sales movement directions in Taiwan was indeed improved by using online sentiment data and the CNN-LSTM model. However, accuracy is an intuitive and great measure only when the dataset is balanced. The collected dataset for forecasting car sales movement directions in Taiwan had an uneven class distribution. Hence, this research further examined three other measures, i.e., precision, recall, and F1-score, to evaluate the performances of the three prediction models. Figure 5 shows the precision of the classical, sentimental, and the CNN-LSTM models for six car brands. As mentioned above, precision is a measure of the ability of a model to correctly identify the observations from the predictions of a specific car sales movement direction. As shown in Figure 5, the precision of the sentimental model for all six brands was higher than that of the classical model. On average, the precision of the sentimental model (0.55) was 0.16 higher than that of the classical model (0.38). Furthermore, the precision of the CNN-LSTM model for all but Lexus was higher than that of the sentimental model. Lexus had high precision of 0.83 in both the sentimental and the CNN-LSTM models.