## 2 Methodology
### Prediction Model Structure Construction
#### 2.2.3 The CNN-LSTM Model  
In the last section, for the sentimental model, online sentiment data were added to the predictor variables of the classical model in an attempt to improve the overall performance of forecasting of car sales movement directions in Taiwan. This section presents the development of the CNN-LSTM model, which integrated the CNN and LSTM networks instead of the MLR method used in the sentimental model to improve the overall performance of forecasting of car sales movement directions in Taiwan. The basic idea of the utilization of these models is that LSTM models are appropriate for dealing with time series data, while CNN models may filter out the noise of the input data and extract more valuable features [34; 35].  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c c c c c c c c c c} \hline \hline \multicolumn{1}{c}{**No**} & \multicolumn{1}{c}{**Rinetical Scale**} & \multicolumn{1}{c}{**Same-Month Last Year Label**} & \multicolumn{1}{c}{**Month Number**} & \multicolumn{1}{c}{**Dissipative Forens**} & \multicolumn{1}{c}{**Social Media**} & \multicolumn{1}{c}{**QKA Publications**} & \multicolumn{1}{c}{**Bings**} & \multicolumn{1}{c}{**Norms**} \\
**0** & **Month** & **Rational Scale** & **Same-Month Last Year Label** & **Month Number** & **P** & **N** & **T** & **P** & **N** & **T** & **P** & **N** & **T**  
Figure 3 presents the architecture of the CNN-LSTM model. A sliding window [36] is first used in pre-processing for data extracting for the CNN-LSTM models. Then, the CNN-LSTM model sequentially consists of a one-dimensional convolutional neural network (1D CNN), a CNN network, an LSTM network, a fully connected network, and an output layer.  
Basically, the CNN-LSTM model used the same predictor variables as the sentimental model with the exception of the historical car sales volumes. After a test of prediction performance, this research chose to use the past 60-day (2-month) data of predictor variables to predict the next-month (1-month) response variable of car sales movement directions. Therefore, this research uses a 90-day (3-month) window with a 30-day (1-month) sliding step to extract the training data for CNN-LSTM model from the collected dataset. Each sliding window includes a set of 2-month predictor variables (\(X_{i}\)) and a 1-month response variable (\(Y_{i}\)). The data of predictor variables was standardized and input via a one-dimensional convolutional neural network (1D CNN) developed for processing data arranged in a single dimension [26; 37]. Since the multi-channel daily sentiment data was collected on a daily basis, its 60-day data was extracted directly to form the 1D 60-day input data. However, because both the same-month-last-year sales movement direction label and the month number were monthly-based, their two monthly data must be duplicated to form the 1D 60-day input data, i.e., one monthly data was duplicated to 30 daily data.  
After receiving data via the 1D CNN layer, a CNN network containing five convolutional and max-pooling layers filtered out the noise of the input sentiment data and extracted sentiment change features for the prediction of car sales movement directions in Taiwan. In addition, both the daily online sentiment data and the monthly car sales data were time series. Moreover, the LSTM network, which has a feedback loop for processing the entire data sequence, is always used for classification, processing, and forecasting based on time series data [27].  
Figure 3: The architecture of the CNN-LSTM model.  
Therefore, an LSTM network and a 4-layer fully connected network following the CNN network were used to generate the output of car sales movement direction prediction. Some considered hyperparameters of the CNN-LSTM model are presented below: the activation function was relu; the loss function was cross_entropy; the optimizer was adam; the learning rate was 0.003. Finally, the earlystop method was used to solve the overfitting problem [38; 39].