# Automotive OEM Demand Forecasting: A Comparative Study of Forecasting Algorithms and Strategies
## 5 Conclusions  
This research compares 21 forecasting techniques (baseline, statistical, and ML algorithms) to provide future demand estimates for an automotive OEM company located in Europe. We use various internal and external data sources that describe the economic context and provide insights on future demand. We considered multiple metrics and criteria to assess forecasting models' performance (R\({}^{2}\)adj, MASE, the ratio of forecasts with less than 30% error, and the ratio of forecasts with under-estimates)--all of them magnitude-agnostic. These metrics and criteria allow us to characterize results to be comparable regardless of the underlying data. We also assess the statistical significance of results, something we missed in most related literature. The obtained results show that grouping products according to their demand patterns or past demand magnitude enhances the performance of ML models. We observed that the best MASE performance was obtained on models created for a group of products with the same demand type. Furthermore, when training global models based on the median of past demand, models usually achieved a better R\({}^{2}\)adj and a better bound on high forecast errors. However, these values were not always statistically significant. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Algorithm Type** & **Algorithm** & **R\({}^{2}\)adj** & **MASE** & **5\% Error** & **10\% Error** & **20\% Error** & **30\% Error** & **UE** & **90\% Error** \\ \hline \multirow{3}{*}{ML batch} & SVR & **0.9212** & 0.2600 & 0.0774 & 0.1101 & 0.2321 & 0.3333 & 0.4077 & 0.3304 \\ \cline{2-10}  & Voting & 0.9059 & 0.2800 & 0.0625 & 0.0923 & 0.1786 & 0.2798 & 0.4792 & 0.3393 \\ \cline{2-10}  & RFR & 0.8953 & 0.2900 & 0.0417 & 0.1012 & 0.2173 & 0.3244 & **0.3423** & 0.3482 \\ \hline \multirow{3}{*}{ML streaming} & ARFR (Experiment 9) & 0.8728 & 0.3300 & 0.0744 & 0.1339 & 0.2500 & 0.3274 & 0.5387 & 0.3452 \\ \cline{2-10}  & ARFR (Experiment 10) & 0.8205 & **0.2200** & 0.0744 & 0.1280 & 0.2232 & 0.3274 & 0.5268 & 0.3423 \\ \hline \multirow{3}{*}{Baseline} & MA(3) & 0.8938 & 0.8800 & 0.1190 & 0.1667 & 0.2530 & 0.3482 & 0.3571 & 0.3065 \\ \cline{2-10}  & Naive & 0.8519 & 1.0000 & **0.2024** & **0.2411** & **0.3423** & **0.4137** & 0.4137 & 0.3214 \\ \hline \multirow{3}{*}{Statistical} & ARIMA(2.1.0) & 0.3846 & 0.4500 & 0.0476 & 0.0774 & 0.1429 & 0.1875 & 0.5536 & 0.5208 \\ \cline{2-10}  & Exponential smoothing & 0.3258 & 0.3600 & 0.0506 & 0.1161 & 0.1905 & 0.2738 & 0.5923 & 0.4434 \\ \cline{2-10}  & ARIMA(1.1.0) & 0.2840 & 0.5200 & 0.0387 & 0.0744 & 0.1012 & 0.1726 & 0.5119 & 0.6071 \\ \cline{2-10}  & Random walk & \(-\)0.6705 & 0.9000 & 0.0327 & 0.0387 & 0.0655 & 0.0923 & 0.3780 & 0.7678 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results we obtained for the top 3 performing models from Experiment 8 (ML batch models), best result for experiments 9–10 (ML streaming models), and baseline and statistical models. We abbreviate _under-estimates_ as _UE_. Our experimental evaluation indicates that the best performing models are SVR, voting ensemble, and RFR trained over product data of the same demand type. The SVR and RFR models achieved near-human performance for the ratio of forecasts under 30% error, and the RFR model scored close to human performance regarding under-estimates. However, none of the models achieved close to human performance on the proportion of forecasts with a high error (more than 90%). How to efficiently detect and bound such cases remains a subject of future research. When comparing batch and streaming ML models' performance, we observed that ML batch models displayed a more robust performance. From the streaming algorithms, the ARFR achieved competitive results, except for a high ratio of under-estimates. This critical aspect must not be overlooked. Models based on the Hoeffding inequality did not learn well and had poor performance, and further research is required to understand the reasons hindering these models' learning process.