## 5 Results
### Evaluation of the Artifacts
#### 5.2.3 Operationality Evaluation  
The sequence diagrams on model training, deployment (Figure 7) and inference (Figure 8) illustrate how decision making of route selection is facilitated with a ML-based model in the emulator. Operationality of the proposed architecture is supported by the online prediction error (\(\sim\)6-8%) being close to the offline accuracy (0.935 \(\pm\) 0.05). Additionally, realized OWD based on the ML-based model was lower in comparison to a heuristic approach (Figure 14).  
Figure 2 illustrated the conceptual view of continuous ML in emulation environments. The execution time of each processing step can be estimated based on the experiments. First, new data had to be collected for model training purposes by running initial emulations, where the vehicles moved and transferred data (steps 1.2-1.3). In practice, the duration of  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**Short Test: 6 min**} & \multicolumn{2}{c}{**Long Test: 1 h**} \\ \hline \hline Vehicle 1 & OWD (ms) & \
\begin{tabular}{c} Prediction \\ error (\%) \\ \end{tabular} & OWD (ms) &
\begin{tabular}{c} Prediction \\ error (\%) \\ \end{tabular} \\ RSSI & 15.40 & & 16.10 & \\ ML & 14.58 & & 14.03 & \\ ML improvement (\%) & 5.34 & 5.70 & 12.85 & 7.52 \\ Vehicle 2 & & & & \\ RSSI & 15.16 & & 15.90 & \\ ML & 13.42 & & 14.04 & \\ ML improvement (\%) & 11.53 & 6.11 & 11.71 & 5.38 \\ \hline \hline \end{tabular}
\end{table}
Table 2: A summary of emulation results when ML-based decision making was compared to the reference (heuristic approach).  
Figure 14: OWD of vehicle 2 in the short emulation experiment (6 min), where decision making was facilitated with a ML-based model. A simple heuristic (RSSI-based) model was used as a reference case.  
an offline emulation was 20 min, and the emulation had to be executed separately for each access point and moving vehicle combination. Thus, execution of the initial emulations (Section 5.2.2) took 12 \(\times\) 20 = 240 min (six Aps and two moving vehicles). Subsequently, training and evaluation of the models were performed (steps 2.1-2.4) with three algorithms (Table 1). The duration of training a model with RandomForestRegressor, as well as saving and conversion of the model to ONNX-format took 78.7 s. The selected ONNX-model (491 kB) was transferred and deployed from MLFlow to RedisAI in a few seconds (step 2.5). Finally, emulations with the selected ML model (and reference case) were executed (step 3.2). The emulations had to be performed separately for each moving vehicle. Thus, the total time required for the online emulations with the ML model and the reference case (Figure 14) was emulation time \(\times\) number of emulations (6 min \(\times\) 4 + 60 min \(\times\) 4 = 4 h 24 min).  
The total execution time (-8 h, 30 min) of the continuous ML cycle was largely dependent on the execution time of emulations. Additionally, manual configuration of the emulations in a distributed computing environment, configuration of ML training jobs, and evaluation of ML-based models required additional effort. Thus, minimization of the total execution time for continuous ML remains an item for future work.