# A Privacy-Preserving Scheme for a Traffic Accident Risk Level Prediction System
## 6 Results
We chose the model based on RF because the difference in accuracy between the model using the default configuration and the one using the optimal configuration is greater, so any trend in values can be identified more easily. We also chose two parameters for hyperparameter tuning: depth and estimators. The results of these experiments are shown in Table 6, and the analysis of these results and the trends of each of the evaluation metrics are shown in Figure 5. Since it is difficult to determine if a configuration makes the model more or less complex than another when both parameters are increasing, we grouped the experiments into different sets (Groups 1 to 4). In these sets, only one of both parameters increases, so it is easy to determine that the current configuration makes the model more complex than the previous one. The results confirmed that the values of the metrics (training, compiling, and inference times) increase as the model gets more complex. For instance, Group 2 (Expts. 8-12) goes from 0.359 to 1.793 s for training time, from 3.801 to 14.107 s for compiling time, and from 64.507 to 410.044 s for inference time. In the case of key generation time, it is not affected \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Experiment**} & \multirow{2}{*}{**Group**} & \multirow{2}{*}{**Parameters**} & \multicolumn{2}{c}{**Training**} & \multirow{2}{*}{**Compiling**} & \multirow{2}{*}{**Key Generation**} & \multirow{2}{*}{**Inference**} & \multirow{2}{*}{**Time**} & \multirow{2}{*}{**Accuracy**} \\ \cline{3-3} \cline{6-9}  & & & & & & & & & \\ \hline \multirow{2}{*}{1} & \multirow{2}{*}{1} & 5 & 5 & 0.155 & 1.987 & 0.747 & 5.267986 & 68.5 \\ \cline{3-3} \cline{6-9}  & & 5 & 10 & 0.159 & 2.116 & 0.711 & 10.328380 & 69.7 \\ \cline{3-3} \cline{6-9}  & & 5 & 20 & 0.312 & 3.299 & 0.719 & 21.743675 & 73.9 \\ \cline{3-3} \cline{6-9}  & & 5 & 30 & 0.300 & 2.498 & 0.778 & 34.756006 & 74.4 \\ \cline{3-3} \cline{6-9}  & & 10 & 5 & 0.359 & 3.801 & 0.545 & 64.507635 & 79.6 \\ \cline{3-3} \cline{6-9}  & & 10 & 10 & 0.542 & 5.642 & 0.501 & 132.744697 & 83.7 \\ \cline{3-3} \cline{6-9}  & & 10 & 20 & 1.093 & 10.623 & 0.554 & 266.056506 & 85.7 \\ \cline{3-3} \cline{6-9}  & & 10 & 30 & 1.793 & 14.107 & 0.521 & 410.044873 & 86.9 \\ \cline{3-3} \cline{6-9}  & & 20 & 5 & 0.533 & 6.089 & 0.562 & 98.318843 & 81.0 \\ \cline{3-3} \cline{6-9} \multirow{3}{*}{3} & & 20 & 10 & 1.255 & 9.475 & 0.533 & 187.311936 & 83.5 \\ \cline{3-3} \cline{6-9}  & & 20 & 20 & 2.903 & 17.774 & 0.531 & 257.538898 & 85.2 \\ \cline{3-3} \cline{6-9}  & & 20 & 30 & 2.992 & 21.486 & 0.642 & 697.009592 & 84.5 \\ \cline{3-3} \cline{6-9}  & & 30 & 5 & 0.550 & 5.317 & 0.518 & 71.353805 & 81.0 \\ \cline{3-3} \cline{6-9}  & & 30 & 10 & 1.026 & 10.651 & 0.559 & 149.668607 & 83.5 \\ \cline{3-3} \cline{6-9}  & & 30 & 20 & 2.189 & 15.220 & 0.537 & 294.049088 & 85.2 \\ \cline{3-3} \cline{6-9}  & & 30 & 30 & 3.072 & 22.392 & 0.653 & 805.031174 & 84.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Evaluation of an FHE-based Random Forest model using different configurations. largely by the hyperparameter tuning. Those times vary between 0.501 s and 0.778 s. As can be seen in Figure 5, key generation time remains stable for all the experiments. Considering that the model used in Experiment 1 (Table 6) is less complex in terms of the number of depths and estimators and the one used in Experiment 8 is more complex, we identified an ascending trend in accuracy (see Figure 5). Thus, it increased from 68.5 (depth = 5, estimators = 5) to 86.9 (depth = 10, estimators = 30). The other metrics also suffered increases; some increases were huge. For instance, the compiling time increased from 1.987 (Expt. 1) to 14.107 s (Expt. 8), and the inference time increased from 5.267 to 410.044 s (see Figure 5). Figure 4: Analysis of the results obtained in evaluating the encrypted models using their optimal configurations (Table 5). Figure 5: Analysis of the results obtained in evaluating the encrypted model based on Random Forest (Table 6).